Not A Blog™

There’s been a lot to do lately regarding the augmentation or expansion of human sensory ability. This is cool stuff; I’ve talked about it before. But recently an article caught my eye that consolidated another idea I’ve had for some time.

First of all, the notion of enhancing human abilities is nothing new. Basically, the reason this works at all is because the brain’s fantastic plasticity. In other words, rather than wiring directly into the brain to stimulate certain areas (something which has been done, though it’s more expensive, more invasive, and not as safe), existing senses can be used to input new information. In other words, some information that we can’t innately feel or sense in some way is converted into something we can.

It’s probably easier to use an example than just talk about it. A few months ago, there was a piercing artist who implanted a magnet in the tip of his finger. (It was a decidedly non-scientific venture.) He discovered that the magnet would resonate in the presence electromagnetic fields, and he could feel this movement through his finger. The sensing of a magnetic field—not something we can do naturally—was mapped to touch instead, and his brain reconfigured itself to accept this new touch input as a sixth sense. It felt different from touching the table.

Another example is someone who created a belt the vibrated in that direction of magnetic north. It was hooked up to a GPS and depending on his orientation, it would vibrate against different parts of his body. He experienced the same thing. The vibration all but disappeared, replaced by a sixth sense of direction.

Finally, the last example is one which I personally find to be the coolest, not to mention it ties into the second part of the article. There has been quite a bit of study to allow blind people to see. As I mentioned, this has been successfully accomplished with invasive procedures—that is, implanting electrodes in the brain—but more practical is accomplishing the same thing without drilling through the skull. One way is by having a computer reinterpret the image from a camera into a predictable pattern of sound, much in the same way sonar works, and input this into the blind person via headphones.

But an even better way has been found by using something called a tongue port. This is a device that zaps the tongue with weak electrical signals in certain places (although reportedly it works if you move it around, too) to interpret the image coming off the camera, mounted on a pair of glasses. Sight to touch. Using this method, non-blind people were able to “see” in only a few hours’ time. In fact, recalling the experience later, they reported remembering a high-contrast picture of black and white, rather than vibrations on the tongue.

Naturally, the first thing I thought while reading this was, “put the glasses on backwards!” If the camera was pointing out behind you, you still get legitimate sensory input representing sight, and by so doing, you’ve just increased your field of view to 360 degrees. This would be so awesome you can’t even imagine it, because we have no experience with seeing “in the round”. But how useful would this be? Very. Almost as useful as having four arms which—while I’m on the subject—would probably be the first mod I’d want (after the BCI for my car’s windshield wipers, of course).

Before I continue, I have to correct an earlier prediction I wrote. Back in Singularity, I made the following statement:

No one is going to go through the process of implanting crap in their body when it gets outdated as fast as today’s hardware does.

If these gadgets are going to be implanted into people, I can still see this as being a problem. Certainly I would think twice about sticking something electronic in myself, given the speed of obsolescence. (Then again, many people wouldn’t, so there you go.) But there are two things working against my earlier statement which will most likely invalidate it.

1. If the gadgets and interface are not implanted then they can be swapped out with no hassle. At that point, there’s really no issues at all because we do this all the time anyway. I will think nothing of dropping fifty bucks on a new pair of headphones for no other reason than the pair I’m currently using has a cord that’s too short.

But even if the gadgets are implanted, we have the following, very interesting, observation:

2. People who used the sensory augmentation devices reported, after taking them off, that they felt a loss. They suddenly were no longer complete. Never mind that no one is born with these extra abilities. It has to do, again, with the brain’s plasticity. After the brain has rewired itself to accept this new input, removing it is something like becoming blind or deaf later in life. You know there’s something missing. I would assume this loss isn’t as traumatic, since the rewiring hasn’t been reinforced very much after a few days of experimentation. But if someone was hooked up to something like this shortly after birth and it stayed with them their entire life, that would be a huge loss.

What I’m getting at is that after someone has an implant for even a few days, not having it starts to no longer be an option. They’re not going to want to “degrade” themselves and lose this capability—and so even with an implant, once it’s in there, it’s not coming out, regardless of how bad of an investment it may be.

With that in mind, this is going to be the industry to jump on for venture capitalists. The old car-salesman saying goes, “the feel of the wheel will seal the deal”: let the customer drive the car—let them take it home for a day—and you’ve got the sale in the bag. They’ve emotionally purchased it.

How much stronger will this be with sensory augmentation? You hook someone up to a device that gives them 360 degree vision (or magnetic field sensing, infrared vision, night vision, cell phone or wifi signal strength… whatever), you give someone that ability for a week and they’re not going to want to take that off, because it will become apart of them in a way that’s even more fundamental than a car is. And I’ve already talked about how the car gets mapped to the body. This is like a hundred times worse! Once people get the ability, they’re going to pay through the nose to keep it.

But I digress. My main point is that, when you get right down to it, the tongue port—or any of these things—is just fancy, computer controlled synesthesia. Most likely you already know what this is. People with synesthesia will see color when they hear music, or see numbers in particular colors. In one thing I read a number of years ago, they gave one synesthetic a page filled with ‘S’s and ‘5’s. To you or I, these characters look very similar, and it takes quite a bit of hunting to find half a dozen fives on a page of S’s. But to the synesthetic, he was able to pick out the fives as soon as he glanced at the page. He didn’t even have to be looking right at the five to see the color.

That’s very interesting, and it got me thinking along a different line. Would it be possible to train oneself to be synesthetic? We already know the brain to be very plastic and flexible. Not only that, but you’ve got the feedback reinforcement going on, too, with reward and punishment systems. I think this would be a fantastic experiment to try.

It would be a simple matter, nowadays, to write a script that codes every letter and number on a computer screen to be a particular color. Even better would be if we could get some sort of statistical data on what colors different synesthetics see certain letters and numbers. It won’t always be the same for everyone, but, for example, if “A” is red for a majority of people, then tell your script to make all “A”s red. It may stick better, this way. (On the other hand, it might not matter at all).

By the same token, you could rig up a pair of glasses with a small set of LEDs in them, that reflects a color into the lens. Then hook this up to a computer that with a microphone. Every time it hears, say, the note “A”, it displays red. Music can be very complex, so maybe it would need to take an average of the tones being heard, or maybe you’d need three or four different lights, but the point is that it could be done.

Run this script for a few weeks. I would be very interested to see if this would cause a bit of synesthesia when you walk away from the input. I think they “coloring letters on the computer screen” bit would be more practical since it doesn’t require a huge complex setup. Not to mention, if most people are anything like me, they’re staring at a computer screen for a good 14 hours a day anyway.

If I knew anything about programming, I’d be all over this. I’d be willing to bet actual, real dollars that it would work, at least to a limited extent (like for a few minutes after you set away from the computer, much like an after-image). I’d also bet that some people would take to it instantly and experience longer-term effects, while for others it would barely make an impression, if it did anything at all.

The great success of augmented sensory systems leads me to believe that my idea of trained synesthesia isn’t really that far fetched of an idea. I’m not sure that it would necessarily be a good idea to train people from a young age to be synesthetic. After all, we’d be doing a disservice to those who actually are synesthetic—especially in the case where what they see differs from the “standard”. So for example, if we made every letter in children’s books a different color based on the mean distribution of letter-color assignment by natural synesthetics, we still end up with the issue of not every synesthetic always seeing those colors. I think this would be a bad idea to try and “retrain” them, or at least “force” them to read the way everyone else does. I see no reason to try and train people this way from birth. Not to mention some people are color blind.

But if it does work, there’s definitely a certain demographic of people (of which I’m apart, naturally), who would be interested in hacking their brain in this manner.

Tying this in to another article I recently wrote, if this sort of thing actually works, we’d do well to train our outer-space colonists this way, as the ability to pick out important (numerical) information from a sheet by colors in one’s perhipheral vision could be the difference between life and death.

Anyway, even if the experiment is a failure, this would still tell us something about the way the brain works. If we can’t encourage the association of disparate senses or sensory input, maybe the brain is only plastic in certain ways. If we can “see” through our tongues, but not “see” ‘A’ as red, why is this, and what parts of the brains are actually being rewired in these circumstances? If we can induce synesthesia, but it only works for a certain period of time afterwords, maybe this means the rewiring, while possible, is not very permanent. Maybe constant reinforcement is needed (in the same way our body’s position relies on constant reinforcement from our eyes to “error check” the innate sense). But then again, it could just be a matter of practice. Even when it doesn’t stick, one who has had experience with it can get back into the state faster than someone doing it for the first time.

Regardless, someone needs to get on that experiment. I can’t think of any good reason why no one’s tried it yet.

-Ted