COMPUTER - BRAIN COMMUNICATION POSSIBILITIES OF A HYBRID MIND

Roberto Etchenique
Departamento de Qumica Inorgnica, Analtica y Qumica Fsica Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires. Ciudad Universitaria, Pabelln 2, Piso 3, INQUIMAE, C1428EHA, Buenos Aires ARGENTINA email: [email protected]

Abstract. Robots and androids has been depicted as having their own intelligence, based in computer like brains. But this approach is not unique. A human brain could be connected to computers and mechanical parts, and the result would be a hybrid organism, even an composite mind. The obligated milestone in this race is to achieve the communication between a human brain and a computer using a common code, but the codification of data inside the brain is far to be known. We will discuss the actual possibilities of connection between biological and electronic brains, focusing in the huge gap that nowadays separate both worlds.

Introduction:
The human brain is capable of multiple and diverse tasks, most of them with almost not awareness by the being that this very brain conforms. Every human trained in basic arithmetics thinks something related with the concept five when asked how much is two plus three. A person with some further skills in the subject will imagine a curve with steep beginning and shallow continuation when thinking in a logarithm. However, probably no one could answer the result of the logarithm of 78, with the possible exception of some idiot savants. The conceptualization and calculations behind all these behaviors are completely unknown in the actual neurosciences. The codification that the brain uses in order to determine the objects, concepts and relations between them is also unknown. Some general principles have been proposed, the most known the putative parallel coding of the algorithms in the brain, based in the anatomic and physiological properties of the neuronal tissue, which presents highly branched and connected structures and slow data processing times. One of the goals of the XXI century neurosciences could be the discovery of the code or codes that represents the data and relations between them in the brain, animal or human. By knowing the codification, the connection between brains and inorganic machines would becomes feasible, and a enormous number of applications could turn into reality.

Roberto Etchenique

The word "robot" brings to the mind the image of a metal-plastic entity that somewhat resembles a human shape. This is a kind of machine that since ScienceFiction popularized, from Kapek to Asimov, and though robot machines paint cars in factories since the '80s, no layman would think of this ugly devices as a real robot. In a robot, the main component is the brain. This brain should be electronic (or "positronic" !) and in this way it establishes a clear difference with the human "biological" brains. If this brain is surrounded with mechanical limbs, we have a robot. If the organs and limbs are mostly organic, the robot appears as a human being, and we have an "android". In order to build a robot or an android, one need to know how to produce "intelligence" within a computarized brain and how to control mechanical or biological parts. This "intelligence" is by much the difficult part of the process, since the control of limbs is nowadays a problem almost solved. But there are other possibilities to build an intelligent machine. One of them consists in using a real biological brain and use it to control mechanical parts. In this way we get the intelligent part as a whole, borrowed from an animal, that can be a human being. A sort of this machine is depicted in Robocop, in which a human brain is embedded into a robotic body. In this case, one need to know the codification that this brain uses to make the interfase between the brain and the limbs, and we obtain also the possibility of communication between that brain and a traditional computer as a side prize. In this approach, the main goal is to obtain the knowledge of the functioning and codification inside the human brain.

The first steps:

Although it is not demonstrated that the human brain behaves internally as some kind of computer, it is evident that humans process information and many computations are indeed done. In this section, I will analog the human brain to a digital computer, just in order to visualize the extreme ignorance of the processes that the brain make, not to declare any internal similarities between brains and computers. Let' s imagine a factory that produces computers. They build 99% of reliable units and 1% of defective ones. These defective units have a problem in the power supply, which gives a lower voltage to the mainboard, that generates random errors and periodical resetting of the computer, making it unusable. Rising the AC input voltage by a 20% the defective power supply can give the right voltage outputs. The simplest solution for the tens of thousands of owners of defective units would be the purchase a voltage elevator capable of rise the voltage from 220 V to 260 V, forgetting the trouble to change all the CPUs. The solution works, but... did anyone say that this is computer science or electronic engineering ? The actual brain medical sciences apply almost always this kind of therapy. Some diseases are known to be related with low levels of the neurotransmitter serotonin inside the brain. The solution is the application of exogenous serotonin in order to reestablish levels ! If your painting robot machine shakes its arms and does not paint well, did you ever think about getting out some part of the circuit from its mainboard by breaking it with a knife ? However, this is one of the surgical methods for Parkinson syndrome [Giladi and Melamed, 2000]. Would you dare to cut a bundle of cables inside your

COMPUTER - BRAIN COMMUNICATION POSSIBILITIES OF A HYBRID MIND

machine to see if something goes better in it ? Well, this is exactly the medical procedure called "corpus callosum sectioning", used as a treatment for certain epileptic disorders [Hodaie et al., 2001]. In brief, as we do not know almost anything about the hardware, firmware and software of our brain, we use the old technique: deep the screwdriver, make some sparks and see what happen. If it worked once, we can try again with a new patient. We humans call to this approach "neuroscience".

Complexity:
When the first microprocessor was presented in 1971, the Intel 4004 contained 2300 2 transistors in a 12 mm area. One of the smallest studied nervous systems, that of the nematode Caenorhabditis elegans comprises 302 neurons in total [White et al., 1986]. A neuron is capable of much more complex calculations than a single transistor, its capabilities are far beyond to that of a operational amplifier. If we assume at least a hundred transistors for neuron (and it could be easily more than a million!), a simple comparison indicates that the first P was much less complex than the simplest living nervous system under study. The last Intel P , the Pentium 4 Extreme Edition, has 6 2 178x10 transistors in a 237 mm area. This complexity is, in our conversion, comparable with a honeybee brain, [Menzel, 2001] that uses around a million neurons. A human brain, on the other hand, is composed by 1011 neurons. In our scale, this 13 5 would be made by no less than 10 transistors, that would occupy 10 Pentium 4 Extreme Edition , which is probably a 1000 fold underestimation of the real complexity, to say nothing about the internal connectivity or the software capable to make this hundreds of thousands of microprocessors work. What about the robustness that presents a nervous system of an animal, including man ? A human brain can have some processes stopped during certain time, but a global crashdown is not acceptable, unless you want to go to a mental hospital and remain there or even become dead. The fact of the minute proportion of mentally ill persons is an unequivocal sample of the incredible power of natural evolved neuronal technology.

Levels of explanation:
A popular screen-saver consists in a dark screen on which some geometrical bodies grow, deform and shrink, while they move all around the screen. Lets assume that this screen-saver is working on a laptop. Its LCD screen is composed by 1280 lines of 1024 dots each, any of them capable to be set to green, blue and red in 256 (8 bits) brilliance levels for each primary color. Now a Martian wants to reproduce the behavior of this laptop, and to do so, he registers the color values of every point in every tenth of a second, during certain time, say 15 minutes. With this data he wants to build a machine that behaves exactly 11 the same way. He will need 1280 x 1024 x 15 x 60 x 10 x 8 x 3 = 3x10 bits of information, and therefore a huge electronic circuit to codify and store those bits.

Roberto Etchenique

However, the laptop that showed this screensaver had just 128x106 memory bits, and was also capable to serve as a word processor, or run a spreadsheet and a minesweeper. As we do not know the codes that lie behind the behavior and computations of a brain, we work like the Martian of the story. Neuroscientists register the firing of individual neurons or groups of them and try to determine whats under that. Many studies of the senses were done on this basis, but almost nothing is yet known about the involved codification.

Inputs:
The inner ear is one of the more complex structures of the body. The main piece of the system is the cochlea, an organ composed by two membranes that contain hair cells, neurons that convert mechanical movements into electrochemical signals. While the anatomy of the cochlea is known since long time ago, just in the last decades the involved mechanisms in the sound perception are being discovered. The knowledge of the codification in the auditory system is crucial for the devising of treatments and prostheses for the profoundly deaf. The information-bearing components of the speech signal include both amplitude and frequency variation over time. Frequency discrimination is generated mechanically in the cochlea (Moller, 1999). According to the rate-place representation, sound is coded by the relative distribution of neural firing patterns as a function of signal frequency: high frequency pure tones elicit a higher rate of firing from auditory nerve fibers at the base of the cochlea and nerve fibers at the apex respond best to low frequency tones. The auditory nerve fibers within the cochlea are therefore described as having a tonotopic organization. Several studies have shown that the tonotopical representation is maintained, and possibly even enhanced, in the auditory cortex (Aitkin, 1990). This is one of the few examples in which the coding is known, and therefore, can be taken into account for technological purposes. The cochlear implant is the only available medical device that can restore partial hearing to patients with profound deafness. The implant bypasses the damaged region of the inner ear and stimulates the auditory nerve directly with an electrical representation of the acoustic signal. Currently there are over 30,000 cochlear implant users worldwide, with some being able to carry on a conversation via telephone. The bionic ear, is nowadays the only existing cybernetic organ that is more than a dumb mechanical part. The retina is a layer of cells, in which some neurons can be excited with light. These neurons are called receptors. The scenes are projected on the retina by means of the cornea and crystalline. There are two kind of receptors: cones and rods. The firsts are in charge of the color vision and the seconds just can see in B&W, or luminance, but with much more accuracy and sensitivity. In darkness, the cones are not sensitive enough and all our vision (scotopic) rely on the rods, and thats why we cannot distinguish the colors in a dark situation. The signals of the receptors are combined in the retina itself, and the compressed information is sent to the visual cortex, in the brain. It is know that there exists a somewhat distorted pattern of neuronal activity in the visual cortex, that resembles the image on the retina. However, this transmission of the visual scene from the retina to

COMPUTER - BRAIN COMMUNICATION POSSIBILITIES OF A HYBRID MIND

the cortex does not imply that this is the vision. Who is in charge of watching at the cortex and see the transmitted retinal image? The way in which the images are codified in the cortex has been investigated since the seminal experiments of Hubel and Wiesel. In these experiments, the researchers demonstrated that in the visual cortex of the cat there exist some neurons that are responsive to vertical, horizontal or slanted lines of different angles. [Hubel and Wiesel, 1959] After that, it was clear that the movement was also codified in function of the firing of certain cells, and the picture of the full codification of the scene in terms of edges, boundaries, movements and colors began to become apparent. Attempts to stimulate vision by electrically stimulating the cerebral cortex go back at least to 1918, when Lowenstein and Borchardt accidentally stimulated the striate (visual) cortex of a man undergoing surgery for a bullet wound to the head. The patient reported a twinkling sensation. Subsequent experiments throughout the century confirmed this finding and showed that the stimulated perceptual flashes, now called phosphenes, always reoccur in the same part of the visual field as long as the same part of the cortex is stimulated. This discovery opened a way towards computer assisted vision to treat the blindness. Electrodes implanted in the visual cortex are connected to a fully implanted stimulator/receiver module. A camera and a small computer system will register an image, digitize it, and transfer it to the cortex, bit by bit, in order to create a perception of the image. Each point in the image is thus represented in the cortex by means of an electrode that stimulates the appropriate location, creating a perceptual phosphene. Figure 1 depicts the mental image we have of using the phosphene method to represent a face. We tend to consider that the mental image would look something like panel B; (medium resolution and an even gray distribution). However, our current best would probably look something like panel C, black and white image (reflecting the fact that we only know how to turn neurons on and off) and a resolution of 8x8 pixels, according with our excitation technology through implanted extracellular electrodes.

Roberto Etchenique

Fig. 1. A real picture as appear to us is depicted in A. B is somewhat we tend to think it would appear with a "bionic eye", gray scale and some poor resolution. However, a more realistic image is that of C, just B&W and about 8x8 pixels.

We would like to obtain something similar to panel B, or even panel A, but the problem is not just that it requires more resolution (say, more electrodes to excite the visual cortex) but rather that the visual system does not work this way. The images we see are not computed in terms of pixels; rather, images are represented in terms of edges, textures, colors and motion. Attempting to stimulate vision through bitmapping is akin to simulate the screensaver with the brute-force approach of our Martian. It simply is the wrong way. No one has ever found a visual neuron that encodes simple spots. On the other hand, most striate neurons are sensitive to orientation, spatial frequency (texture), color, movement direction, and speed. These are the dimensions of vision, so it is logical that to communicate visual data to and from the brain we should be capable to understand that code.

Outputs:
Amyotrophic Lateral Sclerosis and Multiple Sclerosis are simply terrible. The muscles can no longer follow the orders of the brain, and the patient looses progressively the movement capability, in a serious syndrome that can lead to death. Spinal cord injury, of which there are 250,000 cases, and 11,000 new cases each year alone in the United States of America [Reeve, 2002], is its accidental counterpart. Surgical solutions are under study. Prosthesis capable to understand the data sent by the brain to move a mechanical arm or leg would diminish the suffering of those affected for these medical conditions. Although over the last two decades there have been multiple efforts to control a device directly from the thoughts, as our limbs do, just recently some important goals have been achieved. In 2003, Nicolelis group devised a system implanted into a monkeys fronto parietal cortex an electrode ensemble connected through A/D conversion to a computer. The number of electrodes ranged from 96 to 320. Monkeys were trained with a reward mechanism to follow signs that appear in a computer screen. The mental action of the monkey was detected by the computer, translated into a mathematical result and transduced into movements of a small box on the screen. In a more fashionable experiment, a robotic arm could be controlled by the monkey. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move [Carmena et.al., 2003]. In December 2004, human subjects were capable to move a cursor on a computer screen using a noninvasive detection of the brain activity. In this case, the brain signals are monitored from the scalp, using multichannel EEG in 64 electrodes distributed over the entire scalp. The results are impressive, and the control of objects in a computer screen is fast and accurate, with a target hit rate of 92% in 2D point to point experiments. [Wolpaw and Mc.Farland, 2004]. In all these works, no attempt to decode the brain signals have been done. The computer detects the patterns that correspond to certain brain activity and directs that

COMPUTER - BRAIN COMMUNICATION POSSIBILITIES OF A HYBRID MIND

toward the objective, in a way that is modulated by the subject, through a visual feedback.

Conclusions:
Interfasing computers with biological brains is one of the goals that neurosciences are working for. However, almost no knowledge about the codification of the processes in the brain has been achieved. We see the neural processes as the Martian saw a screensaver, knowing that some computation is taking place but ignoring the code and level in which this computation is working. Codification is still the main problem to be solved, although the compatibilization of our silicon-based electronics with the ion-channel based neurobiology is also a huge gap to cross. Next years could be signed by enormous advances in humancomputer connections, but the basis of the brain computing, including memory, emotions, decision making and consciousness is still an unknown universe to be discover. Hybrid minds, generated from the connection of biological and electronic components, can be the ultimate goal of this knowledge. A human-robot entity, capable to perform typical human tasks, as composing, writing, arguing and loving, while having robot-like possibilities as high speed calculations and precise manipulation of different objects could become a reality in the new century.

References:
Aitkin, L (1990). The auditory cortex, (Chapman and Hall, London). Carmena, J. M. et al. (2003), Learning to Control a BrainMachine Interface for Reaching and Grasping by Primates, PLOS Biology, 1, 2, 1-16. Giladi, N. and Melamed, E.(2000), The Role of Functional Neurosurgery in Parkinsons Disease, , IMAJ; 2:455461. Hodaie, M, et al. (2001), Image-Guided, Frameless Stereotactic Sectioning of the Corpus callosum in Children with Intractable Epilepsy, Pediatric Neurosurgery; 34:286-294. Hubel, D. H. and Wiesel. T. N., (1959) Receptive fields of single neurons in the cats striate cortex. J. Physiol. 148:574-591. Menzel, R. (2001), Searching for the Memory Trace in a Mini-Brain, the Honeybee Learning & Memory, 8:5362 Moller, A.R. (1999). Review of the roles of temporal and place coding of frequency in speech discrimination, Acta Otolaryngol, 119, 424-430. Reeve (2002) Christopher Reeve Paralysis Foundation, http://www.christopherreeve.org/.

Roberto Etchenique

Wolpaw, J.R. and McFarland, D.J.(2004), Control of a two-dimensional movement signal by a noninvasive braincomputer interface in humans PNAS, 101, 51, 1784917854.