A new device to help the blind see has been developed by scientists. The electric lollipop or BrainPort vision device captures images using a tiny camera and then converts the image into tiny tingles on the tongue. The tingles are then sent to the brain which then converts the tingles into pictures. After a few days practicing people, who otherwise couldn’t see, were able to make out shapes, read signs and even read letters. This amazing new device may help people to interact with their environment in ways never before experiences.
Using the unique resources of the DOE national laboratories in materials sciences, micro fabrication, microelectrode construction, photochemistry and computer modeling, the project’s goal is to construct the device, capable of restoring vision, with materials that will last for the lifetime of a blind person. Just as blind people read words by touching Braille bumps, some are now able to “see” objects via a special lollipop that stimulates their taste buds. The extraordinary device converts images captured by a tiny camera into a series of electrical tingles, which can be felt on the tongue.
Nerves then send these messages to the brain, which turn the tingles back into pictures.
An electric lollipop that allows the blind to ‘see’ using their tongue has been developed by scientists.
Fig.1 Position of BrainPort vision device
The machine is called the Brain Port vision device and is manufactured by Wicab, a biomedical engineering company based in Middleton, Wis. It relies on sensory substitution, the process in which if one sense is damaged, the part of the brain that would normally control that sense can learn to perform another function.
About two million optic nerves are required to transmit visual signals from the retinathe portion of the eye where light information is decoded or translated into nerve pulsesto the brain’s primary visual cortex. With Brain Port, the device being developed by neuroscientists at Middleton, Wisc.based Wicab, Inc. (a company co-founded by the late Back-y-Rita), visual data are collected through a small digital video camera about 1.5 centimeters in diameter that sits in the center of a pair of sunglasses worn by the user. Bypassing the eyes, the data are transmitted to a handheld base unit, which is a little larger than a cell phone. This unit houses such features as zoom control, light settings and shock intensity levels as well as a central processing unit (CPU), which converts the digital signal into electrical pulses replacing the function of the retina. Part of the challenge of Brain Port is to train the brain to interpret the information it receives through the stimulation device and use it like data from a natural sense. Research from prototype devices showed such training is possible, as patients with severe bilateral vestibular loss could, after time, maintain near-normal posture control while sitting and walking, even on uneven surfaces.
WHY KNOWN AS TASTING DEVICE:
Other than the normal use of tongue for tasting the food, eating, talking there are also many other uses. One of them is for sensing of light. It is called as tasting because it can taste the light and sense the objects. It is this property which is used in BrainPort vision device.
II. PARTS OF DEVICE
1 — Camera on the forehead captures the image of test symbol.
2 — Video output is sent to a processor.
3 — Processor translates output from camera into a pattern of electronic pulses that are sent to an array of electrodes held against the tongue.
4 — Array of electrodes a little over an inch square stimulate receptor cells on the surface of the tongue to the Brain i.e., Tactile or touch receptors on the tongue send impulses to the somatosensory cortex in response to stimulation.
- INTRA ORAL DEVICE(LOLLIPOP)
It consists of three parts
- ELECTRODE ARRAY
Fig.4 electrode array which is placed on tongue
Fig.5 top view of electrode array
The lollipop contains a square grid of 400 electrodes which pulse according to how much light is in that area of the picture. White pixels have a strong pulse while black pixels give no signal. The control unit converts the image into a low resolution black, white and grey picture, which is then recreated as a square grid of 400 electrodes – around the size of a postage stamp – on the lollipop. Each of the electrodes pulses according to how much light is in that area of the picture. It converts pictures into electrical pulses and it is placed on tongue.
Fig.6 Electrode device
A programming device comprising: a user interface; and a processor that presents a user with an interface for selection of one of a constant current mode or a constant voltage mode via the user interface, receives a selection of one of the modes from the user via the user interface, configures a medical device according to the selected mode, and presents the user with either an interface for selection of a voltage amplitude or an interface for selection of a current amplitude via the user interface based on the selected mode, wherein the processor configures the medical device to measure, using impedance measurement circuitry, an impedance presented to stimulation circuitry of the medical device based on the selected mode, and wherein, when the medical device comprises constant current stimulation circuitry and the user selects the constant voltage mode, the processor configures the medical device to measure, using the impedance measurement circuitry, the presented impedance and adjust a stimulation current amplitude based on the measured impedance to deliver stimulation with a substantially constant voltage amplitude.
The other side of electrode array is accelerometer. Named Brain Port, and developed by Wicab, Inc, this experimental device uses an accelerometer to provide head and body position information to the brain through electrotactile stimulation of the tongue. Sensitive nerve fibers on the tongue respond to electrodes to enable a rapid transfer of electrical information.
- SUNGLASSES AND CAMERA
The device is made up of a video camera hidden in a pair of sunglasses, which the user wears. Signals from the camera are sent along a cable to a handheld control unit, about the size of a cell phone, and then to a lollipop-shaped stick, which is placed on the tongue. The inventors claim that blind people using the device, that look like sunglasses attached by cable to a plastic lollipop, blind people can make out shapes and read signs with less than 20 hours training. The Brain Port device collects visual data through a small digital video camera about 1.5 centimeters in diameter that sits in the middle of a pair of sunglasses worn by the user.
- CPU, BATTERY
This unit houses such features as zoom control, light settings and shock intensity levels as well as a central processing unit (CPU), which converts the digital signal into electrical pulsereplacing the function of the retina. It will be a rechargeable battery
- POWER BUTTON
It is used for start and stop.
III.WORKING OF DEVICE
- About two million optic nerves are required to transmit visual signals from the retinathe portion of the eye where light information is decoded or translated into nerve pulsesto the brains primary visual cortex.
- Visual data are collected through a small digital video camera. Bypassing the eyes, the data are transmitted to a handheld base unit, which is a little larger than a cell phone.
- From the CPU, the signals are sent to the tongue via a lollipop, an electrode array about nine square centimeters that sits directly on the tongue.
- Densely packed nerves at the tongue surface receive the incoming electrical signals, which feel a little like Pop Rocks or champagne bubbles to the user.
- These signals from tactile or touch receptors cells are sent to the somatosensory cortex in response to stimulation in the form of pattern impulses.
- Although users initially ‘feel’ the image on their tongue, with practice the signals activate the ‘visual’ parts of the brain for some people.
- In any case, within 15 minutes of using the device, blind people can begin interpreting spatial information via the Brain Port.
IV. WHY DEVICE SHOULD BE PLACED ON TONGUE
Fig.8 position of electrode array on tongue
- Other parts of the body, such as the back, were not sufficiently sensitive. The fingertips were sensitive enough, but people wanted full use of their hands to grip a cane or to grab objects.
- Placing the device on the tongue inside the mouth frees the hands to interact with the environment, Plus, the device can be hidden in the mouth.
- The key to the device may be its utilization of the tongue, which seems to be an ideal organ for sensing electrical current. Saliva there functions as a good conductor.
Fig.9 exact location of electrode array on tongue
Also it might help that the tongue’s nerve fibers are densely packaged and that these fibers are closer to
- the tongue’s surface relative to other touch organs. (The surfaces of fingers, for example, are covered with a layer of dead cells called stratum corneum.)
- The tongue was the ideal place to provide information through tactile stimulation. There is a high level of nerve endings in the tongue, similar to a finger. And the tongue is constantly moist, so there is constant electric conductivity.
- Finger would require 10 times more electric stimulation than the tongue does to produce the same results in the visual cortex.
camera to CPU
- camera captures the image of test symbol and sent to a processor i.e. handled unit about size of cell phone
- CPU translates camera output in to a pattern of electronic pulses & sent to electrode array which is held against the tongue
CPU to tongue
- array electrode stimulate receptor cells(tactile/touch) on the surface of the tongue With practice in use signals activate visual part of brain.
- tongue to brain
- these signals from tactile or touch receptors cells are sent to the somatasensory cortex in response to stimulation in the form of pattern impulses
- With practice in use signals activate visual part of brain.
VI. Test done
This device has been tested on several blind people; one among them is Erik Weihenmayer. A genetic eye condition known as retinoschisis caused him to be visually impaired at birth and completely blind by age 13. In retinoschisis, tiny cysts form within the eyes delicate retinal tissue, eventually causing its layers to split apart. Neither medication nor surgery can restore sight. But with the help and practicing this device he was at least able to identify the obstacles, objects around him and can also read the signs. And by use of this device he has climbed mountains around the worldthe highest peaks, in fact, on every continent.
Fig.11 shows static and dynamic images interpreted by blind people using device
VII. APPLICATIONS OF BRAIN PORT DEVICE
1. One of the applications which has been commercialized is providing vestibular or balance information for people with balance disorders. This is a simple form of sensory substitution, in which the tongue is used to present information from an artificial balance sensor.
2. Another application is providing directional or navigational information for people who operate under central command and control scenarios, such as military and civilian rescue personnel. Providing information via the tongue allows them to fully use their vision and hearing to respond to unforeseen threats or hazards. We have shown in the laboratory that it is possible to navigate a virtual maze (like a simple video game) using only information received on the tongue (i.e., buzz on right side of tongue means turn right, etc.).
3. A third, more ambitious application would be providing very crude visual information through the tongue for persons who are completely blind. Our colleague Eliana Sampaio at the Louis Pasteur University in Strasbourg, France has used our tongue stimulator with a small video camera and demonstrated an equivalent visual acuity of about 20-to-830, which is very poor vision, but possibly useful for certain limited activities with enough practice. Wicab, Inc continues to improve this technology with the aim of commercializing it.
4. A fourth application would be providing tactile feedback to the human operators of robots used for various tasks. For example, UW professor Nicola Ferrier is developing a robot controlled by the tongue of persons with quadriplegia which could incorporate touch sensors into its gripper, relaying the touch information back to the user’s tongue.
- Brain Port device does not replace the sense of sight, it adds to other sensory experiences to give users information about the size, shape, and location of objects.
- Users can operate it independently with a hand-held controller.
- A pair of sunglasses wired to an electric “lollipop” helps the visually impaired regain optical sensations via a different pathway Therefore device is like normal sunglasses hence it does not look bad.
- It uses a rechargeable battery like in normal cell phones.
- This technology can’t be adapted to work on senses the brain doesn’t already have.
- The Brain Port requires training the brain incrementally using daily practice sessions.
- When it comes to market its cost is around $10,000 so it cannot be afforded by common people.
- Occasionally it will produce weak metallic taste sensations, a minor side effect. We have never observed any kind of tissue irritation with the gold-plated electrodes.
Science has always provided mankind with answers and solutions, and science will continue to do so, while simultaneously supplying us with improvements upon previous technologies or new technologies altogether. Today, humanity owes the majority of our commodities, from prosthetic limbs to iPods, to years of scientific research and collaboration between different scientific disciplines. Unfortunately, however much science may have contributed to improving our lives, there is still plenty of headway to be made. We are always looking for areas in which our interdisciplinary strengths can be leveraged to revolutionize areas of science, engineering and technology, and to improve quality of life for millions of people.”
To substitute one sensory input channel for another, you need to correctly encode the nerve signals for the sensory event and send them to the brain through the alternate channel. The brain appears to be flexible when it comes to interpreting sensory input. You can train it to read input from, say, the tactile channel, as visual or balance information, and to act on it accordingly. “It’s a great mystery as to how that process takes place, but the brain can do it if you give it the right information.”
There is a hope that a balance device that uses nerve fibers on the tongue to transmit information about head and body position to the brain can make a serious difference for patients whose sight cannot be replaced.Thus we hope that blind people can also see this colourful world by using this brain port device.