Computational vision

The new study A Feedforward Architecture Accounts for Rapid Categorization, Serre, T., A. Oliva and T. Poggio, PNAS 2007, in press [not online yet] reveals the success of a computational version of vision modeled on the visual cortex processes of immediate recognition of objects. The feedforward model is based on what our vision perceives in the first 100-200 milliseconds of exposure in the ventral stream before cognitive feedback loops kick in. It recognized objects in a database of street scenes with reasonable accuracy and uses a learning algorithm to become better at categorizing new objects. In this study, their system was trained by exposure to images then pitted against human vision and both performed nearly the same, with over 90% accuracy for close-ups and 74% for distant views.

Thomas Serre, Tomaso Poggio and others at the Center for Biological and Computational Learning in the McGovern Institute, the Department of Brain and Cognitive Sciences, and the Computer Science and Artificial Intelligence Lab at MIT collaborated on the system. Another new paper, Robust Object Recognition with Cortex-Like Mechanisms, Serre et al., IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 29 No 3, March 2007 [free PDF], describes the development. The feedforward model uses four layers:

1. Visual processing is hierarchical, aiming to build
   invariance to position and scale first and then to
   viewpoint and other transformations.
   
2. Along the hierarchy, the receptive fields of the neurons
   (i.e., the part of the visual field that could potentially
   elicit a response from the neuron) as well as the
   complexity of their optimal stimuli (i.e., the set of
   stimuli that elicit a response of the neuron) increases.
   
3. The initial processing of information is feedforward
   (for immediate recognition tasks, i.e., when the image
   presentation is rapid and there is no time for eye
   movements or shifts of attention).
   
4. Plasticity and learning probably occurs at all stages
   and certainly at the level of inferotemporal (IT)
   cortex and prefrontal cortex (PFC), the top-most
   layers of the hierarchy.

Poggio said, “We have not solved vision yet, but this model of immediate recognition may provide the skeleton of a theory of vision. The huge task in front of us is to incorporate into the model the effects of attention and top-down beliefs.”

Their next goal is research on the 200-300 milliseconds after the feedforward process of immediate recognition, and a larger one is to incorporate cognitive feedback loops. The feedforward model may ultimately be useful as a front end to more complex processing systems. Bigger implications:

This new study supports a long–held hypothesis that rapid categorization happens without any feedback from cognitive or other areas of the brain. The results also indicate that the model can help neuroscientists make predictions and drive new experiments to explore brain mechanisms involved in human visual perception, cognition, and behavior. Deciphering the relative contribution of feed-forward and feedback processing may eventually help explain neuropsychological disorders such as autism and schizophrenia. The model also bridges the gap between the world of artificial intelligence (AI) and neuroscience because it may lead to better artificial vision systems and augmented sensory prostheses.

Read more.
Download the open source software with StreetScenes dataset.
More research from the MIT CBCL lab.

x-posted to Omni Brain

"The brain is the wildcard"

An eye prosthesis implant has gained FDA approval for clinical trials aiming to restore vision to people blinded by retinal degenerative diseases. Second Sight produced a 16-electrode implant device a few years ago, which five patients are still using with success thanks in part to an ultrananocrystalline diamond (UNCD) film protecting its parts from body fluids. Their new Argus II device with 60 electrodes is an improvement researcher Mark Humayun says, "is like a train and a plane, they're that different."

The device consists of a tiny camera and transmitter mounted in eyeglasses, an implanted receiver, and an electrode-studded array that is secured to the retina with a microtack the width of a human hair. A wireless microprocessor and battery pack worn on the belt powers the entire device.

The camera on the glasses captures an image and sends the information to the video processor, which converts the image to an electronic signal and sends it to the transmitter on the sunglasses. The implanted receiver wirelessly receives this data and sends the signals through a tiny cable to the electrode array, stimulating it to emit electrical pulses. The pulses induce responses in the retina that travel through the optic nerve to the brain, which perceives patterns of light and dark spots corresponding to the electrodes stimulated.

While the implant doesn't restore full vision, patients interpret the light patterns as useful images to interface with the world. As Science Daily reports:

Exactly how much improvement can be achieved won't be clear, he says, until testing is underway. "The brain is the wildcard," [Hunayan] says, noting that his team was surprised by the extent to which the brain was able to "fill in" visual information from the limited number of pixels in the first model.

600 to 1000 pixels are the expected requirement to restore full vision such as facial recognition and reading, and this device doesn't have that capacity. Read Visual Prosthesis in the Annual Review of Biomedical Engineering 2005, for a survey of the challenges involved. Though engineering advancements are needed, the Argus II appears to be an important visionary step towards full artificial sight.

Other implant devices in development include the Retina Implant from Germany, hoping to be on the market in 2009, and the Optobionics Artificial Silicon Retina chip using 5,000 microphotodiode cells. Both are subretinal implants that stimulate remaining healthy retinal cells. The Argus II, in contrast, is an epiretinal device that stimulates the optic nerve with external signals from the camera.

Tagged

Shrink Rap tagged me for a Thinking Blogger "award" and so in turn, here are five thoughtful blogs I recommend.

The Neurocritic
Brain Ethics
Positive Technology Journal
Doc Gm Splash Fly
Machines Like Us

Brain Awareness Week 2007

During last year's Brain Awareness Week I was full of vim and enthusiasm, but this time I've got a flu virus and am feeling listless. Links are the better way to go.

The Dana Alliance for the Brain is the BAW umbrella organization worldwide, and the Society for Neuroscience sponsors many events (like a Brain Bee for young students) in turn. Here it's NeuroScience Canada spearheading celebrations, such as a presentation on neurobiology and psychology in space: Functioning with a Floating Brain by Dr. Luchino Cohen, Program Scientist, Space Life Sciences, Canadian Space Agency. A free, bilingual event at the Montreal Planetarium on March 12. To find goings-on in your area, check the international calendar.

Online, the Encephalon neuroscience blog carnival, hosted this time around at Pharyngula, features a variety of links to intriguing, funny, and informative original writing.

Then there are many, many links compiled at The Neurophilospher's list of neuro/psych blogs. My brain science vlog Channel N was demarcated by an asterisk of recommendation, woot. The number of subscribers soared from 14 to 30 as a result.

I don't do this for fame, y'know. Rather, the prestige and the glory.

Happy Brain Awareness Week!

Neuroethics for Barbie

My Three Shrinks podcast #13: Lost It In Space features psychiatrists ClinkShrink, Dinah and Roy discussing a few questions on neuroethics I'd sent along to ClinkShrink, a forensic psych currently working in a prison. What's the line between the insanity defense and criminal responsibility, considering neuropsychiatric advances? What do they think about forced treatment of offenders, specifically chemical castration of sex offenders?

The trio offers some fun banter, and commentary based on their professional experience with the issues. ClinkShrink also followed up by writing the post Heads, You Lose in their blog, Shrink Rap. Thanks for taking on the topic!

For another view on neuroethical issues check out the excellent new blog The Situationist. There's a great three part series on Situational Sources of Evil by Philip Zimbardo, Professor Emeritus of Psychology at Stanford University.

cross-posted to Omni Brain

Stroke Recovery in Stained Glass

Dr. Jill Bolte Taylor, neuroanatomist and spokesperson for the Harvard Brain Tissue Resource Center at McLean Hospital, suffered an AVM stroke seven years ago that left her without math, language, and other left-hemisphere processing skills. With grittiness, family support and her professional knowledge about the brain and its plasticity, she regained capabilities in innovative ways beyond traditional stroke rehabilitation methods.

By creating stained glass brains art became both a means of expression and a logical therapeutic tool. Their beauty is almost a byproduct - except that understanding the value of being immersed in moments of appreciating beauty was another major lesson.

I asked her a few questions about her art prior to the release of her memoir, My Stroke of Insight: A Brain Scientist's Personal Journey, a fascinating and inspiring account of how and what she recovered and even gained from the experience.

---

In your designs, how much were you drawing from experience looking at and handling brains, compared to images in neuroanatomy books [which she referred to for refreshment after the stroke]?

I have a 3 dimensional picture of the brain in my mind's eye and the stained glass brain image is my artistic impression of the different parts of the organ and how they intersect. It is a composite of how others depict the brain in 2 dimension and what I know to be true about the brain in 3 dimension from dissection.

How much is science, how much is art, and how do they intersect for you?

I am an artist in my heart and chose to apply my art to science in order to make a living. All of my scientific projects are aesthetically beautiful - wait till you see my triple immunofluorescence study where we visualized 3 neurotransmitter systems in the same piece of tissue! Art is beauty to me, the brain is beautiful to me, therefore the brain is beautiful art to me whether in glass or in our heads.

In the photo you sent, your design seems a mix of the two. I recognize some brain structures but some (like the multicoloured areas along the top of the cerebellum) seem to represent something else? Symbolism?

The texture of the glass of the cerebellum is different than any other glass in the brain. I chose a feathered looking glass because the cerebellar tissue looks feathered when viewed under the microscope. The brain is anatomically correct from the level of the cingulate gyrus (orange band) down, the blue is the induseum griseum - the band surrounding the fibers of the corpus callosum (red/orange opaque piece). I chose opaque for the corpus callosum to indicate its density of fibers. Each of the nodules represents something specific - the RED for the amygdala (rage and fear), green for hippocampus (memory and learning), two purples - the pineal (third eye) and the pituitary (hormones). In my newer brains I always make the most posterior portion of the brain in a black/white stripe for the visual cortex V1 of blobs and interblobs, V2 Stripes and interstripes.

This was a form of art therapy; what differences did you notice between your recovery and others who didn't benefit from it? Were certain abilities re-established more easily or quickly?

I cannot compare this project to anyone else's recovery. All I can do is speak to what it helped me with.

1. Balance and equilibrium to stand still in front of a workspace and manipulate the project.
2. Gross motor movement, handling glass is very delicate and dangerous, I was highly motivated to be very careful for both the glass and myself.
3. Fine motor dexterity, cutting glass is a precise activity, grinding glass requires holding my body firm - equilibrium, pushing into the grinder - gross motor and then lining all of the pieces up - fine motor.
4. Cognitive development - this type of a project is a long term project with lots of steps. It helped me in my linear thinking.
5. Cartoon development of the original image required a combination of intuition and sensory organization.
6. Focus and concentration balanced with sleep.
7. Artistry - how does one tweak it all to make it remarkable and beautiful.

What about your creative thinking?

When I lost my left hemisphere I lost all of the normal 'in the box' thinking. When we think about shifts in the brain it is inadequate to focus on the loss because with every loss there is a gain. As a society we do not focus on what someone has gained in the absence of something they have lost. When I lost the ability to define, organize and categorize information, I gained the ability to be intuitive and creative. In the absence of the left mind and its dominating inhibition, I gained a completely uninhibited right mind which processes information in a completely unique way when compared to
the left mind.

---

Thank you Dr. Bolte Taylor! Abject apologies for the inexcusable delay in my posting this interview. It is, however, a timeless testament to resilience and creative spirit.

NOTE: The Harvard Brain Tissue Resource Center is in urgent need of brain tissue from America. Please call 1-800-BRAINBANK to learn more. Also, read My Stroke of Insight for the story of how that toll-free number helped save Dr. Bolte Taylor's life.