FMRI brain scanner reads thoughts letter by letter

Scientists have found a way to use fMRI brain scans to read thoughts letter by letter in real time. They suggest their "brain-scanning speller" has potential for helping paralysed people who can't move or speak, such as those with so-called "locked-in syndrome", to have a conversation. Bettina Sorger of Maastricht University in The Netherlands and colleagues report their work in the 28 June online issue of Current Biology. Human communication depends on being able to move and use a multiplicity of muscles, such as in forming sounds and words and making gestures and facial expressions. To do this the neuromuscular system must be healthy and undamaged. But severely motor-disabled patients, such as those with locked-in syndrome, who are fully conscious and aware, can't have a back-and-forth conversation because their neuromuscular system is not intact. The challenge to scientists trying to find ways to enable such patients to communicate is to tap into those parts of the brain that are performing the mental tasks of communication but are denied the means with which to express them physically, using the motor system or voluntary muscles. fMRI tracks brain activity by measuring changes in blood flow (hemodynamics) and oxygen in the brain. When a brain area is more active it uses more oxygen, and to meet this increase in demand, the blood flow to the area increases. Thus using fMRI, researchers can produce activation maps that show which parts of the brain are involved in particular brain processes. Neuroscientists like Adrian Owen and his team have already used fMRI to assess consciousness in people thought to be in an unconscious vegetative state and thus incapable of thought, and enabled them to respond yes or no to questions. This latest study by Sorger and colleagues takes that work a stage further, as Sorger explained to the press: "The work of Adrian Owen and colleagues led me to wonder whether it might even become possible to use fMRI, mental tasks, and appropriate experimental designs to freely encode thoughts, letter-by-letter, and therewith enable back-and-forth communication in the absence of motor behavior." In their paper, Sorger and colleagues describe how they tested their letter-encoding technique. Participants were asked to pick letters on a screen and for each letter, perform a specific mental task for a set period of time. Each task produced a different fMRI brain pattern which was then encoded to the corresponding letter. By the end of this "training period", the participants produced 27 distinct brain patterns, one for each letter of the alphabet and a space character. Sorger and colleagues write: "By exploiting spatiotemporal characteristics of hemodynamic responses, evoked by performing differently timed mental imagery tasks, our novel letter encoding technique allows translating any freely chosen answer (letter-by-letter) into reliable and differentiable single-trial fMRI signals." Using the brain-scanning speller, the researchers also held mini-conversations with the participants, consisting of two open questions and answers. Everyone they tested successfully produced answers within a single one-hour session. "Because the suggested spelling device requires only little effort and pretraining, it is immediately operational and possesses high potential for clinical applications, both in terms of diagnostics and establishing short-term communication with nonresponsive and severely motor-impaired patients" they write in their conclusions. Sorger said they now want to transfer the fMRI-based technique to a more portable and affordable technology, such as functional near-infrared spectroscopy (fNIRS), which also measures blood flow.