By Celeste Biever: Pictures you are observing can now be recreated with software that uses nothing but scans of your brain. It is the first “mind reading” technology to create such images from scratch, rather than picking them out from a pool of possible images.
Earlier this year Jack Gallant and colleagues at the University of California, Berkeley, showed that they could tell which of a set of images someone was looking at from a brain scan.
To do this, they created software that compared the subject’s brain activity while looking at an image with that captured while they were looking at “training” photographs. The program then picked the most likely match from a set of previously unseen pictures.
Now Yukiyasu Kamitani at ATR Computational Neuroscience Laboratories in Kyoto, Japan has gone a step further: his team has used an image of brain activity taken in a functional MRI scanner to recreate a black-and-white image from scratch.
“By analysing the brain signals when someone is seeing an image, we can reconstruct that image,” says Kamitani.
This means that the mind reading isn’t limited to a selection of existing images, but could potentially be used to “read off” anything that someone was thinking of, without prior knowledge of what that might be.
“It’s absolutely amazing, it really is a very significant step forward,” says John-Dylan Haynes of the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig, Germany.
Dream catcher
Kamitani starts by getting someone to look at a selection of images made up of black and white squares on a 10 by 10 square grid, while having their brain scanned. Software then finds patterns in brain activity that correspond to certain pixels being blacked out. It uses this to record a signature pattern of brain activity for each pixel.
The person then sits in the scanner and is shown fresh patterns. Another piece of software then matches these against the list to reconstruct the pixels on a 10 by 10 grid.
The quality of images that were recreated is quite crude. However, the word “neuron” and several numbers and shapes that people were indeed being shown (see image, top right) could be observed in the reconstructed images. It is an important proof of principle, says Haynes.
As fMRI technology improves, Kamitani adds that an image could potentially be split into many more pixels, producing much higher quality images, and even colour images.
The next step is to find out if it is possible to image things that people are thinking of – as well as what they are looking at – Haynes says it may be possible to “make a videotape of a dream”.
Ethical concerns
Haynes also raises the prospect of “neural marketing”, where advertisers might one day be able to read the thoughts of passers by and use the results to target adverts. “This [new research] specifically doesn’t lead to this – but the whole spirit in which this is done is in line with brain reading and the applications that come with that,” he says.
“If you have a technique that allows you to read out what people are thinking we need clearer ethical guidelines about when and how you are able to do this,” he says. “A lot of people want their minds to be read – take for example a paralysed person. They want us to read their thoughts,” he says. “But it shouldn’t be possible to do this for commercial purposes.”
Kamitani is well aware of the negative potential of the technology. “If the image quality improves, it could have a very serious impact on our privacy and other issues. We will have to discuss with many people – not just scientists – how to apply this technology,” he says.
Journal reference: Neuron (DOI: 10.1016/j.neuron.2008.11.004)' Source:https://www.newscientist.com/
