Instead of using large datasets to predict more details, researchers used the second AI model to associate the keywords from image captions to fMRI scans. But the model struggled with specific objects such as the clock tower and It created abstract and cloudy figures. The first model was able to effectively regenerate the layout and the perspective of the image being viewed.
Original Images (left) and AI-Generated Images for all Four Subjects Together, these models allowed the Stable Diffusion to turn the fMRI data into relatively accurate imitations of the images that were not part of its training achieving an accuracy of almost 80%. One was used to link the brain activity with the fMRI images while the other one was used to link it with the text descriptions of images that the subjects looked at. Two different AI models were trained during the process. A subset of 982 images viewed by all four subjects was used as the test dataset. It contained data obtained from fMRI scans of four subjects who had looked at 10,000 different images. The researchers used the Natural Scenes Dataset ( NSD) provided by the University of Minnesota. Pretty Interesting right? So how does it exactly work? Let's dig deeper into this research paper, its limitations, and its future scope. But the reconstructed image shows him at the crime scene. Imagine a suspect is questioned about where he was during the time of the murder and he answers that he was home.
This technology has the potential to revolutionize fields like psychology, neuroscience, and even the criminal justice system.
Although the paper “ High-resolution image reconstruction with latent diffusion models from human brain activity“ by Yu Takagi and Shinji Nishimotois is not yet peer-reviewed, it has taken the internet by storm as the results are shockingly accurate. Two researchers, from the Graduate School of Frontier Biosciences at Osaka University, proposed a new method using an LDM named Stable Diffusion that accurately reconstructed the images from human brain activity that was obtained by functional magnetic resonance imaging (fMRI). Though neuroscientists still struggle to truly demystify how the human brain converts what our eyes see into mental images, it seems like AI is getting better at this task. It sounds like something straight out of a science fiction movie but with the recent advancements in computer vision and deep learning, it is becoming a reality. Imagine reliving your memories or constructing images of what someone is thinking.
0 kommentar(er)
