Electronic neural networks, one of the key concepts in artificial intelligence research, have drawn inspiration from biological neurons since their inception\u2014as evidenced by their name. New research has now revealed that the influential AI transformer architecture also shares unexpected parallels with human neurobiology.<\/span><\/p>\n In a <\/span>collaborative study<\/span><\/a>, scientists propose that biological <\/span>astrocyte-neuron networks<\/span><\/a> could mimic transformers’ core computations. Or vice versa. The findings\u2014jointly reported by MIT, the MIT-IBM Watson AI Lab, and Harvard Medical School\u2014were published this week in the journal Proceedings of the National Academy of Sciences.<\/span><\/p>\n Astrocyte\u2013neuron networks are networks of cells in the brain that consist of two types of cells: astrocytes and neurons. Astrocytes are cells that support and regulate neurons, which are brain cells that send and receive electrical impulses. Their activity is basically thinking. Astrocytes and neurons talk to each other using chemicals, electricity, and touch.<\/span> The researchers focused on tripartite synapses, which are junctions where astrocytes form connections between a neuron that sends signals (presynaptic neuron) and a neuron that receives signals (postsynaptic neuron).\u00a0<\/span><\/p>\n Using mathematical modeling, they demonstrated how astrocytes\u2019 integration of signals over time could provide the required spatial and temporal memory for self-attention. Their models also show that a biological transformer could be built using calcium signaling between astrocytes and neurons. TL;DR, this study explains how to build an organic transformer.<\/span><\/p>\n \u201cHaving remained electrically silent for over a century of brain recordings, astrocytes are one of the most abundant, yet less explored, cells in the brain,” Konstantinos Michmizos, associate professor of computer science at Rutgers University told MIT<\/a>. “The potential of unleashing the computational power of the other half of our brain is enormous.\u201d<\/span><\/p>\n
<\/span>
<\/span>On the other hand, AI <\/span>transformers\u2014first introduced in 2017\u2014are one of the base technologies behind generative systems like ChatGPT. –in fact, that\u2019s where the \u201cT\u201d in GPT stands from. Unlike neural networks that process inputs sequentially, transformers can directly access all inputs via a mechanism called self-attention. This allows them to learn complex dependencies in data like text.\u00a0<\/span><\/p>\n
![\"Neuron-Astrocyte](\"https:\/\/img.decrypt.co\/insecure\/rs:fit:1536:0:0:0\/plain\/https:\/\/cdn.decrypt.co\/wp-content\/uploads\/2023\/08\/SCR-20230818-qpjk.png@webp\")
<\/span>