{"id":241,"date":"2020-11-30T20:54:37","date_gmt":"2020-11-30T19:54:37","guid":{"rendered":"http:\/\/icator.be\/?p=241"},"modified":"2022-01-27T22:18:35","modified_gmt":"2022-01-27T21:18:35","slug":"researchers-translate-brain-signals-directly-into-speech","status":"publish","type":"post","link":"https:\/\/icator.be\/researchers-translate-brain-signals-directly-into-speech\/","title":{"rendered":"Researchers Translate Brain Signals Directly Into Speech"},"content":{"rendered":"\n<h1 class=\"wp-block-heading\"><\/h1>\n\n\n\n<h3 class=\"wp-block-heading\">Posted on&nbsp;https:\/\/neurosciencenews.com on&nbsp;<time datetime=\"2019-01-29\">January 29, 2019<\/time><\/h3>\n\n\n\n<p><em>Summary: Researchers have developed a new system which utilizes artificial intelligence technology to turn brain signals to recognizable speech. The breakthrough could help restore a voice to those with limited, or no ability, to speak.<\/em><\/p>\n\n\n\n<p><em>Source: Zuckerman Institute.<\/em><\/p>\n\n\n\n<p><strong>In a scientific first, Columbia neuroengineers have created a system that translates thought into intelligible, recognizable speech. By monitoring someone\u2019s brain activity, the technology can reconstruct the words a person hears with unprecedented clarity. This breakthrough, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain. It also lays the groundwork for helping people who cannot speak, such as those living with as amyotrophic lateral sclerosis (ALS) or recovering from stroke, regain their ability to communicate with the outside world.<\/strong><\/p>\n\n\n\n<p>These findings were published today in&nbsp;<em>Scientific Reports.<\/em><\/p>\n\n\n\n<p>\u201cOur voices help connect us to our friends, family and the world around us, which is why losing the power of one\u2019s voice due to injury or disease is so devastating,\u201d said Nima Mesgarani, PhD, the paper\u2019s senior author and a principal investigator at Columbia University\u2019s Mortimer B. Zuckerman Mind Brain Behavior Institute. \u201cWith today\u2019s study, we have a potential way to restore that power. We\u2019ve shown that, with the right technology, these people\u2019s thoughts could be decoded and understood by any listener.\u201d<\/p>\n\n\n\n<p>Decades of research has shown that when people speak \u2014 or even imagine speaking \u2014 telltale patterns of activity appear in their brain. Distinct (but recognizable) pattern of signals also emerge when we listen to someone speak, or imagine listening. Experts, trying to record and decode these patterns, see a future in which thoughts need not remain hidden inside the brain \u2014 but instead could be translated into verbal speech at will.<\/p>\n\n\n\n<p>But accomplishing this feat has proven challenging. Early efforts to decode brain signals by Dr. Mesgarani and others focused on simple computer models that analyzed spectrograms, which are visual representations of sound frequencies.<\/p>\n\n\n\n<p>But because this approach has failed to produce anything resembling intelligible speech, Dr. Mesgarani\u2019s team turned instead to a vocoder, a computer algorithm that can synthesize speech after being trained on recordings of people talking.<\/p>\n\n\n\n<p>\u201cThis is the same technology used by Amazon Echo and Apple Siri to give verbal responses to our questions,\u201d said Dr. Mesgarani, who is also an associate professor of electrical engineering at Columbia\u2019s Fu Foundation School of Engineering and Applied Science.<\/p>\n\n\n\n<p>To teach the vocoder to interpret to brain activity, Dr. Mesgarani teamed up with Ashesh Dinesh Mehta, MD, PhD, a neurosurgeon at Northwell Health Physician Partners Neuroscience Institute and co-author of today\u2019s paper. Dr. Mehta treats epilepsy patients, some of whom must undergo regular surgeries.<\/p>\n\n\n\n<p>\u201cWorking with Dr. Mehta, we asked epilepsy patients already undergoing brain surgery to listen to sentences spoken by different people, while we measured patterns of brain activity,\u201d said Dr. Mesgarani. \u201cThese neural patterns trained the vocoder.\u201d<\/p>\n\n\n\n<p>Next, the researchers asked those same patients to listen to speakers reciting digits between 0 to 9, while recording brain signals that could then be run through the vocoder. The sound produced by the vocoder in response to those signals was analyzed and cleaned up by neural networks, a type of artificial intelligence that mimics the structure of neurons in the biological brain.<\/p>\n\n\n\n<p>The end result was a robotic-sounding voice reciting a sequence of numbers. To test the accuracy of the recording, Dr. Mesgarani and his team tasked individuals to listen to the recording and report what they heard.<\/p>\n\n\n\n<p>\u201cWe found that people could understand and repeat the sounds about 75% of the time, which is well above and beyond any previous attempts,\u201d said Dr. Mesgarani. The improvement in intelligibility was especially evident when comparing the new recordings to the earlier, spectrogram-based attempts. \u201cThe sensitive vocoder and powerful neural networks represented the sounds the patients had originally listened to with surprising accuracy.\u201d<\/p>\n\n\n\n<p>Dr. Mesgarani and his team plan to test more complicated words and sentences next, and they want to run the same tests on brain signals emitted when a person speaks or imagines speaking. Ultimately, they hope their system could be part of an implant, similar to those worn by some epilepsy patients, that translates the wearer\u2019s thoughts directly into words.<\/p>\n\n\n\n<p>\u201cIn this scenario, if the wearer thinks \u2018I need a glass of water,\u2019 our system could take the brain signals generated by that thought, and turn them into synthesized, verbal speech,\u201d said Dr. Mesgarani. \u201cThis would be a game changer. It would give anyone who has lost their ability to speak, whether through injury or disease, the renewed chance to connect to the world around them.\u201d ABOUT THIS NEUROSCIENCE RESEARCH ARTICLE<\/p>\n\n\n\n<p><strong>Funding:<\/strong>&nbsp;This research was supported by the National Institutes of Health (DC014279), the Pew Charitable Trusts and the Pew Biomedical Scholars Program.<\/p>\n\n\n\n<p><strong>Source:<\/strong>&nbsp;Anne Holden \u2013&nbsp;<a rel=\"noreferrer noopener\" href=\"http:\/\/zuckermaninstitute.columbia.edu\/\" target=\"_blank\">Zuckerman Institute<\/a><br><strong>Publisher:<\/strong>&nbsp;Organized by&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/neurosciencenews.com\/brain-signals-speech-10660\" target=\"_blank\">NeuroscienceNews.com.<\/a><br><strong>Image Source:<\/strong>&nbsp;NeuroscienceNews.com image is in the public domain.<br><strong>Original Research:<\/strong>&nbsp;Open access&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/www.nature.com\/articles\/s41598-018-37359-z\" target=\"_blank\">research<\/a>&nbsp;for \u201cTowards reconstructing intelligible speech from the human auditory cortex\u201d by Hassan Akbari, Bahar Khalighinejad, Jose L. Herrero, Ashesh D. Mehta &amp; Nima Mesgarani in&nbsp;<em>Scientific Reports<\/em>. Published January 29 2019.<br><strong>doi:<\/strong><a rel=\"noreferrer noopener\" href=\"https:\/\/dx.doi.org\/10.1038\/s41598-018-37359-z\" target=\"_blank\">10.1038\/s41598-018-37359-z<\/a> Zuckerman Institute\u201dResearchers Translate Brain Signals Directly Into Speech.\u201d NeuroscienceNews. NeuroscienceNews, 29 January 2019.<br>&lt;http:\/\/neurosciencenews.com\/brain-signals-speech-10660\/&gt;. &nbsp; &nbsp;<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Posted on&nbsp;https:\/\/neurosciencenews.com on&nbsp;January 29, 2019 Summary: Researchers have developed a new system which utilizes artificial intelligence technology to turn brain signals to recognizable speech. The breakthrough could help restore a voice to those with limited, or no ability, to speak. Source: Zuckerman Institute. In a scientific first, Columbia neuroengineers have created a system that translates <a class=\"more-link\" href=\"https:\/\/icator.be\/researchers-translate-brain-signals-directly-into-speech\/\">Read More<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10],"tags":[],"_links":{"self":[{"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/posts\/241"}],"collection":[{"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/comments?post=241"}],"version-history":[{"count":2,"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/posts\/241\/revisions"}],"predecessor-version":[{"id":491,"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/posts\/241\/revisions\/491"}],"wp:attachment":[{"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/media?parent=241"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/categories?post=241"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icator.be\/wp-json\/wp\/v2\/tags?post=241"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}