美国加州大学旧金山分校Edward F. Chang和Joseph G. Makin课题组近日取得一项新成果。他们研究开发的一款脑电波编码器-解码器，能够将大脑活动信号直接转化为文本。相关论文发表在2020年3月30日出版的《自然-神经科学》杂志上。

研究人员展示了如何以高准确度和自然语音速率对脑电波进行解码。从机器翻译的最新进展中汲取经验，研究人员优化了一个递归神经网络，将神经活动的每个长句序列编码为抽象代码，然后逐词将该代码解码为英语句子。

对于每个参与者，测试数据包括几次口头重复一组30–50个词的句子和来自分布在Sylvian皮层周围约250个电极的同期信号。保留的重复集中平均单词错误率低至3％。 最后，研究人员展示了如何通过在多个参与者的数据下优化网络的某些层，利用转移学习来改善有限数据的解码。

据悉，在语音从人脑信号中解码出来的十多年后，其准确性和速度仍然远远低于自然语音。

附：英文原文

Title: Machine translation of cortical activity to text with an encoder–decoder framework

Author: Joseph G. Makin, David A. Moses, Edward F. Chang

Issue&Volume: 2020-03-30

Abstract: A decade after speech was first decoded from human brain signals, accuracy and speed remain far below that of natural speech. Here we show how to decode the electrocorticogram with high accuracy and at natural-speech rates. Taking a cue from recent advances in machine translation, we train a recurrent neural network to encode each sentence-length sequence of neural activity into an abstract representation, and then to decode this representation, word by word, into an English sentence. For each participant, data consist of several spoken repeats of a set of 30–50 sentences, along with the contemporaneous signals from ~250 electrodes distributed over peri-Sylvian cortices. Average word error rates across a held-out repeat set are as low as 3%. Finally, we show how decoding with limited data can be improved with transfer learning, by training certain layers of the network under multiple participants’ data.

DOI: 10.1038/s41593-020-0608-8

Source: https://www.nature.com/articles/s41593-020-0608-8