2023年10月30日，《自然—神经科学》杂志在线发表了美国科学家的一项最新研究成果。来自加州大学旧金山分校的Edward F. Chang研究组利用语音深度神经网络剖析人类听觉通路的神经计算。

研究人员使用最先进的深度神经网络（DNN）模型中的语音表征来研究从听觉神经到语音皮层的神经编码。深度神经网络分层表征与整个上升听觉系统的神经活动密切相关。无监督语音模型的表现至少与其他纯监督或微调模型相当。较深的DNN层与高阶听觉皮层的神经活动有更好的相关性，其计算与语音中的音位和音节结构一致。因此，以英语或普通话训练的DNN模型可以预测每种语言的母语使用者的大脑皮层反应。

这些结果揭示了DNN模型表征与生物听觉通路之间的趋同性，并为听觉皮层神经编码建模提供了新方法。

据悉，人类听觉系统从语音信号中提取丰富的语言抽象。理解这一复杂过程的传统方法采用线性特征编码模型，但成效有限。人工神经网络在语音识别任务中表现出色，为语音处理提供了前景广阔的计算模型。

附：英文原文

Title: Dissecting neural computations in the human auditory pathway using deep neural networks for speech

Author: Li, Yuanning, Anumanchipalli, Gopala K., Mohamed, Abdelrahman, Chen, Peili, Carney, Laurel H., Lu, Junfeng, Wu, Jinsong, Chang, Edward F.

Issue&Volume: 2023-10-30

Abstract: The human auditory system extracts rich linguistic abstractions from speech signals. Traditional approaches to understanding this complex process have used linear feature-encoding models, with limited success. Artificial neural networks excel in speech recognition tasks and offer promising computational models of speech processing. We used speech representations in state-of-the-art deep neural network (DNN) models to investigate neural coding from the auditory nerve to the speech cortex. Representations in hierarchical layers of the DNN correlated well with the neural activity throughout the ascending auditory system. Unsupervised speech models performed at least as well as other purely supervised or fine-tuned models. Deeper DNN layers were better correlated with the neural activity in the higher-order auditory cortex, with computations aligned with phonemic and syllabic structures in speech. Accordingly, DNN models trained on either English or Mandarin predicted cortical responses in native speakers of each language. These results reveal convergence between DNN model representations and the biological auditory pathway, offering new approaches for modeling neural coding in the auditory cortex.

DOI: 10.1038/s41593-023-01468-4

Source: https://www.nature.com/articles/s41593-023-01468-4