据悉，近年来，基于流动、扩散或自回归神经网络的强大生成模型得到了发展，在从实例生成数据方面取得了显著的成功，并在广泛的领域得到了应用。然而，对这些方法性能的理论分析和对其局限性的理解仍然具有挑战性。
在本文中，研究人员通过分析这些方法在一类已知概率分布问题上的采样效率，并将其与更传统的方法(如蒙特卡洛马尔可夫链和朗格万动力学)的采样性能进行比较，朝着这个方向迈出了一步。研究人员聚焦于一类在无序系统统计物理中广泛研究的概率分布，涉及自旋玻璃、统计推断和约束满足问题。研究人员利用这样一个事实，即通过基于流动、基于扩散或自回归的网络方法进行采样，可以等效地映射到对修改概率度量的贝叶斯最优去噪的分析。
这项研究结果表明，这些方法在采样过程中遭遇挑战，原因在于算法的去噪路径上存在一阶相变。这一发现具有双向意义：首先，研究人员识别出这些方法在特定参数区域下采样效果不佳，而传统蒙特卡罗或朗之万方法在这些区域可能更为有效；其次，他们也确定了相反的情况，即当标准方法效率低下时，本文所探讨的生成方法却能表现良好。
附：英文原文
Title: Sampling with flows, diffusion, and autoregressive neural networks from a spin-glass perspective
Author: Ghio, Davide, Dandi, Yatin, Krzakala, Florent, Zdeborová, Lenka
Issue&Volume: 2024-6-24
Abstract: Recent years witnessed the development of powerful generative models based on flows, diffusion, or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical analysis of the performance and understanding of the limitations of these methods remain, however, challenging. In this paper, we undertake a step in this direction by analyzing the efficiency of sampling by these methods on a class of problems with a known probability distribution and comparing it with the sampling performance of more traditional methods such as the Monte Carlo Markov chain and Langevin dynamics. We focus on a class of probability distribution widely studied in the statistical physics of disordered systems that relate to spin glasses, statistical inference, and constraint satisfaction problems. We leverage the fact that sampling via flow-based, diffusion-based, or autoregressive networks methods can be equivalently mapped to the analysis of a Bayes optimal denoising of a modified probability measure. Our findings demonstrate that these methods encounter difficulties in sampling stemming from the presence of a first-order phase transition along the algorithm’s denoising path. Our conclusions go both ways: We identify regions of parameters where these methods are unable to sample efficiently, while that is possible using standard Monte Carlo or Langevin approaches. We also identify regions where the opposite happens: standard approaches are inefficient while the discussed generative methods work well.
DOI: 10.1073/pnas.2311810121
Source: https://www.pnas.org/doi/abs/10.1073/pnas.2311810121