APSIPA ASC 2026に当研究室の論文が採録

■書誌情報
Haoyu Zhang, Yuta Oshima, Xingjian Du, Chunfeng Wang, Irene Li, Yusuke Iwasawa, Yutaka Matsuo: SMC-ITA: Sequential Monte Carlo Inference-Time Alignment for Video-to-Audio Generation, Proceedings of the 18th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2026), November 2026
■概要
Video-to-audio (V2A) generation must jointly satisfy audiovisual alignment, semantic consistency, temporal synchronization, and perceptual quality. While prior work has mainly focused on model architecture, multimodal conditioning, and training objectives, inference-time alignment for V2A remains underexplored. In this paper, we study inference-time alignment for flow-matching-based V2A generation and formulate it as a search problem. We propose Sequential Monte Carlo Inference-Time Alignment (SMC-ITA), which combines lookahead-based reward estimation and sequential Monte Carlo resampling to reallocate computation adaptively using multi-dimensional cross-modal rewards. SMC-ITA improves over naive single-trajectory sampling, achieving a 55.67% relative reduction in DeSync, a 20.23% improvement in IB-score, and a 15.44% improvement in Audio Quality. Under matched NFE budgets, it also achieves the best overall trade-off among the compared search baselines, outperforming Best-of-N and Beam Search. Ablation studies further show that lookahead improves the reliability of intermediate reward estimates and that systematic resampling is a strong practical default for V2A inference-time alignment.