This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.
TELEVAL: A Benchmark Designed for Spoken Language Models in Chinese Interactive Scenarios
arXiv:2507.18061v5 Announce Type: replace Abstract: Spoken Language Models (SLMs) are expected to support natural spoken interaction beyond task completion. Existing benchmarks assess a range of semantic tasks and conversational capabilities. However, high task scores do not necessarily imply reliable and appropriate spoken interaction. To address this gap, we introduce TELEVAL, a large-scale benchmark focused on Chinese spoken interaction. It evaluates responses to spoken inputs without additional task-specific prompts or predefined answer options. TELEVAL covers two complementary aspects: (1) Reliable Content Fulfillment, which measures semantic accuracy under diverse acoustic and linguistic conditions, and (2) Interactional Appropriateness, which assesses conversational naturalness and appropriate adaptation to auditory cues. Experiments reveal limitations in both aspects. Strong results on factual question-answering datasets do not consistently extend to locally relevant knowledge…