Newshot

语言模型对齐的理论极限

AI summary

语言模型(LM)对齐在保留基础模型能力的同时改善模型输出以反映人类偏好。最常见的对齐方法是:(i) 强化学习,即在 KL 散度约束下最大化期望奖励;(ii) best-of-NNN 对齐,即从 NNN 个独立样本中选出奖励最高的输出。尽管这些方法被广泛使用,但在 KL 预算下奖励提升的根本极限仍鲜为人知。我们通过推导……的最大值来刻画 KL 正则化对齐的信息论极限。

This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.

Published by Apple Machine Learning
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗