Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF. - Cen, Shicong; Mei, Jincheng; Goshvadi, Katayoon; Dai, Hanjun; Yang, Tong; Yang, Sherry; Schuurmans, Dale; Chi, Yuejie; Dai, Bo (CoRR, 2024)