ترجمه فارسی مقاله یادگیری تقویتی بدون بازخورد انسانی برای آخرین مایل تنظیم دقیق مدل های زبان بزرگ

• کتاب صدها نکته فارسی (خودمونی) – نسخه PDF — زبان ساده و کاربردی
  مشاهده نمونه نسخه نکات ساده
• کتاب صدها نکته رسمی فارسی – نسخه PDF — نگارش استاندارد و علمی
  مشاهده نمونه نسخه نکات رسمی
• کتاب صدها پرسش و پاسخ تشریحی – نسخه PDF
  — هر سؤال همراه با پاسخ کامل برای درک عمیق مفاهیم
  مشاهده نمونه نسخه پرسش و پاسخ
• کتاب پرسش و پاسخ چهارگزینه‌ای – نسخه یادگیری سریع
  — پاسخ‌ها بلافاصله بعد از سؤال برای مرور سریع
  مشاهده نمونه نسخه کوییز سریع
• کتاب پرسش و پاسخ چهارگزینه‌ای – نسخه خودآزمایی
  — پاسخ‌ها در انتهای بخش‌ها برای سنجش واقعی یادگیری
  مشاهده نمونه نسخه آزمونی

🎯 این بسته یک دورهٔ آموزشی کامل و چندلایه است؛ شامل ویدیوهای آموزشی، کتاب‌ها، تمرین‌ها و خودآزمایی.

چکیده

Reinforcement learning is used to align language models with human preference signals after first pre-training the model to predict the next token of text within a large corpus using likelihood maximization. Before being deployed in a specific domain, models are often further fine-tuned on task specific data. Since human preferences are often unavailable for the last step, it is performed using likelihood maximization as that is the typical default method. However, reinforcement learning has other advantages besides facilitating alignment to a human derived reward function. For one, whereas likelihood maximization is a form of imitation learning in which the model is trained on what to do under ideal conditions, reinforcement learning is not limited to demonstrating actions just for optimally reached states and trains a model what to do under a range of scenarios as it explores the policy space. In addition, it also trains a model what not to do, suppressing competitive but poor actions. This work develops a framework for last-mile fine-tuning using reinforcement learning and tests whether it garners performance gains. The experiments center on abstractive summarization, but the framework is general and broadly applicable. Use of the procedure produced significantly better results than likelihood maximization when comparing raw predictions. For the specific data tested, the gap could be bridged by employing post-processing of the maximum likelihood outputs. Nonetheless, the framework offers a new avenue for model optimization in situations where post-processing may be less straightforward or effective, and it can be extended to include more complex classes of undesirable outputs to penalize and train against, such as hallucinations.

چکیده به فارسی (ترجمه ماشینی)