This article analyzes the Rejection sampling Fine-Tuning (RFT) method for enhancing large language models' mathematical reasoning. It details a process where smaller models generate diverse reasoning paths, which are filtered for quality and diversity, then used to fine-tune a larger model (e.g., Llama2-70B). Key findings show RFT significantly improves accuracy over standard Supervised Fine-Tuning (SFT), especially for weaker models, by increasing the variety of unique reasoning paths in the training data.
原文翻译：
本文分析了用于增强大语言模型数学推理能力的拒绝采样微调（RFT）方法。它详细阐述了一个流程：使用小模型生成多样化的推理路径，经过质量和多样性筛选后，用于微调更大的模型（如Llama2-70B）。核心发现表明，通过增加训练数据中独特推理路径的多样性，RFT相比标准监督微调（SFT）能显著提升模型准确率，对于性能较弱的模型提升尤为明显。This article analyzes the Rejection sampling Fine-Tuning (RFT) method for enhancing large language models' mathematical reasoning. It details a process where smaller models generate diverse reasoning paths, which are filtered for quality and diversity, then used to fine-tune a larger model (e.g., Llama2-70B). Key findings show RFT significantly improves accuracy over standard Supervised Fine-Tuning (SFT), especially for weaker models, by increasing the variety of unique reasoning paths in the training data.
原文翻译：
本文分析了用于增强大语言模型数学推理能力的拒绝采样微调（RFT）方法。它详细阐述了一个流程：使用小模型生成多样化的推理路径，经过质量和多样性筛选后，用于微调更大的模型（如Llama2-70B）。核心发现表明，通过增加训练数据中独特推理路径的多样性，RFT相比标准监督微调（SFT）能显著提升模型准确率，对于性能较弱的模型提升尤为明显。
