423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2026-06-21 06:38:54 +08:00
Code Issues Packages Projects Releases Wiki Activity

[refactor] Add KTransformers AMX MoE SFT support via Accelerate (#10430)

Browse Source 
Co-authored-by: mrhaoxx <mr.haoxx@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Peilin Li
2026-05-01 01:47:58 +08:00
committed by GitHub
parent 6b08b948c9
commit 887ee2b121
39 changed files with 287 additions and 1968 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
src/llamafactory/train/dpo/workflow.py
View File

@@ -62,15 +62,7 @@ def run_dpo(
else:
ref_model = None
if model_args.use_kt:
from ktransformers.util.globals import GLOBAL_CONFIG # type: ignore
from .ktrainer import KDPOTrainer as CustomDPOTrainer
GLOBAL_CONFIG._config["mod"] = "sft"
else:
from .trainer import CustomDPOTrainer
from .trainer import CustomDPOTrainer
# Initialize our Trainer
trainer = CustomDPOTrainer(