423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2025-10-15 08:08:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2945 from marko1616/bugfix/lora-model-merge

Browse Source 
修复了在 transformers > 4.36.2 版本中部分模型合并 Lora 模型时因生成配置校验而导致的崩溃问题

Former-commit-id: 95afea730e80f58cc2984592fc07e265504c9491
This commit is contained in:
hoshi-hiyouga 2024-03-25 13:36:08 +08:00 committed by GitHub
commit 39f75c7001
1 changed files with 9 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
src/llmtuner/train/tuner.py
View File

@ -64,6 +64,15 @@ def export_model(args: Optional[Dict[str, Any]] = None):
for param in model.parameters(): for param in model.parameters():
param.data = param.data.to(output_dtype) param.data = param.data.to(output_dtype)
# Configuration check and fix
config = model.generation_config
if not config.do_sample and (
(config.temperature is not None and config.temperature != 1.0) or
(config.top_p is not None and config.top_p != 1.0) or
(config.typical_p is not None and config.typical_p != 1.0)
):
config.do_sample = True
model.save_pretrained( model.save_pretrained(
save_directory=model_args.export_dir, save_directory=model_args.export_dir,
max_shard_size="{}GB".format(model_args.export_size), max_shard_size="{}GB".format(model_args.export_size),