423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2025-12-14 19:06:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

improve lora+ impl.

Browse Source 
Former-commit-id: 72367307df
This commit is contained in:
hiyouga
2024-03-13 23:32:51 +08:00
parent 4000de93ea
commit 8b8671817f
12 changed files with 165 additions and 169 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
examples/extras/lora_plus/sft.sh → examples/extras/loraplus/sft.sh
View File

@@ -9,7 +9,7 @@ CUDA_VISIBLE_DEVICES=0 python ../../src/train_bash.py \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora_plus/sft \
--output_dir ../../saves/LLaMA2-7B/loraplus/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
@@ -30,4 +30,4 @@ CUDA_VISIBLE_DEVICES=0 python ../../src/train_bash.py \
--val_size 0.1 \
--plot_loss \
--fp16 \
--lora_lr_ratio 16.0
--loraplus_lr_ratio 16.0