423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2025-12-16 03:40:34 +08:00
Code Issues Packages Projects Releases Wiki Activity

update example docs

Browse Source
This commit is contained in:
hiyouga
2024-05-06 22:51:02 +08:00
parent 34d33e2257
commit f02f87c6fb
35 changed files with 1048 additions and 594 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
examples/extras/fsdp_qlora/single_node.sh Normal file
View File

@@ -0,0 +1,10 @@
#!/bin/bash
# DO NOT use GPTQ/AWQ model in FSDP+QLoRA
pip install "transformers>=4.39.1"
pip install "accelerate>=0.28.0"
pip install "bitsandbytes>=0.43.0"
CUDA_VISIBLE_DEVICES=0,1 accelerate launch \
--config_file examples/accelerate/fsdp_config.yaml \
src/train.py examples/extras/fsdp_qlora/llama3_lora_sft.yaml