Kingsley
13170577b2
[feat] support megatron-LM training by mcore_adapter ( #9237 )
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Yaowei Zheng <hiyouga@buaa.edu.cn>
2025-10-26 16:21:30 +08:00
Yaowei Zheng
d9d67ba62d
[misc] fix import error ( #9299 )
2025-10-17 17:46:27 +08:00
Ben Feuer
1c44b60e3e
[feat] fp8 training ( #8960 )
...
Co-authored-by: Benjamin Feuer <penfever@gmail.com>
Co-authored-by: Yaowei Zheng <hiyouga@buaa.edu.cn>
2025-10-01 14:32:53 +08:00
Yaowei Zheng
52488ac974
[deps] upgrade transformers to 4.56.1 ( #9128 )
2025-09-14 02:26:39 +08:00
Yaowei Zheng
2c31279316
[assets] update wechat ( #8962 )
2025-08-19 02:55:09 +08:00
Zeju Qiu
003a2acb1a
[feature] adding orthogononal finetuning (OFT) to llama factory ( #8623 )
...
Co-authored-by: Zeju <zqiu@g003.internal.cluster.is.localnet>
Co-authored-by: Zeju <zqiu@login2.is.localnet>
Co-authored-by: Yaowei Zheng <hiyouga@buaa.edu.cn>
2025-08-18 18:22:47 +08:00
XLXW
1ada15981a
[feature] add support for dft loss ( #8917 )
2025-08-15 23:29:57 +08:00
Kingsley
936f4fd78e
[feature] Support MPO ( #8930 )
2025-08-15 15:09:59 +08:00
golangboy
ef507ae8e0
[file] Resolve file lock issue when deleting safetensors on Windows ( #8839 )
2025-08-08 14:59:54 +08:00
Yaowei Zheng
2c26ce6ac4
Merge commit from fork
2025-06-26 13:55:42 +08:00
Yaowei Zheng
9a2d1dec62
[assets] update wechat ( #8385 )
2025-06-16 18:23:22 +08:00
Aman Gupta
8e4ac78607
[trainer] Add LD-DPO objective ( #8362 )
2025-06-12 16:10:38 +08:00
Ze-Yi LIN
c4e51d40e0
[tracking] swanlab add llamafactory tag ( #8258 )
2025-06-03 18:42:29 +08:00
hoshi-hiyouga
9ae17cd173
[deps] update to transformers 4.52 ( #8125 )
2025-05-21 05:16:18 +08:00
hoshi-hiyouga
beae231af6
[doc] add no build isolation ( #8103 )
2025-05-19 19:25:13 +08:00
Ma, Xiaochen
a0b4b91577
[trainer] fix KeyError at end of pretrain ( #8099 )
2025-05-19 18:01:26 +08:00
Eric Tang
ef03832cd4
[ray] add storage filesystem to ray config ( #7854 )
2025-04-27 22:12:40 +08:00
hoshi-hiyouga
fddcd43c88
[trainer] support early stop ( #7797 )
2025-04-22 01:59:33 +08:00
hoshi-hiyouga
b07628dea5
[example] add bash usage ( #7794 )
2025-04-22 00:25:51 +08:00
Juanxi Tian
12ada72ed4
[trainer] Add Muon Optimizer ( #7749 )
...
Co-authored-by: hoshi-hiyouga <hiyouga@buaa.edu.cn>
2025-04-21 23:38:37 +08:00
hoshi-hiyouga
416853dd25
[parser] support omegaconf ( #7793 )
2025-04-21 23:30:30 +08:00
hoshi-hiyouga
39169986ef
[trainer] fix pt loss ( #7748 )
...
* fix pt loss
* robust
* fix
* test
2025-04-17 03:15:35 +08:00
hoshi-hiyouga
86ebb219d6
[breaking] bump transformers to 4.45.0 & improve ci ( #7746 )
...
* update ci
* fix
* fix
* fix
* fix
* fix
2025-04-17 02:36:48 +08:00
Eric Tang
bb8d79bae2
[ray] allow for specifying ray.init kwargs (i.e. runtime_env) ( #7647 )
...
* ray init kwargs
* Update trainer_utils.py
* fix ray args
---------
Co-authored-by: hoshi-hiyouga <hiyouga@buaa.edu.cn>
2025-04-10 11:31:05 +08:00
hoshi-hiyouga
1abd71b551
[assets] update readme ( #7644 )
2025-04-09 01:06:06 +08:00
Shawn Tao
acb09fa3a3
[trainer] fix key error ( #7635 )
2025-04-08 18:39:50 +08:00
hoshi-hiyouga
c3c0efbaa0
[misc] fix packing and eval plot ( #7623 )
2025-04-07 18:20:57 +08:00
hoshi-hiyouga
831e7f1cfd
[model] add llama4 ( #7611 )
2025-04-06 13:42:31 +08:00
gechengze
7b9deb9410
[trainer] fix batch processing in PPO trainer ( #7576 )
2025-04-02 21:17:48 +08:00
Xu-pixel
b578a7d5b6
[3rdparty] support swanlab lark notification ( #7481 )
2025-03-27 01:52:01 +08:00
Kdump
24afceddb7
[trainer] fix wsd scheduler ( #7304 )
...
* [trainer] Warmup_stable_decay supports setting the number of stable and decay steps according to the warmup_ratio ratio
* Update trainer_utils.py
---------
Co-authored-by: hoshi-hiyouga <hiyouga@buaa.edu.cn>
2025-03-26 15:25:02 +08:00
hoshi-hiyouga
0583d06676
[model] add qwen2vl 32b & upgrade peft ( #7469 )
...
* add qwen2vl 32b
* fix ci
* upgrade peft to 0.15
* fix ci
* fix ci
2025-03-25 12:15:58 +08:00
hoshi-hiyouga
7203365b80
[trainer] fix vlm loss for transformers 4.49 ( #7448 )
2025-03-24 10:24:05 +08:00
hoshi-hiyouga
05b19d6952
[deps] upgrade transformers to 4.50.0 ( #7437 )
...
* upgrade transformers
* fix hf cache
* fix dpo trainer
2025-03-23 17:44:27 +08:00
Eric Tang
db0a08db6f
[3rdparty] fix redundant process group destroy for ray ( #7395 )
...
* fix redundant process group destroy for ray
* Update tuner.py
---------
Co-authored-by: hoshi-hiyouga <hiyouga@buaa.edu.cn>
2025-03-21 10:56:47 +08:00
hoshi-hiyouga
650a9a9057
[misc] update format ( #7277 )
2025-03-13 02:53:08 +08:00
hoshi-hiyouga
264538cb26
[misc] upgrade format to py39 ( #7256 )
2025-03-12 00:08:41 +08:00
hiyouga
1890d3dafe
release v0.9.2
...
Former-commit-id: e7ed1782d4a006400de6fc0f864abd01f7fadeea
2025-03-11 14:49:13 +08:00
Ze-Yi LIN
18968405d0
[tracking] add swanlab_logdir param ( #7219 )
...
* feat: add swanlab_logdir param
* fix
Former-commit-id: 9215ad488b6ac6cd57fe8fa4acdacceb63f68ca5
2025-03-11 00:53:07 +08:00
hoshi-hiyouga
16419b2834
[data] fix loader ( #7207 )
...
* fix dataloader
* add test case
* fix type
* fix ci
* fix ci
* fix ci
* disable overwrite cache in ci
Former-commit-id: e84af0e140b1aafd1a6d6fe185a8e41c8fc5f831
2025-03-07 17:20:46 +08:00
hoshi-hiyouga
a255c3a476
[misc] fix cli ( #7204 )
...
Former-commit-id: 999f57133ca163c7108d2d5ee8194eca9b2109b4
2025-03-07 15:01:18 +08:00
hoshi-hiyouga
9f16c50155
[model] add QwQ 32b ( #7179 )
...
Former-commit-id: 8897e48b8cd55407812453ddd4ff98ac7bdc4e91
2025-03-06 11:58:36 +08:00
Ze-Yi LIN
25bb9f5ad9
[trainer] fix swanlab callback ( #7176 )
...
Former-commit-id: 6d9acf4bd30db24499118aee16bd19cb19ba9e3d
2025-03-06 00:33:37 +08:00
hoshi-hiyouga
7b985f55db
[trainer] update config ( #7174 )
...
Former-commit-id: 9f535d0e3c4ee3cd0f1b65218c2eee5d03f43c6f
2025-03-05 23:32:54 +08:00
Ze-Yi LIN
11672f760d
[webui] display swanlab exp link ( #7089 )
...
* webui add swanlab link
* change callback name
* update
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 27a4b93871c63b839c92940766bd7e0177972c9b
2025-02-27 19:40:54 +08:00
Eric Tang
76f9bd1820
[ray] specify ray storage path ( #6920 )
...
Former-commit-id: 4be6b66b1eaa79955e936ce2b747a8837ecd1e49
2025-02-14 21:55:41 +08:00
Billy Cao
58e9ca8aa0
[trainer] fix gen_kwarg to eval during training ( #5451 )
...
* Correctly pass gen_kwarg to eval during model runs
* fix
* fix
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 845d16122496311e08263610a6a922f82604de7b
2025-02-13 02:35:06 +08:00
marko1616
0c0cdc26bc
[trainer] fix llama3.2 vision kto train ( #6904 )
...
Former-commit-id: 1563e89adc8988fc6e4250634a3f1e385979b0e5
2025-02-12 19:09:14 +08:00
hoshi-hiyouga
86063e27ea
[data] fix ollama template ( #6902 )
...
* fix ollama template
* add meta info
* use half precision
Former-commit-id: 1304bbea69d8c8ca57140017515dee7ae2ee6536
2025-02-11 22:43:09 +08:00
hoshi-hiyouga
88eafd865b
[misc] support export ollama modelfile ( #6899 )
...
* support export ollama modelfile
* update config
* add system and num ctx
Former-commit-id: 8c2af7466f4015f300b51841db11bcd2505ebf20
2025-02-11 19:52:25 +08:00
