hiyouga
|
f4c9555760
|
fix lint
Former-commit-id: b170165679317af2b3f03633afac27661b3deb06
|
2024-06-13 00:48:44 +08:00 |
|
hiyouga
|
f7b66af7af
|
fix #4242
Former-commit-id: cf260e7af03f49aa5e3d6daf3b27738ff9b9bcb8
|
2024-06-12 16:50:11 +08:00 |
|
Arthur Kim
|
45712c6251
|
Support vllm==0.5.0
Former-commit-id: e7a8ffd7af21bc3759f055033ba2209fa7a1be0e
|
2024-06-12 16:49:12 +09:00 |
|
hoshi-hiyouga
|
2640315e63
|
Merge pull request #4204 from dignfei/main
fixbug：llama3在增量预训练时应该使用<|end_of_text|>标识文本的结束
Former-commit-id: e566342636faf0031a0ba5d5dd4fcff8401a2b76
|
2024-06-11 17:06:10 +08:00 |
|
hoshi-hiyouga
|
6625bf6b33
|
Update pretrain.py
Former-commit-id: e2317b2a84149e39fddfd6366be3de23dfb71f82
|
2024-06-11 17:02:14 +08:00 |
|
hiyouga
|
a7233181f2
|
fix deepspeed version
Former-commit-id: 938a69bb07d4de7d82928ff01c582032162c1480
|
2024-06-11 16:52:36 +08:00 |
|
d
|
dfac202c7d
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: ef470561f742b16eaa0f99c4cadecd7c84ce6bd2
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
95f95bef60
|
fix #4198
Former-commit-id: 945d2c6cc73542adf9272ebd9aa332ea2c1c7361
|
2024-06-11 15:38:38 +08:00 |
|
hiyouga
|
b7458a24b6
|
tiny fix
Former-commit-id: c4b2e263d9cefbad0fbc5de72422e4ef8edbcb54
|
2024-06-11 12:48:53 +08:00 |
|
hiyouga
|
8c7943c4de
|
tiny fix
Former-commit-id: b5e9711ef375cc323fc083e742cccfc974550416
|
2024-06-11 01:04:16 +08:00 |
|
hiyouga
|
e540759f4f
|
set dev version
Former-commit-id: 16c47cc15226119e33e46ba0f2f6ccb37072257f
|
2024-06-11 00:50:53 +08:00 |
|
hiyouga
|
41eadf5459
|
release v0.8.1
Former-commit-id: 875a34f492701d1c644facbe9ede411af2931513
|
2024-06-11 00:44:26 +08:00 |
|
hiyouga
|
68df064c1f
|
fix #4160
The split heads should be concatenated in dim=2
Former-commit-id: 4b3f247f270d44df9fe226cfe0dabfb7fcd2deda
|
2024-06-11 00:37:17 +08:00 |
|
hiyouga
|
bc86e70af9
|
update evaluator
Former-commit-id: bb8661e62481ff7027b8969f3d8a6a17290c9da3
|
2024-06-10 23:56:00 +08:00 |
|
hiyouga
|
7474e8035f
|
fix #2666
Former-commit-id: f121d5c4f94af9f165132c4309cb9bdc8217d985
|
2024-06-10 21:24:15 +08:00 |
|
hiyouga
|
a2acefea6e
|
fix llamafactory-cli env
Former-commit-id: b0515e5f42831b67d1f4d049999ecb68756e66db
|
2024-06-08 07:15:45 +08:00 |
|
hiyouga
|
088292e84a
|
set dev version
Former-commit-id: 08b7fe1c452cc99264ff0312e310b579590c6a45
|
2024-06-08 06:46:09 +08:00 |
|
hiyouga
|
cabe5ca7d0
|
release v0.8.0
Former-commit-id: 004db680b9e3996ec511ee818df6c0c02bf13603
|
2024-06-08 05:20:54 +08:00 |
|
hiyouga
|
35a36d96e5
|
reorganize adapter code
Former-commit-id: b26c2df9d97f4efffccbf7d28de13619b43f10dd
|
2024-06-08 00:47:23 +08:00 |
|
hoshi-hiyouga
|
17c66e9502
|
fix #4139
Former-commit-id: c025a4d74f293c14c2705e68af20a82a84608520
|
2024-06-08 00:45:02 +08:00 |
|
hiyouga
|
5606780ab6
|
add resume args in webui
Former-commit-id: 1d86ad768b1f36e54b4c2a9f18f6ea5a7df04c90
|
2024-06-08 00:22:16 +08:00 |
|
hiyouga
|
4377ad4391
|
fix #4137
Former-commit-id: cdc0d6f5a2e5040e145c82c4801f37bd76529047
|
2024-06-07 19:16:06 +08:00 |
|
hiyouga
|
4785c723f8
|
tiny fix
Former-commit-id: 0621bcad1dfbe8ce2464f741d4256c5df2a8d1b6
|
2024-06-07 05:19:21 +08:00 |
|
hiyouga
|
90f706c05b
|
fix ppo trainer save zero3 model
accelerator.get_state_dict(ds_model) should be called at all ranks
Former-commit-id: 3a0f60f0aa072531e4ae5819ec00c8fa42aa0913
|
2024-06-07 05:14:19 +08:00 |
|
hiyouga
|
7b7dc2f423
|
fix ppo in trl 0.8.6
Former-commit-id: 5e0d66a0d80b4bd4a8506e2317209d8fb9d25ff6
|
2024-06-07 04:48:29 +08:00 |
|
hiyouga
|
8cc3bbdc62
|
fix #4120
Former-commit-id: 2a44da678a5e360a9c0f9056397ac9e801329321
|
2024-06-07 04:18:05 +08:00 |
|
hiyouga
|
72c91be943
|
update data processors
Former-commit-id: 04b138cbcb8b9a72e4bbda6c65843bb459e525e7
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
a4cc4413e3
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 5ded166b39a75a98ded5733678f5a1eab7d4cc71
|
2024-06-07 03:48:46 +08:00 |
|
hoshi-hiyouga
|
4a4b8d0b39
|
Update supervised.py
Former-commit-id: 04b6c2a754e602e0b698cfe6c255c2f2486d8865
|
2024-06-07 03:42:08 +08:00 |
|
hoshi-hiyouga
|
9de42c4b3a
|
Update supervised.py
Former-commit-id: 49993c4f4e1f871a22ff0196afe60026b668a4dc
|
2024-06-07 03:38:23 +08:00 |
|
hoshi-hiyouga
|
b5421ac8eb
|
Update supervised.py
Former-commit-id: 67625b5278a839c12a3e4245f9e90af67d8b11b4
|
2024-06-07 03:38:04 +08:00 |
|
hiyouga
|
093abed7cc
|
add qwen2 models
Former-commit-id: 49cb694d02c876e3740a003a8b332349f4310ad3
|
2024-06-07 00:22:57 +08:00 |
|
hiyouga
|
0b1f4a34f8
|
rename files
Former-commit-id: e1a8431770fc36c0c9ee7fed4abbc3d7fdcc5efd
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
3e274dfb19
|
add DISABLE_TORCHRUN option
Former-commit-id: bcc574b479c2101438723aadead42743d4378776
|
2024-06-06 23:44:58 +08:00 |
|
hoshi-hiyouga
|
19b5b09642
|
Merge pull request #4082 from MengqingCao/bugfix
Fix #4077
Former-commit-id: 288028c3fb6bb1b58d1b7f4e8b90108c9bbf27d1
|
2024-06-06 23:38:40 +08:00 |
|
hoshi-hiyouga
|
969d0f7cdd
|
Update cli.py
Former-commit-id: 32190507534adf5f505858b3af2b592ca6568ac7
|
2024-06-06 23:38:09 +08:00 |
|
hiyouga
|
ee2c360167
|
fix ppo+zero3 #3108
Former-commit-id: 33a93cc29e3e57bf001515000c0a70c112573dea
|
2024-06-06 23:30:07 +08:00 |
|
hiyouga
|
d3a378ffea
|
fix torch gc
Former-commit-id: e173799d057598e5692a407601c30d8ce1513461
|
2024-06-06 20:30:25 +08:00 |
|
hiyouga
|
56a6db6d84
|
fix ppo dataset bug #4012
Former-commit-id: 7fc51b2e93698ae5e012566af8481f4d861c873d
|
2024-06-06 19:03:20 +08:00 |
|
hiyouga
|
67246f52f2
|
update trainers
Former-commit-id: b7f6c4a171293cf4f3e88f15a811f847342f84ee
|
2024-06-06 18:45:49 +08:00 |
|
hiyouga
|
e963a470c1
|
fix base64 image read #4061
Former-commit-id: 66ccb2a27a04296b4600f2c85f428071bf14eeb0
|
2024-06-06 17:29:19 +08:00 |
|
hiyouga
|
990dd6d44c
|
lora modules: all by default
Former-commit-id: 52c4ae87c7f4312704c31ef26b079b2c5b95ea5f
|
2024-06-06 03:53:28 +08:00 |
|
hiyouga
|
8d9f3022d2
|
add codestral 22B
Former-commit-id: b011c7f527a57cb1d21c4e2c9631c2fb62bb835e
|
2024-06-06 03:42:50 +08:00 |
|
hiyouga
|
e9f9b1f250
|
lint
Former-commit-id: 9030501eaef97ea249347198272adf0d709503ec
|
2024-06-06 03:33:44 +08:00 |
|
hoshi-hiyouga
|
fbc1168294
|
Merge pull request #4066 from injet-zhou/main
add throughput entry to training log
Former-commit-id: d2816f343f405f3fab09f2a8eade774b886e8f92
|
2024-06-06 03:32:04 +08:00 |
|
hoshi-hiyouga
|
dd5e377b8e
|
Merge pull request #4080 from MengqingCao/npu
Add npu option for model exporting
Former-commit-id: 07fc67193ef6bcb8e8a392aff0c57a2eb36832bf
|
2024-06-06 03:15:44 +08:00 |
|
hoshi-hiyouga
|
f002409e6a
|
Update export.py
Former-commit-id: 694833c1104d13929d4f181f014a121f25955dc5
|
2024-06-06 03:14:46 +08:00 |
|
hoshi-hiyouga
|
79a39ca703
|
Update model_args.py
Former-commit-id: 09c0afd94a8a5f5b45a61b32c983d50e1b9e2941
|
2024-06-06 03:14:23 +08:00 |
|
hoshi-hiyouga
|
2c1951f00e
|
Merge pull request #4053 from hzhaoy/feature/add_select_config_file
Support selecting saved configuration files
Former-commit-id: 568ef3cf2a793f268cbe01c39dec418a13e61ecd
|
2024-06-06 03:06:03 +08:00 |
|
hiyouga
|
c955a332d7
|
add vllm_dtype arg #3387 #3717
Former-commit-id: a0dd3a6351bb78541d40fec1d2fc457d803c86a4
|
2024-06-06 02:53:27 +08:00 |
|