hiyouga
|
ab66ae8cd2
|
fix #4295
Former-commit-id: 78589cf90c
|
2024-06-15 04:34:55 +08:00 |
|
hiyouga
|
a3f4925c2c
|
add test cases
Former-commit-id: b27269bd2b
|
2024-06-15 04:05:54 +08:00 |
|
hiyouga
|
27777c8e68
|
add quant check in webui export tab
Former-commit-id: c94e6c9411
|
2024-06-13 03:19:18 +08:00 |
|
hiyouga
|
8fccaf20c5
|
fix #4221
Former-commit-id: 6baafd4eb3
|
2024-06-13 02:48:21 +08:00 |
|
hiyouga
|
81ed4d8abf
|
fix #4209
DeepSpeed ZeRO3 has inflight param error when calling model.eval()
Former-commit-id: cf9f2d6c42
|
2024-06-13 02:25:50 +08:00 |
|
hiyouga
|
833aa324c2
|
clean code
Former-commit-id: 2ed8270112
|
2024-06-13 01:58:16 +08:00 |
|
hoshi-hiyouga
|
7366647b43
|
Merge pull request #4246 from hzhaoy/adapt-vllm-v0.5.0
adapt vllm==0.5.0
Former-commit-id: 1f23f25226
|
2024-06-13 01:54:02 +08:00 |
|
hiyouga
|
99ce085415
|
fix lint
Former-commit-id: 713fde4259
|
2024-06-13 00:48:44 +08:00 |
|
hzhaoy
|
65f2ba3802
|
adapt vllm==0.5.0
Former-commit-id: 8fb6366ebe
|
2024-06-12 18:29:03 +08:00 |
|
hiyouga
|
f8497921fe
|
fix #4242
Former-commit-id: 577de2fa07
|
2024-06-12 16:50:11 +08:00 |
|
Arthur Kim
|
bebca6d01c
|
Support vllm==0.5.0
Former-commit-id: d65a3f7cb6
|
2024-06-12 16:49:12 +09:00 |
|
hoshi-hiyouga
|
fecb9c9a76
|
Merge pull request #4204 from dignfei/main
fixbug：llama3在增量预训练时应该使用<|end_of_text|>标识文本的结束
Former-commit-id: 9049aab911
|
2024-06-11 17:06:10 +08:00 |
|
hoshi-hiyouga
|
bf3de9bfe8
|
Update pretrain.py
Former-commit-id: 0c29233237
|
2024-06-11 17:02:14 +08:00 |
|
hiyouga
|
8c574eb3cb
|
fix deepspeed version
Former-commit-id: cca6f35108
|
2024-06-11 16:52:36 +08:00 |
|
d
|
da39715085
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: 6979f3f848
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
5834651c4a
|
fix #4198
Former-commit-id: 89f2bd8c8c
|
2024-06-11 15:38:38 +08:00 |
|
hiyouga
|
53de7f7cc3
|
tiny fix
Former-commit-id: 90e14a960d
|
2024-06-11 12:48:53 +08:00 |
|
hiyouga
|
e3baa5aa08
|
tiny fix
Former-commit-id: 3f24337a8a
|
2024-06-11 01:04:16 +08:00 |
|
hiyouga
|
d6632fefc9
|
set dev version
Former-commit-id: 91e62a098f
|
2024-06-11 00:50:53 +08:00 |
|
hiyouga
|
75e1bbf128
|
release v0.8.1
Former-commit-id: 2b6ebd6b51
|
2024-06-11 00:44:26 +08:00 |
|
hiyouga
|
2f164c2c41
|
fix #4160
The split heads should be concatenated in dim=2
Former-commit-id: a793e8456b
|
2024-06-11 00:37:17 +08:00 |
|
hiyouga
|
144544cd37
|
update evaluator
Former-commit-id: 0012762b04
|
2024-06-10 23:56:00 +08:00 |
|
hiyouga
|
3b244a69dc
|
fix #2666
Former-commit-id: c907d81667
|
2024-06-10 21:24:15 +08:00 |
|
hiyouga
|
1a261add61
|
fix llamafactory-cli env
Former-commit-id: 972ec9c668
|
2024-06-08 07:15:45 +08:00 |
|
hiyouga
|
de3400a521
|
set dev version
Former-commit-id: 3ac11e77cc
|
2024-06-08 06:46:09 +08:00 |
|
hiyouga
|
ce40d12692
|
release v0.8.0
Former-commit-id: 5aa4ce4756
|
2024-06-08 05:20:54 +08:00 |
|
hiyouga
|
4f0ce9be4e
|
reorganize adapter code
Former-commit-id: 54cd743ebf
|
2024-06-08 00:47:23 +08:00 |
|
hoshi-hiyouga
|
bad35d1730
|
fix #4139
Former-commit-id: cfd62283a9
|
2024-06-08 00:45:02 +08:00 |
|
hiyouga
|
a8318723a4
|
add resume args in webui
Former-commit-id: 06e5d136a4
|
2024-06-08 00:22:16 +08:00 |
|
hiyouga
|
d79222894c
|
fix #4137
Former-commit-id: 8bf9da659c
|
2024-06-07 19:16:06 +08:00 |
|
hiyouga
|
ca9468ff04
|
tiny fix
Former-commit-id: f8d8690bf4
|
2024-06-07 05:19:21 +08:00 |
|
hiyouga
|
4f3c89a6eb
|
fix ppo trainer save zero3 model
accelerator.get_state_dict(ds_model) should be called at all ranks
Former-commit-id: 4489d73ac7
|
2024-06-07 05:14:19 +08:00 |
|
hiyouga
|
f76d427332
|
fix ppo in trl 0.8.6
Former-commit-id: 2702d7e952
|
2024-06-07 04:48:29 +08:00 |
|
hiyouga
|
d3196318be
|
fix #4120
Former-commit-id: f9e818d79c
|
2024-06-07 04:18:05 +08:00 |
|
hiyouga
|
c6f5f69644
|
update data processors
Former-commit-id: ccc8b64cc2
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
4953ded639
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 181dbb0d05
|
2024-06-07 03:48:46 +08:00 |
|
hoshi-hiyouga
|
e3ef239bc0
|
Update supervised.py
Former-commit-id: c09ad8bab3
|
2024-06-07 03:42:08 +08:00 |
|
hoshi-hiyouga
|
fd7bd911a6
|
Update supervised.py
Former-commit-id: 788e8232fc
|
2024-06-07 03:38:23 +08:00 |
|
hoshi-hiyouga
|
21df5f0bd0
|
Update supervised.py
Former-commit-id: 8cecade708
|
2024-06-07 03:38:04 +08:00 |
|
hiyouga
|
8a0263551d
|
add qwen2 models
Former-commit-id: 8e95648850
|
2024-06-07 00:22:57 +08:00 |
|
hiyouga
|
8da149ba40
|
rename files
Former-commit-id: 74f96efef9
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
fd2c64315b
|
add DISABLE_TORCHRUN option
Former-commit-id: 45d8be8f93
|
2024-06-06 23:44:58 +08:00 |
|
hoshi-hiyouga
|
77b70664e5
|
Merge pull request #4082 from MengqingCao/bugfix
Fix #4077
Former-commit-id: 55c18c49b0
|
2024-06-06 23:38:40 +08:00 |
|
hoshi-hiyouga
|
9bebdeabda
|
Update cli.py
Former-commit-id: 751dd77bc0
|
2024-06-06 23:38:09 +08:00 |
|
hiyouga
|
368695483d
|
fix ppo+zero3 #3108
Former-commit-id: 76c61905b2
|
2024-06-06 23:30:07 +08:00 |
|
hiyouga
|
6cbc66a602
|
fix torch gc
Former-commit-id: 451b6693c0
|
2024-06-06 20:30:25 +08:00 |
|
hiyouga
|
e0aadd4b34
|
fix ppo dataset bug #4012
Former-commit-id: 149610c636
|
2024-06-06 19:03:20 +08:00 |
|
hiyouga
|
e898d8bbc4
|
update trainers
Former-commit-id: fad2591e31
|
2024-06-06 18:45:49 +08:00 |
|
hiyouga
|
f2edacb02d
|
fix base64 image read #4061
Former-commit-id: 67aa78cde0
|
2024-06-06 17:29:19 +08:00 |
|
hiyouga
|
cceff9f520
|
lora modules: all by default
Former-commit-id: cae4737907
|
2024-06-06 03:53:28 +08:00 |
|