hiyouga
|
f8497921fe
|
fix #4242
Former-commit-id: 577de2fa078ee62d61bca24b75fa59590e2ef66b
|
2024-06-12 16:50:11 +08:00 |
|
hoshi-hiyouga
|
bc5e97295e
|
Merge pull request #4234 from kimdwkimdw/patch-1
Support vllm==0.5.0
Former-commit-id: 656b2bbdafa0208c28801d41ac4887518d3f0ce5
|
2024-06-12 16:39:09 +08:00 |
|
Arthur Kim
|
bebca6d01c
|
Support vllm==0.5.0
Former-commit-id: d65a3f7cb6dcd34e9a3ff9e30dffc52a7671d50b
|
2024-06-12 16:49:12 +09:00 |
|
ancv
|
045eb155a2
|
implement efficient packing without cross-contamination attention
Former-commit-id: b2c367bc61c2778dc359613dca496d9e134c2743
|
2024-06-12 11:56:01 +07:00 |
|
codingma
|
c7e021a837
|
update wechat_npu.jpg
Former-commit-id: 557891debb8a64b73eea012f99780a7b76424cd5
|
2024-06-12 10:39:05 +08:00 |
|
hoshi-hiyouga
|
fecb9c9a76
|
Merge pull request #4204 from dignfei/main
fixbug：llama3在增量预训练时应该使用<|end_of_text|>标识文本的结束
Former-commit-id: 9049aab911fa3fbe9cd463ae3805f81b484da052
|
2024-06-11 17:06:10 +08:00 |
|
hoshi-hiyouga
|
bf3de9bfe8
|
Update pretrain.py
Former-commit-id: 0c292332374fb96c3fc753abde42d070a0c1dca8
|
2024-06-11 17:02:14 +08:00 |
|
hiyouga
|
8c574eb3cb
|
fix deepspeed version
Former-commit-id: cca6f351081903ca3b5f79f10accc1bbbae0ee61
|
2024-06-11 16:52:36 +08:00 |
|
d
|
da39715085
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: 6979f3f8480755604d8aea8164f6418126e094c5
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
6c9cc199ef
|
Update bug-report.yml
Former-commit-id: 53b74361d3d76c602f80854320efc555164afe44
|
2024-06-11 15:40:21 +08:00 |
|
hiyouga
|
5834651c4a
|
fix #4198
Former-commit-id: 89f2bd8c8c035181927bd530a7ffc733407d674c
|
2024-06-11 15:38:38 +08:00 |
|
hiyouga
|
53de7f7cc3
|
tiny fix
Former-commit-id: 90e14a960d1437a16d35dbabbb8aa50714583d3a
|
2024-06-11 12:48:53 +08:00 |
|
hoshi-hiyouga
|
4ab902d558
|
Merge pull request #4191 from iamthebot/al--add_manifest_for_reqs
Add MANIFEST.in so requirements.txt is present in sdist
Former-commit-id: 796699f8672e45d53ca2e6eb635b6dee2113d3ef
|
2024-06-11 10:41:15 +08:00 |
|
Alfredo Luque
|
fb20b8dcc8
|
add manifest so requirements.txt in sdist
Former-commit-id: 5f7b3b3ff6a0965bb5036868f7ca5612780d6766
|
2024-06-11 00:07:06 +00:00 |
|
hiyouga
|
e3baa5aa08
|
tiny fix
Former-commit-id: 3f24337a8a995b145b1e8075bc23878eaa363844
|
2024-06-11 01:04:16 +08:00 |
|
hiyouga
|
d6632fefc9
|
set dev version
Former-commit-id: 91e62a098fd997d0d1d12baef64d089aabc01fba
|
2024-06-11 00:50:53 +08:00 |
|
hiyouga
|
75e1bbf128
|
release v0.8.1
Former-commit-id: 2b6ebd6b51133cf114d6f0e8605ad2bb26aa6d65
v0.8.1
|
2024-06-11 00:44:26 +08:00 |
|
hiyouga
|
2f164c2c41
|
fix #4160
The split heads should be concatenated in dim=2
Former-commit-id: a793e8456b664ea0b48f0ba162999f18d06b4c2f
|
2024-06-11 00:37:17 +08:00 |
|
hiyouga
|
d984776d35
|
fix #4145
Fix the docker image
Former-commit-id: 949e9908ad634874cf5449ee9904745c7acda611
|
2024-06-11 00:19:17 +08:00 |
|
hiyouga
|
144544cd37
|
update evaluator
Former-commit-id: 0012762b04d4082a3a7d98889cc3024adc92fc44
|
2024-06-10 23:56:00 +08:00 |
|
hiyouga
|
3b244a69dc
|
fix #2666
Former-commit-id: c907d816670975daa900898660d3503708b7fc37
|
2024-06-10 21:24:15 +08:00 |
|
hoshi-hiyouga
|
55b40b3d62
|
Merge pull request #4167 from yzoaim/branch
fix README
Former-commit-id: ef4afdaf0e718ef7c316bbee4c618a1f92dadaa6
|
2024-06-10 16:24:33 +08:00 |
|
mMrBun
|
b6d63b3324
|
Optimize the handling of QWEN2 in scenarios involving multiple tool calls.
Former-commit-id: 950e360ca00c29febadc14d5995de7d57b5c43a7
|
2024-06-10 02:00:14 +08:00 |
|
mMrBun
|
3f11ab800f
|
Removed unnecessary comments.
Former-commit-id: 6ed0b0c800d416379acf8395aa852c188c107eb9
|
2024-06-09 18:25:22 +08:00 |
|
mMrBun
|
daf472994d
|
Merge branch 'hiyouga:main' into main
Former-commit-id: 0f2609ce19492f0bab9b4880ded228b5513e5907
|
2024-06-09 18:17:24 +08:00 |
|
mMrBun
|
18a86ea104
|
Implemented the tool_formatter and tool_extractor for glm4 tool_format
Former-commit-id: cb1cbcb293917e960cad8f0eac7a11a122ab644a
|
2024-06-09 18:16:15 +08:00 |
|
-.-
|
b187450340
|
fix README
Former-commit-id: 483cdd9b6ad42bc43a97df8ce867e3a9ef9bf5bc
|
2024-06-08 23:51:56 +08:00 |
|
hiyouga
|
e4b9c80104
|
add pr ci
Former-commit-id: b61d25cd705db821664f87adf37d2ed56c68130f
|
2024-06-08 21:25:35 +08:00 |
|
hiyouga
|
0c44309e15
|
Update tests.yml
Former-commit-id: 25c635ef2849fd173617b6a8d8d3f5d4f800a893
|
2024-06-08 21:15:36 +08:00 |
|
hiyouga
|
1c31809652
|
update git workflows
Former-commit-id: 4c4f950f3936edbce84796c9a5e11b61ea07864e
|
2024-06-08 21:11:32 +08:00 |
|
hiyouga
|
1a261add61
|
fix llamafactory-cli env
Former-commit-id: 972ec9c668de1a9b6d872187dbc0c1d94f6fec6b
|
2024-06-08 07:15:45 +08:00 |
|
hiyouga
|
de3400a521
|
set dev version
Former-commit-id: 3ac11e77cccf686e0da499bd152997133b49a265
|
2024-06-08 06:46:09 +08:00 |
|
hiyouga
|
ce40d12692
|
release v0.8.0
Former-commit-id: 5aa4ce47567146cd97c61623018153b41d7c1278
v0.8.0
|
2024-06-08 05:20:54 +08:00 |
|
hiyouga
|
3547a26f86
|
add ultrafeedback and fineweb #4085 #4132
Former-commit-id: 12d79f89c5082eb29842b501e1cb88433a248ba3
|
2024-06-08 02:42:34 +08:00 |
|
hiyouga
|
de9e773764
|
fix ci
Former-commit-id: 42d9b26fc81d66cb0485a2ae148c3e719df59b7d
|
2024-06-08 02:00:44 +08:00 |
|
hiyouga
|
d3eb985bb6
|
fix ci
Former-commit-id: 7f20e4722ae6ac907b36a3219dcd09d2ff5d071a
|
2024-06-08 01:57:36 +08:00 |
|
hiyouga
|
6a5e3816cf
|
add ci
Former-commit-id: aa2578bea072354656dbdb62b970e16a8b86f16b
|
2024-06-08 01:48:30 +08:00 |
|
hiyouga
|
4f3e680b57
|
init unittest
Former-commit-id: 1c7f0ab51906b20190f8d4db932623cff76efc01
|
2024-06-08 01:35:58 +08:00 |
|
hiyouga
|
64cf35cccc
|
Delete .readthedocs.yaml
Former-commit-id: 4b55f35662c9f424f07196ada7f94746f78f95e7
|
2024-06-08 00:58:10 +08:00 |
|
hiyouga
|
4f0ce9be4e
|
reorganize adapter code
Former-commit-id: 54cd743ebfbd296ae9eaf10c33f59e127f451785
|
2024-06-08 00:47:23 +08:00 |
|
hoshi-hiyouga
|
bad35d1730
|
fix #4139
Former-commit-id: cfd62283a9772fc854b852d2a1b71699f79a0048
|
2024-06-08 00:45:02 +08:00 |
|
hiyouga
|
a8318723a4
|
add resume args in webui
Former-commit-id: 06e5d136a4916413d1c116e341ba7d5136d7748a
|
2024-06-08 00:22:16 +08:00 |
|
hiyouga
|
d79222894c
|
fix #4137
Former-commit-id: 8bf9da659cc789dbdd909930f79ac59539b51127
|
2024-06-07 19:16:06 +08:00 |
|
hiyouga
|
4b72592594
|
Update wechat.jpg
Former-commit-id: cce0fad91c0ef1758b021a94c02e9fdd97b73183
|
2024-06-07 19:04:16 +08:00 |
|
hiyouga
|
ca9468ff04
|
tiny fix
Former-commit-id: f8d8690bf4c2981f3151b4ccf07daeb4f3cd38a9
|
2024-06-07 05:19:21 +08:00 |
|
hiyouga
|
4f3c89a6eb
|
fix ppo trainer save zero3 model
accelerator.get_state_dict(ds_model) should be called at all ranks
Former-commit-id: 4489d73ac75c8dbc002fc16c854148994d432c3a
|
2024-06-07 05:14:19 +08:00 |
|
hiyouga
|
f76d427332
|
fix ppo in trl 0.8.6
Former-commit-id: 2702d7e952523b584d67c8901888b492d4a79b14
|
2024-06-07 04:48:29 +08:00 |
|
hiyouga
|
d3196318be
|
fix #4120
Former-commit-id: f9e818d79cf686cb34789327add7ed1f749966c6
|
2024-06-07 04:18:05 +08:00 |
|
hiyouga
|
c6f5f69644
|
update data processors
Former-commit-id: ccc8b64cc21417cfd88d9b5e9bd62e15c1e834a1
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
4953ded639
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 181dbb0d05655c6b46a3859cb47cfa3f86ef8abd
|
2024-06-07 03:48:46 +08:00 |
|