1692 Commits

Author SHA1 Message Date
hiyouga
d4ce280fbc Update README.md
Former-commit-id: 2d43b8bb49057e14a9f79146acdcc0cfa94bcc5a
2024-06-13 16:02:21 +08:00
hiyouga
f81a839197 update examples
Former-commit-id: 892e561c28cbbab1ee38a8022ddd9b397c873563
2024-06-13 03:26:10 +08:00
hiyouga
27777c8e68 add quant check in webui export tab
Former-commit-id: c94e6c9411cc1165ea8180d377611a3ae47956e6
2024-06-13 03:19:18 +08:00
hiyouga
4c40171c55 Update llama3_full_sft_ds3.yaml
Former-commit-id: a19cdd39feafa91fdacae4818393f82e9724c53f
2024-06-13 03:16:20 +08:00
hiyouga
0926d81053 update examples
Former-commit-id: b6e008c152421db668c971b0828cbee6a80b16bc
2024-06-13 03:15:06 +08:00
hiyouga
8fccaf20c5 fix #4221
Former-commit-id: 6baafd4eb3147ad9f7d2952b8eb27c5486940f36
2024-06-13 02:48:21 +08:00
hiyouga
c51f5c2a0a update wechat
Former-commit-id: 9419f96609d791397f02b411966cd1ef779f2f97
2024-06-13 02:31:45 +08:00
hiyouga
81ed4d8abf fix #4209
DeepSpeed ZeRO3 has inflight param error when calling model.eval()


Former-commit-id: cf9f2d6c42b5a37038c9eededbb767eae6a3f67d
2024-06-13 02:25:50 +08:00
hiyouga
833aa324c2 clean code
Former-commit-id: 2ed8270112755971e3f2dfd2f29c5939b077330a
2024-06-13 01:58:16 +08:00
hoshi-hiyouga
7366647b43 Merge pull request #4246 from hzhaoy/adapt-vllm-v0.5.0
adapt vllm==0.5.0

Former-commit-id: 1f23f25226f43bb2b2116474b9c6ab2d72e7e260
2024-06-13 01:54:02 +08:00
hiyouga
e89d1b1ec3 add neo-sft dataset
Former-commit-id: c7a5620ccc72b7574255ea764693ccb866c48263
2024-06-13 01:00:56 +08:00
hiyouga
99ce085415 fix lint
Former-commit-id: 713fde4259233af645bade7790211064a07a2a6f
2024-06-13 00:48:44 +08:00
hiyouga
b2b0b96051 fix docker compose usage
Former-commit-id: 947a34f53b74e4cd2b964941cf1580bcabde2228
2024-06-13 00:07:48 +08:00
hzhaoy
65f2ba3802 adapt vllm==0.5.0
Former-commit-id: 8fb6366ebec432c1333c80eb51007460ac5896bd
2024-06-12 18:29:03 +08:00
hiyouga
77e4dc255f update readme
Former-commit-id: 2ce2e5bc478f6ffcafe8e6451b1fef4e8994694c
2024-06-12 17:39:12 +08:00
hiyouga
f8497921fe fix #4242
Former-commit-id: 577de2fa078ee62d61bca24b75fa59590e2ef66b
2024-06-12 16:50:11 +08:00
hoshi-hiyouga
bc5e97295e Merge pull request #4234 from kimdwkimdw/patch-1
Support vllm==0.5.0

Former-commit-id: 656b2bbdafa0208c28801d41ac4887518d3f0ce5
2024-06-12 16:39:09 +08:00
Arthur Kim
bebca6d01c Support vllm==0.5.0
Former-commit-id: d65a3f7cb6dcd34e9a3ff9e30dffc52a7671d50b
2024-06-12 16:49:12 +09:00
codingma
c7e021a837 update wechat_npu.jpg
Former-commit-id: 557891debb8a64b73eea012f99780a7b76424cd5
2024-06-12 10:39:05 +08:00
hoshi-hiyouga
fecb9c9a76 Merge pull request #4204 from dignfei/main
fixbug:llama3在增量预训练时应该使用<|end_of_text|>标识文本的结束

Former-commit-id: 9049aab911fa3fbe9cd463ae3805f81b484da052
2024-06-11 17:06:10 +08:00
hoshi-hiyouga
bf3de9bfe8 Update pretrain.py
Former-commit-id: 0c292332374fb96c3fc753abde42d070a0c1dca8
2024-06-11 17:02:14 +08:00
hiyouga
8c574eb3cb fix deepspeed version
Former-commit-id: cca6f351081903ca3b5f79f10accc1bbbae0ee61
2024-06-11 16:52:36 +08:00
d
da39715085 经过大量的增量预训练,进行对比试验,发现这个bug:llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ,这里在每条数据后面也得用这个,而不是'<|eot_id|>',否则很容易导致严重的性能下降
Former-commit-id: 6979f3f8480755604d8aea8164f6418126e094c5
2024-06-11 16:23:40 +08:00
hiyouga
6c9cc199ef Update bug-report.yml
Former-commit-id: 53b74361d3d76c602f80854320efc555164afe44
2024-06-11 15:40:21 +08:00
hiyouga
5834651c4a fix #4198
Former-commit-id: 89f2bd8c8c035181927bd530a7ffc733407d674c
2024-06-11 15:38:38 +08:00
hiyouga
53de7f7cc3 tiny fix
Former-commit-id: 90e14a960d1437a16d35dbabbb8aa50714583d3a
2024-06-11 12:48:53 +08:00
hoshi-hiyouga
4ab902d558 Merge pull request #4191 from iamthebot/al--add_manifest_for_reqs
Add MANIFEST.in so requirements.txt is present in sdist

Former-commit-id: 796699f8672e45d53ca2e6eb635b6dee2113d3ef
2024-06-11 10:41:15 +08:00
Alfredo Luque
fb20b8dcc8 add manifest so requirements.txt in sdist
Former-commit-id: 5f7b3b3ff6a0965bb5036868f7ca5612780d6766
2024-06-11 00:07:06 +00:00
hiyouga
e3baa5aa08 tiny fix
Former-commit-id: 3f24337a8a995b145b1e8075bc23878eaa363844
2024-06-11 01:04:16 +08:00
hiyouga
d6632fefc9 set dev version
Former-commit-id: 91e62a098fd997d0d1d12baef64d089aabc01fba
2024-06-11 00:50:53 +08:00
hiyouga
75e1bbf128 release v0.8.1
Former-commit-id: 2b6ebd6b51133cf114d6f0e8605ad2bb26aa6d65
v0.8.1
2024-06-11 00:44:26 +08:00
hiyouga
2f164c2c41 fix #4160
The split heads should be concatenated in dim=2


Former-commit-id: a793e8456b664ea0b48f0ba162999f18d06b4c2f
2024-06-11 00:37:17 +08:00
hiyouga
d984776d35 fix #4145
Fix the docker image


Former-commit-id: 949e9908ad634874cf5449ee9904745c7acda611
2024-06-11 00:19:17 +08:00
hiyouga
144544cd37 update evaluator
Former-commit-id: 0012762b04d4082a3a7d98889cc3024adc92fc44
2024-06-10 23:56:00 +08:00
hiyouga
3b244a69dc fix #2666
Former-commit-id: c907d816670975daa900898660d3503708b7fc37
2024-06-10 21:24:15 +08:00
hoshi-hiyouga
55b40b3d62 Merge pull request #4167 from yzoaim/branch
fix README

Former-commit-id: ef4afdaf0e718ef7c316bbee4c618a1f92dadaa6
2024-06-10 16:24:33 +08:00
-.-
b187450340 fix README
Former-commit-id: 483cdd9b6ad42bc43a97df8ce867e3a9ef9bf5bc
2024-06-08 23:51:56 +08:00
hiyouga
e4b9c80104 add pr ci
Former-commit-id: b61d25cd705db821664f87adf37d2ed56c68130f
2024-06-08 21:25:35 +08:00
hiyouga
0c44309e15 Update tests.yml
Former-commit-id: 25c635ef2849fd173617b6a8d8d3f5d4f800a893
2024-06-08 21:15:36 +08:00
hiyouga
1c31809652 update git workflows
Former-commit-id: 4c4f950f3936edbce84796c9a5e11b61ea07864e
2024-06-08 21:11:32 +08:00
hiyouga
1a261add61 fix llamafactory-cli env
Former-commit-id: 972ec9c668de1a9b6d872187dbc0c1d94f6fec6b
2024-06-08 07:15:45 +08:00
hiyouga
de3400a521 set dev version
Former-commit-id: 3ac11e77cccf686e0da499bd152997133b49a265
2024-06-08 06:46:09 +08:00
hiyouga
ce40d12692 release v0.8.0
Former-commit-id: 5aa4ce47567146cd97c61623018153b41d7c1278
v0.8.0
2024-06-08 05:20:54 +08:00
hiyouga
3547a26f86 add ultrafeedback and fineweb #4085 #4132
Former-commit-id: 12d79f89c5082eb29842b501e1cb88433a248ba3
2024-06-08 02:42:34 +08:00
hiyouga
de9e773764 fix ci
Former-commit-id: 42d9b26fc81d66cb0485a2ae148c3e719df59b7d
2024-06-08 02:00:44 +08:00
hiyouga
d3eb985bb6 fix ci
Former-commit-id: 7f20e4722ae6ac907b36a3219dcd09d2ff5d071a
2024-06-08 01:57:36 +08:00
hiyouga
6a5e3816cf add ci
Former-commit-id: aa2578bea072354656dbdb62b970e16a8b86f16b
2024-06-08 01:48:30 +08:00
hiyouga
4f3e680b57 init unittest
Former-commit-id: 1c7f0ab51906b20190f8d4db932623cff76efc01
2024-06-08 01:35:58 +08:00
hiyouga
64cf35cccc Delete .readthedocs.yaml
Former-commit-id: 4b55f35662c9f424f07196ada7f94746f78f95e7
2024-06-08 00:58:10 +08:00
hiyouga
4f0ce9be4e reorganize adapter code
Former-commit-id: 54cd743ebfbd296ae9eaf10c33f59e127f451785
2024-06-08 00:47:23 +08:00