hoshi-hiyouga
|
28b0499d78
|
Merge pull request #4307 from hiyouga/pissa
Support pissa
Former-commit-id: e7c0eefe96540c106162f5d252476b10b97ae696
|
2024-06-16 01:41:50 +08:00 |
|
hiyouga
|
d5a0cc93a2
|
fix tol
Former-commit-id: bdb54bcb477126687db789bd89f2df84e424a2a3
|
2024-06-16 01:38:44 +08:00 |
|
hiyouga
|
1a66bb1438
|
Update tests.yml
Former-commit-id: 82e83615a706293abbf266d11c57caedafdd4c5b
|
2024-06-16 01:22:23 +08:00 |
|
hiyouga
|
bf46a8ca3e
|
increase tol
Former-commit-id: c29071445e34aed23123fdf883a4d877744a1b0e
|
2024-06-16 01:21:06 +08:00 |
|
hiyouga
|
0b571f84b4
|
support pissa
Former-commit-id: ef8e45f2eaf466c54e9a671512a2974575677b08
|
2024-06-16 01:08:12 +08:00 |
|
hiyouga
|
640372cb66
|
tiny fix
Former-commit-id: f7f440986b0ae3b38ea9f2da80789629d4f79ea1
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
c5e1dfb3a0
|
remove some unused params
Former-commit-id: fef8132c50505a5fb6a246bd024491bd31798a3c
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
f9653ac29c
|
use fixture
Former-commit-id: 10761985691b9f934f7689c1f82aa6dd68febcca
|
2024-06-15 20:06:17 +08:00 |
|
hiyouga
|
4851ef85b7
|
add tests
Former-commit-id: 484634ee9c982e82e919ff67d507e0210345182d
|
2024-06-15 19:51:20 +08:00 |
|
hiyouga
|
61aaab22c9
|
add minicpm #4227
Former-commit-id: e1bb18ce60be9a1b203989def30f1b9194286325
|
2024-06-15 17:58:52 +08:00 |
|
hiyouga
|
acfae2e677
|
add license
Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
8b523554d6
|
update readme
Former-commit-id: a43d302aa79cbfb9b0606e855b4c1af6865d8e68
|
2024-06-15 05:13:16 +08:00 |
|
hiyouga
|
ab4b3931fd
|
fix #4271
Former-commit-id: 03707e78d29bfcf5d395a64bb38632bdb3ff47ce
|
2024-06-15 05:11:33 +08:00 |
|
hiyouga
|
c9959df3c7
|
disable DP
Former-commit-id: c18fd609d268389f3e65274992045a6c9f8e6c1f
|
2024-06-15 04:57:19 +08:00 |
|
hiyouga
|
d2137c7099
|
fix #4292
Former-commit-id: 4cd4c179d24eab0fcaec2b29b9dd71970f877fe8
|
2024-06-15 04:47:13 +08:00 |
|
hiyouga
|
43cfbde0d1
|
fix #4295
Former-commit-id: 08f657868f9d605b837c5d8c2946a25cc05c8735
|
2024-06-15 04:34:55 +08:00 |
|
hiyouga
|
bbeb3b10aa
|
add test cases
Former-commit-id: 731176ff34cdf0cbf6b41c40c69f4ceb54c2daf6
|
2024-06-15 04:05:54 +08:00 |
|
hiyouga
|
6306f3e2ea
|
Update README.md
Former-commit-id: f8d701cd3ce2e56f95b4f5439b8b48d5b62e0d2b
|
2024-06-13 16:02:21 +08:00 |
|
hiyouga
|
db6e74884c
|
update examples
Former-commit-id: d6bf6231290d79eb3a63e711f18fa711ef18a4f6
|
2024-06-13 03:26:10 +08:00 |
|
hiyouga
|
48d87a2dec
|
add quant check in webui export tab
Former-commit-id: 6455ca07061ae9858cd7bc996b28be1fde697a3d
|
2024-06-13 03:19:18 +08:00 |
|
hiyouga
|
e06568de98
|
Update llama3_full_sft_ds3.yaml
Former-commit-id: e715af62d521112d9c155cfa91fbb42fa0e77710
|
2024-06-13 03:16:20 +08:00 |
|
hiyouga
|
554c84f8d3
|
update examples
Former-commit-id: 19681f93db399d695aa8e35f8ec2a9e720875baa
|
2024-06-13 03:15:06 +08:00 |
|
hiyouga
|
e8885443a9
|
fix #4221
Former-commit-id: 05a3be4853b941909e7d193c31e8d62c8c5f879b
|
2024-06-13 02:48:21 +08:00 |
|
hiyouga
|
045cef901e
|
fix #4209
DeepSpeed ZeRO3 has inflight param error when calling model.eval()
Former-commit-id: 4be013f18ea6a35b5a11db98db5f0670ffb41619
|
2024-06-13 02:25:50 +08:00 |
|
hiyouga
|
344d1192ac
|
clean code
Former-commit-id: f54cafd5c7f0383370d1a2f357834a61a97397ce
|
2024-06-13 01:58:16 +08:00 |
|
hoshi-hiyouga
|
e1a33ab25f
|
Merge pull request #4246 from hzhaoy/adapt-vllm-v0.5.0
adapt vllm==0.5.0
Former-commit-id: 1068e25fc8b89f11cc79b164ee4aef9ce137ad4c
|
2024-06-13 01:54:02 +08:00 |
|
hiyouga
|
39e3d3fed6
|
add neo-sft dataset
Former-commit-id: 34863fa7cb641ceca92e3a2eec914126db537b62
|
2024-06-13 01:00:56 +08:00 |
|
hiyouga
|
f4c9555760
|
fix lint
Former-commit-id: b170165679317af2b3f03633afac27661b3deb06
|
2024-06-13 00:48:44 +08:00 |
|
hiyouga
|
7d3a9b10b7
|
fix docker compose usage
Former-commit-id: 59a5bd5d5c8d2a44e2dad26b74e77a45e109c8d6
|
2024-06-13 00:07:48 +08:00 |
|
hzhaoy
|
d98900d804
|
adapt vllm==0.5.0
Former-commit-id: 02afd9ff64f23e6707ac739ae1269f41bd70c340
|
2024-06-12 18:29:03 +08:00 |
|
hiyouga
|
8ad6a70a57
|
update readme
Former-commit-id: a436aaa83f0cf12c8f404459e5486f9369d538ec
|
2024-06-12 17:39:12 +08:00 |
|
hiyouga
|
f7b66af7af
|
fix #4242
Former-commit-id: cf260e7af03f49aa5e3d6daf3b27738ff9b9bcb8
|
2024-06-12 16:50:11 +08:00 |
|
hoshi-hiyouga
|
b7f3919c44
|
Merge pull request #4234 from kimdwkimdw/patch-1
Support vllm==0.5.0
Former-commit-id: 0a9da057c9e7ef11cd709b20263c3d2e4c2d72ed
|
2024-06-12 16:39:09 +08:00 |
|
Arthur Kim
|
45712c6251
|
Support vllm==0.5.0
Former-commit-id: e7a8ffd7af21bc3759f055033ba2209fa7a1be0e
|
2024-06-12 16:49:12 +09:00 |
|
ancv
|
4463a5227a
|
implement efficient packing without cross-contamination attention
Former-commit-id: a64a5305c0da5ef092d4cc26faf829bb44de65d1
|
2024-06-12 11:56:01 +07:00 |
|
hoshi-hiyouga
|
2640315e63
|
Merge pull request #4204 from dignfei/main
fixbug：llama3在增量预训练时应该使用<|end_of_text|>标识文本的结束
Former-commit-id: e566342636faf0031a0ba5d5dd4fcff8401a2b76
|
2024-06-11 17:06:10 +08:00 |
|
hoshi-hiyouga
|
6625bf6b33
|
Update pretrain.py
Former-commit-id: e2317b2a84149e39fddfd6366be3de23dfb71f82
|
2024-06-11 17:02:14 +08:00 |
|
hiyouga
|
a7233181f2
|
fix deepspeed version
Former-commit-id: 938a69bb07d4de7d82928ff01c582032162c1480
|
2024-06-11 16:52:36 +08:00 |
|
d
|
dfac202c7d
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: ef470561f742b16eaa0f99c4cadecd7c84ce6bd2
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
3f52c233b1
|
Update bug-report.yml
Former-commit-id: bb022cd867ebf2593e40fc6ba43b768603b129a3
|
2024-06-11 15:40:21 +08:00 |
|
hiyouga
|
95f95bef60
|
fix #4198
Former-commit-id: 945d2c6cc73542adf9272ebd9aa332ea2c1c7361
|
2024-06-11 15:38:38 +08:00 |
|
hiyouga
|
b7458a24b6
|
tiny fix
Former-commit-id: c4b2e263d9cefbad0fbc5de72422e4ef8edbcb54
|
2024-06-11 12:48:53 +08:00 |
|
hoshi-hiyouga
|
ca435ca2d3
|
Merge pull request #4191 from iamthebot/al--add_manifest_for_reqs
Add MANIFEST.in so requirements.txt is present in sdist
Former-commit-id: fd6d1c3fce855d1ef7396cf33af9f12eadc5a878
|
2024-06-11 10:41:15 +08:00 |
|
Alfredo Luque
|
4f33de245c
|
add manifest so requirements.txt in sdist
Former-commit-id: b501a3c56c51786c3006a2aca15a145641a4556c
|
2024-06-11 00:07:06 +00:00 |
|
hiyouga
|
8c7943c4de
|
tiny fix
Former-commit-id: b5e9711ef375cc323fc083e742cccfc974550416
|
2024-06-11 01:04:16 +08:00 |
|
hiyouga
|
e540759f4f
|
set dev version
Former-commit-id: 16c47cc15226119e33e46ba0f2f6ccb37072257f
|
2024-06-11 00:50:53 +08:00 |
|
hiyouga
|
41eadf5459
|
release v0.8.1
Former-commit-id: 875a34f492701d1c644facbe9ede411af2931513
|
2024-06-11 00:44:26 +08:00 |
|
hiyouga
|
68df064c1f
|
fix #4160
The split heads should be concatenated in dim=2
Former-commit-id: 4b3f247f270d44df9fe226cfe0dabfb7fcd2deda
|
2024-06-11 00:37:17 +08:00 |
|
hiyouga
|
0e7c15d2bd
|
fix #4145
Fix the docker image
Former-commit-id: a9838281156fe870bfcde5d1f7afc15264fd4aad
|
2024-06-11 00:19:17 +08:00 |
|
hiyouga
|
bc86e70af9
|
update evaluator
Former-commit-id: bb8661e62481ff7027b8969f3d8a6a17290c9da3
|
2024-06-10 23:56:00 +08:00 |
|