hiyouga
|
030b4811c7
|
update patcher
Former-commit-id: 3b040e8e0f78dbb6bc1409a1b2b788e1affc7458
|
2024-06-19 21:27:00 +08:00 |
|
hiyouga
|
80e9f8e000
|
set dev version
Former-commit-id: 42e69a3c634ccae792bd8ffb4642061ee475e836
|
2024-06-19 21:08:16 +08:00 |
|
hiyouga
|
9c1b04cd11
|
release v0.8.2
Former-commit-id: 71327ba85a3a1bb2d2d20c86951c6c7c0ba98829
|
2024-06-19 20:42:09 +08:00 |
|
hiyouga
|
3d72b1a856
|
fix jinja template
Former-commit-id: 2b596fb55ff689d2e488d9a9bbab98f70f356c3c
|
2024-06-19 20:03:50 +08:00 |
|
hiyouga
|
7735456561
|
fix templates
Former-commit-id: 4cff6a4ad55b24bf57db6be5cf817180c1ea5626
|
2024-06-19 17:44:05 +08:00 |
|
Jonery
|
c779899f7b
|
Cleaner integration.
Former-commit-id: 5c2ff1b749a265dd3c979189ec491d8ac911a6f6
|
2024-06-19 12:29:40 +08:00 |
|
hiyouga
|
c9557241f6
|
fix bug
Former-commit-id: 6d2bf216ac3a48450e861148ce664dad717fd019
|
2024-06-19 03:49:23 +08:00 |
|
hiyouga
|
e73a235a38
|
use prefix to replace force system
Former-commit-id: 4f22eae8f405de918237d406e5e9847592925565
|
2024-06-19 03:39:52 +08:00 |
|
hiyouga
|
bccc852f76
|
fix tool formatter, allow parallel function #4362
Former-commit-id: cd75b1fe9d91fb52a9ae6de7435302ff06b4d933
|
2024-06-19 03:23:51 +08:00 |
|
hoshi-hiyouga
|
6db02615d4
|
Merge pull request #4173 from mMrBun/main
Implemented the tool_formatter and tool_extractor for glm4 and Qwen2 tool_format
Former-commit-id: c0ca42566c6aeccd8d384377510690eafef10995
|
2024-06-19 03:18:55 +08:00 |
|
hiyouga
|
e3bf22f61b
|
add deepseek coder v2 #4346
Former-commit-id: a233fbc258d38c62d78b9d1eaf034720361795e6
|
2024-06-18 22:53:54 +08:00 |
|
hiyouga
|
5156114981
|
fix #4357
Former-commit-id: 4bd77d8563aa85230af65caf901214247e214bed
|
2024-06-18 22:42:45 +08:00 |
|
hiyouga
|
15a5eb6647
|
fix #4335
Former-commit-id: c96264bc477d65276557e9059cac7c550c4835a8
|
2024-06-18 22:08:56 +08:00 |
|
Jonery
|
c2734108e7
|
fix typo
Former-commit-id: 8f7c78b64138602406af748b0e15948ebbd2dcb5
|
2024-06-18 12:39:26 +08:00 |
|
Jonery
|
3a5eacb4cf
|
Support distributed BAdam.
Former-commit-id: 0f72aac8c9227e33ad20d2b1641b1c9faae16a5f
|
2024-06-18 12:27:47 +08:00 |
|
hiyouga
|
19bf21efba
|
lint
Former-commit-id: 24c160df3d575843e5ad5f1b47246d04430a79f0
|
2024-06-17 22:35:56 +08:00 |
|
hiyouga
|
3d85217464
|
update chat engine #4335
Former-commit-id: 7857c0990b37d37e58691257eea13fe5e244aaa0
|
2024-06-17 19:07:17 +08:00 |
|
Jonery
|
5d59f6562a
|
Merge remote-tracking branch 'upstream/main'
Former-commit-id: ea1f3ba5e030504e07053484f50f4cbdb37808bc
|
2024-06-17 18:44:51 +08:00 |
|
Jonery
|
756566342d
|
adapt for badam with ds zero3
Former-commit-id: 33b437277846d4f0b64c13a0bc892ef4f345a21e
|
2024-06-17 18:18:10 +08:00 |
|
hiyouga
|
7ef169ed39
|
fix #4326
Former-commit-id: e2665e71c7428014d46d91542b01a58c1064d05a
|
2024-06-17 18:17:48 +08:00 |
|
ancv
|
988231026a
|
update packing with sdpa and eager attention mode
Former-commit-id: 238f5c3d99809c6ae2571b59bdce8d8ea3c700b9
|
2024-06-16 02:25:47 +07:00 |
|
hoshi-hiyouga
|
06bbc29614
|
Update parser.py
Former-commit-id: 29c1f31baa442e35714b18b7e51896274a828cae
|
2024-06-16 02:57:00 +08:00 |
|
hiyouga
|
ce4a27a5f7
|
fix tol
Former-commit-id: 46093b5786611d99adf1fd3d42926a728fc629f8
|
2024-06-16 01:38:44 +08:00 |
|
hiyouga
|
f25b8626bf
|
support pissa
Former-commit-id: 8c1046d78ac6c8f9429b73617e35e1eccb35138f
|
2024-06-16 01:08:12 +08:00 |
|
hiyouga
|
c0c6b8075a
|
tiny fix
Former-commit-id: 38b6b0f52edeb8ba45aa03b415b3c0c1b0e0c1e4
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
9d9f8c6531
|
remove some unused params
Former-commit-id: 04315c3d92ecc25537e45d5807cb38bc290dcb16
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
96b82ccd4d
|
use fixture
Former-commit-id: 80a9e6bf94cf14fa63e6b6cdf7e1ce13722c8b5e
|
2024-06-15 20:06:17 +08:00 |
|
hiyouga
|
8053929b20
|
add tests
Former-commit-id: 1b834f50be64ae9b5123da0e6f528cfbd5167477
|
2024-06-15 19:51:20 +08:00 |
|
hiyouga
|
f0d6e63f55
|
add minicpm #4227
Former-commit-id: 572d8bbfdd73c1a00b432f0d0411f46fad6aa1a6
|
2024-06-15 17:58:52 +08:00 |
|
hiyouga
|
2946153cea
|
add license
Former-commit-id: d87108daa68bd40174b262be1ca65fe6e1b7ab56
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
fcbfa70c19
|
disable DP
Former-commit-id: d519b4d76d39b21a21b1d2f6f7ce6b3af9525d03
|
2024-06-15 04:57:19 +08:00 |
|
hiyouga
|
ba9aa7e2aa
|
fix #4292
Former-commit-id: 9092f963dba6f2e0eeade53b5b35e6b770f8348e
|
2024-06-15 04:47:13 +08:00 |
|
hiyouga
|
ab66ae8cd2
|
fix #4295
Former-commit-id: 78589cf90c6e12e612f269b1c771f19f3dad83d2
|
2024-06-15 04:34:55 +08:00 |
|
hiyouga
|
a3f4925c2c
|
add test cases
Former-commit-id: b27269bd2b52fb9d43cde8a8b7f293099b0127a2
|
2024-06-15 04:05:54 +08:00 |
|
hiyouga
|
27777c8e68
|
add quant check in webui export tab
Former-commit-id: c94e6c9411cc1165ea8180d377611a3ae47956e6
|
2024-06-13 03:19:18 +08:00 |
|
hiyouga
|
8fccaf20c5
|
fix #4221
Former-commit-id: 6baafd4eb3147ad9f7d2952b8eb27c5486940f36
|
2024-06-13 02:48:21 +08:00 |
|
hiyouga
|
81ed4d8abf
|
fix #4209
DeepSpeed ZeRO3 has inflight param error when calling model.eval()
Former-commit-id: cf9f2d6c42b5a37038c9eededbb767eae6a3f67d
|
2024-06-13 02:25:50 +08:00 |
|
hiyouga
|
833aa324c2
|
clean code
Former-commit-id: 2ed8270112755971e3f2dfd2f29c5939b077330a
|
2024-06-13 01:58:16 +08:00 |
|
hoshi-hiyouga
|
7366647b43
|
Merge pull request #4246 from hzhaoy/adapt-vllm-v0.5.0
adapt vllm==0.5.0
Former-commit-id: 1f23f25226f43bb2b2116474b9c6ab2d72e7e260
|
2024-06-13 01:54:02 +08:00 |
|
hiyouga
|
99ce085415
|
fix lint
Former-commit-id: 713fde4259233af645bade7790211064a07a2a6f
|
2024-06-13 00:48:44 +08:00 |
|
hzhaoy
|
65f2ba3802
|
adapt vllm==0.5.0
Former-commit-id: 8fb6366ebec432c1333c80eb51007460ac5896bd
|
2024-06-12 18:29:03 +08:00 |
|
hiyouga
|
f8497921fe
|
fix #4242
Former-commit-id: 577de2fa078ee62d61bca24b75fa59590e2ef66b
|
2024-06-12 16:50:11 +08:00 |
|
Arthur Kim
|
bebca6d01c
|
Support vllm==0.5.0
Former-commit-id: d65a3f7cb6dcd34e9a3ff9e30dffc52a7671d50b
|
2024-06-12 16:49:12 +09:00 |
|
ancv
|
045eb155a2
|
implement efficient packing without cross-contamination attention
Former-commit-id: b2c367bc61c2778dc359613dca496d9e134c2743
|
2024-06-12 11:56:01 +07:00 |
|
hoshi-hiyouga
|
fecb9c9a76
|
Merge pull request #4204 from dignfei/main
fixbug：llama3在增量预训练时应该使用<|end_of_text|>标识文本的结束
Former-commit-id: 9049aab911fa3fbe9cd463ae3805f81b484da052
|
2024-06-11 17:06:10 +08:00 |
|
hoshi-hiyouga
|
bf3de9bfe8
|
Update pretrain.py
Former-commit-id: 0c292332374fb96c3fc753abde42d070a0c1dca8
|
2024-06-11 17:02:14 +08:00 |
|
hiyouga
|
8c574eb3cb
|
fix deepspeed version
Former-commit-id: cca6f351081903ca3b5f79f10accc1bbbae0ee61
|
2024-06-11 16:52:36 +08:00 |
|
d
|
da39715085
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: 6979f3f8480755604d8aea8164f6418126e094c5
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
5834651c4a
|
fix #4198
Former-commit-id: 89f2bd8c8c035181927bd530a7ffc733407d674c
|
2024-06-11 15:38:38 +08:00 |
|
hiyouga
|
53de7f7cc3
|
tiny fix
Former-commit-id: 90e14a960d1437a16d35dbabbb8aa50714583d3a
|
2024-06-11 12:48:53 +08:00 |
|