hoshi-hiyouga
|
0ac7824822
|
Merge pull request #4892 from piamo/main
update deepseek template
Former-commit-id: 8a2846cfe1266bedde84da175805dac78439c20e
|
2024-07-26 11:49:34 +08:00 |
|
hiyouga
|
d63beb7a24
|
fix #4928
Former-commit-id: 091010492b2afc8ad25359323aba760523a4c5af
|
2024-07-24 17:00:29 +08:00 |
|
hiyouga
|
8d06679a3f
|
fix flashattn + packing
Former-commit-id: 4135e69406c09f3892276e4fee7e5757a52a42d6
|
2024-07-21 17:07:45 +08:00 |
|
huangpan.foo
|
ee4c3f32d1
|
update deepseek template
Former-commit-id: 44e48e2b82929888b0880c00519102da4eb38ca8
|
2024-07-19 15:02:54 +08:00 |
|
hiyouga
|
34f16cc635
|
follow #4878 fix #4684
Former-commit-id: 779aae83d253de0a86201ff87543b5d695e28d23
|
2024-07-18 22:06:12 +08:00 |
|
Shiyu Zhang
|
249adacc4d
|
仅仅训练最后一轮对话
Former-commit-id: 1e7b396ff2489055574fd3365425d26360d73897
|
2024-07-18 15:30:25 +08:00 |
|
hoshi-hiyouga
|
2b22a7da48
|
Merge pull request #4691 from codemayq/feature-suppot-eval-dataset
add eval dataset support
Former-commit-id: 15b399a82f45b08fc07d2957884fb7821eba9fd9
|
2024-07-15 01:00:34 +08:00 |
|
hoshi-hiyouga
|
30a3c6e886
|
Update preprocess.py
Former-commit-id: df52fb05b1b08887288bbaab7c612b7ac27c2290
|
2024-07-15 00:55:36 +08:00 |
|
hoshi-hiyouga
|
eed7cbb453
|
Update parser.py
Former-commit-id: 84e4047f8a1f78256be65f3f7bddce358ed9e882
|
2024-07-15 00:55:21 +08:00 |
|
hoshi-hiyouga
|
5633c0ab1e
|
Update data_utils.py
Former-commit-id: 97a0e291c79f145950b54a11d03d81ada4784d22
|
2024-07-15 00:54:34 +08:00 |
|
hoshi-hiyouga
|
2e9c9471da
|
Update loader.py
Former-commit-id: a5b809516e7de1d6d5f4583089fee3028d0db01d
|
2024-07-15 00:50:06 +08:00 |
|
hoshi-hiyouga
|
140b512426
|
Update parser.py
Former-commit-id: 3d39d74003c4ca36f9c9b77f622d366383b0af7e
|
2024-07-14 23:04:34 +08:00 |
|
hiyouga
|
0b26011181
|
fix gemma2 attention
Former-commit-id: 2f6af73da28c4f8321b625fd09ddec8bd4977b08
|
2024-07-13 23:33:45 +08:00 |
|
hiyouga
|
3baf1d1f7e
|
fix #4792
Former-commit-id: 6b48308ef9be34d072f3e6bb2444e186a38c2779
|
2024-07-13 22:07:58 +08:00 |
|
hiyouga
|
7fcffb860d
|
add codegeex4, internlm2.5
Former-commit-id: 53b1002fb74123095e7466c75b941a31a7cfba4d
|
2024-07-06 16:16:47 +08:00 |
|
codingma
|
74f0d02eb8
|
1. add custom eval dataset support
2. merge load dataset and split dataset function
Former-commit-id: 76f3bbcfc0e11aa41f8f5cbebc60b77b987f7901
|
2024-07-05 15:52:10 +08:00 |
|
hiyouga
|
8379a39776
|
fix processors
Former-commit-id: 9f33f1edf544807e498f60881f30b00149fe570f
|
2024-07-05 08:33:22 +08:00 |
|
hiyouga
|
9aa3403687
|
fix #4683
Former-commit-id: e43809bced009323b3bac9accdd3baa3a2836fdb
|
2024-07-05 00:58:05 +08:00 |
|
hzhaoy
|
6d892dbc23
|
tiny fix
Former-commit-id: 738df477485de3633049651a9f1d498adf95a3d5
|
2024-07-04 10:20:28 +08:00 |
|
hiyouga
|
cbb93a2b47
|
tiny fix
Former-commit-id: 44747cebd28d0b800196f032e18d2f4ff51ee5b3
|
2024-07-04 03:02:23 +08:00 |
|
hiyouga
|
4987aa32ba
|
fix data map for packing
Former-commit-id: b5d101e1bf435731e6b8e5aed8727ddfb021e4f0
|
2024-07-04 03:01:31 +08:00 |
|
hiyouga
|
7b3c1f29ff
|
fix packing for eager/sdpa attn
Former-commit-id: 6fd6aa4530f81a2ed306eeb2a5167607288b62c6
|
2024-07-04 01:52:43 +08:00 |
|
hoshi-hiyouga
|
a38ff842d0
|
Merge pull request #4224 from chuan298/main
Implement efficient packing without cross-contamination attention
Former-commit-id: 87d9b2d00513c163335d3f2e2bb3cb3299cecdaa
|
2024-07-04 01:18:54 +08:00 |
|
hiyouga
|
bfdaadcc40
|
update packing
Former-commit-id: cce7083024bed4c7429ddc8288d1c9190fde29f5
|
2024-07-04 01:10:55 +08:00 |
|
hiyouga
|
ff6fc666c1
|
update hparams
Former-commit-id: 575a02a23d9b41d00ca6291d8a40b5bdb3cbeeec
|
2024-07-03 23:18:58 +08:00 |
|
hiyouga
|
cc31014002
|
improve rlhf
Former-commit-id: c47ab6c07287fb260ea49b8b7af46bdd416f88f7
|
2024-07-02 22:23:08 +08:00 |
|
ancv
|
7f42932957
|
move efficient_packing from data_args to model_args
Former-commit-id: e8e13b09423dd08a31a3bde8f85833c6e5d43ee5
|
2024-07-02 18:37:55 +07:00 |
|
hoshi-hiyouga
|
2452f57cd7
|
Merge branch 'main' into main
Former-commit-id: e8e6af26514272e29a50649b38182beb4db4ebfa
|
2024-07-01 21:01:09 +08:00 |
|
hiyouga
|
ca7b65439d
|
fix #4402 #4617
Deprecate reserved_label_len arg
Former-commit-id: 1771251ce3f6887b301dac10f3de7a253c5e5884
|
2024-07-01 01:19:27 +08:00 |
|
hiyouga
|
654116c0b1
|
fix #4556
Former-commit-id: 59e0b4f616736ede37cc37a13346b547f5a2d4e7
|
2024-06-26 19:43:16 +08:00 |
|
hiyouga
|
d519c2fde5
|
tiny fix
Former-commit-id: 41086059b12ecb7827eb390294e315068ff9c2e6
|
2024-06-25 01:15:19 +08:00 |
|
hoshi-hiyouga
|
709bbc1d92
|
Merge pull request #4417 from mMrBun/main
Add tool_format parameter to rewrite templates for different function call formats.
Former-commit-id: def6d280db3a9fe468b05503bcd9929c83c6c19b
|
2024-06-24 23:17:55 +08:00 |
|
hoshi-hiyouga
|
b7f5cfde6e
|
Update template.py
Former-commit-id: 1240bd57d8a21540c636a6da839e6b3112d1395a
|
2024-06-24 23:12:59 +08:00 |
|
hoshi-hiyouga
|
673f27a59e
|
Update loader.py
Former-commit-id: dddfd516ee66e9937e21f05300832aab45034b12
|
2024-06-24 23:06:18 +08:00 |
|
hiyouga
|
47651a94a3
|
fix #4410
Former-commit-id: fca893d73c3d7bbb87a816522f2e1568d3e9c612
|
2024-06-24 22:34:31 +08:00 |
|
mMrBun
|
c0e005e2ea
|
Add tool_format to overwrite tool formatter template
Former-commit-id: 20e2e6fdcb0cd1771906be035745a2d9fcd3e138
|
2024-06-22 02:13:23 +08:00 |
|
hiyouga
|
98abb5c900
|
remove dup template
Former-commit-id: db9a1912e3551394039cc57b4913f03e8f9aa29d
|
2024-06-22 01:31:32 +08:00 |
|
hiyouga
|
3d72b1a856
|
fix jinja template
Former-commit-id: 2b596fb55ff689d2e488d9a9bbab98f70f356c3c
|
2024-06-19 20:03:50 +08:00 |
|
hiyouga
|
7735456561
|
fix templates
Former-commit-id: 4cff6a4ad55b24bf57db6be5cf817180c1ea5626
|
2024-06-19 17:44:05 +08:00 |
|
hiyouga
|
c9557241f6
|
fix bug
Former-commit-id: 6d2bf216ac3a48450e861148ce664dad717fd019
|
2024-06-19 03:49:23 +08:00 |
|
hiyouga
|
e73a235a38
|
use prefix to replace force system
Former-commit-id: 4f22eae8f405de918237d406e5e9847592925565
|
2024-06-19 03:39:52 +08:00 |
|
hiyouga
|
bccc852f76
|
fix tool formatter, allow parallel function #4362
Former-commit-id: cd75b1fe9d91fb52a9ae6de7435302ff06b4d933
|
2024-06-19 03:23:51 +08:00 |
|
hoshi-hiyouga
|
6db02615d4
|
Merge pull request #4173 from mMrBun/main
Implemented the tool_formatter and tool_extractor for glm4 and Qwen2 tool_format
Former-commit-id: c0ca42566c6aeccd8d384377510690eafef10995
|
2024-06-19 03:18:55 +08:00 |
|
hiyouga
|
c0c6b8075a
|
tiny fix
Former-commit-id: 38b6b0f52edeb8ba45aa03b415b3c0c1b0e0c1e4
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
9d9f8c6531
|
remove some unused params
Former-commit-id: 04315c3d92ecc25537e45d5807cb38bc290dcb16
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
2946153cea
|
add license
Former-commit-id: d87108daa68bd40174b262be1ca65fe6e1b7ab56
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
8fccaf20c5
|
fix #4221
Former-commit-id: 6baafd4eb3147ad9f7d2952b8eb27c5486940f36
|
2024-06-13 02:48:21 +08:00 |
|
ancv
|
045eb155a2
|
implement efficient packing without cross-contamination attention
Former-commit-id: b2c367bc61c2778dc359613dca496d9e134c2743
|
2024-06-12 11:56:01 +07:00 |
|
hoshi-hiyouga
|
bf3de9bfe8
|
Update pretrain.py
Former-commit-id: 0c292332374fb96c3fc753abde42d070a0c1dca8
|
2024-06-11 17:02:14 +08:00 |
|
d
|
da39715085
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: 6979f3f8480755604d8aea8164f6418126e094c5
|
2024-06-11 16:23:40 +08:00 |
|