hiyouga
|
34f16cc635
|
follow #4878 fix #4684
Former-commit-id: 779aae83d2
|
2024-07-18 22:06:12 +08:00 |
|
Shiyu Zhang
|
249adacc4d
|
仅仅训练最后一轮对话
Former-commit-id: 1e7b396ff2
|
2024-07-18 15:30:25 +08:00 |
|
hoshi-hiyouga
|
2b22a7da48
|
Merge pull request #4691 from codemayq/feature-suppot-eval-dataset
add eval dataset support
Former-commit-id: 15b399a82f
|
2024-07-15 01:00:34 +08:00 |
|
hoshi-hiyouga
|
30a3c6e886
|
Update preprocess.py
Former-commit-id: df52fb05b1
|
2024-07-15 00:55:36 +08:00 |
|
hoshi-hiyouga
|
eed7cbb453
|
Update parser.py
Former-commit-id: 84e4047f8a
|
2024-07-15 00:55:21 +08:00 |
|
hoshi-hiyouga
|
5633c0ab1e
|
Update data_utils.py
Former-commit-id: 97a0e291c7
|
2024-07-15 00:54:34 +08:00 |
|
hoshi-hiyouga
|
2e9c9471da
|
Update loader.py
Former-commit-id: a5b809516e
|
2024-07-15 00:50:06 +08:00 |
|
hoshi-hiyouga
|
140b512426
|
Update parser.py
Former-commit-id: 3d39d74003
|
2024-07-14 23:04:34 +08:00 |
|
hiyouga
|
0b26011181
|
fix gemma2 attention
Former-commit-id: 2f6af73da2
|
2024-07-13 23:33:45 +08:00 |
|
hiyouga
|
3baf1d1f7e
|
fix #4792
Former-commit-id: 6b48308ef9
|
2024-07-13 22:07:58 +08:00 |
|
hiyouga
|
7fcffb860d
|
add codegeex4, internlm2.5
Former-commit-id: 53b1002fb7
|
2024-07-06 16:16:47 +08:00 |
|
codingma
|
74f0d02eb8
|
1. add custom eval dataset support
2. merge load dataset and split dataset function
Former-commit-id: 76f3bbcfc0
|
2024-07-05 15:52:10 +08:00 |
|
hiyouga
|
8379a39776
|
fix processors
Former-commit-id: 9f33f1edf5
|
2024-07-05 08:33:22 +08:00 |
|
hiyouga
|
9aa3403687
|
fix #4683
Former-commit-id: e43809bced
|
2024-07-05 00:58:05 +08:00 |
|
hzhaoy
|
6d892dbc23
|
tiny fix
Former-commit-id: 738df47748
|
2024-07-04 10:20:28 +08:00 |
|
hiyouga
|
cbb93a2b47
|
tiny fix
Former-commit-id: 44747cebd2
|
2024-07-04 03:02:23 +08:00 |
|
hiyouga
|
4987aa32ba
|
fix data map for packing
Former-commit-id: b5d101e1bf
|
2024-07-04 03:01:31 +08:00 |
|
hiyouga
|
7b3c1f29ff
|
fix packing for eager/sdpa attn
Former-commit-id: 6fd6aa4530
|
2024-07-04 01:52:43 +08:00 |
|
hoshi-hiyouga
|
a38ff842d0
|
Merge pull request #4224 from chuan298/main
Implement efficient packing without cross-contamination attention
Former-commit-id: 87d9b2d005
|
2024-07-04 01:18:54 +08:00 |
|
hiyouga
|
bfdaadcc40
|
update packing
Former-commit-id: cce7083024
|
2024-07-04 01:10:55 +08:00 |
|
hiyouga
|
ff6fc666c1
|
update hparams
Former-commit-id: 575a02a23d
|
2024-07-03 23:18:58 +08:00 |
|
hiyouga
|
cc31014002
|
improve rlhf
Former-commit-id: c47ab6c072
|
2024-07-02 22:23:08 +08:00 |
|
ancv
|
7f42932957
|
move efficient_packing from data_args to model_args
Former-commit-id: e8e13b0942
|
2024-07-02 18:37:55 +07:00 |
|
hoshi-hiyouga
|
2452f57cd7
|
Merge branch 'main' into main
Former-commit-id: e8e6af2651
|
2024-07-01 21:01:09 +08:00 |
|
hiyouga
|
ca7b65439d
|
fix #4402 #4617
Deprecate reserved_label_len arg
Former-commit-id: 1771251ce3
|
2024-07-01 01:19:27 +08:00 |
|
hiyouga
|
654116c0b1
|
fix #4556
Former-commit-id: 59e0b4f616
|
2024-06-26 19:43:16 +08:00 |
|
hiyouga
|
d519c2fde5
|
tiny fix
Former-commit-id: 41086059b1
|
2024-06-25 01:15:19 +08:00 |
|
hoshi-hiyouga
|
709bbc1d92
|
Merge pull request #4417 from mMrBun/main
Add tool_format parameter to rewrite templates for different function call formats.
Former-commit-id: def6d280db
|
2024-06-24 23:17:55 +08:00 |
|
hoshi-hiyouga
|
b7f5cfde6e
|
Update template.py
Former-commit-id: 1240bd57d8
|
2024-06-24 23:12:59 +08:00 |
|
hoshi-hiyouga
|
673f27a59e
|
Update loader.py
Former-commit-id: dddfd516ee
|
2024-06-24 23:06:18 +08:00 |
|
hiyouga
|
47651a94a3
|
fix #4410
Former-commit-id: fca893d73c
|
2024-06-24 22:34:31 +08:00 |
|
mMrBun
|
c0e005e2ea
|
Add tool_format to overwrite tool formatter template
Former-commit-id: 20e2e6fdcb
|
2024-06-22 02:13:23 +08:00 |
|
hiyouga
|
98abb5c900
|
remove dup template
Former-commit-id: db9a1912e3
|
2024-06-22 01:31:32 +08:00 |
|
hiyouga
|
3d72b1a856
|
fix jinja template
Former-commit-id: 2b596fb55f
|
2024-06-19 20:03:50 +08:00 |
|
hiyouga
|
7735456561
|
fix templates
Former-commit-id: 4cff6a4ad5
|
2024-06-19 17:44:05 +08:00 |
|
hiyouga
|
c9557241f6
|
fix bug
Former-commit-id: 6d2bf216ac
|
2024-06-19 03:49:23 +08:00 |
|
hiyouga
|
e73a235a38
|
use prefix to replace force system
Former-commit-id: 4f22eae8f4
|
2024-06-19 03:39:52 +08:00 |
|
hiyouga
|
bccc852f76
|
fix tool formatter, allow parallel function #4362
Former-commit-id: cd75b1fe9d
|
2024-06-19 03:23:51 +08:00 |
|
hoshi-hiyouga
|
6db02615d4
|
Merge pull request #4173 from mMrBun/main
Implemented the tool_formatter and tool_extractor for glm4 and Qwen2 tool_format
Former-commit-id: c0ca42566c
|
2024-06-19 03:18:55 +08:00 |
|
hiyouga
|
c0c6b8075a
|
tiny fix
Former-commit-id: 38b6b0f52e
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
9d9f8c6531
|
remove some unused params
Former-commit-id: 04315c3d92
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
2946153cea
|
add license
Former-commit-id: d87108daa6
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
8fccaf20c5
|
fix #4221
Former-commit-id: 6baafd4eb3
|
2024-06-13 02:48:21 +08:00 |
|
ancv
|
045eb155a2
|
implement efficient packing without cross-contamination attention
Former-commit-id: b2c367bc61
|
2024-06-12 11:56:01 +07:00 |
|
hoshi-hiyouga
|
bf3de9bfe8
|
Update pretrain.py
Former-commit-id: 0c29233237
|
2024-06-11 17:02:14 +08:00 |
|
d
|
da39715085
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: 6979f3f848
|
2024-06-11 16:23:40 +08:00 |
|
mMrBun
|
b6d63b3324
|
Optimize the handling of QWEN2 in scenarios involving multiple tool calls.
Former-commit-id: 950e360ca0
|
2024-06-10 02:00:14 +08:00 |
|
mMrBun
|
3f11ab800f
|
Removed unnecessary comments.
Former-commit-id: 6ed0b0c800
|
2024-06-09 18:25:22 +08:00 |
|
mMrBun
|
daf472994d
|
Merge branch 'hiyouga:main' into main
Former-commit-id: 0f2609ce19
|
2024-06-09 18:17:24 +08:00 |
|
mMrBun
|
18a86ea104
|
Implemented the tool_formatter and tool_extractor for glm4 tool_format
Former-commit-id: cb1cbcb293
|
2024-06-09 18:16:15 +08:00 |
|