hiyouga
|
8fccaf20c5
|
fix #4221
Former-commit-id: 6baafd4eb3
|
2024-06-13 02:48:21 +08:00 |
|
hoshi-hiyouga
|
bf3de9bfe8
|
Update pretrain.py
Former-commit-id: 0c29233237
|
2024-06-11 17:02:14 +08:00 |
|
d
|
da39715085
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: 6979f3f848
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
ce40d12692
|
release v0.8.0
Former-commit-id: 5aa4ce4756
|
2024-06-08 05:20:54 +08:00 |
|
hiyouga
|
c6f5f69644
|
update data processors
Former-commit-id: ccc8b64cc2
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
4953ded639
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 181dbb0d05
|
2024-06-07 03:48:46 +08:00 |
|
hoshi-hiyouga
|
e3ef239bc0
|
Update supervised.py
Former-commit-id: c09ad8bab3
|
2024-06-07 03:42:08 +08:00 |
|
hoshi-hiyouga
|
fd7bd911a6
|
Update supervised.py
Former-commit-id: 788e8232fc
|
2024-06-07 03:38:23 +08:00 |
|
hoshi-hiyouga
|
21df5f0bd0
|
Update supervised.py
Former-commit-id: 8cecade708
|
2024-06-07 03:38:04 +08:00 |
|
hiyouga
|
8da149ba40
|
rename files
Former-commit-id: 74f96efef9
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
e0aadd4b34
|
fix ppo dataset bug #4012
Former-commit-id: 149610c636
|
2024-06-06 19:03:20 +08:00 |
|
hiyouga
|
94c37490d1
|
support glm-4
Former-commit-id: f48f5e646e
|
2024-06-05 15:16:38 +08:00 |
|
hiyouga
|
0eff6a66d5
|
tiny fix
Former-commit-id: 5a13b3baa6
|
2024-06-04 00:31:10 +08:00 |
|
hiyouga
|
8ecf606230
|
fix #3992
Former-commit-id: a18acf2abe
|
2024-06-04 00:17:36 +08:00 |
|
hiyouga
|
64d24842fe
|
fix data loader hint
Former-commit-id: 49b1e88e3d
|
2024-06-03 18:28:27 +08:00 |
|
ylfeng
|
62d55b71a3
|
remove empty line
Former-commit-id: b47e317447
|
2024-05-31 21:43:08 +08:00 |
|
ylfeng
|
0feb2ad35c
|
fix eos
Former-commit-id: 84aee57901
|
2024-05-31 21:40:41 +08:00 |
|
ylfeng
|
8350e508d3
|
supervised packing with greedy knapsack algorithm
Former-commit-id: f9db439cb7
|
2024-05-31 15:33:54 +08:00 |
|
hoshi-hiyouga
|
9b6bdf9449
|
Merge pull request #3829 from seanzhang-zhichen/add_dataset_sample_num
Add dataset sample num
Former-commit-id: 483eb47e5d
|
2024-05-30 00:25:45 +08:00 |
|
hoshi-hiyouga
|
7b83c550ab
|
Update loader.py
Former-commit-id: ca5dd7c6c1
|
2024-05-30 00:20:20 +08:00 |
|
hoshi-hiyouga
|
9fc713da89
|
Update loader.py
Former-commit-id: f9a88b89ca
|
2024-05-30 00:17:21 +08:00 |
|
hoshi-hiyouga
|
c0f11a280e
|
Update loader.py
Former-commit-id: b55fb611c5
|
2024-05-30 00:12:12 +08:00 |
|
hoshi-hiyouga
|
69a51cacb1
|
Update parser.py
Former-commit-id: 51dd454337
|
2024-05-30 00:05:20 +08:00 |
|
hiyouga
|
19a3262387
|
fix cohere system
Former-commit-id: d0aa36b8ad
|
2024-05-29 20:58:23 +08:00 |
|
hiyouga
|
c05cb3769f
|
fix #3965
Former-commit-id: 0930f58699
|
2024-05-29 20:55:51 +08:00 |
|
hiyouga
|
a71a6a05c3
|
update readme
Former-commit-id: 89ca832740
|
2024-05-29 18:39:11 +08:00 |
|
hzhaoy
|
ce1be3da4b
|
add TeleChat-12B/TeleChat-12B-v2 models
Former-commit-id: 0dd632fe9e
|
2024-05-29 15:00:37 +08:00 |
|
Yimi81
|
7324984127
|
fix yi template
Former-commit-id: dc07413e7d
|
2024-05-27 13:11:25 +00:00 |
|
hiyouga
|
0706dbf7e6
|
tiny fix
Former-commit-id: c1fdf81df6
|
2024-05-27 20:54:26 +08:00 |
|
hoshi-hiyouga
|
eceec1d7fd
|
Update template.py
Former-commit-id: f1002b9f93
|
2024-05-27 20:51:56 +08:00 |
|
hoshi-hiyouga
|
b7b8223230
|
Update template.py
Former-commit-id: 122213a7a7
|
2024-05-27 20:51:26 +08:00 |
|
Jianbai Ye
|
d2c1df7f3d
|
add openchat-3.6-8B support
Former-commit-id: cff815391f
|
2024-05-27 20:42:08 +08:00 |
|
hiyouga
|
df33548b39
|
update readme
Former-commit-id: 5581cb2e4e
|
2024-05-27 18:14:02 +08:00 |
|
seanzhang-zhichen
|
9c8d79fbe3
|
Merge branch 'main' into add_dataset_sample_num
Former-commit-id: 27cb51f7f8
|
2024-05-24 15:57:47 +08:00 |
|
hiyouga
|
3e729798df
|
refactor data preprocessing, fix mllm rlhf
Former-commit-id: 3a023bca2a
|
2024-05-24 04:08:25 +08:00 |
|
hiyouga
|
d3490aceb7
|
fix paligemma sft
requires transformers>=4.41.1
Former-commit-id: de0e67aff1
|
2024-05-24 00:23:40 +08:00 |
|
hiyouga
|
4ddc1c9c16
|
fix paligemma sft
Former-commit-id: 7134fb02bb
|
2024-05-21 20:03:09 +08:00 |
|
hiyouga
|
a935c5105d
|
fix paligemma data preprocess
Former-commit-id: e55c85ac72
|
2024-05-20 23:51:32 +08:00 |
|
hiyouga
|
446c681b58
|
fix paligemma inference
Former-commit-id: 542229abb3
|
2024-05-20 23:36:43 +08:00 |
|
zhangzc
|
4b90f04c1f
|
fix conflict
Former-commit-id: d956041640
|
2024-05-20 17:10:01 +08:00 |
|
hiyouga
|
864da49139
|
fix chat engines
do not use pop(key, default) since api assigns None to dict values
Former-commit-id: d52fae2fa8
|
2024-05-20 00:36:43 +08:00 |
|
hiyouga
|
0e57bb201c
|
fix jinja template
Former-commit-id: 10573e1639
|
2024-05-19 23:38:30 +08:00 |
|
hiyouga
|
519d2511ae
|
improve data process logger
Former-commit-id: a851056229
|
2024-05-18 22:02:42 +08:00 |
|
hiyouga
|
1e867c0fa0
|
fix #3803
Former-commit-id: 0edc16769f
|
2024-05-18 16:13:14 +08:00 |
|
hiyouga
|
13d7b48efe
|
improve KTO impl., replace datasets
Former-commit-id: c450ee87a3
|
2024-05-18 03:44:56 +08:00 |
|
enji.zhou
|
03956053b8
|
add kto
Former-commit-id: db1d5a4f51
|
2024-05-17 13:09:17 +08:00 |
|
hiyouga
|
cae823ddf0
|
rename package
Former-commit-id: 308edbc426
|
2024-05-16 18:39:08 +08:00 |
|