hiyouga
|
49b58fd6af
|
fix #4221
Former-commit-id: 05a3be4853b941909e7d193c31e8d62c8c5f879b
|
2024-06-13 02:48:21 +08:00 |
|
hoshi-hiyouga
|
cc9717e2f2
|
Update pretrain.py
Former-commit-id: e2317b2a84149e39fddfd6366be3de23dfb71f82
|
2024-06-11 17:02:14 +08:00 |
|
d
|
77bf3d66c7
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: ef470561f742b16eaa0f99c4cadecd7c84ce6bd2
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
c0c387e4db
|
release v0.8.0
Former-commit-id: 004db680b9e3996ec511ee818df6c0c02bf13603
|
2024-06-08 05:20:54 +08:00 |
|
hiyouga
|
8c4c2e580c
|
update data processors
Former-commit-id: 04b138cbcb8b9a72e4bbda6c65843bb459e525e7
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
07f33e7641
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 5ded166b39a75a98ded5733678f5a1eab7d4cc71
|
2024-06-07 03:48:46 +08:00 |
|
hoshi-hiyouga
|
1998c641af
|
Update supervised.py
Former-commit-id: 04b6c2a754e602e0b698cfe6c255c2f2486d8865
|
2024-06-07 03:42:08 +08:00 |
|
hoshi-hiyouga
|
be1e5f9d62
|
Update supervised.py
Former-commit-id: 49993c4f4e1f871a22ff0196afe60026b668a4dc
|
2024-06-07 03:38:23 +08:00 |
|
hoshi-hiyouga
|
fdeec6db52
|
Update supervised.py
Former-commit-id: 67625b5278a839c12a3e4245f9e90af67d8b11b4
|
2024-06-07 03:38:04 +08:00 |
|
hiyouga
|
fcb134e144
|
rename files
Former-commit-id: e1a8431770fc36c0c9ee7fed4abbc3d7fdcc5efd
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
ca95e98ca0
|
fix ppo dataset bug #4012
Former-commit-id: 7fc51b2e93698ae5e012566af8481f4d861c873d
|
2024-06-06 19:03:20 +08:00 |
|
hiyouga
|
7528bc1bc0
|
support glm-4
Former-commit-id: a10f4718fbf3f3c89dc7eb31cb8e1a46ca6adda5
|
2024-06-05 15:16:38 +08:00 |
|
hiyouga
|
2ac2cde03e
|
tiny fix
Former-commit-id: f9d50501aac1f60a3b445ca3fee9aa60995461ee
|
2024-06-04 00:31:10 +08:00 |
|
hiyouga
|
f4f5d7e3ce
|
fix #3992
Former-commit-id: a48321fbf5196b88a11106cf74a74fbcea2ea50b
|
2024-06-04 00:17:36 +08:00 |
|
hiyouga
|
123a845209
|
fix data loader hint
Former-commit-id: 25b56126a11591b0155e2f72b673dd8f45a6c8c9
|
2024-06-03 18:28:27 +08:00 |
|
ylfeng
|
b1b7d735b3
|
remove empty line
Former-commit-id: 3164710971a6d6545629f5bf133f98de5ff0991a
|
2024-05-31 21:43:08 +08:00 |
|
ylfeng
|
230c69f7ce
|
fix eos
Former-commit-id: 6e236c952958cbfe50b5dcb7b8eff6aea8477922
|
2024-05-31 21:40:41 +08:00 |
|
ylfeng
|
bfc43558ef
|
supervised packing with greedy knapsack algorithm
Former-commit-id: 24d12396c9aabd49da0b08719068f24679111cc6
|
2024-05-31 15:33:54 +08:00 |
|
hoshi-hiyouga
|
4f7c850115
|
Merge pull request #3829 from seanzhang-zhichen/add_dataset_sample_num
Add dataset sample num
Former-commit-id: ab38cf74ce48ea4f1800e077ca287f2eb9336135
|
2024-05-30 00:25:45 +08:00 |
|
hoshi-hiyouga
|
391eca66cf
|
Update loader.py
Former-commit-id: 0aa59322906d91c5e385c9c02ebb5dd64ba060f3
|
2024-05-30 00:20:20 +08:00 |
|
hoshi-hiyouga
|
a67199246d
|
Update loader.py
Former-commit-id: aa7f335e3ad5a78e4ed5f99c120be28e9733ea2e
|
2024-05-30 00:17:21 +08:00 |
|
hoshi-hiyouga
|
5f67fdaac9
|
Update loader.py
Former-commit-id: 19d8fd62c18ee3ba0e431fc241f7d315cb716fef
|
2024-05-30 00:12:12 +08:00 |
|
hoshi-hiyouga
|
05e6fe4287
|
Update parser.py
Former-commit-id: 310cc11e8c83f16fc5bccc349c38fea347ea9a97
|
2024-05-30 00:05:20 +08:00 |
|
hiyouga
|
f90c4ca672
|
fix cohere system
Former-commit-id: 5d629b29e705c8ff8dd4521719d9c0e67a3fe0a2
|
2024-05-29 20:58:23 +08:00 |
|
hiyouga
|
a922e85a5c
|
fix #3965
Former-commit-id: 37d15ac55d0be0ff47d6a88f07e2d823117a4a36
|
2024-05-29 20:55:51 +08:00 |
|
hiyouga
|
9a65820592
|
update readme
Former-commit-id: 440e9de66986ef7736361ce8ec3e23ce68655a56
|
2024-05-29 18:39:11 +08:00 |
|
hzhaoy
|
29cb4a1327
|
add TeleChat-12B/TeleChat-12B-v2 models
Former-commit-id: e0675385c88af03aaef8d51586c8a282829c4051
|
2024-05-29 15:00:37 +08:00 |
|
Yimi81
|
d9fbd675d5
|
fix yi template
Former-commit-id: b3669c8989c3adda305416245e32e9e5a3b7caac
|
2024-05-27 13:11:25 +00:00 |
|
hiyouga
|
0206e7b9de
|
tiny fix
Former-commit-id: 4c47b3dcef9e400a1c35fce1ad53619a0a86fe81
|
2024-05-27 20:54:26 +08:00 |
|
hoshi-hiyouga
|
8c9b929bb0
|
Update template.py
Former-commit-id: f4dabce0a71c9978e051e70886941b64b928ffe2
|
2024-05-27 20:51:56 +08:00 |
|
hoshi-hiyouga
|
1bb1ae834e
|
Update template.py
Former-commit-id: af869e4c48eb426c4078415533f6dab89123a9d8
|
2024-05-27 20:51:26 +08:00 |
|
Jianbai Ye
|
0d9e364a90
|
add openchat-3.6-8B support
Former-commit-id: b66f39d50d896d7597a1506e67ec210b31c9b700
|
2024-05-27 20:42:08 +08:00 |
|
hiyouga
|
97a23e1cbe
|
update readme
Former-commit-id: b8d0170fe0d094acce85dcb5f91775e4685ee055
|
2024-05-27 18:14:02 +08:00 |
|
seanzhang-zhichen
|
a3b52fd380
|
Merge branch 'main' into add_dataset_sample_num
Former-commit-id: 26300127c45f24e63b91f1b0cc73e46c3a936a91
|
2024-05-24 15:57:47 +08:00 |
|
hiyouga
|
bf59383783
|
refactor data preprocessing, fix mllm rlhf
Former-commit-id: 53ff2dd24f9121ea30c95063bb72e49a9b31e980
|
2024-05-24 04:08:25 +08:00 |
|
hiyouga
|
e6fc0ac8fe
|
fix paligemma sft
requires transformers>=4.41.1
Former-commit-id: 80b3030569cd606ac0de43e9a682478f5bd7b727
|
2024-05-24 00:23:40 +08:00 |
|
hiyouga
|
2b65f8bd5c
|
fix paligemma sft
Former-commit-id: 60682d04414be37e611d6470618a8d599703942b
|
2024-05-21 20:03:09 +08:00 |
|
hiyouga
|
f5962f8128
|
fix paligemma data preprocess
Former-commit-id: 71b85437301739d9d96d3881d4a34b37c0f69db8
|
2024-05-20 23:51:32 +08:00 |
|
hiyouga
|
b31d808655
|
fix paligemma inference
Former-commit-id: 46357b7a677e8ba2e0a7c9d4ec1974abd061569c
|
2024-05-20 23:36:43 +08:00 |
|
zhangzc
|
de9f1583c2
|
fix conflict
Former-commit-id: 6922b23a748c2459147bf44b96d86daa89f2c96c
|
2024-05-20 17:10:01 +08:00 |
|
hiyouga
|
ab48653e63
|
fix chat engines
do not use pop(key, default) since api assigns None to dict values
Former-commit-id: 3ebbd0b55ea07de2897c27ca54eeab5c3b319419
|
2024-05-20 00:36:43 +08:00 |
|
hiyouga
|
3453a8eebb
|
fix jinja template
Former-commit-id: 353561f0e3914de3f81499c4e4b831ae0a6383b6
|
2024-05-19 23:38:30 +08:00 |
|
hiyouga
|
0aa072a155
|
improve data process logger
Former-commit-id: 33d0b012b56dbafc9fff87b821c2d1bf1409dbb5
|
2024-05-18 22:02:42 +08:00 |
|
hiyouga
|
9c1c59e481
|
fix #3803
Former-commit-id: 1ef12c95059d14a1717c82ce04e529e7ad6435ed
|
2024-05-18 16:13:14 +08:00 |
|
hiyouga
|
2bff90719b
|
improve KTO impl., replace datasets
Former-commit-id: e56a57ddcf061de6e4acc8679f7dbf0b68364986
|
2024-05-18 03:44:56 +08:00 |
|
enji.zhou
|
66b5634ebf
|
add kto
Former-commit-id: ec51986cf70b0bdd79b8141e45916670fb97a08e
|
2024-05-17 13:09:17 +08:00 |
|
hiyouga
|
dfa686b617
|
rename package
Former-commit-id: a07ff0c083558cfe6f474d13027642d3052fee08
|
2024-05-16 18:39:08 +08:00 |
|