hiyouga
|
c0c6b8075a
|
tiny fix
Former-commit-id: 38b6b0f52edeb8ba45aa03b415b3c0c1b0e0c1e4
|
2024-06-16 01:06:41 +08:00 |
|
hiyouga
|
2946153cea
|
add license
Former-commit-id: d87108daa68bd40174b262be1ca65fe6e1b7ab56
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
8fccaf20c5
|
fix #4221
Former-commit-id: 6baafd4eb3147ad9f7d2952b8eb27c5486940f36
|
2024-06-13 02:48:21 +08:00 |
|
hoshi-hiyouga
|
bf3de9bfe8
|
Update pretrain.py
Former-commit-id: 0c292332374fb96c3fc753abde42d070a0c1dca8
|
2024-06-11 17:02:14 +08:00 |
|
d
|
da39715085
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: 6979f3f8480755604d8aea8164f6418126e094c5
|
2024-06-11 16:23:40 +08:00 |
|
hiyouga
|
c6f5f69644
|
update data processors
Former-commit-id: ccc8b64cc21417cfd88d9b5e9bd62e15c1e834a1
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
4953ded639
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 181dbb0d05655c6b46a3859cb47cfa3f86ef8abd
|
2024-06-07 03:48:46 +08:00 |
|
hoshi-hiyouga
|
e3ef239bc0
|
Update supervised.py
Former-commit-id: c09ad8bab38bc2f151da3a924eba225111af2481
|
2024-06-07 03:42:08 +08:00 |
|
hoshi-hiyouga
|
fd7bd911a6
|
Update supervised.py
Former-commit-id: 788e8232fc4ed58ab2439a9bc2e38f64e12c6eb3
|
2024-06-07 03:38:23 +08:00 |
|
hoshi-hiyouga
|
21df5f0bd0
|
Update supervised.py
Former-commit-id: 8cecade7082a52f413517ea20b1c5dd812db8e53
|
2024-06-07 03:38:04 +08:00 |
|
hiyouga
|
8da149ba40
|
rename files
Former-commit-id: 74f96efef9bcd63f65d0190c901ff9be54ccd350
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
e0aadd4b34
|
fix ppo dataset bug #4012
Former-commit-id: 149610c636bbb974e546d13fa302884ea65a6d38
|
2024-06-06 19:03:20 +08:00 |
|
ylfeng
|
62d55b71a3
|
remove empty line
Former-commit-id: b47e3174472f458a3a8b84a66b475da8fce6db79
|
2024-05-31 21:43:08 +08:00 |
|
ylfeng
|
0feb2ad35c
|
fix eos
Former-commit-id: 84aee579013f0c095a918a8c61611ccbb1d7fc84
|
2024-05-31 21:40:41 +08:00 |
|
ylfeng
|
8350e508d3
|
supervised packing with greedy knapsack algorithm
Former-commit-id: f9db439cb7511b12aa3524d5fdcc45864aebda91
|
2024-05-31 15:33:54 +08:00 |
|
hiyouga
|
df33548b39
|
update readme
Former-commit-id: 5581cb2e4e59f3f8109e2acd4611789f9e50bfca
|
2024-05-27 18:14:02 +08:00 |
|
hiyouga
|
3e729798df
|
refactor data preprocessing, fix mllm rlhf
Former-commit-id: 3a023bca2a502810a436cfba7708df164754ea62
|
2024-05-24 04:08:25 +08:00 |
|