17 Commits

Author	SHA1	Message	Date
hiyouga	c0c6b8075a	tiny fix ... Former-commit-id: 38b6b0f52e	2024-06-16 01:06:41 +08:00
hiyouga	2946153cea	add license ... Former-commit-id: d87108daa6	2024-06-15 17:54:33 +08:00
hiyouga	8fccaf20c5	fix #4221 ... Former-commit-id: 6baafd4eb3	2024-06-13 02:48:21 +08:00
hoshi-hiyouga	bf3de9bfe8	Update pretrain.py ... Former-commit-id: 0c29233237	2024-06-11 17:02:14 +08:00
d	da39715085	经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降 ... Former-commit-id: 6979f3f848	2024-06-11 16:23:40 +08:00
hiyouga	c6f5f69644	update data processors ... Former-commit-id: ccc8b64cc2	2024-06-07 04:15:40 +08:00
hoshi-hiyouga	4953ded639	Merge pull request #4009 from AlongWY/main ... supervised packing with greedy knapsack algorithm Former-commit-id: 181dbb0d05	2024-06-07 03:48:46 +08:00
hoshi-hiyouga	e3ef239bc0	Update supervised.py ... Former-commit-id: c09ad8bab3	2024-06-07 03:42:08 +08:00
hoshi-hiyouga	fd7bd911a6	Update supervised.py ... Former-commit-id: 788e8232fc	2024-06-07 03:38:23 +08:00
hoshi-hiyouga	21df5f0bd0	Update supervised.py ... Former-commit-id: 8cecade708	2024-06-07 03:38:04 +08:00
hiyouga	8da149ba40	rename files ... Former-commit-id: 74f96efef9	2024-06-07 00:09:06 +08:00
hiyouga	e0aadd4b34	fix ppo dataset bug #4012 ... Former-commit-id: 149610c636	2024-06-06 19:03:20 +08:00
ylfeng	62d55b71a3	remove empty line ... Former-commit-id: b47e317447	2024-05-31 21:43:08 +08:00
ylfeng	0feb2ad35c	fix eos ... Former-commit-id: 84aee57901	2024-05-31 21:40:41 +08:00
ylfeng	8350e508d3	supervised packing with greedy knapsack algorithm ... Former-commit-id: f9db439cb7	2024-05-31 15:33:54 +08:00
hiyouga	df33548b39	update readme ... Former-commit-id: 5581cb2e4e	2024-05-27 18:14:02 +08:00
hiyouga	3e729798df	refactor data preprocessing, fix mllm rlhf ... Former-commit-id: 3a023bca2a	2024-05-24 04:08:25 +08:00