7 Commits

Author	SHA1	Message	Date
hiyouga	05f3a3c944	tiny fix ... Former-commit-id: f7f440986b0ae3b38ea9f2da80789629d4f79ea1	2024-06-16 01:06:41 +08:00
hiyouga	bb88536166	add license ... Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0	2024-06-15 17:54:33 +08:00
hiyouga	49b58fd6af	fix #4221 ... Former-commit-id: 05a3be4853b941909e7d193c31e8d62c8c5f879b	2024-06-13 02:48:21 +08:00
hoshi-hiyouga	cc9717e2f2	Update pretrain.py ... Former-commit-id: e2317b2a84149e39fddfd6366be3de23dfb71f82	2024-06-11 17:02:14 +08:00
d	77bf3d66c7	经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降 ... Former-commit-id: ef470561f742b16eaa0f99c4cadecd7c84ce6bd2	2024-06-11 16:23:40 +08:00
hiyouga	ca95e98ca0	fix ppo dataset bug #4012 ... Former-commit-id: 7fc51b2e93698ae5e012566af8481f4d861c873d	2024-06-06 19:03:20 +08:00
hiyouga	bf59383783	refactor data preprocessing, fix mllm rlhf ... Former-commit-id: 53ff2dd24f9121ea30c95063bb72e49a9b31e980	2024-05-24 04:08:25 +08:00