109 Commits

Author SHA1 Message Date
hiyouga
2b0fced03b fix Baichuan-13B
Former-commit-id: 08439d29b2031ffbe77fe581c148e6d94e68bfc4
2023-07-13 23:08:45 +08:00
zxbsmk
3b15aacf02 Support for WebNovel dataset
Former-commit-id: 4955dc9eed33a904e4e2b9d5985b3fda87c3674a
2023-07-12 17:29:47 +08:00
hiyouga
92070c3d7a add open assistant dataset
Former-commit-id: 3154fec979aba48f54b7afde3740c4990d445a41
2023-06-28 23:09:33 +08:00
hiyouga
9155401bf9 add belle multiturn dataset
Former-commit-id: 334d1a6d26a0c814b86bdfe68fe291c0513123fd
2023-06-16 20:01:16 +08:00
hiyouga
1fbda5d139 support RM metrics, add generating Args
Former-commit-id: cec6524d6b1be65c5d171a5b3dcaae7818132bc5
2023-06-12 15:48:48 +08:00
BUAADreamer
53727aee3e add code for reading from multi files in one directory
Former-commit-id: 3dd5f9a874d66353bb4379bbe39a89cd425dac3d
2023-06-10 15:53:47 +08:00
hiyouga
ddb456bbcb remove dummy code
Former-commit-id: a72492e6490c44a7edccd572da73c47d6f278cc7
2023-05-30 16:28:00 +08:00
hiyouga
4c7c96e656 add pre-training script
Former-commit-id: 8ff96509fa621054368919988a15b50da1891852
2023-05-29 21:37:22 +08:00
hiyouga
54b8ce7b63 Initial commit
Former-commit-id: 769c6ab56be0c9d26e9289f61ac54a4068d935c1
2023-05-28 18:09:04 +08:00