37 Commits

Author SHA1 Message Date
hiyouga
868444e124 allow non-packing pretraining
Former-commit-id: bdb496644ce2c18806fc4fdae1fedcb3e5b5f808
2024-03-09 22:21:46 +08:00
hiyouga
34533b2f35 support vllm
Former-commit-id: d07ad5cc1cdbc13879afd84f653afdfee03a6933
2024-03-07 20:26:31 +08:00
hiyouga
96265ec154 support llama pro #2338 , add rslora
Former-commit-id: 7924ffc55d98e33bfbfbca303e46c8f476435673
2024-02-15 02:27:36 +08:00
hiyouga
23dd337ac2 lint
Former-commit-id: 88a1bc97736bf06f292cd768fc8b61503aca1988
2024-02-07 01:10:04 +08:00
hiyouga
91b7fd1ec3 fix reserved label len
Former-commit-id: db0ab4d601b5e68660c00f07998ff26a8745ba90
2024-02-04 17:54:26 +08:00
hiyouga
b27e91222c format style
Former-commit-id: 638234ceee1b19716e45b6e5f4ea54d9122da4df
2024-01-20 20:15:56 +08:00
hiyouga
4e3bfb799d support function calling
Former-commit-id: d9f1cae35150cce594a7abd96dd2beb811fa33f2
2024-01-18 09:54:23 +08:00
hiyouga
11a6c8a9a0 fix #1909
Former-commit-id: 0bbf7118dfeb936baf03e74269211fa593003cfe
2023-12-23 14:42:20 +08:00
hiyouga
1a0bdd305c support system column #1765
Former-commit-id: 0a9c6e0146ebc71d5438c837463d6ab236e227c4
2023-12-12 19:45:59 +08:00
hiyouga
cefc0b2f03 fix modelscope data hub
Former-commit-id: d5b2c57a356539df9993e4774b856231eca8a6da
2023-12-12 18:33:06 +08:00
hoshi-hiyouga
b67085e13a Merge branch 'main' into feat/support_ms
Former-commit-id: 6382efec52f6be3daa5db0bd280a96162009fca1
2023-12-12 17:55:32 +08:00
hiyouga
b641e9e97e fix #1784
Former-commit-id: 28d5de7e785f31b223a4646c9c1c770f43e187ec
2023-12-09 20:53:18 +08:00
yuze.zyz
c523613f0a support ms dataset
Former-commit-id: 9c2247d700763f480d88a5dd46480cb32cfc174e
2023-12-08 18:00:57 +08:00
hiyouga
e964fa7df7 fix err hint
Former-commit-id: a5a248d569f8bf97cb9be71221783d97c666583c
2023-12-01 17:13:22 +08:00
hiyouga
8387f3011c fix #1494
Former-commit-id: d125ef55358837d4d76943739afeb6c70a901cd7
2023-11-14 18:07:20 +08:00
hiyouga
125587b187 refactor evaluation, upgrade trl to 074
Former-commit-id: 442aefb925c4ff02b98aa30c49c2e01d04f6496a
2023-11-13 22:20:35 +08:00
hiyouga
a9db89a025 update data readme (zh)
Former-commit-id: cc8ffa10d877f5893f3940204e5bec6f3266559f
2023-11-02 23:42:49 +08:00
hiyouga
a1b0655457 support sharegpt format, add datasets
Former-commit-id: a8371724130db2fbd7273a480e2acb251e382aec
2023-11-02 23:10:04 +08:00
hiyouga
15cef791ba fix #1356
Former-commit-id: dff128c7e38dd079a5840ea4e73ee3e9bbd1c3c9
2023-11-02 16:51:52 +08:00
hiyouga
22b3c913e9 fix #1325
Former-commit-id: 083787dbfe41f58ff59cb16ddde02df98593aef5
2023-11-01 23:38:49 +08:00
hiyouga
fcfcac4858 support dataset cache
Former-commit-id: 3fe7df628db4093d7b3c121ececff60be0aa3a8a
2023-10-26 21:48:45 +08:00
hiyouga
95697652f1 fix #1232
Former-commit-id: b665e9e133bf2f6f10346c374eb0de8a96dd5c7e
2023-10-20 23:28:52 +08:00
hiyouga
4930118761 fix #1218
Former-commit-id: 7a11a42dfd414d140cd83b7a74760715d2ae2078
2023-10-19 16:17:41 +08:00
hiyouga
e585c789ce fix #1184
Former-commit-id: af18b0dce7a4ef10b30da069d454010eddd269af
2023-10-14 19:20:11 +08:00
hiyouga
108c31e1fc support LongLoRA
Former-commit-id: 90375f600d5601866836123597fa3ef52008eeef
2023-09-27 21:55:50 +08:00
hiyouga
4581d09fa6 fix #944
Former-commit-id: 338b8664edea5ae65192ac657bb013581245ae15
2023-09-21 19:51:02 +08:00
hiyouga
9ed4bb63d4 change to right-padding, update reward score #803
Former-commit-id: 8ea32e4046d75ddfa9517669e9de9f48fea720c6
2023-09-08 20:04:31 +08:00
hiyouga
a4fd976048 refactor dataset_attr, add eos in pt, fix #757
Former-commit-id: a9d1fb72f791ae57a4d12f4e3a7e2abccf6a7077
2023-09-01 19:00:45 +08:00
codemayq
2b979d39f2 add stage in DatasetAttr
Former-commit-id: ba94c8729dd8c90bedf7079a6978e150fc92b737
2023-08-23 20:54:53 +08:00
hiyouga
edc15c62fa fix system prompt
Former-commit-id: 7407d9daa16bf6b3cd5002e16b2c53e402d2bc39
2023-08-16 01:35:52 +08:00
hiyouga
abdfa26d06 support DPO training (2305.18290)
Former-commit-id: 3ec4351cfdaf2aefcc7d13345e19d79874ed61d3
2023-08-11 03:02:53 +08:00
hiyouga
6404167ab7 support val set in streaming mode
Former-commit-id: d86ea314a197fd821770d895e988c48d46679047
2023-08-09 23:00:26 +08:00
hiyouga
b32ed1d7be support interleave probs
Former-commit-id: 69744c17e8180e0ad549b57d575454724b820d01
2023-08-04 21:27:35 +08:00
hiyouga
e80b75b560 support streaming data, fix #284 #274 #268
Former-commit-id: 0411a4b3e122e7907441bc7a64b004948741a620
2023-07-31 23:33:00 +08:00
hiyouga
2a664783c3 Update data_args.py
Former-commit-id: 513e1f1ec952dc5901d90dd34e32440230bca3f0
2023-07-28 17:42:41 +08:00
hiyouga
b8b38a9ade create chat model
Former-commit-id: 657cf0f55a7f0886bc837bdd44528971dc5e5caa
2023-07-15 19:26:20 +08:00
hiyouga
a696148d6b modity code structure
Former-commit-id: f75137661358f9070bc70c341dfa2cc5fd69cf94
2023-07-15 16:54:28 +08:00