31 Commits

Author SHA1 Message Date
hiyouga
f776e738f8 tiny fix
Former-commit-id: 352693e2dcc8fc039b5d574e1a5709563929b0ce
2024-03-11 00:17:18 +08:00
hiyouga
4a4e4b4354 support layerwise galore
Former-commit-id: 8664262cde3919e10eaecbd66e8c5d356856362e
2024-03-10 00:24:11 +08:00
hiyouga
34533b2f35 support vllm
Former-commit-id: d07ad5cc1cdbc13879afd84f653afdfee03a6933
2024-03-07 20:26:31 +08:00
hiyouga
eeb34e5696 auto set chat template
Former-commit-id: 9e56eaf2d3e42d3985b0e68929a3a32e1db157ef
2024-03-05 02:41:20 +08:00
hiyouga
db2051684b improve aligner
Former-commit-id: 7d2dc83c5e2085da6273241269c9e9d7509ae51b
2024-02-10 16:39:19 +08:00
hiyouga
36f092b53f improve fix tokenizer
Former-commit-id: 54ea9684ed05ac7fa5bbaa04d39636eda37cd34e
2024-02-09 14:53:14 +08:00
hiyouga
961368a8c4 fix #2397
Former-commit-id: 51df8657348ac3153e1a44143dc61008f10ab186
2024-02-03 23:45:31 +08:00
hiyouga
b27e91222c format style
Former-commit-id: 638234ceee1b19716e45b6e5f4ea54d9122da4df
2024-01-20 20:15:56 +08:00
hiyouga
2f7684a8ee fix tests
Former-commit-id: f6d6e00337ebef8740d180836dcb18d0e6a3c59a
2024-01-20 19:58:04 +08:00
hiyouga
397eb8fbb9 fix #2249
Former-commit-id: 12043aab9c9ea2acaa3ff80232ce0451ccaa557d
2024-01-19 21:44:32 +08:00
hiyouga
4e3bfb799d support function calling
Former-commit-id: d9f1cae35150cce594a7abd96dd2beb811fa33f2
2024-01-18 09:54:23 +08:00
hiyouga
11a6c8a9a0 fix #1909
Former-commit-id: 0bbf7118dfeb936baf03e74269211fa593003cfe
2023-12-23 14:42:20 +08:00
hiyouga
c1233ab65f add max_memory for gptq #1923
Former-commit-id: c4a3977ad7278b59a9b3dadcb446bb4c99da5c9d
2023-12-20 18:15:17 +08:00
hiyouga
f64be8ee84 optimize data loading logic
Former-commit-id: ec1fe1daa98c61f62c753b22847de028b5c5cded
2023-12-20 16:15:41 +08:00
hiyouga
7dbc670902 support quantization in export model
Former-commit-id: 3524aa1e58da94ab00e9a2024952ea1b4119b2af
2023-12-15 23:44:50 +08:00
hiyouga
c9a2a7a3b3 fix sharegpt loading
Former-commit-id: 2c8e88f9c14e030f79778893eaeeb7abda97dd0b
2023-12-13 00:56:16 +08:00
hiyouga
1a0bdd305c support system column #1765
Former-commit-id: 0a9c6e0146ebc71d5438c837463d6ab236e227c4
2023-12-12 19:45:59 +08:00
hiyouga
cefc0b2f03 fix modelscope data hub
Former-commit-id: d5b2c57a356539df9993e4774b856231eca8a6da
2023-12-12 18:33:06 +08:00
hoshi-hiyouga
b67085e13a Merge branch 'main' into feat/support_ms
Former-commit-id: 6382efec52f6be3daa5db0bd280a96162009fca1
2023-12-12 17:55:32 +08:00
xingjun.wang
6cb2c99e7d add use_streaming
Former-commit-id: adc98c86dad64f1a793017fa628b5cf19abbdd01
2023-12-12 14:23:05 +08:00
xingjun.wang
1bd75afae8 fix cache dir
Former-commit-id: 1909f0d11732bd99fadc6c1191e026137c6a7dff
2023-12-12 14:21:33 +08:00
xingjun.wang
c1974c91e5 add print info for test
Former-commit-id: 168321a4da7612620b9528860306f03bf65d019a
2023-12-12 14:14:40 +08:00
xingjun.wang
e17f2a3f7f update cache dir
Former-commit-id: edc82b923a3fb03c5af100b5357e10f0c18b4523
2023-12-12 13:08:18 +08:00
xingjun.wang
879209829e update args for MsDataset.load
Former-commit-id: 09533e95edc5fa65a38b2f04c6d88506196021b3
2023-12-12 13:02:54 +08:00
xingjun.wang
6520aecef1 update
Former-commit-id: cfba1009d0fc31b5933b558b249d89248f723d6b
2023-12-12 12:03:23 +08:00
xingjun.wang
1d65d24071 for test
Former-commit-id: 5b979147f093e86f44c4228ab34d04bdae94f89f
2023-12-12 11:52:59 +08:00
xingjun.wang
2918743520 for test
Former-commit-id: 8a908a8c644f4a961001cdd8388a3a7fea992c55
2023-12-12 11:47:59 +08:00
hiyouga
b641e9e97e fix #1784
Former-commit-id: 28d5de7e785f31b223a4646c9c1c770f43e187ec
2023-12-09 20:53:18 +08:00
yuze.zyz
c523613f0a support ms dataset
Former-commit-id: 9c2247d700763f480d88a5dd46480cb32cfc174e
2023-12-08 18:00:57 +08:00
hiyouga
32545bd6d9 better data streaming
Former-commit-id: 00baaa990e099d6b75436eaa7a922a07646afa26
2023-11-19 23:32:47 +08:00
hiyouga
06a4820836 disentangle model from tuner and rename modules
Former-commit-id: 4736344eb1595ee023a50d49e8118f4eee46305f
2023-11-15 16:29:09 +08:00