51 Commits

Author SHA1 Message Date
hiyouga
baaa0acdf1 fix #2260
Former-commit-id: cf818a25982c8a2808cf21f22e1e5eb4dd7e0713
2024-01-20 23:22:09 +08:00
hiyouga
4ac36c14ae release v0.5.0
Former-commit-id: 71cfdc2658e346e3fd05b3a8a678c811b3ba1bd5
2024-01-20 20:21:39 +08:00
hiyouga
b27e91222c format style
Former-commit-id: 638234ceee1b19716e45b6e5f4ea54d9122da4df
2024-01-20 20:15:56 +08:00
hiyouga
2f7684a8ee fix tests
Former-commit-id: f6d6e00337ebef8740d180836dcb18d0e6a3c59a
2024-01-20 19:58:04 +08:00
hiyouga
397eb8fbb9 fix #2249
Former-commit-id: 12043aab9c9ea2acaa3ff80232ce0451ccaa557d
2024-01-19 21:44:32 +08:00
hiyouga
a9fe47a848 fix function formatter
Former-commit-id: 35aef8b28761d787d42e548567fc6689a1502ab5
2024-01-18 16:01:07 +08:00
hiyouga
f7fcfeaa47 fix templates
Former-commit-id: a73a979afd660afd3f2091c7b813a82b3a28645d
2024-01-18 14:49:52 +08:00
hiyouga
f266d473b8 fix pretrain data loader
Former-commit-id: 5ff10fac4fd7fa0545ec7bcaa6268764f77342cf
2024-01-18 14:42:52 +08:00
hiyouga
e95c0242a8 fix dataset
Former-commit-id: 487dee066f51caea86dc65847bd326e13445de09
2024-01-18 12:59:30 +08:00
hiyouga
7f12aedc08 enable cutoff len
Former-commit-id: f1067d2b585cf24b3e48463692ac99a8222161c9
2024-01-18 12:25:42 +08:00
hiyouga
1323b8f102 add tool test
Former-commit-id: 83dbfce8c30bc1183a5c86fb77a5676bf0b753af
2024-01-18 10:26:26 +08:00
hiyouga
4e3bfb799d support function calling
Former-commit-id: d9f1cae35150cce594a7abd96dd2beb811fa33f2
2024-01-18 09:54:23 +08:00
hiyouga
21020e51ca support solar 10.7B #1907
Former-commit-id: bf73224f33c93647a6101d36bde5bf8ddfc91438
2024-01-14 00:30:30 +08:00
hiyouga
4dc16df570 fix #2147
Former-commit-id: 9aa1a2fc172730eabc4f14bcb08c0d66fcb2d844
2024-01-12 03:30:56 +08:00
hiyouga
a78cd5c5ad fix qwen template
Former-commit-id: ed216bbc4693860eaf27c2a95108f705d530cbdb
2024-01-05 16:14:56 +08:00
hiyouga
c14bf033a3 fix #2014
Former-commit-id: ce2156eaa83e2b572f4198a1eb77a835fb502319
2023-12-29 15:17:22 +08:00
hiyouga
16688b773a fix export format
Former-commit-id: e165354facf7e69f535f9b7d99438f03dbf0293d
2023-12-28 18:40:46 +08:00
hiyouga
11a6c8a9a0 fix #1909
Former-commit-id: 0bbf7118dfeb936baf03e74269211fa593003cfe
2023-12-23 14:42:20 +08:00
hiyouga
9337568e23 fix stop words
Former-commit-id: dec360d5aee58c11804b8e45dd8d4c375086887f
2023-12-20 19:06:43 +08:00
hiyouga
622d31e398 fix yi template #1895
Former-commit-id: 5af8841c4f6c97df522d2cf4e283d5ef0af21a18
2023-12-20 18:58:16 +08:00
hiyouga
c1233ab65f add max_memory for gptq #1923
Former-commit-id: c4a3977ad7278b59a9b3dadcb446bb4c99da5c9d
2023-12-20 18:15:17 +08:00
hiyouga
f64be8ee84 optimize data loading logic
Former-commit-id: ec1fe1daa98c61f62c753b22847de028b5c5cded
2023-12-20 16:15:41 +08:00
hiyouga
633624dc3c fix #1909
Former-commit-id: c6abbbfe90dcb0e832f73f0c611fc32eaa7ea78d
2023-12-20 16:11:07 +08:00
hiyouga
3981927d6b add codegeex template
Former-commit-id: a67a440644687dc2262134c0f2895f3ae42cae19
2023-12-18 19:52:35 +08:00
hiyouga
7dbc670902 support quantization in export model
Former-commit-id: 3524aa1e58da94ab00e9a2024952ea1b4119b2af
2023-12-15 23:44:50 +08:00
hiyouga
bd03307bbd refactor adapter hparam
Former-commit-id: 0716f5e470afffd2df5a815712b552a4b4797153
2023-12-15 20:53:11 +08:00
hiyouga
c9a2a7a3b3 fix sharegpt loading
Former-commit-id: 2c8e88f9c14e030f79778893eaeeb7abda97dd0b
2023-12-13 00:56:16 +08:00
hiyouga
1a0bdd305c support system column #1765
Former-commit-id: 0a9c6e0146ebc71d5438c837463d6ab236e227c4
2023-12-12 19:45:59 +08:00
hiyouga
cefc0b2f03 fix modelscope data hub
Former-commit-id: d5b2c57a356539df9993e4774b856231eca8a6da
2023-12-12 18:33:06 +08:00
hoshi-hiyouga
b67085e13a Merge branch 'main' into feat/support_ms
Former-commit-id: 6382efec52f6be3daa5db0bd280a96162009fca1
2023-12-12 17:55:32 +08:00
xingjun.wang
6cb2c99e7d add use_streaming
Former-commit-id: adc98c86dad64f1a793017fa628b5cf19abbdd01
2023-12-12 14:23:05 +08:00
xingjun.wang
1bd75afae8 fix cache dir
Former-commit-id: 1909f0d11732bd99fadc6c1191e026137c6a7dff
2023-12-12 14:21:33 +08:00
xingjun.wang
c1974c91e5 add print info for test
Former-commit-id: 168321a4da7612620b9528860306f03bf65d019a
2023-12-12 14:14:40 +08:00
xingjun.wang
e17f2a3f7f update cache dir
Former-commit-id: edc82b923a3fb03c5af100b5357e10f0c18b4523
2023-12-12 13:08:18 +08:00
xingjun.wang
879209829e update args for MsDataset.load
Former-commit-id: 09533e95edc5fa65a38b2f04c6d88506196021b3
2023-12-12 13:02:54 +08:00
xingjun.wang
6520aecef1 update
Former-commit-id: cfba1009d0fc31b5933b558b249d89248f723d6b
2023-12-12 12:03:23 +08:00
xingjun.wang
1d65d24071 for test
Former-commit-id: 5b979147f093e86f44c4228ab34d04bdae94f89f
2023-12-12 11:52:59 +08:00
xingjun.wang
2918743520 for test
Former-commit-id: 8a908a8c644f4a961001cdd8388a3a7fea992c55
2023-12-12 11:47:59 +08:00
hiyouga
b7d99ad5f4 support mixtral
Former-commit-id: 96380f5e1887bb166be339e58ab8f65e464d4010
2023-12-12 11:39:04 +08:00
hiyouga
b641e9e97e fix #1784
Former-commit-id: 28d5de7e785f31b223a4646c9c1c770f43e187ec
2023-12-09 20:53:18 +08:00
yuze.zyz
c523613f0a support ms dataset
Former-commit-id: 9c2247d700763f480d88a5dd46480cb32cfc174e
2023-12-08 18:00:57 +08:00
hiyouga
9b84a706af add models
Former-commit-id: e25f7bae16b7ea41a4a1fd1e8db1b961e55d0c5b
2023-12-06 13:33:18 +08:00
hiyouga
f8376b228a add xuanyuan models
Former-commit-id: 6e7af11b989e4cf97ffacbab4736e3434ff6c925
2023-12-02 00:35:29 +08:00
hiyouga
1c43fb6a41 add models
Former-commit-id: 509abe8864ada29ac7fa0f636b662531c8dd3a33
2023-11-30 19:16:13 +08:00
hiyouga
5f2943dc84 support Yi-34B-Chat models
Former-commit-id: ff1c289229ee382d3e76578bbb6a5e299b969ded
2023-11-23 19:31:49 +08:00
hiyouga
4966bd7911 support GPTQ tuning #729 #1481 #1545 , fix chatglm template #1453 #1480 #1569
Former-commit-id: 9ea93801459b0d271d21a2d730c44abae9106c51
2023-11-20 22:52:11 +08:00
hiyouga
32545bd6d9 better data streaming
Former-commit-id: 00baaa990e099d6b75436eaa7a922a07646afa26
2023-11-19 23:32:47 +08:00
hiyouga
8d82d7e994 fix Mistral template
https://github.com/lm-sys/FastChat/pull/2547

Former-commit-id: bfb94331657f385f4653ddcb8f7b57d1c052804d
2023-11-19 16:29:30 +08:00
hiyouga
5de45bf989 fix chatglm template
Former-commit-id: ed9f7705efbed0accf4dc5c9dfa9e3e7e15e1174
2023-11-16 22:54:15 +08:00
hiyouga
3e0b76650a update readme and constants
Former-commit-id: 1e19cf242a1f843b590feefbe24b2cc0a17712b5
2023-11-15 18:04:37 +08:00