73 Commits

Author SHA1 Message Date
hiyouga
45af1a951f fix template for tiktoken
Former-commit-id: 1afa51c2fa9839056644803eedef4e9d1af0d51e
2023-08-05 13:42:42 +08:00
hiyouga
1a1caf2116 remove redundant code
Former-commit-id: 53d95725c588a9858e699e3e591cb0d3c1441208
2023-08-05 00:27:27 +08:00
hiyouga
ab95f569a4 fix template
Former-commit-id: c183b3551d8d190965d2403e27fa2cedf8ac7bff
2023-08-05 00:25:00 +08:00
hiyouga
7a89fce4c7 fix llama2 template
Former-commit-id: e4a15f863c879f28a716a90f7c928ac02f059b6e
2023-08-05 00:07:54 +08:00
hiyouga
65369ecf48 fix bos and eos token
Former-commit-id: d87c8fd8ab84c9f58c0b1f3fb4ad0adf98b25715
2023-08-04 23:55:57 +08:00
hiyouga
dbb284b5a2 fix encode
Former-commit-id: 8172ad1b5e3fa0b224d761ce6069d0db4397da2d
2023-08-04 23:27:55 +08:00
hiyouga
ea045b0e5b support chatml safe encoding
Former-commit-id: b4852f94065a11c8cd00ffa7e71ac0e0b2bf477a
2023-08-04 23:14:28 +08:00
hiyouga
9c84c4ed5d support Qwen-7B, fix InternLM-7B inference
Former-commit-id: 87f8f830e20aa839e089559c1d038954742000ef
2023-08-03 15:53:32 +08:00
hiyouga
534e3320b5 release v0.1.5
Former-commit-id: c689857bbb82ecaa317bfc22d831c7025fe39cc7
2023-08-02 16:10:31 +08:00
YC Chen
bb2b38a31f [fix] Remove useless code
Former-commit-id: ca125da0eb81d93622b9cb9ac9183956e3e51582
2023-08-02 14:35:35 +08:00
YC Chen
bf844e8a99 [feature] Fix template of Llama2 to match the offical template
Former-commit-id: 432377308968202758904955e64199c1fa8d4fdd
2023-08-02 14:10:15 +08:00
hiyouga
e80b75b560 support streaming data, fix #284 #274 #268
Former-commit-id: 0411a4b3e122e7907441bc7a64b004948741a620
2023-07-31 23:33:00 +08:00
hiyouga
94f2fd634f update UI, fix #212
Former-commit-id: 4d1641c1bff1b2b53c0e9b80b3e3ac7979223ccd
2023-07-20 22:09:06 +08:00
hiyouga
64b4f71673 simplify code
Former-commit-id: 67a27730744b71795b10260d050501bfe2329c26
2023-07-20 15:08:57 +08:00
hiyouga
fa47c99fa9 add datasets
Former-commit-id: 7159bc54ed0f1bba974662a87ba5039d9aacadee
2023-07-19 20:59:15 +08:00
hiyouga
f7f2accf05 support LLaMA-2
Former-commit-id: 7a3ade8c699ff1cd2d17590e2f8df79e1738cee2
2023-07-19 16:42:14 +08:00
hiyouga
18c5eb4a44 add LLaMA2 template
Former-commit-id: 38eb1aaf550b6b5952d757c37eb274587c59dece
2023-07-19 00:44:49 +08:00
hiyouga
35d52c4100 update baichuan template
Former-commit-id: bdf91846da12c9d0aee26bf17f5431e4c0eb4bc8
2023-07-18 16:43:51 +08:00
hiyouga
4a18759864 fix template
Former-commit-id: d1ae428c6ec2a50e0cfc70657111fc5b67eff51e
2023-07-18 16:37:23 +08:00
hiyouga
92c393dc2a fix webUI, fix #171 #177
Former-commit-id: 6f9360c0bdc36ed2c6d0881136b191c96535ffc0
2023-07-18 15:51:48 +08:00
hiyouga
091805d38e release v0.1.0
Former-commit-id: f8193e8009451cf569a28a10eb4bd88831844441
2023-07-18 00:18:25 +08:00
hiyouga
22e979c482 update stream_chat
Former-commit-id: 8528a84e7486351af6b53db91649fa5772a99d13
2023-07-15 19:51:02 +08:00
hiyouga
a696148d6b modity code structure
Former-commit-id: f75137661358f9070bc70c341dfa2cc5fd69cf94
2023-07-15 16:54:28 +08:00