hoshi-hiyouga
28a9176784
Merge pull request #1436 from lvzii/main
...
fix tokenizer config changed after pretrain
Former-commit-id: 7ca32d8e69d5a2790b8ac323f6e6d0aea42600e7
2023-11-09 14:30:50 +08:00
hiyouga
f9ebc718d0
support parquet format #1446
...
Former-commit-id: 3df90b988bd987daa4e3a991c32ae53446481dcd
2023-11-09 14:17:40 +08:00
hiyouga
b9f42172dd
fix #1438 #1439
...
Former-commit-id: 33422e1feff706bb7cfb235b9e40bd2e124222be
2023-11-09 13:45:10 +08:00
lvzi
13eb365eb7
fix tokenizer config changed after pretrain
...
Changing tokenizer's attribute at preprocessing stage will result in saving a wrong tokenizer.
for example, baichuan2
Former-commit-id: 043c316ac8913e10b2274867033f194ea92bfcd6
2023-11-08 15:50:46 +08:00
hiyouga
91f406cc99
fix ppo train and dpo eval
...
Former-commit-id: 01260d975477ebb8570933a1bd7f547b4dba607f
2023-11-07 22:48:51 +08:00
hiyouga
100dc4c458
fix #1422
...
Former-commit-id: 11c1e1e1570d3712109dd4dce831674a98841bd5
2023-11-07 19:42:01 +08:00
hiyouga
ed584b9f52
fix reward model loading
...
Former-commit-id: c52336d14435bc3bd98b6070cc1309b5e7d706c4
2023-11-07 17:20:51 +08:00
hiyouga
b446582bfd
fix args
...
Former-commit-id: d92f112951a8d8b28b180c3e2f504a094a9885dd
2023-11-07 16:36:06 +08:00
hiyouga
53fcc531b5
update info
...
Former-commit-id: 17c64a05796cac70fc76ed728705cd60efa41cae
2023-11-07 16:28:21 +08:00
hiyouga
1f2c56bff9
delete file
...
Former-commit-id: 479d0af2dc4ab8282b9d55aba1b03ab3a54f400b
2023-11-07 16:20:12 +08:00
hiyouga
d843efc413
fix #1418
...
Former-commit-id: 7ebd63a609d4bc4f8a645c5c60be77842ebac825
2023-11-07 16:17:22 +08:00
hiyouga
3d40bdb600
upgrade peft, fix #1088 #1411
...
Former-commit-id: b2a60905f384ada92618bf21301fe96dac1c10bf
2023-11-07 16:13:36 +08:00
hiyouga
936297aeac
update requirements
...
Former-commit-id: 66a91e1fe39483b83c7636c8199c8a87cf6a599e
2023-11-06 19:01:21 +08:00
hiyouga
5c19786f7c
use seed in evaluate.py
...
Former-commit-id: de95b6928293e79c7e204be307c1784ce146c1b1
2023-11-06 18:17:51 +08:00
hiyouga
2627f95ac3
update readme (list in alphabetical order)
...
Former-commit-id: e1e04cb1f1a0d37cac47c228bd3d5bbcda912aff
2023-11-06 17:18:12 +08:00
hiyouga
a919b6a478
update templates
...
Former-commit-id: a7eeb8e17c2f23f16732f5a5d767b39bcc1ac517
2023-11-06 12:25:47 +08:00
hiyouga
ecdea0036c
fix #1383
...
Former-commit-id: 2e77a5718a06f013685e673a19a201653c4cad03
2023-11-06 11:42:23 +08:00
hiyouga
034b658348
fix deepseek template
...
Former-commit-id: d08f5e8a147f1929567d42b6bed8bc998c2a866d
2023-11-05 13:08:46 +08:00
hiyouga
04107b7af6
support deepseek coder #1378
...
Former-commit-id: 2a8a25819524e84a5e6e907923c47693f8b7a48d
2023-11-05 12:51:03 +08:00
hiyouga
0d488085b8
fix #1365
...
Former-commit-id: 63ff909310eec1db64bd4bcc1ce013dbd96f0f99
2023-11-05 12:21:07 +08:00
hiyouga
c2302a3e70
Update wechat.jpg
...
Former-commit-id: 5227e18c4436762a2d50a606dba4c3aad186e469
2023-11-05 10:25:59 +08:00
hiyouga
983779c474
tiny fix
...
Former-commit-id: 05d9fc7eff8f4f056c48b66bf89f1834ade27968
2023-11-03 01:26:06 +08:00
hiyouga
574a7d175c
fix #1290
...
Former-commit-id: eb9d9e104ad5cdfc23f9cc48f20068c674762c47
2023-11-03 00:44:53 +08:00
hiyouga
5507014392
fix bug in data loader, support dpo eval
...
Former-commit-id: b355f6cac99592b66890ccc04e77a9993de0447d
2023-11-03 00:34:26 +08:00
hiyouga
b2bf10661b
update data readme
...
Former-commit-id: 2b5e33c338e6e8b10c4cbaa68ed26ef3b38ad5f9
2023-11-03 00:15:23 +08:00
hiyouga
a9db89a025
update data readme (zh)
...
Former-commit-id: cc8ffa10d877f5893f3940204e5bec6f3266559f
2023-11-02 23:42:49 +08:00
hiyouga
a1b0655457
support sharegpt format, add datasets
...
Former-commit-id: a8371724130db2fbd7273a480e2acb251e382aec
2023-11-02 23:10:04 +08:00
hiyouga
89c1a80920
support pagination in webui preview
...
Former-commit-id: c1edb0cf1b2a4d52506fc9e15353dfbe513e5d8f
2023-11-02 21:21:45 +08:00
hiyouga
c3fab5307b
fix webui
...
Former-commit-id: 34d8b2e56c36aea2d8d2ecf1f5e89f8cdfb776a3
2023-11-02 18:03:14 +08:00
hiyouga
11ff5d1e43
support warning in webui
...
Former-commit-id: 9cde5e8af6e1a00d9ad084a847c16d016e2ef5ef
2023-11-02 17:57:04 +08:00
hiyouga
a465785df0
fix #1349
...
Former-commit-id: f8703aac08b45d23b341209b47db74679fba2c65
2023-11-02 17:02:44 +08:00
hiyouga
15cef791ba
fix #1356
...
Former-commit-id: dff128c7e38dd079a5840ea4e73ee3e9bbd1c3c9
2023-11-02 16:51:52 +08:00
hiyouga
22b3c913e9
fix #1325
...
Former-commit-id: 083787dbfe41f58ff59cb16ddde02df98593aef5
2023-11-01 23:38:49 +08:00
hiyouga
d57f70bf8a
fix chat
...
Former-commit-id: 8b912690e34b040b98c4eac5eef5c56876345b1d
2023-11-01 23:07:58 +08:00
hiyouga
e3d64e119e
update gradio, support multiple resp in api
...
Former-commit-id: 84af10cec912309b82fc16e20879eec1c2e3729a
2023-11-01 23:02:16 +08:00
hiyouga
abd6ec174f
fix SFT trainer
...
Former-commit-id: d8cf8cfdeb861152771d22f873d8ed17b41538ce
2023-10-31 21:52:52 +08:00
hiyouga
6493f6d2e9
fix #1316
...
Former-commit-id: f4e4a04529a60b4c4ccc66cbf67f6e951fbc68d3
2023-10-31 11:32:08 +08:00
hiyouga
8fb1c94632
Update wechat.jpg
...
Former-commit-id: 9093cb1a2e16d1a7fde5abdd15c2527033e33143
2023-10-30 14:01:08 +08:00
hiyouga
b3931f73cc
update projects
...
Former-commit-id: 640a5201089f2ed5c6721ee29ee8b7a9be95847b
2023-10-29 22:53:47 +08:00
hiyouga
dc8739066b
add projects
...
Former-commit-id: 59f342e76f9d1ad04cfa7979b98d1eb9a3cebc99
2023-10-29 22:07:13 +08:00
hiyouga
d48478ef88
update constants
...
Former-commit-id: f28a034a9b74630a56314446bd0f103c086bda60
2023-10-29 13:30:20 +08:00
hiyouga
bf0faf129d
fix vicuna template
...
Former-commit-id: 52fc24d1664bc701f43e2bff8b3faded795b929c
2023-10-27 22:15:25 +08:00
hiyouga
5705c82cd8
fix chatglm3 template
...
Former-commit-id: 4117f388279ca43eb46def195c21e7051aefd0c7
2023-10-27 21:12:06 +08:00
hiyouga
8babfe3b4c
update readme
...
Former-commit-id: 4600c29e932b3c46561c2e8727e4cbdf270a72bb
2023-10-27 19:19:03 +08:00
hiyouga
8a76b1e499
support chatglm3
...
Former-commit-id: 1c0ab9a908dedf0ad69ad5741a23465da02006d9
2023-10-27 19:16:28 +08:00
hiyouga
fcfcac4858
support dataset cache
...
Former-commit-id: 3fe7df628db4093d7b3c121ececff60be0aa3a8a
2023-10-26 21:48:45 +08:00
hiyouga
4906092811
fix #1287
...
Former-commit-id: 838ed9aa8737e8f74ec7e8c660254cabbbbcf6ef
2023-10-26 17:49:41 +08:00
hiyouga
4d6606a72c
fix #1285
...
Former-commit-id: aff9363ce399fd9f1b29d5a088b49c865001f37a
2023-10-26 16:34:52 +08:00
hiyouga
596bda345c
Update wechat.jpg
...
Former-commit-id: d357e08b5819a6281adcf6410c57ff50e3a3e3ef
2023-10-24 16:02:12 +08:00
hiyouga
84e27a1c0b
remove filter in preprocess
...
Former-commit-id: 2caf91f824320b226daa4666eda2da7cb853db9c
2023-10-23 23:46:02 +08:00
