Commit Graph

1808 Commits

Author SHA1 Message Date
hiyouga
cf08bcf3d9 truncate readme
Former-commit-id: baac22f4f4
2023-09-10 21:04:20 +08:00
hiyouga
17bc66fce0 update readme
Former-commit-id: 63611de7ae
2023-09-10 21:01:20 +08:00
hiyouga
7a715aac55 update readme
Former-commit-id: 34005252df
2023-09-10 20:52:21 +08:00
hiyouga
8ab5566dc0 support FlashAttention2
Former-commit-id: d8aa1404be
2023-09-10 20:43:56 +08:00
hiyouga
85aa16f6c6 fix #850
Former-commit-id: 815b92e698
2023-09-10 14:22:03 +08:00
hiyouga
f865d0bd51 fix lora target
Former-commit-id: a51b7c98ac
2023-09-09 17:04:45 +08:00
hiyouga
c818a7ff60 support lora target auto find
Former-commit-id: bca1a247bc
2023-09-09 15:38:37 +08:00
hiyouga
c6265e6969 fix chatglm2 tokenizer
Former-commit-id: d8d82ca281
2023-09-09 13:50:29 +08:00
hiyouga
f803c7c309 add baichuan2 convert script
Former-commit-id: d2015c8e12
2023-09-08 22:59:41 +08:00
hiyouga
43a20c67d4 fix bug in DPO data collator
Former-commit-id: 90bd085ae4
2023-09-08 20:45:07 +08:00
hiyouga
405df0f63d fix #761
Former-commit-id: b34797a845
2023-09-08 20:22:18 +08:00
hiyouga
9ed4bb63d4 change to right-padding, update reward score #803
Former-commit-id: 8ea32e4046
2023-09-08 20:04:31 +08:00
hiyouga
62941919e8 fix chatglm template
Former-commit-id: 8aaaa132d4
2023-09-08 14:45:58 +08:00
hiyouga
2f243d6e73 Update wechat.jpg
Former-commit-id: 8b66247d2f
2023-09-07 20:56:07 +08:00
hiyouga
f225a71445 update requirements
Former-commit-id: f5351c18e1
2023-09-07 19:26:25 +08:00
hiyouga
091326dc9f fix #818
Former-commit-id: 5a9970dbef
2023-09-07 19:19:53 +08:00
hiyouga
5030f05126 add deepspeed check in PPO training
Former-commit-id: ed1c2c5557
2023-09-07 19:12:40 +08:00
hiyouga
e6fa0229f4 fix #809
Former-commit-id: e2bf7c3bad
2023-09-07 19:04:32 +08:00
hiyouga
f74b980650 fix baichuan templates
Former-commit-id: 85b1f6632a
2023-09-07 18:54:14 +08:00
hiyouga
51f662860d update baichuan2 template
Former-commit-id: 0531886e1f
2023-09-06 21:43:06 +08:00
hiyouga
0ba72273d2 add Baichuan2 models
Former-commit-id: 60603a94c6
2023-09-06 18:40:11 +08:00
hiyouga
f9aee17f9d add Baichuan2 models
Former-commit-id: 62ce65c628
2023-09-06 18:36:04 +08:00
hoshi-hiyouga
5b4f59c3f9 Merge pull request #786 from kinghuin/patch-1
fix utils.py bug

Former-commit-id: 9224db90ea
2023-09-05 10:49:34 +08:00
Q
a46ffa68ff fix utils.py bug
Former-commit-id: a19fc2ebf7
2023-09-05 10:38:01 +08:00
hiyouga
3b5a9c60b6 fix #763
Former-commit-id: 370bdb6e43
2023-09-01 23:13:05 +08:00
hiyouga
a4fd976048 refactor dataset_attr, add eos in pt, fix #757
Former-commit-id: a9d1fb72f7
2023-09-01 19:00:45 +08:00
hiyouga
173022473d Update wechat.jpg
Former-commit-id: cf106e9d65
2023-09-01 15:02:23 +08:00
codingma
5e1a8b1c74 Merge pull request #741 from hiyouga/feature-addDatasetCheck
Feature add dataset check

Former-commit-id: 701a9d60cb
2023-08-31 20:57:36 +08:00
codemayq
ea74e5a81b update llama2 template
Former-commit-id: 0bcc489c42
2023-08-30 16:23:56 +08:00
codemayq
c955d9267c add dataset stage check
Former-commit-id: f7fdc088d4
2023-08-30 16:23:08 +08:00
codingma
00c3cd5454 Merge pull request #651 from hiyouga/feature-dataset_stage
add dataset stage

Former-commit-id: 9b4d16d040
2023-08-28 16:03:45 +08:00
codingma
b094b3edb8 Merge pull request #678 from hiyouga/feature-txt_preview
add text format dataset preview in webui

Former-commit-id: 01dfba85b4
2023-08-28 16:03:02 +08:00
codemayq
d9b9d9d1fe add ad gen dataset
Former-commit-id: 604f85487b
2023-08-27 20:35:32 +08:00
hiyouga
45405a4ec1 Update wechat.jpg
Former-commit-id: 15cec650c7
2023-08-27 01:21:31 +08:00
codemayq
292abfd438 add text format dataset preview in webui
Former-commit-id: 24e68d29f2
2023-08-24 19:45:36 +08:00
codemayq
2b979d39f2 add stage in DatasetAttr
Former-commit-id: ba94c8729d
2023-08-23 20:54:53 +08:00
hiyouga
0d0232479f fix import error
Former-commit-id: 2de1a7610a
2023-08-23 20:45:03 +08:00
hiyouga
38080233a5 fix #649
Former-commit-id: 57146c101f
2023-08-23 20:21:15 +08:00
codemayq
b032dc4c4e add readme for dataset
Former-commit-id: cece66d48a
2023-08-23 19:55:45 +08:00
codemayq
4b29d9d2b0 add dataset stage and filter dataset when stage chosen in webui
Former-commit-id: c0e4d1e81b
2023-08-23 18:54:23 +08:00
hiyouga
2accde2cb7 fix webui
Former-commit-id: 1c702ad538
2023-08-23 11:03:35 +08:00
hoshi-hiyouga
1512891efc Merge pull request #644 from hiyouga/fix-quantization_bit
fix quantization bit is ""

Former-commit-id: c562307476
2023-08-23 10:45:45 +08:00
codemayq
433a96b8c2 fix quantization bit is ""
Former-commit-id: a7cc6c4140
2023-08-23 10:08:17 +08:00
codemayq
7b82ea708d fix quantization is ""
Former-commit-id: ec2047b064
2023-08-23 10:04:03 +08:00
hiyouga
802494e20a update template
Former-commit-id: 4318347d3f
2023-08-22 19:46:09 +08:00
hoshi-hiyouga
1bbdd60843 Merge pull request #629 from panpan0000/main
add rm dataset explanation

Former-commit-id: 4da719c830
2023-08-22 13:41:44 +08:00
Peter Pan
23443e9696 add rm dataset explanation
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>

Former-commit-id: b0ca8fe634
2023-08-22 01:33:59 -04:00
hoshi-hiyouga
a80cd5d86b Merge pull request #619 from hiyouga/feature-templateTest
add template encode test

Former-commit-id: bc7795655f
2023-08-21 20:56:34 +08:00
codemayq
4b83d692b1 add template encode test
Former-commit-id: cbbee7933e
2023-08-21 20:51:24 +08:00
hiyouga
d05d535a58 fix #617
Former-commit-id: 5235b15c91
2023-08-21 18:16:11 +08:00