Commit Graph

77 Commits

Author SHA1 Message Date
hiyouga
13d7b48efe improve KTO impl., replace datasets
Former-commit-id: c450ee87a3
2024-05-18 03:44:56 +08:00
enji.zhou
03956053b8 add kto
Former-commit-id: db1d5a4f51
2024-05-17 13:09:17 +08:00
hiyouga
51e0f095a9 remove checksum and fix ui args
Former-commit-id: 58c522cd5c
2024-05-12 01:10:30 +08:00
codingma
e017fb67d0 fix sha1 of glaive_toolcall dataset
Former-commit-id: d5520b6017
2024-05-09 16:33:45 +08:00
hiyouga
38c6ce9311 remove big file
Former-commit-id: 1ccbfe562d
2024-05-07 22:14:06 +08:00
hiyouga
175a7ea951 fix stop param
Former-commit-id: 09f3ef1de4
2024-05-07 00:41:04 +08:00
hoshi-hiyouga
14c3c8cc8f Merge pull request #3588 from ZeyuTeng96/patch-1
update hf_hub_url for nectar_rm in dataset_info

Former-commit-id: d6ca7853fa
2024-05-07 00:06:11 +08:00
hoshi-hiyouga
a13bdb9a2b Update dataset_info.json
Former-commit-id: c3910ab98a
2024-05-07 00:05:45 +08:00
hiyouga
92cafef325 update example docs
Former-commit-id: f02f87c6fb
2024-05-06 22:51:02 +08:00
ZeyuTeng96
96354ca55f update hf_hub_url for nectar_rm in dataset_info
Hi there,

I cannot find the "mlinmg/RLAIF-Nectar" on hf, seems like it changed as "AstraMindAI/RLAIF-Nectar". So, making a PR for updating.

See: https://huggingface.co/datasets/AstraMindAI/RLAIF-Nectar
Former-commit-id: 044af36442
2024-05-06 16:44:50 +08:00
hiyouga
c9fce361fb update readme
Former-commit-id: 5ee04d418c
2024-04-26 23:39:19 +08:00
hoshi-hiyouga
76f767d5b0 Merge pull request #3471 from BUAADreamer/main
add llava_150k en/zh mllm sft data

Former-commit-id: 8f91420223
2024-04-26 23:36:41 +08:00
hoshi-hiyouga
5ad1c3dd36 Update dataset_info.json
Former-commit-id: c29b257007
2024-04-26 23:34:34 +08:00
BUAADreamer
044668af10 add llava_150k en/zh mllm sft data
Former-commit-id: a177872010
2024-04-26 23:18:58 +08:00
hiyouga
eb14501a52 release v0.7.0
Former-commit-id: 168f56683a
2024-04-26 23:18:00 +08:00
hiyouga
d2df4c22ab support mllm hf inference
Former-commit-id: e057c8de48
2024-04-26 05:34:58 +08:00
hoshi-hiyouga
3e832e53be Update dataset_info.json
Former-commit-id: f8c26e6a34
2024-04-26 03:03:36 +08:00
BUAADreamer
56028422e8 merge data part to the text stream
Former-commit-id: 42c90c8183
2024-04-25 19:58:47 +08:00
BUAADreamer
b6d78b2a64 merge data part to the text stream
Former-commit-id: c6dd89918f
2024-04-25 19:19:59 +08:00
BUAADreamer
31bce63a10 add llava and instructblip
Former-commit-id: cfb485eddf
2024-04-25 00:22:43 +08:00
BUAADreamer
175b56bced add multimodal LLM BLIP-2 and InstructBLIP
Former-commit-id: 4dcb11eab7
2024-04-23 18:45:43 +08:00
hiyouga
12290955d8 add dpo mix dataset
Former-commit-id: 6339edefff
2024-04-20 01:31:38 +08:00
hiyouga
db42378f29 fix #3247
Former-commit-id: d1fb6c72b5
2024-04-12 17:41:33 +08:00
li.yunhao
0330ba2ae6 fix pile datset hf hub url
Former-commit-id: 9c2ef9cdf4
2024-03-30 16:06:10 +08:00
hiyouga
6646e18c02 add orca_dpo_pairs dataset
Former-commit-id: 3271af2afc
2024-03-20 20:09:06 +08:00
hiyouga
9ae1514a75 update readme, add starcoder2, cosmopedia
Former-commit-id: 894d183214
2024-03-03 01:01:46 +08:00
hiyouga
3b16912235 update data
Former-commit-id: 32884523c5
2024-03-02 19:37:18 +08:00
hiyouga
7e2d8b170a fix #2533
Former-commit-id: 1630a4cb8f
2024-02-21 22:47:48 +08:00
hiyouga
62b78001b7 fix #2481
Former-commit-id: 22acab8aff
2024-02-15 19:07:47 +08:00
hiyouga
db2051684b improve aligner
Former-commit-id: 7d2dc83c5e
2024-02-10 16:39:19 +08:00
Mark Mueller
4bd7b8375e Slim Orca data parsing
Former-commit-id: 1d3598afa1
2024-02-08 19:32:20 +01:00
Johann-Peter Hartmann
ace1770085 WS fix
Former-commit-id: 49c69ea4b9
2024-02-06 20:13:04 +01:00
Johann-Peter Hartmann
6ff4e9e62c add ranking to dpo dataset
Former-commit-id: 1126563505
2024-02-06 20:12:36 +01:00
Johann-Peter Hartmann
77746ad86c remove comma
Former-commit-id: 870182c3a9
2024-02-03 08:48:39 +01:00
Johann-Peter Hartmann
c264eb4793 Add support for german datasets
Former-commit-id: d9a8301ed4
2024-01-30 10:18:01 +01:00
hiyouga
cd4d38e0cc Update dataset_info.json
Former-commit-id: dbaaa4546e
2024-01-23 00:10:32 +08:00
hiyouga
7f12aedc08 enable cutoff len
Former-commit-id: f1067d2b58
2024-01-18 12:25:42 +08:00
hiyouga
4e3bfb799d support function calling
Former-commit-id: d9f1cae351
2024-01-18 09:54:23 +08:00
hiyouga
a52aafdbdc tiny update
Former-commit-id: 5b93d545e2
2023-12-25 18:29:34 +08:00
hiyouga
f0f9d253d8 support autogptq in llama board #246
Former-commit-id: 71389be37c
2023-12-16 16:31:30 +08:00
hiyouga
1a0bdd305c support system column #1765
Former-commit-id: 0a9c6e0146
2023-12-12 19:45:59 +08:00
hiyouga
cefc0b2f03 fix modelscope data hub
Former-commit-id: d5b2c57a35
2023-12-12 18:33:06 +08:00
hoshi-hiyouga
b67085e13a Merge branch 'main' into feat/support_ms
Former-commit-id: 6382efec52
2023-12-12 17:55:32 +08:00
xingjun.wang
e331e8c200 modify guanaco
Former-commit-id: e80a989d49
2023-12-12 15:00:37 +08:00
xingjun.wang
277790d868 update dataset info
Former-commit-id: 73b50a26b9
2023-12-12 14:53:59 +08:00
xingjun.wang
879209829e update args for MsDataset.load
Former-commit-id: 09533e95ed
2023-12-12 13:02:54 +08:00
xingjun.wang
9f17d36ccf add new datasets
Former-commit-id: fe4acc66b0
2023-12-12 12:44:15 +08:00
xingjun.wang
92fb73abd4 add open orca
Former-commit-id: 0ce18a3782
2023-12-12 12:34:04 +08:00
hiyouga
b641e9e97e fix #1784
Former-commit-id: 28d5de7e78
2023-12-09 20:53:18 +08:00
yuze.zyz
9c30cdb53d fix typo
Former-commit-id: e4cf2a75ca
2023-12-08 18:13:26 +08:00