Yaowei Zheng
7242caf0ff
[assets] update readme ( #8461 )
2025-06-25 22:15:03 +08:00
hoshi-hiyouga
763fbc294b
[misc] update data readme ( #8128 )
2025-05-21 22:41:18 +08:00
hoshi-hiyouga
b0c8ba73e0
[deps] update to transformers 4.52 ( #8125 )
2025-05-21 05:16:18 +08:00
hoshi-hiyouga
b83a38eb98
[data] qwen3 fixes ( #8109 )
2025-05-20 02:00:30 +08:00
hoshi-hiyouga
8d472c20cb
[model] add seed coder and qwen3 quant models ( #8039 )
2025-05-13 15:59:55 +08:00
Eric Tang
6c53471de2
[data] support for specifying a dataset in cloud storage ( #7567 )
...
* add support for loading datasets from s3/gcs
* add comments to readme
* run linter and address comments
* add option to pass in kwargs to ray init (i.e. runtime env)
* address comment
* revert mixed up changes
2025-04-10 11:31:35 +08:00
hoshi-hiyouga
34fdabe005
[data] add coig-p dataset ( #7657 )
2025-04-09 21:18:25 +08:00
Zhangchi Feng
01915eaf40
[model] support audio ( #6701 )
...
* support qwen2_audio
* improve code
* lint
* fix
* fix
* fix
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 24c78429489809873a1269a735ea5421340b32a2
2025-02-05 04:59:09 +08:00
hiyouga
dec6ff046b
update data readme
...
Former-commit-id: 70e36ff2f4b500d987160f3a57d5fb3d4d2007d5
2024-09-05 04:44:49 +08:00
hiyouga
c4d7d76358
update data readme
...
Former-commit-id: 6055fe02deb3585b4330a7902bf8821dd41ea5cb
2024-09-05 04:25:27 +08:00
hiyouga
51a0016873
optimize predict vram
...
Former-commit-id: a244f143f48a01910ce1cd56c0855ef11d62a72a
2024-08-30 23:08:45 +08:00
hoshi-hiyouga
ddbd848e49
Update README.md
...
Former-commit-id: 9d64507bd5d47f096e81c90bfb347690afaaec2b
2024-07-14 21:27:04 +08:00
codingma
74f0d02eb8
1. add custom eval dataset support
...
2. merge load dataset and split dataset function
Former-commit-id: 76f3bbcfc0e11aa41f8f5cbebc60b77b987f7901
2024-07-05 15:52:10 +08:00
hoshi-hiyouga
eb7ee82f16
Update README.md
...
Former-commit-id: 3761d7d5dd97ce2fe0098284e6d4821fc0d63d30
2024-05-30 00:04:26 +08:00
zhangzc
4b90f04c1f
fix conflict
...
Former-commit-id: d956041640d9abc5e59919a227d27270fb513a7e
2024-05-20 17:10:01 +08:00
hiyouga
c53e626c9a
update data readme
...
Former-commit-id: ca48f90f1eb9828300635bdaee6c10d6cc632d3d
2024-05-18 21:37:38 +08:00
hiyouga
68c07d3e1e
update data readme
...
Former-commit-id: 18cbf8561d6c3fdceac47991ed16d35471823187
2024-05-18 21:15:20 +08:00
hiyouga
13d7b48efe
improve KTO impl., replace datasets
...
Former-commit-id: c450ee87a35ff9235f9b695b0de2e042b2971178
2024-05-18 03:44:56 +08:00
hoshi-hiyouga
2186deceac
Update README.md
...
Former-commit-id: b072ec9d1b18f7e9d5d2c9529eac55d29ca832c8
2024-05-02 02:13:46 +08:00
khazic
6f0b412265
added the second sharegpt format
...
Former-commit-id: d1ba32e4bb70489a9e6f5d3657988c9b7553a157
2024-04-28 14:27:45 +08:00
hiyouga
d2df4c22ab
support mllm hf inference
...
Former-commit-id: e057c8de486bfbc829240924f9238d6212c917f1
2024-04-26 05:34:58 +08:00
hiyouga
2f878bde11
support ORPO
...
Former-commit-id: 17bf8a2c3a7bb5b83071c8659cfd8751e894e692
2024-03-31 18:29:50 +08:00
zhangzc
05afeb304d
Supports custom data set sampling quantity
...
Former-commit-id: 449e2aa38e3a6cf301a43c12c121ac24ebf12027
2024-03-27 14:22:50 +08:00
hiyouga
9cf5d89bd1
update data/readme
...
Former-commit-id: a754f6e9ec157ba76178fa8ea8111e0c7b06008b
2024-02-10 21:04:29 +08:00
hiyouga
db2051684b
improve aligner
...
Former-commit-id: 7d2dc83c5e2085da6273241269c9e9d7509ae51b
2024-02-10 16:39:19 +08:00
Mark Mueller
4bd7b8375e
Slim Orca data parsing
...
Former-commit-id: 1d3598afa10797ba0ce30d44f52e7994587c0ce8
2024-02-08 19:32:20 +01:00
hiyouga
7beeae2209
fix autoset attn impl, update data readme
...
Former-commit-id: 521ad765521bb65aff5a29a8125a2b26ef00bff4
2024-01-31 11:58:07 +08:00
hiyouga
48cab43cb5
add array param format
...
Former-commit-id: 486cc8d3600397812e3927d43ab4181f4e86f5dd
2024-01-21 22:17:48 +08:00
hiyouga
1af13cb737
add models
...
Former-commit-id: 709ac8870a17a96e786b32c75ad8c4e573148cee
2023-12-18 19:09:31 +08:00
hiyouga
1a0bdd305c
support system column #1765
...
Former-commit-id: 0a9c6e0146ebc71d5438c837463d6ab236e227c4
2023-12-12 19:45:59 +08:00
hiyouga
b641e9e97e
fix #1784
...
Former-commit-id: 28d5de7e785f31b223a4646c9c1c770f43e187ec
2023-12-09 20:53:18 +08:00
hiyouga
b2bf10661b
update data readme
...
Former-commit-id: 2b5e33c338e6e8b10c4cbaa68ed26ef3b38ad5f9
2023-11-03 00:15:23 +08:00
hiyouga
a9db89a025
update data readme (zh)
...
Former-commit-id: cc8ffa10d877f5893f3940204e5bec6f3266559f
2023-11-02 23:42:49 +08:00
hiyouga
a1b0655457
support sharegpt format, add datasets
...
Former-commit-id: a8371724130db2fbd7273a480e2acb251e382aec
2023-11-02 23:10:04 +08:00
hiyouga
a4fd976048
refactor dataset_attr, add eos in pt, fix #757
...
Former-commit-id: a9d1fb72f791ae57a4d12f4e3a7e2abccf6a7077
2023-09-01 19:00:45 +08:00
codemayq
b032dc4c4e
add readme for dataset
...
Former-commit-id: cece66d48a770e3e418496445d4040e3cafa9411
2023-08-23 19:55:45 +08:00
hiyouga
802494e20a
update template
...
Former-commit-id: 4318347d3f1982c773dad1074636ec7b550770fd
2023-08-22 19:46:09 +08:00
Peter Pan
23443e9696
add rm dataset explanation
...
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Former-commit-id: b0ca8fe634c35073bb156447ff45c5a8eb54aca1
2023-08-22 01:33:59 -04:00
hiyouga
261ca840d0
update readme, fix web ui postprocess
...
Former-commit-id: 035c966d5c1a2c7b9e9cba8ad06182a6672eabd4
2023-07-22 14:29:22 +08:00
hiyouga
fa47c99fa9
add datasets
...
Former-commit-id: 7159bc54ed0f1bba974662a87ba5039d9aacadee
2023-07-19 20:59:15 +08:00
hiyouga
54b8ce7b63
Initial commit
...
Former-commit-id: 769c6ab56be0c9d26e9289f61ac54a4068d935c1
2023-05-28 18:09:04 +08:00
