32 Commits

Author	SHA1	Message	Date
hiyouga	21db8ed2f4	use pre-commit	2024-10-29 09:07:46 +00:00
hiyouga	3af57795dd	tiny fix	2024-10-11 23:51:54 +08:00
huniu20	7b91be33c9	add om_hub_token argument	2024-10-10 17:16:46 +08:00
huniu20	0f669f221a	1. add model and dataset info to support webui	2024-10-10 16:46:34 +08:00
huniu20	24ebe187e3	1. add modelers hub support	2024-10-09 17:21:37 +08:00
hiyouga	54c6905937	add docstrings, refactor logger	2024-09-08 00:56:56 +08:00
hiyouga	dabad5570b	update get template	2024-09-04 22:36:20 +08:00
hoshi-hiyouga	8f441c2b3a	Merge pull request #5323 from naem1023/feat/add-dataset-map-batch-size-argument ... Add batch size of map function in the preprocessed dataset	2024-09-04 22:09:36 +08:00
hoshi-hiyouga	44d6947e55	fix #5228	2024-09-04 19:10:30 +08:00
hiyouga	47ea97fb1b	lazy image load	2024-09-04 02:27:08 +08:00
naem1023	209313eeea	feat: add batch size of map function in the preprocessed dataset	2024-09-02 13:52:47 +09:00
hiyouga	c87023d539	follow #5115	2024-08-09 18:03:00 +08:00
“Wzw”	2fa1e0b2ad	mask_history args verify valid	2024-08-08 10:12:01 +08:00
hoshi-hiyouga	a5b809516e	Update loader.py	2024-07-15 00:50:06 +08:00
codingma	76f3bbcfc0	1. add custom eval dataset support ... 2. merge load dataset and split dataset function	2024-07-05 15:52:10 +08:00
hoshi-hiyouga	dddfd516ee	Update loader.py	2024-06-24 23:06:18 +08:00
hiyouga	d87108daa6	add license	2024-06-15 17:54:33 +08:00
hiyouga	6baafd4eb3	fix #4221	2024-06-13 02:48:21 +08:00
hiyouga	74f96efef9	rename files	2024-06-07 00:09:06 +08:00
hiyouga	149610c636	fix ppo dataset bug #4012	2024-06-06 19:03:20 +08:00
hiyouga	5a13b3baa6	tiny fix	2024-06-04 00:31:10 +08:00
hiyouga	a18acf2abe	fix #3992	2024-06-04 00:17:36 +08:00
hiyouga	49b1e88e3d	fix data loader hint	2024-06-03 18:28:27 +08:00
hoshi-hiyouga	ca5dd7c6c1	Update loader.py	2024-05-30 00:20:20 +08:00
hoshi-hiyouga	f9a88b89ca	Update loader.py	2024-05-30 00:17:21 +08:00
hoshi-hiyouga	b55fb611c5	Update loader.py	2024-05-30 00:12:12 +08:00
seanzhang-zhichen	27cb51f7f8	Merge branch 'main' into add_dataset_sample_num	2024-05-24 15:57:47 +08:00
hiyouga	3a023bca2a	refactor data preprocessing, fix mllm rlhf	2024-05-24 04:08:25 +08:00
zhangzc	d956041640	fix conflict	2024-05-20 17:10:01 +08:00
hiyouga	c450ee87a3	improve KTO impl., replace datasets	2024-05-18 03:44:56 +08:00
enji.zhou	db1d5a4f51	add kto	2024-05-17 13:09:17 +08:00
hiyouga	308edbc426	rename package	2024-05-16 18:39:08 +08:00