19 Commits

Author	SHA1	Message	Date
hiyouga	c4a3977ad7	add max_memory for gptq #1923	2023-12-20 18:15:17 +08:00
hiyouga	ec1fe1daa9	optimize data loading logic	2023-12-20 16:15:41 +08:00
hiyouga	3524aa1e58	support quantization in export model	2023-12-15 23:44:50 +08:00
hiyouga	2c8e88f9c1	fix sharegpt loading	2023-12-13 00:56:16 +08:00
hiyouga	0a9c6e0146	support system column #1765	2023-12-12 19:45:59 +08:00
hiyouga	d5b2c57a35	fix modelscope data hub	2023-12-12 18:33:06 +08:00
hoshi-hiyouga	6382efec52	Merge branch 'main' into feat/support_ms	2023-12-12 17:55:32 +08:00
xingjun.wang	adc98c86da	add use_streaming	2023-12-12 14:23:05 +08:00
xingjun.wang	1909f0d117	fix cache dir	2023-12-12 14:21:33 +08:00
xingjun.wang	168321a4da	add print info for test	2023-12-12 14:14:40 +08:00
xingjun.wang	edc82b923a	update cache dir	2023-12-12 13:08:18 +08:00
xingjun.wang	09533e95ed	update args for MsDataset.load	2023-12-12 13:02:54 +08:00
xingjun.wang	cfba1009d0	update	2023-12-12 12:03:23 +08:00
xingjun.wang	5b979147f0	for test	2023-12-12 11:52:59 +08:00
xingjun.wang	8a908a8c64	for test	2023-12-12 11:47:59 +08:00
hiyouga	28d5de7e78	fix #1784	2023-12-09 20:53:18 +08:00
yuze.zyz	9c2247d700	support ms dataset	2023-12-08 18:00:57 +08:00
hiyouga	00baaa990e	better data streaming	2023-11-19 23:32:47 +08:00
hiyouga	4736344eb1	disentangle model from tuner and rename modules	2023-11-15 16:29:09 +08:00