50 Commits

Author	SHA1	Message	Date
BUAADreamer	4dcb11eab7	add multimodal LLM BLIP-2 and InstructBLIP	2024-04-23 18:45:43 +08:00
hiyouga	f58425ab45	fix mod stuff	2024-04-21 18:11:10 +08:00
Marco	620add7b9f	Added Mixture of Depths	2024-04-18 20:31:24 +02:00
hiyouga	e0dbac2845	support cohere commandR #3184	2024-04-15 23:26:42 +08:00
hiyouga	efc345c4b0	fix #3273	2024-04-15 15:32:58 +08:00
hiyouga	b267aeb53f	add moe aux loss control #3085	2024-04-02 14:26:31 +08:00
hiyouga	eb259cc573	support infer 4bit model on GPUs #3023	2024-04-01 17:34:04 +08:00
hiyouga	96ce76cd27	fix kv cache	2024-03-13 01:21:50 +08:00
hiyouga	8d8956bad5	fix #2802	2024-03-12 17:08:34 +08:00
hiyouga	07f9b754a7	fix #2782 #2798	2024-03-12 15:53:29 +08:00
hiyouga	bdb496644c	allow non-packing pretraining	2024-03-09 22:21:46 +08:00
hiyouga	5d956e2a51	fix chat engine, update webui	2024-03-08 03:01:53 +08:00
hiyouga	d07ad5cc1c	support vllm	2024-03-07 20:26:31 +08:00
hiyouga	f74f804a71	fix #2735	2024-03-07 16:15:53 +08:00
hiyouga	259af60d28	improve aqlm optim	2024-03-05 20:49:50 +08:00
hiyouga	7924ffc55d	support llama pro #2338 , add rslora	2024-02-15 02:27:36 +08:00
hiyouga	88a1bc9773	lint	2024-02-07 01:10:04 +08:00
hiyouga	638234ceee	format style	2024-01-20 20:15:56 +08:00
hiyouga	8cbe4e9609	add upcast_lmhead option	2024-01-19 23:54:25 +08:00
hiyouga	42859f0734	support export push_to_hub #2183	2024-01-16 23:59:42 +08:00
hiyouga	65c5b0477c	fix args	2023-12-28 18:47:19 +08:00
hiyouga	e165354fac	fix export format	2023-12-28 18:40:46 +08:00
hiyouga	6629087e12	update loader	2023-12-24 19:10:23 +08:00
hiyouga	e44b82ee24	update patcher	2023-12-23 15:24:27 +08:00
hiyouga	7aad0b889d	support unsloth	2023-12-23 00:14:33 +08:00
hiyouga	71a9c16171	fix tokenizer for Yi chat models #1617 #1875	2023-12-18 17:18:11 +08:00
hiyouga	0716f5e470	refactor adapter hparam	2023-12-15 20:53:11 +08:00
hoshi-hiyouga	6382efec52	Merge branch 'main' into feat/support_ms	2023-12-12 17:55:32 +08:00
xingjun.wang	09533e95ed	update args for MsDataset.load	2023-12-12 13:02:54 +08:00
hiyouga	bd42c229b0	patch modelscope	2023-12-01 22:53:15 +08:00
yuze.zyz	d38a2e7341	support ms	2023-11-29 20:36:55 +08:00
hiyouga	ce78303600	support full-parameter PPO	2023-11-16 02:08:04 +08:00
hiyouga	a0c31c68c4	add todo	2023-11-10 14:38:18 +08:00
hiyouga	01260d9754	fix ppo train and dpo eval	2023-11-07 22:48:51 +08:00
hiyouga	b2a60905f3	upgrade peft, fix #1088 #1411	2023-11-07 16:13:36 +08:00
hiyouga	7b4acf7265	reimplement neftune	2023-10-22 16:15:08 +08:00
hiyouga	ea82f8a82a	refactor export, fix #1190	2023-10-15 16:01:48 +08:00
hiyouga	af18b0dce7	fix #1184	2023-10-14 19:20:11 +08:00
hiyouga	2818af0b09	refactor model_dtype, fix PPO trainer	2023-10-11 23:16:01 +08:00
hiyouga	84b7486885	fix layer norm dtype	2023-09-28 00:25:55 +08:00
hiyouga	90375f600d	support LongLoRA	2023-09-27 21:55:50 +08:00
hiyouga	338b8664ed	fix #944	2023-09-21 19:51:02 +08:00
hiyouga	d8aa1404be	support FlashAttention2	2023-09-10 20:43:56 +08:00
hiyouga	8ea32e4046	change to right-padding, update reward score #803	2023-09-08 20:04:31 +08:00
hiyouga	a9d1fb72f7	refactor dataset_attr, add eos in pt, fix #757	2023-09-01 19:00:45 +08:00
hiyouga	fa940c17b8	support rope scaling, fix #475 #476 #478	2023-08-12 20:46:27 +08:00
hiyouga	a48cb0d474	Release v0.1.6	2023-08-11 23:25:57 +08:00
hiyouga	3ec4351cfd	support DPO training (2305.18290)	2023-08-11 03:02:53 +08:00
jiongxuc	3e000c2b60	huggingface login for projects must login while running	2023-08-10 14:57:12 +08:00
hiyouga	f751376613	modity code structure	2023-07-15 16:54:28 +08:00