56 Commits

Author	SHA1	Message	Date
hiyouga	b3247d6a16	support olmo	2024-03-12 18:30:38 +08:00
hiyouga	bdb496644c	allow non-packing pretraining	2024-03-09 22:21:46 +08:00
hiyouga	921ee82267	fix chatglm3 template	2024-03-07 14:26:16 +08:00
hiyouga	9658c63cd9	fix add tokens	2024-03-06 15:04:02 +08:00
hiyouga	9e56eaf2d3	auto set chat template	2024-03-05 02:41:20 +08:00
hiyouga	24a79bd50f	update readme	2024-03-04 19:29:26 +08:00
hiyouga	4cc2781efe	fix #2629	2024-02-29 00:37:29 +08:00
hiyouga	38d8b2cef8	update chatglm3 template	2024-02-28 21:11:23 +08:00
hiyouga	3ba1054593	update readme	2024-02-26 17:25:47 +08:00
Rayrtfr	6e0fba60b3	Support Atom Model	2024-02-26 10:44:10 +08:00
hiyouga	354f13c01a	fix data entry	2024-02-23 18:29:24 +08:00
hiyouga	6bf4c1274f	fix gemma template	2024-02-23 13:49:53 +08:00
hiyouga	a87838ded1	fix template	2024-02-22 12:09:21 +08:00
hiyouga	c375a20230	fix template	2024-02-22 12:06:48 +08:00
hiyouga	c99e19641a	support gemma	2024-02-21 23:27:36 +08:00
hiyouga	a5fb2806cd	update default template	2024-02-10 16:44:47 +08:00
hiyouga	7d2dc83c5e	improve aligner	2024-02-10 16:39:19 +08:00
hiyouga	54ea9684ed	improve fix tokenizer	2024-02-09 14:53:14 +08:00
hoshi-hiyouga	d0daaa01f9	Merge pull request #2423 from mayflower/main ... Support for german sft and dpo	2024-02-07 15:58:20 +08:00
hiyouga	85622ae757	add models	2024-02-06 14:57:23 +08:00
Johann-Peter Hartmann	1ecea9de63	Merge branch 'hiyouga:main' into main	2024-02-04 13:55:00 +00:00
hiyouga	3dc86c4af9	fix #2421	2024-02-04 21:02:55 +08:00
Johann-Peter Hartmann	63da6294dd	Merge branch 'hiyouga:main' into main	2024-02-04 12:51:25 +00:00
hiyouga	db0ab4d601	fix reserved label len	2024-02-04 17:54:26 +08:00
hiyouga	901faa16cc	support minicpm #2404	2024-02-03 22:36:46 +08:00
Johann-Peter Hartmann	b0ffde6e98	add simple german chatml template chatml_de	2024-02-03 09:01:15 +01:00
Fallen Angel	3399c0d645	fix eos_token_id=0 bug ... when eos_token_id=0, will never add eos_token	2024-02-02 17:34:48 +08:00
hiyouga	6fc2d5cc03	add orion models	2024-01-22 21:26:53 +08:00
hiyouga	55f707196e	fix api	2024-01-21 00:03:09 +08:00
hiyouga	a9c18255aa	fix internlm2 template	2024-01-20 23:33:50 +08:00
hiyouga	c550987a72	fix cli_demo	2024-01-20 23:27:10 +08:00
hiyouga	cf818a2598	fix #2260	2024-01-20 23:22:09 +08:00
hiyouga	638234ceee	format style	2024-01-20 20:15:56 +08:00
hiyouga	12043aab9c	fix #2249	2024-01-19 21:44:32 +08:00
hiyouga	a73a979afd	fix templates	2024-01-18 14:49:52 +08:00
hiyouga	f1067d2b58	enable cutoff len	2024-01-18 12:25:42 +08:00
hiyouga	83dbfce8c3	add tool test	2024-01-18 10:26:26 +08:00
hiyouga	d9f1cae351	support function calling	2024-01-18 09:54:23 +08:00
hiyouga	bf73224f33	support solar 10.7B #1907	2024-01-14 00:30:30 +08:00
hiyouga	9aa1a2fc17	fix #2147	2024-01-12 03:30:56 +08:00
hiyouga	ed216bbc46	fix qwen template	2024-01-05 16:14:56 +08:00
hiyouga	e165354fac	fix export format	2023-12-28 18:40:46 +08:00
hiyouga	dec360d5ae	fix stop words	2023-12-20 19:06:43 +08:00
hiyouga	5af8841c4f	fix yi template #1895	2023-12-20 18:58:16 +08:00
hiyouga	a67a440644	add codegeex template	2023-12-18 19:52:35 +08:00
hiyouga	0716f5e470	refactor adapter hparam	2023-12-15 20:53:11 +08:00
hiyouga	96380f5e18	support mixtral	2023-12-12 11:39:04 +08:00
hiyouga	e25f7bae16	add models	2023-12-06 13:33:18 +08:00
hiyouga	6e7af11b98	add xuanyuan models	2023-12-02 00:35:29 +08:00
hiyouga	509abe8864	add models	2023-11-30 19:16:13 +08:00