329 Commits

Author	SHA1	Message	Date
niuba	2ec68d3398	add last_checkpoint support	2023-08-09 16:39:27 +08:00
hiyouga	df946e6949	fix sft trainer	2023-08-09 16:35:03 +08:00
hiyouga	39cd8b6989	fix rm #420, fix template #426, fix #423	2023-08-09 16:23:31 +08:00
hoshi-hiyouga	2d90685358	fix llama2 template	2023-08-09 00:58:27 +08:00
hoshi-hiyouga	32fa5e8d70	fix tokenizer	2023-08-09 00:54:54 +08:00
hiyouga	3a720aac66	update webui	2023-08-09 00:26:11 +08:00
hiyouga	eecc4b2131	fix tokenizer #417	2023-08-08 23:59:41 +08:00
hiyouga	caa0eda27d	fix bug	2023-08-08 21:28:28 +08:00
hiyouga	4b841a6b35	fix bug	2023-08-08 17:55:55 +08:00
hiyouga	a9980617f5	fix chatml template #408	2023-08-08 17:44:39 +08:00
hiyouga	5453b93db0	update args spec	2023-08-07 15:23:35 +08:00
hiyouga	20cf27976f	update readme	2023-08-07 15:02:02 +08:00
hiyouga	081345baca	fix #376	2023-08-07 13:58:59 +08:00
hiyouga	220175ab24	update trainer	2023-08-07 13:34:35 +08:00
hiyouga	e21ae01356	fix qwen eos token	2023-08-06 13:31:17 +08:00
hiyouga	7f18d2a335	fix qwen tokenizer #361	2023-08-05 17:06:05 +08:00
hiyouga	1afa51c2fa	fix template for tiktoken	2023-08-05 13:42:42 +08:00
hiyouga	53d95725c5	remove redundant code	2023-08-05 00:27:27 +08:00
hiyouga	c183b3551d	fix template	2023-08-05 00:25:00 +08:00
hiyouga	e4a15f863c	fix llama2 template	2023-08-05 00:07:54 +08:00
hiyouga	d87c8fd8ab	fix bos and eos token	2023-08-04 23:55:57 +08:00
hiyouga	8172ad1b5e	fix encode	2023-08-04 23:27:55 +08:00
hiyouga	b4852f9406	support chatml safe encoding	2023-08-04 23:14:28 +08:00
hiyouga	69744c17e8	support interleave probs	2023-08-04 21:27:35 +08:00
hiyouga	d6b14658d4	fix webui export model	2023-08-04 14:20:27 +08:00
hiyouga	a0173c427d	fix mtloader	2023-08-03 19:29:02 +08:00
hiyouga	ff98f1cba8	tiny fix	2023-08-03 17:42:28 +08:00
hiyouga	2780792754	fix qwen inference	2023-08-03 16:31:55 +08:00
hiyouga	ea30da4794	fix qwen inference	2023-08-03 16:15:38 +08:00
hiyouga	87f8f830e2	support Qwen-7B, fix InternLM-7B inference	2023-08-03 15:53:32 +08:00
hiyouga	53d6dc396d	update web demo	2023-08-03 13:28:28 +08:00
hiyouga	e23a3a366c	fix webui	2023-08-03 12:43:12 +08:00
hiyouga	08f180e788	modify code structure	2023-08-02 23:17:36 +08:00
hiyouga	1d8a1878ea	fix PPO trainer	2023-08-02 19:10:23 +08:00
hiyouga	b5ba87952a	update ppo trainer	2023-08-02 18:46:41 +08:00
hiyouga	286f7be346	fix memory leak of PPO trainer	2023-08-02 17:41:34 +08:00
hiyouga	c689857bbb	release v0.1.5	2023-08-02 16:10:31 +08:00
YC Chen	ca125da0eb	[fix] Remove useless code	2023-08-02 14:35:35 +08:00
YC Chen	4323773089	[feature] Fix template of Llama2 to match the offical template	2023-08-02 14:10:15 +08:00
hiyouga	968ce0dcce	fix bug in preprocessing	2023-08-02 01:10:28 +08:00
hiyouga	e3f80774c4	fix #296	2023-08-01 18:43:53 +08:00
hiyouga	e6a3894b99	Fix #294	2023-08-01 18:13:03 +08:00
hiyouga	befaab5720	fix webui	2023-08-01 12:11:37 +08:00
hiyouga	ac88ce5233	fix RM save model	2023-08-01 11:56:17 +08:00
hiyouga	973a638665	release v0.1.4	2023-08-01 10:08:47 +08:00
hiyouga	d3a0692d4d	fix inference	2023-08-01 00:06:48 +08:00
hiyouga	9cb1f119a4	fix arg check	2023-07-31 23:48:57 +08:00
hiyouga	62dca5bb82	update readme	2023-07-31 23:42:32 +08:00
hiyouga	0411a4b3e1	support streaming data, fix #284 #274 #268	2023-07-31 23:33:00 +08:00
hiyouga	513e1f1ec9	Update data_args.py	2023-07-28 17:42:41 +08:00