56 Commits

Author	SHA1	Message	Date
hiyouga	7b3c1f29ff	fix packing for eager/sdpa attn ... Former-commit-id: 6fd6aa4530	2024-07-04 01:52:43 +08:00
hiyouga	3595d98b4c	fix #4609 ... unwrap_model_for_generation(reward_model) is necessary for zero3 training Former-commit-id: 8845e94f91	2024-07-03 19:45:51 +08:00
hiyouga	104151d558	tiny fix ... Former-commit-id: 8b1172b910	2024-07-03 02:31:50 +08:00
hiyouga	c9e9beee4e	tiny fix ... Former-commit-id: 71cdf8956e	2024-07-02 23:06:13 +08:00
hiyouga	ea2d3f6c18	remove rlhf support for chatglm2&3 ... Former-commit-id: 821bb6660e	2024-07-02 23:03:17 +08:00
hiyouga	4828bed837	upcast logits ... Former-commit-id: c13ae2df19	2024-07-02 22:32:05 +08:00
hiyouga	cc31014002	improve rlhf ... Former-commit-id: c47ab6c072	2024-07-02 22:23:08 +08:00
hiyouga	08296f4092	fix ppo callbacks ... Former-commit-id: 4c296001c4	2024-07-02 17:34:56 +08:00
hiyouga	2cf03017a0	tiny fix ... Former-commit-id: 73280b7dc7	2024-07-01 05:43:17 +08:00
hiyouga	54e786346e	add eval acc ... Former-commit-id: 1856a08e87	2024-07-01 03:51:20 +08:00
hiyouga	2b006beab1	loose gemma2 attention ... Former-commit-id: 2f4b89ace1	2024-06-29 01:42:14 +08:00
hiyouga	d3b7c489f2	add Gemma2 models ... Former-commit-id: 6f63050e1b	2024-06-28 01:26:50 +08:00
hiyouga	835f0578c2	refactor pissa, improve llamaboard ... Former-commit-id: 8baf3b22b0	2024-06-28 01:04:24 +08:00
hzhaoy	e1751f6398	fix #4579 ... Former-commit-id: 677c86594e	2024-06-27 13:49:57 +08:00
hiyouga	28e613efd0	fix #4458 ... Former-commit-id: 8d6cd69ac4	2024-06-26 19:52:35 +08:00
hiyouga	ad0304e147	fix #4379 ... Former-commit-id: cc016461e6	2024-06-25 02:31:44 +08:00
hiyouga	a225b5a70c	tiny fix about badam ... Former-commit-id: 095fab58d3	2024-06-25 01:54:53 +08:00
hoshi-hiyouga	fe6ef6400c	Merge pull request #4352 from Ledzy/main ... [Enhancement] Support ZeRO-3 when using BAdam Former-commit-id: d0f953bf5b	2024-06-25 01:49:13 +08:00
hiyouga	7be502c5c5	update readme ... Former-commit-id: e507e60638	2024-06-24 18:22:12 +08:00
hiyouga	7735456561	fix templates ... Former-commit-id: 4cff6a4ad5	2024-06-19 17:44:05 +08:00
Jonery	c779899f7b	Cleaner integration. ... Former-commit-id: 5c2ff1b749	2024-06-19 12:29:40 +08:00
Jonery	3a5eacb4cf	Support distributed BAdam. ... Former-commit-id: 0f72aac8c9	2024-06-18 12:27:47 +08:00
Jonery	5d59f6562a	Merge remote-tracking branch 'upstream/main' ... Former-commit-id: ea1f3ba5e0	2024-06-17 18:44:51 +08:00
Jonery	756566342d	adapt for badam with ds zero3 ... Former-commit-id: 33b4372778	2024-06-17 18:18:10 +08:00
hiyouga	ce4a27a5f7	fix tol ... Former-commit-id: 46093b5786	2024-06-16 01:38:44 +08:00
hiyouga	f25b8626bf	support pissa ... Former-commit-id: 8c1046d78a	2024-06-16 01:08:12 +08:00
hiyouga	c0c6b8075a	tiny fix ... Former-commit-id: 38b6b0f52e	2024-06-16 01:06:41 +08:00
hiyouga	2946153cea	add license ... Former-commit-id: d87108daa6	2024-06-15 17:54:33 +08:00
hiyouga	ab66ae8cd2	fix #4295 ... Former-commit-id: 78589cf90c	2024-06-15 04:34:55 +08:00
hiyouga	a3f4925c2c	add test cases ... Former-commit-id: b27269bd2b	2024-06-15 04:05:54 +08:00
hiyouga	8fccaf20c5	fix #4221 ... Former-commit-id: 6baafd4eb3	2024-06-13 02:48:21 +08:00
hiyouga	81ed4d8abf	fix #4209 ... DeepSpeed ZeRO3 has inflight param error when calling model.eval() Former-commit-id: cf9f2d6c42	2024-06-13 02:25:50 +08:00
hiyouga	833aa324c2	clean code ... Former-commit-id: 2ed8270112	2024-06-13 01:58:16 +08:00
hiyouga	5834651c4a	fix #4198 ... Former-commit-id: 89f2bd8c8c	2024-06-11 15:38:38 +08:00
hiyouga	ca9468ff04	tiny fix ... Former-commit-id: f8d8690bf4	2024-06-07 05:19:21 +08:00
hiyouga	4f3c89a6eb	fix ppo trainer save zero3 model ... accelerator.get_state_dict(ds_model) should be called at all ranks Former-commit-id: 4489d73ac7	2024-06-07 05:14:19 +08:00
hiyouga	f76d427332	fix ppo in trl 0.8.6 ... Former-commit-id: 2702d7e952	2024-06-07 04:48:29 +08:00
hiyouga	d3196318be	fix #4120 ... Former-commit-id: f9e818d79c	2024-06-07 04:18:05 +08:00
hiyouga	8da149ba40	rename files ... Former-commit-id: 74f96efef9	2024-06-07 00:09:06 +08:00
hiyouga	368695483d	fix ppo+zero3 #3108 ... Former-commit-id: 76c61905b2	2024-06-06 23:30:07 +08:00
hiyouga	e0aadd4b34	fix ppo dataset bug #4012 ... Former-commit-id: 149610c636	2024-06-06 19:03:20 +08:00
hiyouga	e898d8bbc4	update trainers ... Former-commit-id: fad2591e31	2024-06-06 18:45:49 +08:00
hiyouga	a16786d8ba	fix #4090 ... Former-commit-id: 67fe822324	2024-06-06 00:50:32 +08:00
hiyouga	6f7b6ae0c3	remove gc warnings in DPO&KTO ... Former-commit-id: f9a206509e	2024-06-03 22:53:54 +08:00
hoshi-hiyouga	5d96cf146e	Update trainer.py ... Former-commit-id: 24499f40dc	2024-06-03 22:08:38 +08:00
enji.zhou	e58aca0602	fix KTO Trainer Sampler ... Former-commit-id: 34a2c5087a	2024-06-03 21:32:38 +08:00
Uminosachi	0de4e1e9e2	Set scheduler_specific_kwargs to get_scheduler ... Former-commit-id: 14e97dc119	2024-05-31 13:45:39 +09:00
hiyouga	468d0e7ed1	10x generate in ppo w/ zero3 ... https://github.com/huggingface/trl/pull/1483 Former-commit-id: 65cd8bdbdb	2024-05-29 00:23:23 +08:00
hiyouga	bfac965f9c	update dpo, kto trainer ... Former-commit-id: 7c8e01bb74	2024-05-29 00:14:29 +08:00
hiyouga	14f6cc2b7c	clean kto trainer ... Former-commit-id: 900e1ea622	2024-05-28 21:43:26 +08:00