46 Commits

Author	SHA1	Message	Date
hiyouga	f8c11bd540	update examples ... Former-commit-id: 0a690ada6f	2024-08-09 20:13:46 +08:00
hiyouga	5eacd17090	add adam_mini to readme ... Former-commit-id: e2a28f51c6	2024-08-09 20:02:03 +08:00
hiyouga	25b9cfa163	update scripts ... Former-commit-id: 86f7099fa3	2024-08-09 19:16:23 +08:00
hiyouga	fae881b854	fix #4944 ... Former-commit-id: 1bbd49faae	2024-07-24 16:42:51 +08:00
hiyouga	d97bb11821	update pissa example ... Former-commit-id: c9bb0757ec	2024-07-06 15:47:32 +08:00
hiyouga	2105cf6000	update examples ... Former-commit-id: 2f78b5d62a	2024-06-28 01:17:07 +08:00
hiyouga	a225b5a70c	tiny fix about badam ... Former-commit-id: 095fab58d3	2024-06-25 01:54:53 +08:00
Jonery	bc1c082bc2	add example ... Former-commit-id: 97c5235160	2024-06-18 13:50:26 +08:00
hiyouga	004f289074	tiny fix ... Former-commit-id: 2bf2863a58	2024-06-17 17:47:25 +08:00
hiyouga	f25b8626bf	support pissa ... Former-commit-id: 8c1046d78a	2024-06-16 01:08:12 +08:00
hiyouga	0926d81053	update examples ... Former-commit-id: b6e008c152	2024-06-13 03:15:06 +08:00
hiyouga	cceff9f520	lora modules: all by default ... Former-commit-id: cae4737907	2024-06-06 03:53:28 +08:00
hiyouga	00b3fb4d14	update train hparams ... Former-commit-id: dc4a00dd63	2024-06-06 01:49:20 +08:00
hiyouga	0eff6a66d5	tiny fix ... Former-commit-id: 5a13b3baa6	2024-06-04 00:31:10 +08:00
hiyouga	e4ce59243b	fix #4005 #4013 ... Former-commit-id: eed33862bc	2024-06-03 19:12:29 +08:00
hiyouga	13d7b48efe	improve KTO impl., replace datasets ... Former-commit-id: c450ee87a3	2024-05-18 03:44:56 +08:00
hiyouga	947f0e9964	update badam example #3764 ... Former-commit-id: e5bba7cf1b	2024-05-17 02:21:10 +08:00
hiyouga	dfff5119b4	update examples ... Former-commit-id: ddec9e1b84	2024-05-17 01:02:00 +08:00
hiyouga	6e6267f17c	fix #3694 ... Former-commit-id: 2a67ab3925	2024-05-16 00:35:28 +08:00
hiyouga	3318b6e188	update examples ... Former-commit-id: dae83f4199	2024-05-13 20:39:36 +08:00
hiyouga	92cafef325	update example docs ... Former-commit-id: f02f87c6fb	2024-05-06 22:51:02 +08:00
hiyouga	eb21a527a6	update docs ... Former-commit-id: 34d33e2257	2024-05-06 21:47:00 +08:00
Oscar	c57a42164c	Fix badam example outdated argument ... Former-commit-id: eeb415f6fa	2024-05-05 23:35:19 +08:00
hiyouga	289d1f3679	update webui and add CLIs ... Former-commit-id: 245fe47ece	2024-05-03 02:58:23 +08:00
hiyouga	d8deb0f99e	update readme and examples ... Former-commit-id: a1f1fac33b	2024-04-22 00:37:32 +08:00
hiyouga	92e24a73cb	remove extras ... Former-commit-id: ddbd29d777	2024-04-22 00:35:41 +08:00
hiyouga	9e45f82be7	fix bug in galore optimizer ... Former-commit-id: 5c62881c5a	2024-04-21 18:53:22 +08:00
hiyouga	ec81d45d27	fix mod stuff ... Former-commit-id: f58425ab45	2024-04-21 18:11:10 +08:00
Marco	639297a5ef	Added Mixture of Depths ... Former-commit-id: 620add7b9f	2024-04-18 20:31:24 +02:00
hoshi-hiyouga	507ab397f5	Update sft.sh ... Former-commit-id: 57dcd91e17	2024-04-16 17:25:40 +08:00
Jonery	b3260c7456	resolve gradient checkpointing issue. ... Former-commit-id: 7ecb61822b	2024-04-16 12:05:27 +08:00
Jonery	025f329445	Feature BAdam ... Former-commit-id: 06c8908d3f	2024-04-15 23:15:27 +08:00
hiyouga	fb385b8c26	update examples ... Former-commit-id: cce52351b5	2024-04-15 22:14:34 +08:00
hiyouga	e341fa59fe	update examples ... Former-commit-id: f22eaeb5bc	2024-04-02 20:51:21 +08:00
hiyouga	9df316931b	update examples ... Former-commit-id: 31ffbde24d	2024-04-02 20:41:49 +08:00
hiyouga	135c4e3512	update readme ... Former-commit-id: 11a6c1bad6	2024-04-02 20:37:37 +08:00
hiyouga	bf5ffeeae0	simplify readme ... Former-commit-id: 92dab8a90b	2024-04-02 20:07:43 +08:00
hiyouga	89c400633a	update trainers ... Former-commit-id: 8c77b10912	2024-03-28 18:16:27 +08:00
hiyouga	8b8671817f	improve lora+ impl. ... Former-commit-id: 72367307df	2024-03-13 23:32:51 +08:00
齐保元	24c9277488	[FEATURE]: ADD LORA+ ALGORITHM ... Former-commit-id: a0965cd62c	2024-03-13 19:43:27 +08:00
hiyouga	4a4e4b4354	support layerwise galore ... Former-commit-id: 8664262cde	2024-03-10 00:24:11 +08:00
hiyouga	eb363b04b9	update examples ... Former-commit-id: 4c00bcdcae	2024-03-09 02:30:37 +08:00
hiyouga	398c261c7c	fix aqlm version ... Former-commit-id: 10be2f0ecc	2024-03-09 00:09:09 +08:00
hiyouga	ccec17f773	fix example params ... Former-commit-id: 8a45213440	2024-03-08 20:41:43 +08:00
hiyouga	5b50458acf	fix galore ... Former-commit-id: 33a4c24a8a	2024-03-08 00:44:51 +08:00
hiyouga	cb2bf680c9	add galore examples ... Former-commit-id: 7230e1177d	2024-03-07 22:53:45 +08:00