25 Commits

Author SHA1 Message Date
hiyouga
eb21a527a6 update docs
Former-commit-id: 34d33e22570338da709b8499830adb06b202095c
2024-05-06 21:47:00 +08:00
Oscar
c57a42164c Fix badam example outdated argument
Former-commit-id: eeb415f6fa81ca9093ad0419d1343bd5f780a688
2024-05-05 23:35:19 +08:00
hiyouga
289d1f3679 update webui and add CLIs
Former-commit-id: 245fe47ece22a4b7822449b126715aaa8ec25aba
2024-05-03 02:58:23 +08:00
hiyouga
d8deb0f99e update readme and examples
Former-commit-id: a1f1fac33b2a727b38e8ba52d68a224814d4848b
2024-04-22 00:37:32 +08:00
hiyouga
92e24a73cb remove extras
Former-commit-id: ddbd29d77702f7b82051d930e3eac1b47f5c6d35
2024-04-22 00:35:41 +08:00
hiyouga
9e45f82be7 fix bug in galore optimizer
Former-commit-id: 5c62881c5a59cfcc5a76d365263c8ad8c817ce49
2024-04-21 18:53:22 +08:00
hiyouga
ec81d45d27 fix mod stuff
Former-commit-id: f58425ab45727f7859583d4b9fda776715e27ff6
2024-04-21 18:11:10 +08:00
Marco
639297a5ef Added Mixture of Depths
Former-commit-id: 620add7b9f634de1a711f7b87b16050adf735e9b
2024-04-18 20:31:24 +02:00
hoshi-hiyouga
507ab397f5 Update sft.sh
Former-commit-id: 57dcd91e17833a0eeb8d99af92ac73c132a77648
2024-04-16 17:25:40 +08:00
Jonery
b3260c7456 resolve gradient checkpointing issue.
Former-commit-id: 7ecb61822b37f5d71060d696495830ff98edaa06
2024-04-16 12:05:27 +08:00
Jonery
025f329445 Feature BAdam
Former-commit-id: 06c8908d3fe48907ddb585c5fa15677fc5416f94
2024-04-15 23:15:27 +08:00
hiyouga
fb385b8c26 update examples
Former-commit-id: cce52351b54f70904f33902d9c17411134f9f6eb
2024-04-15 22:14:34 +08:00
hiyouga
e341fa59fe update examples
Former-commit-id: f22eaeb5bc5329146feb0cc5455fae8ce10380f8
2024-04-02 20:51:21 +08:00
hiyouga
9df316931b update examples
Former-commit-id: 31ffbde24dd2e30c3d06331ac4b47d966fc2a191
2024-04-02 20:41:49 +08:00
hiyouga
135c4e3512 update readme
Former-commit-id: 11a6c1bad65a86b0f3d9c5e5df84d246d7d368df
2024-04-02 20:37:37 +08:00
hiyouga
bf5ffeeae0 simplify readme
Former-commit-id: 92dab8a90bdd82a72a06559943467b56dde12c71
2024-04-02 20:07:43 +08:00
hiyouga
89c400633a update trainers
Former-commit-id: 8c77b1091296e204dc3c8c1f157c288ca5b236bd
2024-03-28 18:16:27 +08:00
hiyouga
8b8671817f improve lora+ impl.
Former-commit-id: 72367307dfadf936fb989ebe8bc9f0ff229fb933
2024-03-13 23:32:51 +08:00
齐保元
24c9277488 [FEATURE]: ADD LORA+ ALGORITHM
Former-commit-id: a0965cd62c85545aa2364e244295df2963308354
2024-03-13 19:43:27 +08:00
hiyouga
4a4e4b4354 support layerwise galore
Former-commit-id: 8664262cde3919e10eaecbd66e8c5d356856362e
2024-03-10 00:24:11 +08:00
hiyouga
eb363b04b9 update examples
Former-commit-id: 4c00bcdcaeb675c9fdb3e977c27c3604d7895ae2
2024-03-09 02:30:37 +08:00
hiyouga
398c261c7c fix aqlm version
Former-commit-id: 10be2f0eccc3963a985afcd24e5b8b8fc638b1c3
2024-03-09 00:09:09 +08:00
hiyouga
ccec17f773 fix example params
Former-commit-id: 8a45213440ffc960947dd69ecf3b092aa724bef3
2024-03-08 20:41:43 +08:00
hiyouga
5b50458acf fix galore
Former-commit-id: 33a4c24a8a3c153bc62edf74b9246699a0ae3233
2024-03-08 00:44:51 +08:00
hiyouga
cb2bf680c9 add galore examples
Former-commit-id: 7230e1177daf4d96a1205565ab9335085cc8f3a7
2024-03-07 22:53:45 +08:00