165 Commits

Author SHA1 Message Date
hiyouga
289d1f3679 update webui and add CLIs
Former-commit-id: 245fe47ece22a4b7822449b126715aaa8ec25aba
2024-05-03 02:58:23 +08:00
hiyouga
ed8d9e0881 fix badam configs
Former-commit-id: 9433c8c215881692f318b89df03af97b4eda4dd5
2024-05-02 02:47:04 +08:00
hiyouga
eb14501a52 release v0.7.0
Former-commit-id: 168f56683ae4909ae50edd4859032fad60149d00
2024-04-26 23:18:00 +08:00
hiyouga
12790601c1 fix llava qlora
Former-commit-id: fc67b736ba3abd1a321c3498548d58de48921626
2024-04-26 18:00:23 +08:00
hiyouga
70bf2a2247 update readme
Former-commit-id: 27ba1b63ce22d6fc2198c517141051705b2e77f6
2024-04-26 05:44:30 +08:00
hiyouga
d2df4c22ab support mllm hf inference
Former-commit-id: e057c8de486bfbc829240924f9238d6212c917f1
2024-04-26 05:34:58 +08:00
hoshi-hiyouga
6ef8ee5988 update hparam name
Former-commit-id: 860549b99b520ae9509559b18646f83fd95933b8
2024-04-26 02:49:39 +08:00
BUAADreamer
a0be27fc9b modify some style
Former-commit-id: 2d4ded535faa44b460f88028d49e4b8c8b430db5
2024-04-25 21:58:18 +08:00
BUAADreamer
549f35b1fd modify style
Former-commit-id: fc0fa9f048aa7ac2f74da15c0250bae5c46cf386
2024-04-25 21:27:48 +08:00
BUAADreamer
e6cf251fb8 modify style
Former-commit-id: 1dcabafe72fe21c7f9122a6bc1a1ccc4f5d08fdd
2024-04-25 21:15:16 +08:00
BUAADreamer
f42c0b26d1 Merge branch 'main' of https://github.com/BUAADreamer/LLaMA-Factory
Former-commit-id: 43d7ad5eccfc04fd7f31a481b278a7101c64a2fa
2024-04-25 21:08:40 +08:00
hiyouga
7b4e1ca788 vllm + lora support
Former-commit-id: 28571da80a35a7cf302a2134a05592505128a0d6
2024-04-25 20:24:31 +08:00
BUAADreamer
ad7d8a6525 Merge branch 'hiyouga:main' into main
Former-commit-id: 68cdd9a020902e276efec6f431ecaf7cf3ded6da
2024-04-25 20:02:50 +08:00
BUAADreamer
b6d78b2a64 merge data part to the text stream
Former-commit-id: c6dd89918feb25fe8c07857162421ad1706f791f
2024-04-25 19:19:59 +08:00
hiyouga
84031c5bf9 add export_device in webui #3333
Former-commit-id: 3a7c1286ce7fdfe5db60b17720ce734408913b7f
2024-04-25 19:02:32 +08:00
BUAADreamer
4e032ff95e merge model part to the text stream
Former-commit-id: 838eb87a961894b072bf79a60bbda63516670d6f
2024-04-25 08:20:41 +08:00
BUAADreamer
ff8d729b59 remove conflicts
Former-commit-id: 7ffee907995095220e93c282d8b57137c0e6c018
2024-04-25 00:34:22 +08:00
BUAADreamer
31bce63a10 add llava and instructblip
Former-commit-id: cfb485eddff0130422416b50c50e171fccc8103e
2024-04-25 00:22:43 +08:00
hiyouga
ce490c65ae support new special token #3420
Former-commit-id: 297fb8ead3daf154152d9826b49bb4d769fbaaa9
2024-04-24 23:39:31 +08:00
hiyouga
80c8586534 reenable sdpa and fast tok by default
Former-commit-id: 07737a3d2d026c973ab964f948953d6ce0e1f2a9
2024-04-24 02:18:44 +08:00
BUAADreamer
175b56bced add multimodal LLM BLIP-2 and InstructBLIP
Former-commit-id: 4dcb11eab7bbeac866043d2a7c748b8d06fbd243
2024-04-23 18:45:43 +08:00
hiyouga
d5d6fb3970 update readme
Former-commit-id: db7f3b9784d21ef5c18a11679ad995bb97d61f7c
2024-04-22 17:09:17 +08:00
hiyouga
6a3ee1edd9 fix optimizers
Former-commit-id: fbbe0dba2f244557fff8ac0076a9f312d0aa01ab
2024-04-21 20:40:54 +08:00
hiyouga
ec81d45d27 fix mod stuff
Former-commit-id: f58425ab45727f7859583d4b9fda776715e27ff6
2024-04-21 18:11:10 +08:00
Marco
b6a87e0a3f fix small typo
Former-commit-id: 4fb7e046b3599aebeeba5b150acb6369f70ecb49
2024-04-18 20:33:29 +02:00
Marco
639297a5ef Added Mixture of Depths
Former-commit-id: 620add7b9f634de1a711f7b87b16050adf735e9b
2024-04-18 20:31:24 +02:00
hiyouga
038d524fcd Update parser.py
Former-commit-id: c00f0771a5ab2a422e0300dcb6d88e8609a3b997
2024-04-16 18:09:31 +08:00
hoshi-hiyouga
496396b3bc Merge pull request #3287 from Ledzy/badam
[Feature] Add BAdam algorithm

Former-commit-id: 4d660c5ade384df4444fa0543a39edce6220903d
2024-04-16 17:32:16 +08:00
hoshi-hiyouga
0f6d691aa6 Update parser.py
Former-commit-id: 4660703674233949a8ba8c76bdb17dafc9d620d4
2024-04-16 17:27:25 +08:00
hoshi-hiyouga
d68212d016 Update parser.py
Former-commit-id: 5b59ff421204115ced405a2e3d56ac0ee8c5b788
2024-04-16 17:27:02 +08:00
hoshi-hiyouga
5f48dd545e Update finetuning_args.py
Former-commit-id: ec899cccf3b8710510e496a3cd8e4c302bb99a19
2024-04-16 17:26:30 +08:00
hiyouga
2dc3343b1c support cohere commandR #3184
Former-commit-id: e0dbac28450a0e1e0b84e1577ef785fc762c0b46
2024-04-15 23:26:42 +08:00
Jonery
025f329445 Feature BAdam
Former-commit-id: 06c8908d3fe48907ddb585c5fa15677fc5416f94
2024-04-15 23:15:27 +08:00
hiyouga
fb385b8c26 update examples
Former-commit-id: cce52351b54f70904f33902d9c17411134f9f6eb
2024-04-15 22:14:34 +08:00
hiyouga
ceccad3419 fix #3273
Former-commit-id: efc345c4b0095ec959ea23bbe54c344278780cbe
2024-04-15 15:32:58 +08:00
hiyouga
c9d3cc181a fix #3238
Former-commit-id: 232642a6215de9b57a1627f39b4efea127948a01
2024-04-12 14:28:11 +08:00
hiyouga
88d9f47a0b fix #3116
Former-commit-id: ce77d98872fa377fd4bc961701b07982f4b51491
2024-04-03 14:47:59 +08:00
hiyouga
bf5ffeeae0 simplify readme
Former-commit-id: 92dab8a90bdd82a72a06559943467b56dde12c71
2024-04-02 20:07:43 +08:00
hiyouga
f4be51f356 add moe aux loss control #3085
Former-commit-id: b267aeb53fc49d2eeb0f3fc5ebe55e643f5db377
2024-04-02 14:26:31 +08:00
hiyouga
b7468ea0a8 support infer 4bit model on GPUs #3023
Former-commit-id: eb259cc5738dfb383e4cc5d32579501c580e11b1
2024-04-01 17:34:04 +08:00
hiyouga
2f878bde11 support ORPO
Former-commit-id: 17bf8a2c3a7bb5b83071c8659cfd8751e894e692
2024-03-31 18:29:50 +08:00
hiyouga
e4f3d583df fix #2982
Former-commit-id: 8d603f8820efd1617557f2bc5d9674143abe7c57
2024-03-28 20:22:31 +08:00
hiyouga
c311375b50 fix bug
Former-commit-id: 3164b4f11b72684c8aa2105037cb36c47b6acfd4
2024-03-26 17:30:12 +08:00
hiyouga
ec94e5e876 fix #2961
Former-commit-id: 511f6754026fbbf48bd481018015338a6a3ad92f
2024-03-26 17:26:14 +08:00
hiyouga
196a33cca4 tiny fix
Former-commit-id: 98a42cbdaa4a90dbe5edda1c412c17e628324f52
2024-03-25 23:28:52 +08:00
hiyouga
b18749fb01 add arg check
Former-commit-id: 1484f76a95bcf40e4c668d52fed68d68c9745a75
2024-03-25 22:42:58 +08:00
hiyouga
8b8671817f improve lora+ impl.
Former-commit-id: 72367307dfadf936fb989ebe8bc9f0ff229fb933
2024-03-13 23:32:51 +08:00
齐保元
24c9277488 [FEATURE]: ADD LORA+ ALGORITHM
Former-commit-id: a0965cd62c85545aa2364e244295df2963308354
2024-03-13 19:43:27 +08:00
hiyouga
a74426df0f fix kv cache
Former-commit-id: 96ce76cd2753bc91c781ad13aa8f7a972abe815a
2024-03-13 01:21:50 +08:00
hiyouga
bbf272f96e support QDoRA
Former-commit-id: 19ef4826490b79e0c2aee20ad67430aa0e4724a7
2024-03-12 22:12:42 +08:00