hiyouga
|
2b006beab1
|
loose gemma2 attention
Former-commit-id: 2f4b89ace15b7a4d2adf16eeba9feb7de9e25d43
|
2024-06-29 01:42:14 +08:00 |
|
hiyouga
|
87e60f8bac
|
bf16 by default, gemma2 attns
Gemma2 finetuning cannot work until merging https://github.com/huggingface/transformers/pull/31674
Former-commit-id: 4d35e218b1d60ff24b368ff5bc608be9c85411de
|
2024-06-28 06:00:26 +08:00 |
|
hiyouga
|
58607ec1b0
|
add quant checks
Former-commit-id: 96a5044394bff75ca8ef17bd7d07d4da66f797f0
|
2024-06-27 01:12:25 +08:00 |
|
hiyouga
|
d2d9fa4abb
|
support HQQ/EETQ #4113
Former-commit-id: ad144c2265cdee0d23014dbb3d017ea257cb26ed
|
2024-06-27 00:29:42 +08:00 |
|
hiyouga
|
6b2733ce12
|
improve autogptq integration
Former-commit-id: addca926de42f91366185a47eb8e777ed44a8e77
|
2024-06-26 22:11:44 +08:00 |
|
hiyouga
|
0ae1302e41
|
fix #4432
Former-commit-id: 1e9d0aa1e45fac52614e79a9fe87e8f1d3757333
|
2024-06-25 02:34:04 +08:00 |
|
hiyouga
|
47651a94a3
|
fix #4410
Former-commit-id: fca893d73c3d7bbb87a816522f2e1568d3e9c612
|
2024-06-24 22:34:31 +08:00 |
|
stceum
|
9aa640f27b
|
Bug Fix: off is parsed as False in yaml file, changed to disabled to avoid this.
Former-commit-id: 3ed063f281d1c2563df1b9eb3800543208c9dc16
|
2024-06-24 20:39:31 +08:00 |
|
ancv
|
5319447aa5
|
move configure_packing to llamafactory.model.patcher and fix constants
Former-commit-id: 770f75dc8363bfa284a72159ff8ad25ec9abe4e0
|
2024-06-21 00:45:06 +07:00 |
|
hiyouga
|
0844750bb9
|
tiny fix
Former-commit-id: 8d4f5093cfcccfe9df173b4c4f7ec0125aecf198
|
2024-06-20 22:56:05 +08:00 |
|
hiyouga
|
030b4811c7
|
update patcher
Former-commit-id: 3b040e8e0f78dbb6bc1409a1b2b788e1affc7458
|
2024-06-19 21:27:00 +08:00 |
|
hiyouga
|
5156114981
|
fix #4357
Former-commit-id: 4bd77d8563aa85230af65caf901214247e214bed
|
2024-06-18 22:42:45 +08:00 |
|
hiyouga
|
7ef169ed39
|
fix #4326
Former-commit-id: e2665e71c7428014d46d91542b01a58c1064d05a
|
2024-06-17 18:17:48 +08:00 |
|
ancv
|
988231026a
|
update packing with sdpa and eager attention mode
Former-commit-id: 238f5c3d99809c6ae2571b59bdce8d8ea3c700b9
|
2024-06-16 02:25:47 +07:00 |
|
hiyouga
|
f25b8626bf
|
support pissa
Former-commit-id: 8c1046d78ac6c8f9429b73617e35e1eccb35138f
|
2024-06-16 01:08:12 +08:00 |
|
hiyouga
|
c0c6b8075a
|
tiny fix
Former-commit-id: 38b6b0f52edeb8ba45aa03b415b3c0c1b0e0c1e4
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
9d9f8c6531
|
remove some unused params
Former-commit-id: 04315c3d92ecc25537e45d5807cb38bc290dcb16
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
2946153cea
|
add license
Former-commit-id: d87108daa68bd40174b262be1ca65fe6e1b7ab56
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
a3f4925c2c
|
add test cases
Former-commit-id: b27269bd2b52fb9d43cde8a8b7f293099b0127a2
|
2024-06-15 04:05:54 +08:00 |
|
hiyouga
|
833aa324c2
|
clean code
Former-commit-id: 2ed8270112755971e3f2dfd2f29c5939b077330a
|
2024-06-13 01:58:16 +08:00 |
|
ancv
|
045eb155a2
|
implement efficient packing without cross-contamination attention
Former-commit-id: b2c367bc61c2778dc359613dca496d9e134c2743
|
2024-06-12 11:56:01 +07:00 |
|
hiyouga
|
8c574eb3cb
|
fix deepspeed version
Former-commit-id: cca6f351081903ca3b5f79f10accc1bbbae0ee61
|
2024-06-11 16:52:36 +08:00 |
|
hiyouga
|
5834651c4a
|
fix #4198
Former-commit-id: 89f2bd8c8c035181927bd530a7ffc733407d674c
|
2024-06-11 15:38:38 +08:00 |
|
hiyouga
|
e3baa5aa08
|
tiny fix
Former-commit-id: 3f24337a8a995b145b1e8075bc23878eaa363844
|
2024-06-11 01:04:16 +08:00 |
|
hiyouga
|
2f164c2c41
|
fix #4160
The split heads should be concatenated in dim=2
Former-commit-id: a793e8456b664ea0b48f0ba162999f18d06b4c2f
|
2024-06-11 00:37:17 +08:00 |
|
hiyouga
|
3b244a69dc
|
fix #2666
Former-commit-id: c907d816670975daa900898660d3503708b7fc37
|
2024-06-10 21:24:15 +08:00 |
|
hiyouga
|
4f0ce9be4e
|
reorganize adapter code
Former-commit-id: 54cd743ebfbd296ae9eaf10c33f59e127f451785
|
2024-06-08 00:47:23 +08:00 |
|
hoshi-hiyouga
|
bad35d1730
|
fix #4139
Former-commit-id: cfd62283a9772fc854b852d2a1b71699f79a0048
|
2024-06-08 00:45:02 +08:00 |
|
hiyouga
|
a8318723a4
|
add resume args in webui
Former-commit-id: 06e5d136a4916413d1c116e341ba7d5136d7748a
|
2024-06-08 00:22:16 +08:00 |
|
hiyouga
|
8da149ba40
|
rename files
Former-commit-id: 74f96efef9bcd63f65d0190c901ff9be54ccd350
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
6cbc66a602
|
fix torch gc
Former-commit-id: 451b6693c0cb86cc9ac03d1a9389cf1fd2b918ec
|
2024-06-06 20:30:25 +08:00 |
|
hiyouga
|
3fcb678d00
|
support train from scratch #4033 #4075
Former-commit-id: a12a506c3d2ba85975a5990c46d2e055cdfe0f2e
|
2024-06-06 02:43:19 +08:00 |
|
hiyouga
|
b88ecd71fd
|
fix full/freeze tuning for mllm
Former-commit-id: 08564838bd02651668845ed74e2e60561e5b6d8c
|
2024-05-27 20:37:57 +08:00 |
|
BUAADreamer
|
606240aec0
|
add regex of only tune lm and mm_proj
Former-commit-id: 57eb13b75d8597d748e84d3549a0b08876b669db
|
2024-05-27 18:59:00 +08:00 |
|
BUAADreamer
|
3eaf371a22
|
Merge branch 'hiyouga:main' into main
Former-commit-id: 60170a1da42a395cf440bbd3825c4e295c31ac38
|
2024-05-25 14:18:49 +08:00 |
|
hiyouga
|
e5d2ef4434
|
fix #3853
Former-commit-id: 063f91cc80193853d17c55fe092fb33683f5d39c
|
2024-05-24 23:29:45 +08:00 |
|
BUAADreamer
|
071d674065
|
support pretraining of llava
Former-commit-id: 29a6d5bdb8610be8f796eed65eede9ba7b503527
|
2024-05-21 08:57:14 +08:00 |
|
hiyouga
|
8d4a5ebf6e
|
fix zero2 high ram usage
Former-commit-id: 31a0564d4f4886db03250f2c6daee6e042dc3eb4
|
2024-05-19 21:53:54 +08:00 |
|
hiyouga
|
df4aec7e72
|
fix #3807
Former-commit-id: 1ebc890a5ff7b034c112bc9cf5cd8a6936613572
|
2024-05-19 17:07:57 +08:00 |
|
hiyouga
|
7130efff54
|
fix jetmoe z3 block
Former-commit-id: d43822fcc220806b9eb7cbf9336ef42a0e6b2a51
|
2024-05-18 22:28:45 +08:00 |
|
hiyouga
|
780a1f5a4e
|
better dtype handle in loading
Former-commit-id: d9f190ff1ea1cc4dd061e8b03d429caea037bca4
|
2024-05-17 02:14:56 +08:00 |
|
hiyouga
|
cae823ddf0
|
rename package
Former-commit-id: 308edbc4260d45907b4a9d3a45ec21d83e48aacb
|
2024-05-16 18:39:08 +08:00 |
|