hiyouga
|
f849d03533
|
update func name
Former-commit-id: ed93ac0829fa656194fd32e1ac063843f475746f
|
2024-07-03 23:29:33 +08:00 |
|
hiyouga
|
7c08a4a82a
|
update arg name
Former-commit-id: 1509ed550b2060f946ce20e3c5a9e5c49e86e3ab
|
2024-07-03 23:23:24 +08:00 |
|
hoshi-hiyouga
|
9174675ba9
|
Merge branch 'main' into main
Former-commit-id: 7be442f37d53a0c6324728fa1fa8e2c84d7f0fa5
|
2024-07-01 21:01:09 +08:00 |
|
hiyouga
|
711ffd0aaf
|
tiny fix
Former-commit-id: 19e43c3a9ed771e991cb273d394ab28fb923f868
|
2024-07-01 03:55:20 +08:00 |
|
hiyouga
|
f7a4f3d9c0
|
loose gemma2 attention
Former-commit-id: a0b645017a2de3d58b6cbc71bd91ec96fc7a818b
|
2024-06-29 01:42:14 +08:00 |
|
hiyouga
|
6ce0b5891b
|
bf16 by default, gemma2 attns
Gemma2 finetuning cannot work until merging https://github.com/huggingface/transformers/pull/31674
Former-commit-id: da66c32c7be0adc28d2185b23e9f62d56acb961c
|
2024-06-28 06:00:26 +08:00 |
|
hiyouga
|
2381fb68a4
|
add quant checks
Former-commit-id: 15bb053e3549739b1a2134640a659b0f35df7de7
|
2024-06-27 01:12:25 +08:00 |
|
hiyouga
|
28c2c7fba5
|
support HQQ/EETQ #4113
Former-commit-id: b7cb51ddb394f04fe4646b2c297fc8d918c9979e
|
2024-06-27 00:29:42 +08:00 |
|
hiyouga
|
4041aa024b
|
improve autogptq integration
Former-commit-id: d68408c7b123b8ff92014db35cac0b24b414a6f4
|
2024-06-26 22:11:44 +08:00 |
|
stceum
|
0bf750ade8
|
Bug Fix: off is parsed as False in yaml file, changed to disabled to avoid this.
Former-commit-id: 171289d8e4c111fdca2b100282b64c74a04a4726
|
2024-06-24 20:39:31 +08:00 |
|
ancv
|
4d345f7901
|
move configure_packing to llamafactory.model.patcher and fix constants
Former-commit-id: 9c5e972c9c81957f2e9e30bf284ef1c076de9fd0
|
2024-06-21 00:45:06 +07:00 |
|
hiyouga
|
0680f18633
|
update patcher
Former-commit-id: afb365e515d615dd62f791622450debab60ce5cc
|
2024-06-19 21:27:00 +08:00 |
|
hiyouga
|
650bb45954
|
fix #4357
Former-commit-id: a6741bba8cebd16a6a3f97a2dc81057d0e27eb39
|
2024-06-18 22:42:45 +08:00 |
|
hiyouga
|
bb8c7e7048
|
fix #4326
Former-commit-id: 3c2c45812a720d92f7f5b15b9f03370fe6bf069e
|
2024-06-17 18:17:48 +08:00 |
|
ancv
|
84e1f06e45
|
update packing with sdpa and eager attention mode
Former-commit-id: 285636ba3a57a1038b2f2fd4cf909a1ca07708d4
|
2024-06-16 02:25:47 +07:00 |
|
hiyouga
|
640372cb66
|
tiny fix
Former-commit-id: f7f440986b0ae3b38ea9f2da80789629d4f79ea1
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
c5e1dfb3a0
|
remove some unused params
Former-commit-id: fef8132c50505a5fb6a246bd024491bd31798a3c
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
acfae2e677
|
add license
Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
344d1192ac
|
clean code
Former-commit-id: f54cafd5c7f0383370d1a2f357834a61a97397ce
|
2024-06-13 01:58:16 +08:00 |
|
ancv
|
4463a5227a
|
implement efficient packing without cross-contamination attention
Former-commit-id: a64a5305c0da5ef092d4cc26faf829bb44de65d1
|
2024-06-12 11:56:01 +07:00 |
|
hiyouga
|
a7233181f2
|
fix deepspeed version
Former-commit-id: 938a69bb07d4de7d82928ff01c582032162c1480
|
2024-06-11 16:52:36 +08:00 |
|
hiyouga
|
8c7943c4de
|
tiny fix
Former-commit-id: b5e9711ef375cc323fc083e742cccfc974550416
|
2024-06-11 01:04:16 +08:00 |
|
hiyouga
|
68df064c1f
|
fix #4160
The split heads should be concatenated in dim=2
Former-commit-id: 4b3f247f270d44df9fe226cfe0dabfb7fcd2deda
|
2024-06-11 00:37:17 +08:00 |
|
hiyouga
|
0b1f4a34f8
|
rename files
Former-commit-id: e1a8431770fc36c0c9ee7fed4abbc3d7fdcc5efd
|
2024-06-07 00:09:06 +08:00 |
|