hiyouga
|
5ab997d484
|
fix gemma2 attention
Former-commit-id: aeafc68e169ae0ea5939cc81cb0cf89f0ca044b6
|
2024-07-13 23:33:45 +08:00 |
|
hiyouga
|
8567dab167
|
tiny fix
Former-commit-id: 9b211861eba19ae9fc360bc96eeb8ad67ba40c49
|
2024-07-04 03:47:05 +08:00 |
|
hiyouga
|
3d219b91b9
|
fix packing for eager/sdpa attn
Former-commit-id: 735a033ceb7f2da6da71d138ea091d8a665411a9
|
2024-07-04 01:52:43 +08:00 |
|
hoshi-hiyouga
|
a90c6306f8
|
Merge pull request #4224 from chuan298/main
Implement efficient packing without cross-contamination attention
Former-commit-id: ac382cc9fe4ec483658fd54f07f9a123788ce1b1
|
2024-07-04 01:18:54 +08:00 |
|
hiyouga
|
60558388ec
|
update packing
Former-commit-id: f3d9c31efa0e64317bdd5b4ed6f78653cf3b5ba4
|
2024-07-04 01:10:55 +08:00 |
|
hoshi-hiyouga
|
b29a7f8cd6
|
Update packing.py
Former-commit-id: 3cc11aa88839c5b99cfd83d9225770a33d0eb6fd
|
2024-07-03 23:36:01 +08:00 |
|
hiyouga
|
a1501591e8
|
update func name
Former-commit-id: ed93ac0829fa656194fd32e1ac063843f475746f
|
2024-07-03 23:29:33 +08:00 |
|
hiyouga
|
1408aa078d
|
update arg name
Former-commit-id: 1509ed550b2060f946ce20e3c5a9e5c49e86e3ab
|
2024-07-03 23:23:24 +08:00 |
|
hiyouga
|
a42671c2d7
|
tiny fix
Former-commit-id: d944020257f363f38e62de6279b337e399b7c65e
|
2024-07-03 02:31:50 +08:00 |
|
ancv
|
20fdf177e8
|
move efficient_packing from data_args to model_args
Former-commit-id: 7b61659c707480bcf8c802c73e10d12ad5b9b965
|
2024-07-02 18:37:55 +07:00 |
|
hoshi-hiyouga
|
a715490c2a
|
Merge branch 'main' into main
Former-commit-id: 7be442f37d53a0c6324728fa1fa8e2c84d7f0fa5
|
2024-07-01 21:01:09 +08:00 |
|
hiyouga
|
4357e42391
|
tiny fix
Former-commit-id: 19e43c3a9ed771e991cb273d394ab28fb923f868
|
2024-07-01 03:55:20 +08:00 |
|
hiyouga
|
188b4be64d
|
fix #4398 #4592
Former-commit-id: 8c92d268903c00392c8bd75a731daa1f107d6202
|
2024-06-30 21:28:51 +08:00 |
|
hiyouga
|
3c4f8eaa55
|
loose gemma2 attention
Former-commit-id: a0b645017a2de3d58b6cbc71bd91ec96fc7a818b
|
2024-06-29 01:42:14 +08:00 |
|
hiyouga
|
fda2cf677b
|
bf16 by default, gemma2 attns
Gemma2 finetuning cannot work until merging https://github.com/huggingface/transformers/pull/31674
Former-commit-id: da66c32c7be0adc28d2185b23e9f62d56acb961c
|
2024-06-28 06:00:26 +08:00 |
|
hiyouga
|
d1aad72826
|
add quant checks
Former-commit-id: 15bb053e3549739b1a2134640a659b0f35df7de7
|
2024-06-27 01:12:25 +08:00 |
|
hiyouga
|
8aaf1185a5
|
support HQQ/EETQ #4113
Former-commit-id: b7cb51ddb394f04fe4646b2c297fc8d918c9979e
|
2024-06-27 00:29:42 +08:00 |
|
hiyouga
|
08fa707085
|
improve autogptq integration
Former-commit-id: d68408c7b123b8ff92014db35cac0b24b414a6f4
|
2024-06-26 22:11:44 +08:00 |
|
hiyouga
|
f51b435bcf
|
fix #4432
Former-commit-id: 972a3b469c600bc6528aef3a49b6fdec63d65803
|
2024-06-25 02:34:04 +08:00 |
|
hiyouga
|
a79e93f335
|
fix #4410
Former-commit-id: f49adc4ab5eade21d7a9e029212f17688ee9b0cf
|
2024-06-24 22:34:31 +08:00 |
|
stceum
|
16e950454e
|
Bug Fix: off is parsed as False in yaml file, changed to disabled to avoid this.
Former-commit-id: 171289d8e4c111fdca2b100282b64c74a04a4726
|
2024-06-24 20:39:31 +08:00 |
|
ancv
|
6c185a2c57
|
move configure_packing to llamafactory.model.patcher and fix constants
Former-commit-id: 9c5e972c9c81957f2e9e30bf284ef1c076de9fd0
|
2024-06-21 00:45:06 +07:00 |
|
hiyouga
|
af2cb33bb2
|
tiny fix
Former-commit-id: 2d8d47f6126d68db1701ed18fc31310c6f14dd49
|
2024-06-20 22:56:05 +08:00 |
|
hiyouga
|
5f5d4c1923
|
update patcher
Former-commit-id: afb365e515d615dd62f791622450debab60ce5cc
|
2024-06-19 21:27:00 +08:00 |
|
hiyouga
|
4bc0bea0e9
|
fix #4357
Former-commit-id: a6741bba8cebd16a6a3f97a2dc81057d0e27eb39
|
2024-06-18 22:42:45 +08:00 |
|
hiyouga
|
60d9896a70
|
fix #4326
Former-commit-id: 3c2c45812a720d92f7f5b15b9f03370fe6bf069e
|
2024-06-17 18:17:48 +08:00 |
|
ancv
|
dd7a1dbfae
|
update packing with sdpa and eager attention mode
Former-commit-id: 285636ba3a57a1038b2f2fd4cf909a1ca07708d4
|
2024-06-16 02:25:47 +07:00 |
|
hiyouga
|
32f45c9e91
|
support pissa
Former-commit-id: ef8e45f2eaf466c54e9a671512a2974575677b08
|
2024-06-16 01:08:12 +08:00 |
|
hiyouga
|
05f3a3c944
|
tiny fix
Former-commit-id: f7f440986b0ae3b38ea9f2da80789629d4f79ea1
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
f91fe10985
|
remove some unused params
Former-commit-id: fef8132c50505a5fb6a246bd024491bd31798a3c
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
bb88536166
|
add license
Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
3ff9b87012
|
add test cases
Former-commit-id: 731176ff34cdf0cbf6b41c40c69f4ceb54c2daf6
|
2024-06-15 04:05:54 +08:00 |
|
hiyouga
|
0a75224f62
|
clean code
Former-commit-id: f54cafd5c7f0383370d1a2f357834a61a97397ce
|
2024-06-13 01:58:16 +08:00 |
|
ancv
|
c7ab302c69
|
implement efficient packing without cross-contamination attention
Former-commit-id: a64a5305c0da5ef092d4cc26faf829bb44de65d1
|
2024-06-12 11:56:01 +07:00 |
|
hiyouga
|
08f2f99f4b
|
fix deepspeed version
Former-commit-id: 938a69bb07d4de7d82928ff01c582032162c1480
|
2024-06-11 16:52:36 +08:00 |
|
hiyouga
|
820b6e7b32
|
fix #4198
Former-commit-id: 945d2c6cc73542adf9272ebd9aa332ea2c1c7361
|
2024-06-11 15:38:38 +08:00 |
|
hiyouga
|
2723438531
|
tiny fix
Former-commit-id: b5e9711ef375cc323fc083e742cccfc974550416
|
2024-06-11 01:04:16 +08:00 |
|
hiyouga
|
4d7dd0330d
|
fix #4160
The split heads should be concatenated in dim=2
Former-commit-id: 4b3f247f270d44df9fe226cfe0dabfb7fcd2deda
|
2024-06-11 00:37:17 +08:00 |
|
hiyouga
|
784088db3f
|
fix #2666
Former-commit-id: f121d5c4f94af9f165132c4309cb9bdc8217d985
|
2024-06-10 21:24:15 +08:00 |
|
hiyouga
|
ff9ba70bb8
|
reorganize adapter code
Former-commit-id: b26c2df9d97f4efffccbf7d28de13619b43f10dd
|
2024-06-08 00:47:23 +08:00 |
|
hoshi-hiyouga
|
adbebb0e3f
|
fix #4139
Former-commit-id: c025a4d74f293c14c2705e68af20a82a84608520
|
2024-06-08 00:45:02 +08:00 |
|
hiyouga
|
3f6b3eed98
|
add resume args in webui
Former-commit-id: 1d86ad768b1f36e54b4c2a9f18f6ea5a7df04c90
|
2024-06-08 00:22:16 +08:00 |
|
hiyouga
|
fcb134e144
|
rename files
Former-commit-id: e1a8431770fc36c0c9ee7fed4abbc3d7fdcc5efd
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
80f716bc10
|
fix torch gc
Former-commit-id: e173799d057598e5692a407601c30d8ce1513461
|
2024-06-06 20:30:25 +08:00 |
|
hiyouga
|
765715af21
|
support train from scratch #4033 #4075
Former-commit-id: 1290b9d01077e62f8de7a23637daa2586cc82bfa
|
2024-06-06 02:43:19 +08:00 |
|
hiyouga
|
3b28c003dd
|
fix full/freeze tuning for mllm
Former-commit-id: df5860ddb593d5b82163a585d12160b41dbce0f3
|
2024-05-27 20:37:57 +08:00 |
|
BUAADreamer
|
932f0d5c20
|
add regex of only tune lm and mm_proj
Former-commit-id: 38d540b3e69bceabafafab524fcfc78aeb05612d
|
2024-05-27 18:59:00 +08:00 |
|
BUAADreamer
|
5c51ab7e1f
|
Merge branch 'hiyouga:main' into main
Former-commit-id: fd5420c43e1414bcd3fadb6239f4e5d42e6ac10e
|
2024-05-25 14:18:49 +08:00 |
|
hiyouga
|
26f293d587
|
fix #3853
Former-commit-id: 465a5500bae1f30744d4b9b3db40aaf9171da2cb
|
2024-05-24 23:29:45 +08:00 |
|
BUAADreamer
|
1f093334d1
|
support pretraining of llava
Former-commit-id: 6a4c8cf0a6a1674c693b9337f018ff8df7477f8f
|
2024-05-21 08:57:14 +08:00 |
|