hiyouga
|
a62d17d009
|
fix export on cpu device
Former-commit-id: cda2ff87272797a062c7addb1bf840ac46208dfd
|
2024-03-04 17:35:09 +08:00 |
|
hiyouga
|
d1e6e02461
|
fix #2649
Former-commit-id: 4e5fae2fac85227641bd16159cf296a32e0b18b4
|
2024-03-01 13:02:41 +08:00 |
|
hiyouga
|
3787d13816
|
fix #2642
Former-commit-id: c0be617195f43d972681dd59727857b1247eeb7e
|
2024-02-29 18:32:54 +08:00 |
|
hiyouga
|
1853b5c172
|
tiny fix
Former-commit-id: 4a871e80e205466262534cdc710b0495954b153e
|
2024-02-29 17:28:50 +08:00 |
|
hiyouga
|
8e7d50dae4
|
release v0.5.3
Former-commit-id: fa5ab21ebc0ab738178c0c57578db3bda995ae06
|
2024-02-29 00:34:19 +08:00 |
|
hiyouga
|
5abbca70d3
|
support DoRA, AWQ, AQLM #2512
Former-commit-id: cfefacaa37453a15c55866d019887f24e886a577
|
2024-02-28 19:53:28 +08:00 |
|
hiyouga
|
0fcb931f18
|
support lora for llama pro
Former-commit-id: 9aeb404a946795d6c4fa3cb45e3e96ffeec13646
|
2024-02-21 02:17:22 +08:00 |
|
hiyouga
|
62b78001b7
|
fix #2481
Former-commit-id: 22acab8aff8cadbba2a67e56af5701c0261ade49
|
2024-02-15 19:07:47 +08:00 |
|
hiyouga
|
96265ec154
|
support llama pro #2338 , add rslora
Former-commit-id: 7924ffc55d98e33bfbfbca303e46c8f476435673
|
2024-02-15 02:27:36 +08:00 |
|
younesbelkada
|
6b98435a53
|
add v1 hf tags
Former-commit-id: 0ca0f08162b18d326787939f12bb5ba07904fb4a
|
2024-02-13 05:58:49 +00:00 |
|
hiyouga
|
75adbfec79
|
add option to disable version check
Former-commit-id: 91d09a01ac3b5da29d284b8d51cdfe4252b391e0
|
2024-02-10 22:31:23 +08:00 |
|
hiyouga
|
bbe5ff0570
|
update gc kwargs
Former-commit-id: 0ae9a16b9d13bc1093662aa0b9bd990400ec2646
|
2024-02-07 00:38:24 +08:00 |
|
hiyouga
|
caeffc780d
|
fix #2438
Former-commit-id: ebf31b62eb1b75399cff7c7542c45ac72f6f41dd
|
2024-02-06 15:23:08 +08:00 |
|
hiyouga
|
f6b2bcfa16
|
fix #2420
Former-commit-id: 19d33ede137a687417bb8697f10e1749781d0d23
|
2024-02-04 15:51:47 +08:00 |
|
hiyouga
|
b1064d2f9b
|
bump up transformers version
Former-commit-id: 38e63bfd28179fbf4b06adc0d127c358fd297e58
|
2024-02-04 00:01:16 +08:00 |
|
hiyouga
|
0fc8612b97
|
add hint for freeze #2412
Former-commit-id: 6545c02790e39395a87d664682ab73e0e3191099
|
2024-02-03 23:38:56 +08:00 |
|
hiyouga
|
a9e58740f5
|
fix #2376
Former-commit-id: 4ecadc35122340b3e520804270c1c1d16c696830
|
2024-02-03 23:14:31 +08:00 |
|
hiyouga
|
7beeae2209
|
fix autoset attn impl, update data readme
Former-commit-id: 521ad765521bb65aff5a29a8125a2b26ef00bff4
|
2024-01-31 11:58:07 +08:00 |
|
hiyouga
|
b8a827faeb
|
fix #2320
Former-commit-id: 2bc30763e9a40a82484c27b9a472425fdb9b3bd8
|
2024-01-24 16:19:18 +08:00 |
|
ldwang
|
323ec3f89f
|
Add patch_mixtral_replace_moe_impl for full training Mitral using DeepSpeed Zero3.
Signed-off-by: ldwang <ftgreat@gmail.com>
Former-commit-id: c284665425e8eefcea2d0dd1c835883e7ce18c97
|
2024-01-24 15:25:31 +08:00 |
|
ldwang
|
db500a2bb6
|
Add patch_mixtral_replace_moe_impl for full training Mitral using DeepSpeed Zero3.
Signed-off-by: ldwang <ftgreat@gmail.com>
Former-commit-id: 18923b14026d88cac2631be1e5d05ba001f69ae6
|
2024-01-24 14:43:16 +08:00 |
|
hiyouga
|
60a042cc16
|
add hint
Former-commit-id: e4ba1deedfc366c984f0095b250ed42a57c3dfba
|
2024-01-22 23:32:01 +08:00 |
|
hoshi-hiyouga
|
68977c8ca4
|
Update patcher.py
Former-commit-id: bdc9eff635cbea670722e672fd329ac35a01a812
|
2024-01-22 23:27:39 +08:00 |
|
A-Cepheus
|
f00ad6b4f8
|
🐞 fix: typo
Former-commit-id: b06a31e76ac623ea89ecc567dd9e9ec88dfdb585
|
2024-01-22 16:04:39 +08:00 |
|
A-Cepheus
|
39d9aba166
|
🐞 fix: typo, move MoE fix to patcher
Former-commit-id: 319a72b48d79c5887f39fc09c66761d5d230a317
|
2024-01-22 16:01:58 +08:00 |
|
A-Cepheus
|
8985c43033
|
fix: ZeRO3 does not work with MoE models
Former-commit-id: e1d5c9851922522f45314c3058d4658198631875
|
2024-01-22 15:21:14 +08:00 |
|
hiyouga
|
fb2d563be5
|
fix #2268
Former-commit-id: e0a717aa3aa97ca5a5d6a2ad8c1711a1f92b001b
|
2024-01-21 14:11:38 +08:00 |
|
hiyouga
|
b27e91222c
|
format style
Former-commit-id: 638234ceee1b19716e45b6e5f4ea54d9122da4df
|
2024-01-20 20:15:56 +08:00 |
|
hiyouga
|
69e8925249
|
support longlora for main branch
Former-commit-id: 38af076a75c33da26d641780820694e4b7342d92
|
2024-01-20 19:25:22 +08:00 |
|
hoshi-hiyouga
|
b36dea7e7b
|
Merge pull request #2201 from liu-zichen/token_embed_resize
support resize embed for zero3
Former-commit-id: bb92cdd0db1e7dd1ea9ec38a0b1d9494a8b5801d
|
2024-01-20 17:45:38 +08:00 |
|
hiyouga
|
edf725208d
|
add upcast_lmhead option
Former-commit-id: 8cbe4e960983ace47f8b956cdc31411347592129
|
2024-01-19 23:54:25 +08:00 |
|
hiyouga
|
ffde3d94bf
|
set use_reentrant=False
Former-commit-id: 0ff9a1fb4fdf42d91095f2dc113d9844bfe4368a
|
2024-01-19 23:29:54 +08:00 |
|
hiyouga
|
e199967391
|
add bf16 lora option
Former-commit-id: b6ec112bebcb379caa32617a135df4d5d3cf865b
|
2024-01-19 16:29:03 +08:00 |
|
hiyouga
|
4e3bfb799d
|
support function calling
Former-commit-id: d9f1cae35150cce594a7abd96dd2beb811fa33f2
|
2024-01-18 09:54:23 +08:00 |
|
liuzc
|
61bc5bd0dd
|
support resize embed for zero3
Former-commit-id: a5f6a7f4fb057511428011c37422c535f31b79d2
|
2024-01-16 15:16:20 +08:00 |
|
hiyouga
|
61960189b2
|
fix #1789
Former-commit-id: 4571068e1e00dc234c9131185fe0924c726add84
|
2024-01-09 18:31:27 +08:00 |
|
hiyouga
|
f53bc7d9a0
|
fix #2127
Former-commit-id: ebee4f6a2a20a39ad7fbb87edfa63e244da7b5e6
|
2024-01-09 14:49:13 +08:00 |
|
hiyouga
|
6bbcf5ad16
|
improve model export
Former-commit-id: d2a676c8ba550e1dd7f4e12cb397a32e01831d85
|
2024-01-05 18:51:49 +08:00 |
|
hiyouga
|
7c50959fe6
|
fix #2098
Former-commit-id: f6fdd83f8a6bf3e48cf08fd098ec7b08d34d16d7
|
2024-01-05 17:11:26 +08:00 |
|
hiyouga
|
dc8714a003
|
fix #2081
Former-commit-id: 33f2c0d4f89cf76671c0fdfbcee79d732b6a020e
|
2024-01-04 23:19:08 +08:00 |
|
hiyouga
|
3fff67e3c7
|
fix dispatch
Former-commit-id: 1696698eb980fef36a44f860105711bc06d674c9
|
2024-01-03 16:33:16 +08:00 |
|
hiyouga
|
eb021ca748
|
fix valuehead patch
Former-commit-id: 24d8d6f224ccb98387ec72e688fa32f5f308dd07
|
2024-01-03 16:19:23 +08:00 |
|
hiyouga
|
1a86cc3078
|
fix rm server
Former-commit-id: 55021097d565536a68113ee33af31beaff38334e
|
2024-01-03 15:30:46 +08:00 |
|
hiyouga
|
afa3c86391
|
fix version
Former-commit-id: 47da742fc971520a697ed5ce611f8d6de61ab22b
|
2023-12-29 04:53:36 +08:00 |
|
hiyouga
|
498b83d77a
|
fix bug
Former-commit-id: e4bb846c43c087bfdd99d0c9eb0318e95b943977
|
2023-12-24 19:20:12 +08:00 |
|
hiyouga
|
2a3980d6ba
|
update loader
Former-commit-id: 6629087e12f64f2635f24311234202077814083c
|
2023-12-24 19:10:23 +08:00 |
|
hiyouga
|
5d440f978e
|
update patcher
Former-commit-id: e44b82ee245a7ee99057c7b58b1edef5c222dc1f
|
2023-12-23 15:24:27 +08:00 |
|
hiyouga
|
a7fc20cb2d
|
fix unsloth dtype
Former-commit-id: 779cfefb7841d00fc712a5f5addf0fe3eb14c6fd
|
2023-12-23 01:59:49 +08:00 |
|
hiyouga
|
938c4cb132
|
fix dpo trainer
Former-commit-id: 074745b1707f98e092749f57041d866c5d55bc04
|
2023-12-23 01:51:55 +08:00 |
|
hiyouga
|
f0d405f392
|
support unsloth
Former-commit-id: 7aad0b889d9a316fffd65f32a419078418fc0986
|
2023-12-23 00:14:33 +08:00 |
|