hiyouga
|
59a9a5994e
|
fix #2649
Former-commit-id: 1c850de660c671d92f0bc63f230d338b60b7c0bd
|
2024-03-01 13:02:41 +08:00 |
|
hiyouga
|
88fddb879d
|
fix #2642
Former-commit-id: d8435e7f1850532310e1bee069b45f38cd666e48
|
2024-02-29 18:32:54 +08:00 |
|
hiyouga
|
30855b924a
|
tiny fix
Former-commit-id: 3b6e1132c4d203e6d5376cf97e81cc160697c822
|
2024-02-29 17:28:50 +08:00 |
|
hiyouga
|
544e7a491b
|
release v0.5.3
Former-commit-id: f6bc89581b3cd129448da2defc23848de6f494ed
|
2024-02-29 00:34:19 +08:00 |
|
hiyouga
|
b392e6cfb9
|
support DoRA, AWQ, AQLM #2512
Former-commit-id: 6614cc1f08aa944db083e27e451bbdd733f7dd97
|
2024-02-28 19:53:28 +08:00 |
|
hiyouga
|
bc16c9a54a
|
support lora for llama pro
Former-commit-id: f74c78ba95f0545aae89e603e466f494705ad024
|
2024-02-21 02:17:22 +08:00 |
|
hiyouga
|
6bf0fe4913
|
fix #2481
Former-commit-id: 2a4e3e4a26a2fad77ccc476be7d45434b8af4a55
|
2024-02-15 19:07:47 +08:00 |
|
hiyouga
|
596b6828cb
|
support llama pro #2338 , add rslora
Former-commit-id: 40d659b7f30dd5a004703c176ec1f22dc864e505
|
2024-02-15 02:27:36 +08:00 |
|
younesbelkada
|
590b6c2143
|
add v1 hf tags
Former-commit-id: a29cc9f4472c95cd6a43ea350ab728e0a8069c6e
|
2024-02-13 05:58:49 +00:00 |
|
hiyouga
|
5f83860aa1
|
add option to disable version check
Former-commit-id: fd769cb2de696aee3c5e882237e16eace6a9d675
|
2024-02-10 22:31:23 +08:00 |
|
hiyouga
|
f67f781fed
|
update gc kwargs
Former-commit-id: 0cb81c156bc8c21a4bbdd3289a491f78dfcaf730
|
2024-02-07 00:38:24 +08:00 |
|
hiyouga
|
b564b97b7e
|
fix #2438
Former-commit-id: 412d856eeada2abcea598fac0a8d35ae90cc9c01
|
2024-02-06 15:23:08 +08:00 |
|
hiyouga
|
5589d0296a
|
fix #2420
Former-commit-id: 7a34087e4db62e603c9a9a26d8ff3910d7b10c40
|
2024-02-04 15:51:47 +08:00 |
|
hiyouga
|
f2e7122a96
|
bump up transformers version
Former-commit-id: 82f4d4301ed9f31b160d6313a1d2d44a22865f4d
|
2024-02-04 00:01:16 +08:00 |
|
hiyouga
|
a2ae5bd867
|
add hint for freeze #2412
Former-commit-id: 9600c93633629605573d908019563fa3870ad6f8
|
2024-02-03 23:38:56 +08:00 |
|
hiyouga
|
5fa52e87cb
|
fix #2376
Former-commit-id: 8e2cfa7cca21b7fd4538d72114e36f704bcc82fe
|
2024-02-03 23:14:31 +08:00 |
|
hiyouga
|
5b8712d061
|
fix autoset attn impl, update data readme
Former-commit-id: 34a6e5f82baf45cc8dbb11f9f7ab4a480ab7ec5c
|
2024-01-31 11:58:07 +08:00 |
|
hiyouga
|
1ace676170
|
fix #2320
Former-commit-id: e0b0c4415aaf80e75f6dd4f3777a0616b0e60f84
|
2024-01-24 16:19:18 +08:00 |
|
ldwang
|
786a2f1103
|
Add patch_mixtral_replace_moe_impl for full training Mitral using DeepSpeed Zero3.
Signed-off-by: ldwang <ftgreat@gmail.com>
Former-commit-id: 5f50c02f0e425737cd80abdf8fde9e25abf13083
|
2024-01-24 15:25:31 +08:00 |
|
ldwang
|
36ac14a566
|
Add patch_mixtral_replace_moe_impl for full training Mitral using DeepSpeed Zero3.
Signed-off-by: ldwang <ftgreat@gmail.com>
Former-commit-id: d1413dcec8a3b1d671f240b82a689c72b54d7b93
|
2024-01-24 14:43:16 +08:00 |
|
hiyouga
|
7a048fc91d
|
add hint
Former-commit-id: c540ef41bda61993b83ef8cfe3c84b1d169e984c
|
2024-01-22 23:32:01 +08:00 |
|
hoshi-hiyouga
|
b36c4b99cc
|
Update patcher.py
Former-commit-id: 33556cc6b0b65cc6db02e66f4f6e75112c33d966
|
2024-01-22 23:27:39 +08:00 |
|
A-Cepheus
|
882a6a1d51
|
🐞 fix: typo
Former-commit-id: 57a3687ecd23237559aee0e8e811b782846f2415
|
2024-01-22 16:04:39 +08:00 |
|
A-Cepheus
|
712ab4ae7a
|
🐞 fix: typo, move MoE fix to patcher
Former-commit-id: 4ff28e99ff9b48df7150591c6bbd3723f22b7715
|
2024-01-22 16:01:58 +08:00 |
|
A-Cepheus
|
18ad259fb3
|
fix: ZeRO3 does not work with MoE models
Former-commit-id: b2844c049a88ea89f8e1812e2d2e8662b4002965
|
2024-01-22 15:21:14 +08:00 |
|
hiyouga
|
96531a0ef8
|
fix #2268
Former-commit-id: 300ecf9b9d7fd99fbb68f3d086e3ad973c2f894e
|
2024-01-21 14:11:38 +08:00 |
|
hiyouga
|
66e0e651b9
|
format style
Former-commit-id: 53b683531b83cd1d19de97c6565f16c1eca6f5e1
|
2024-01-20 20:15:56 +08:00 |
|
hiyouga
|
80637fc06d
|
support longlora for main branch
Former-commit-id: f869501ad4c368df26534c41f62c6d63c6be17dd
|
2024-01-20 19:25:22 +08:00 |
|
hoshi-hiyouga
|
8efc055511
|
Merge pull request #2201 from liu-zichen/token_embed_resize
support resize embed for zero3
Former-commit-id: c0d1b5e3aef70da6b115614bd1ed539a76d6547a
|
2024-01-20 17:45:38 +08:00 |
|
hiyouga
|
be61bfda93
|
add upcast_lmhead option
Former-commit-id: 7ef69a1697c11ff13e7503360e40ef36cfb1c345
|
2024-01-19 23:54:25 +08:00 |
|
hiyouga
|
1a39f529c0
|
set use_reentrant=False
Former-commit-id: efa2e27d5ef6eaeb7baa7551c651ef10ab31400c
|
2024-01-19 23:29:54 +08:00 |
|
hiyouga
|
384f0e7678
|
add bf16 lora option
Former-commit-id: 58e7d7ff0cf9bf30e53b3eb12576f38d31976413
|
2024-01-19 16:29:03 +08:00 |
|
hiyouga
|
a423274fd9
|
support function calling
Former-commit-id: 66533b3f65babf2429c92c0f8fafe4eff5e0ff63
|
2024-01-18 09:54:23 +08:00 |
|
liuzc
|
aa72a4349e
|
support resize embed for zero3
Former-commit-id: b5464f5699b13bb118ac57ebc40b3cf9eb030396
|
2024-01-16 15:16:20 +08:00 |
|
hiyouga
|
4d6669c268
|
fix #1789
Former-commit-id: d86455f685fa531e651333e00b4fe54d895cf2e4
|
2024-01-09 18:31:27 +08:00 |
|
hiyouga
|
89f4ae51f9
|
fix #2127
Former-commit-id: 5a1aa33fa9b546ab520f0ba4cb9d996b87eb71ca
|
2024-01-09 14:49:13 +08:00 |
|
hiyouga
|
75fe1404b1
|
improve model export
Former-commit-id: 31255147a566a23ce1a48402662d14af8ac267ab
|
2024-01-05 18:51:49 +08:00 |
|
hiyouga
|
b460c9372f
|
fix #2098
Former-commit-id: e62d9158cffbf1044396597ddaf15b1c0bc5f954
|
2024-01-05 17:11:26 +08:00 |
|
hiyouga
|
04ae80a52e
|
fix #2081
Former-commit-id: ec4b539b6c0be11e15d273025c414b694bbd6c9a
|
2024-01-04 23:19:08 +08:00 |
|
hiyouga
|
8c74851b70
|
fix dispatch
Former-commit-id: deda82638716506dc690902c51276bb1eb0ddd5e
|
2024-01-03 16:33:16 +08:00 |
|
hiyouga
|
7168392a51
|
fix valuehead patch
Former-commit-id: d9cb98362b58b28ae0ee207e7c07e75e5d810876
|
2024-01-03 16:19:23 +08:00 |
|
hiyouga
|
ccc5b324fe
|
fix rm server
Former-commit-id: 81bc1638682a9fd01518f9f25250a6b584d2a9e6
|
2024-01-03 15:30:46 +08:00 |
|
hiyouga
|
ebb32e85f8
|
fix version
Former-commit-id: dd7500b65d0d548441eece101b60d51fa619cc0f
|
2023-12-29 04:53:36 +08:00 |
|
hiyouga
|
c33fbea469
|
fix bug
Former-commit-id: b06faa1be3f5aa5e0fa31aa31314c213c36c3442
|
2023-12-24 19:20:12 +08:00 |
|
hiyouga
|
921f593632
|
update loader
Former-commit-id: 080d8eab858217ca58bffe719d5ffde7579c5bda
|
2023-12-24 19:10:23 +08:00 |
|
hiyouga
|
940403720a
|
update patcher
Former-commit-id: d6d7b6670847ce4ea10353c5b126214542b45c2b
|
2023-12-23 15:24:27 +08:00 |
|
hiyouga
|
306a70c7ba
|
fix unsloth dtype
Former-commit-id: fd22e6546ce5f38a6a075cf894aafc3d206b2fcd
|
2023-12-23 01:59:49 +08:00 |
|
hiyouga
|
d358d955e5
|
fix dpo trainer
Former-commit-id: c160dd7cd86e296e32775ace2e4258a473449c41
|
2023-12-23 01:51:55 +08:00 |
|
hiyouga
|
6faf9c35a9
|
support unsloth
Former-commit-id: b857f00234b90b785d82ca7cdb29af3d948b1a7b
|
2023-12-23 00:14:33 +08:00 |
|
ShaneTian
|
d05febe5de
|
Fix slow model initialization in bfloat16 dtype.
Former-commit-id: cf2e2f6f9b7f09b1e2faf6fbc413e3f62e3846c7
|
2023-12-22 16:27:28 +08:00 |
|