hiyouga
|
4ef67ed4dd
|
improve lora+ impl.
Former-commit-id: 332bad25455a70ad9204e7dd384bb086d789aa39
|
2024-03-13 23:32:51 +08:00 |
|
齐保元
|
3c15d4cbec
|
[FEATURE]: ADD LORA+ ALGORITHM
Former-commit-id: c35b3c3b1e27171f8a703f88ede1dc8a84c80a56
|
2024-03-13 19:43:27 +08:00 |
|
hiyouga
|
0aacc41252
|
support layerwise galore
Former-commit-id: d43a4da0947897d0be3f62fad3107754d4c89f2b
|
2024-03-10 00:24:11 +08:00 |
|
hiyouga
|
72608bbcb3
|
update hardware requirements
Former-commit-id: 604b3d10fc1448f702943114b66b97bded21e080
|
2024-03-09 03:58:18 +08:00 |
|
hiyouga
|
a44beb2b20
|
fix chat engine, update webui
Former-commit-id: 8b32dddd7d883bae07735796a517927c79d1c33b
|
2024-03-08 03:01:53 +08:00 |
|
hiyouga
|
a0b2d41dc5
|
update galore args
Former-commit-id: c7479a7976f773feb36aab4fdb0500be53d83b6a
|
2024-03-08 01:17:32 +08:00 |
|
hiyouga
|
b5187e4104
|
fix galore
Former-commit-id: 62a3ceeef8f60caef43ccc7f971a0c9184e21296
|
2024-03-08 00:44:51 +08:00 |
|
hiyouga
|
3f56782ffa
|
support galore
Former-commit-id: b67a4a46a88d83bb2a3459b3317b66cda15e0171
|
2024-03-07 22:41:36 +08:00 |
|
hiyouga
|
ddabd699ca
|
support vllm
Former-commit-id: 889f6e910e654d8ec3922c2185042d737ffbf1c3
|
2024-03-07 20:26:31 +08:00 |
|
hiyouga
|
4aa6db78fb
|
fix version checking
Former-commit-id: 5780da8d640609cca388f55983d0251e5547209a
|
2024-03-06 14:51:51 +08:00 |
|
hiyouga
|
98a0c8e8bf
|
support DoRA, AWQ, AQLM #2512
Former-commit-id: 6614cc1f08aa944db083e27e451bbdd733f7dd97
|
2024-02-28 19:53:28 +08:00 |
|
hiyouga
|
c7875bdf27
|
fix #2532
Former-commit-id: 23a8e64f1c47cd473c627effbe271233c136369c
|
2024-02-21 21:55:14 +08:00 |
|
hiyouga
|
8fd6803294
|
support lora for llama pro
Former-commit-id: f74c78ba95f0545aae89e603e466f494705ad024
|
2024-02-21 02:17:22 +08:00 |
|
hiyouga
|
9ee8aaae5e
|
update webui
Former-commit-id: 9e0f7c362d40b78d57e77d52eaa96e678cebadcd
|
2024-02-19 16:49:58 +08:00 |
|
hiyouga
|
562b9d0167
|
support llama pro #2338 , add rslora
Former-commit-id: 40d659b7f30dd5a004703c176ec1f22dc864e505
|
2024-02-15 02:27:36 +08:00 |
|
hiyouga
|
a52372df01
|
add option to disable version check
Former-commit-id: fd769cb2de696aee3c5e882237e16eace6a9d675
|
2024-02-10 22:31:23 +08:00 |
|
hiyouga
|
d4da4d55e4
|
lint
Former-commit-id: 6b1f89b6494e9b6b087fe90600617a3024e014e5
|
2024-02-07 01:10:04 +08:00 |
|
hiyouga
|
80c0ad44a3
|
add hint for freeze #2412
Former-commit-id: 9600c93633629605573d908019563fa3870ad6f8
|
2024-02-03 23:38:56 +08:00 |
|
hiyouga
|
c0e4eebf17
|
format style
Former-commit-id: 53b683531b83cd1d19de97c6565f16c1eca6f5e1
|
2024-01-20 20:15:56 +08:00 |
|
hiyouga
|
ec88dd2930
|
add bf16 lora option
Former-commit-id: 58e7d7ff0cf9bf30e53b3eb12576f38d31976413
|
2024-01-19 16:29:03 +08:00 |
|
hiyouga
|
c7132c3261
|
fix #2195
Former-commit-id: 801f7279693a0c785480ea67d663d99f4ca653da
|
2024-01-16 23:53:50 +08:00 |
|
hiyouga
|
422cabc3f8
|
update loader
Former-commit-id: 080d8eab858217ca58bffe719d5ffde7579c5bda
|
2023-12-24 19:10:23 +08:00 |
|
hiyouga
|
9cdaa43d1c
|
support unsloth
Former-commit-id: b857f00234b90b785d82ca7cdb29af3d948b1a7b
|
2023-12-23 00:14:33 +08:00 |
|
hiyouga
|
b616365c4b
|
fix param type
Former-commit-id: 11b99f344416ade1cdac52e11ba7f36fcf689221
|
2023-12-21 17:33:01 +08:00 |
|
hiyouga
|
3eec0052cc
|
support dpo-ftx
Former-commit-id: 86dfa04f9821556019fa777106787f73eb70b452
|
2023-12-16 19:21:41 +08:00 |
|
hiyouga
|
a08089f449
|
support quantization in export model
Former-commit-id: f32500ae6edccab7d14df4c92467e15986866def
|
2023-12-15 23:44:50 +08:00 |
|
hiyouga
|
8432e50396
|
refactor adapter hparam
Former-commit-id: f82aece9ebd6df83a7a005cc7cbbcec07fa6e14d
|
2023-12-15 20:53:11 +08:00 |
|
hiyouga
|
8dc554a56b
|
remove loftq
Former-commit-id: e175c0a1c631296117abda2403a4b87bbdd35a66
|
2023-12-13 01:53:46 +08:00 |
|
hiyouga
|
0507637911
|
support loftq
Former-commit-id: e7ac2eb7f7daae17525a278ffbe2f82c0fbd8093
|
2023-12-12 22:47:06 +08:00 |
|
hiyouga
|
e88b100ce2
|
update readme
Former-commit-id: 42e042a4206aeb5177ddde56386e9655b0c06460
|
2023-12-12 11:44:30 +08:00 |
|
hiyouga
|
74851e7033
|
implement rm server #1543
Former-commit-id: 2e5bb6888c86079493456c2ddd525f8c52b9963e
|
2023-12-03 20:52:54 +08:00 |
|
hiyouga
|
a073c3824a
|
fix #1659
Former-commit-id: e4123129aae59f4123d53c1f5320e3d5e09ae26d
|
2023-11-28 20:52:28 +08:00 |
|
hiyouga
|
7796b04ddb
|
support export size setting
Former-commit-id: 1a4de54586c21cdbbc89f8a716ca5a54c87a6120
|
2023-11-26 18:34:09 +08:00 |
|
hiyouga
|
78e6ac0156
|
update ppo trainer
Former-commit-id: caa525a5c6f228b9ad71387d1fe4f1c2ffa2479e
|
2023-11-20 21:39:15 +08:00 |
|
Yuchen Han
|
632158ce8e
|
Update finetuning_args.py
Former-commit-id: 30e3430553f1f7e09cd57ef2c9843b549746c618
|
2023-11-17 00:15:51 -08:00 |
|
hiyouga
|
e83b0cbafb
|
fix rlhf callback
Former-commit-id: f5485452d660caef56474cb7dc37abbe4f34599e
|
2023-11-16 03:26:19 +08:00 |
|
hiyouga
|
71fe9ccdd4
|
fix bug in PPO training
Former-commit-id: 2e99f0e53ce6de0acbcab85dd50aef874e8c6336
|
2023-11-16 02:32:54 +08:00 |
|
hiyouga
|
685d0c975a
|
support full-parameter PPO
Former-commit-id: 4af967d69475e1c9fdf1a7983cd6b83bd431abff
|
2023-11-16 02:08:04 +08:00 |
|
hiyouga
|
ddd0e61e57
|
support multiple modules in freeze training #1514
Former-commit-id: 60abac70dfd778df2ae8b3a2e960ed8b607d7ab6
|
2023-11-15 17:08:18 +08:00 |
|
hiyouga
|
c6dfbfa62c
|
refactor evaluation, upgrade trl to 074
Former-commit-id: ed09ebe2c1926ffdb0520b3866f7fd03a9aed046
|
2023-11-13 22:20:35 +08:00 |
|
hiyouga
|
3e3837f363
|
tiny fix
Former-commit-id: 97ba2027bb1ddc01a3c824c40d5a180828810c2c
|
2023-11-09 17:20:49 +08:00 |
|
Yanqing
|
3df8b4fe96
|
Update finetuning_args.py
更新 chatglm/falcon/bloom 的 lora_target 的名称
Former-commit-id: 06606739af035a80ae9ddba9d12c965ed289305d
|
2023-11-09 17:04:40 +08:00 |
|
hiyouga
|
a3973795a0
|
fix ppo train and dpo eval
Former-commit-id: ced863031836632cb5920e22ae6991f251372118
|
2023-11-07 22:48:51 +08:00 |
|
hiyouga
|
08df3dca98
|
upgrade peft, fix #1088 #1411
Former-commit-id: aa7d104f8e050d12cb8f585bc8a52c850995500f
|
2023-11-07 16:13:36 +08:00 |
|
hiyouga
|
31969e78f7
|
reimplement neftune
Former-commit-id: efe9e5a194d3a9f052701d904715238816e4c09e
|
2023-10-22 16:15:08 +08:00 |
|
anvie
|
202cd43027
|
add NEFTune optimization
Former-commit-id: 603e0298af64116ac07130fe6661a9ba823c186c
|
2023-10-21 13:24:10 +07:00 |
|
hiyouga
|
cb68572bce
|
refactor export, fix #1190
Former-commit-id: 30e60e37023a7c4a2db033ffec0542efa3d5cdfb
|
2023-10-15 16:01:48 +08:00 |
|
hiyouga
|
19582ee70f
|
fix ppo args
Former-commit-id: 0f12899951808f53a482082eb116bda309775930
|
2023-10-11 23:40:50 +08:00 |
|
hiyouga
|
867e513e18
|
refactor finetuning Args
Former-commit-id: be425a70a4c8f051717cf1e4464dbd79dae4c0b5
|
2023-09-27 22:28:06 +08:00 |
|
hiyouga
|
538c404cc0
|
update template
Former-commit-id: a95f3a4d62de1073a78125401cf4289ec0523156
|
2023-08-22 19:46:09 +08:00 |
|