hiyouga
e645b9f42b
fix #3022
...
Former-commit-id: dac2f617bda9470ac8d85c7e9def09cc04970506
2024-04-02 13:58:39 +08:00
hiyouga
f0e657ec99
set dev version
...
Former-commit-id: 922ecae89210e5b8d62d78774f123a6d75c525ba
2024-04-01 23:24:08 +08:00
hiyouga
357a32d7a0
fix #3083
...
Former-commit-id: ff9a3f73961a362d0ddc22079f80a85465fffda8
2024-04-01 22:53:52 +08:00
hiyouga
ff301c08d6
add qwen1.5 moe
...
Former-commit-id: 3ea94f0d12cec25ac694a2c4ae8971c356990b61
2024-04-01 21:49:40 +08:00
hiyouga
8365522ce2
fix #3077
...
Former-commit-id: d0340391e8075cff0d84b3ef879c2101b66ca1dc
2024-04-01 21:35:18 +08:00
hiyouga
4e3ee3b703
support infer 4bit model on GPUs #3023
...
Former-commit-id: 950a9dab9055839990656b2b40956792b253573d
2024-04-01 17:34:04 +08:00
hiyouga
99294220db
update webui
...
Former-commit-id: e96d260917a35ad2068f7b28b4f0b334b808ccc2
2024-04-01 16:23:28 +08:00
hiyouga
0116675349
fix ORPO loss
...
Former-commit-id: 5544ddde9087f00f9e20b78d0079f20c2f5d1604
2024-04-01 14:42:41 +08:00
hiyouga
bf10e23dc8
fix IPO and ORPO loss
...
Former-commit-id: fc27955732aedbb12003faf19b760e2768b228f2
2024-04-01 14:37:53 +08:00
hiyouga
f59dd028e5
fix plots
...
Former-commit-id: 81355671296b84d438967463bb2a92934ff31aae
2024-03-31 19:43:48 +08:00
hiyouga
899e9e091c
use log1p in orpo loss
...
https://github.com/huggingface/trl/pull/1491
Former-commit-id: 3b15d495264b00a4f8716bafea334778874963d7
2024-03-31 19:27:08 +08:00
hiyouga
cd05e1bb8b
update readme
...
Former-commit-id: 297b01f16ac78cde15a5d85a9a5b82ea20bfaf23
2024-03-31 18:46:34 +08:00
hiyouga
cdde212184
support orpo in webui
...
Former-commit-id: dd5cc78d4fb18dd0a2e9d57f0f046cfe9f0dc2c9
2024-03-31 18:34:59 +08:00
hiyouga
e6c7e6e667
support ORPO
...
Former-commit-id: f44a4c27e2461cdaa1b16865f597a31033c0e6d9
2024-03-31 18:29:50 +08:00
hiyouga
0c96919aa5
tiny fix
...
Former-commit-id: ba4a9b3c01e2f7467fbc5be268f47c0d003caa65
2024-03-31 00:10:29 +08:00
marko1616
e9060f37e4
fix blank line contains whitespace
...
Former-commit-id: 7bc3bcc64353d5a1d4870c6a9509b64cff710492
2024-03-30 23:46:55 +08:00
marko1616
fb6e653443
Fix Llama model save for full param train
...
Former-commit-id: ca17b5db4f97c3ec9fe2004877f150e8f51ab4b5
2024-03-30 23:45:04 +08:00
hiyouga
d7ea73957f
support save args in webui #2807 #3046
...
some ideas are borrowed from @marko1616
Former-commit-id: b5a062aa2d4a37670007e8b3dae5b6f5b7ffb15c
2024-03-30 23:09:12 +08:00
hiyouga
0f38af9365
upgrade gradio to 4.21.0
...
Former-commit-id: 63eecbeb967d849e1d03d8d03fb6421c0ee89257
2024-03-30 20:37:08 +08:00
hiyouga
cc249506a8
release v0.6.1
...
Former-commit-id: a59d823f554505b2e649e6e111b9dee8306d3ad8
2024-03-29 11:36:08 +08:00
hiyouga
5c8794ad1a
fix #2982
...
Former-commit-id: e5e6a0c50c7a1c0052ed6b459450b9735ff2c9a1
2024-03-28 20:22:31 +08:00
hiyouga
66420ea460
fix #3010
...
Former-commit-id: a5e823ae75556eaa3b52ce7a887a6e7838a1eb5f
2024-03-28 18:31:17 +08:00
hiyouga
27f5c967e4
update trainers
...
Former-commit-id: d0dd6eefed0b86895ed00a7cafb331e5193db645
2024-03-28 18:16:27 +08:00
zhangzc
5e1cb05b95
Supports custom data set sampling quantity
...
Former-commit-id: fa8325401df27595de4611a89dfcc14644956abd
2024-03-27 14:22:50 +08:00
hoshi-hiyouga
702ddf0749
fix ds optimizer
...
Former-commit-id: 2675127070a1e7584e71039a11c1ebac54ddd1db
2024-03-26 23:39:56 +08:00
hiyouga
8da4e1625f
fix bug
...
Former-commit-id: f513e1415cc3fe87f600318fba855d1286b6d007
2024-03-26 17:30:12 +08:00
hiyouga
52eb06e2ee
fix #2961
...
Former-commit-id: 616917bb3be7f71073b56ad8c7bc4e164b08b9b5
2024-03-26 17:26:14 +08:00
hiyouga
f62e99a71d
release v0.6.0 (real)
...
Former-commit-id: 34e06bf408ccd21e674f896703f1c7b62e97e1ca
2024-03-25 23:37:48 +08:00
hiyouga
247666b665
tiny fix
...
Former-commit-id: bf2455e420cf35c6596528f319c1b18408b5519a
2024-03-25 23:28:52 +08:00
hiyouga
83709c5dd2
add arg check
...
Former-commit-id: 86e0d5a5a50ae34307f5176c7c4a6ab9d0c224b9
2024-03-25 22:42:58 +08:00
hiyouga
80861ca109
release v0.6.0
...
Former-commit-id: 51910d5803eb718e4976da0b3bfcdc5eeeea48eb
2024-03-25 22:38:56 +08:00
hiyouga
046bd94ce4
tiny fix
...
Former-commit-id: c39cf3439a3025f703d50ac414c10ef3c8486a1f
2024-03-25 21:18:08 +08:00
marko1616
e1493e0192
pass ruff check
...
Former-commit-id: 8534b069a05121eb041371a6becccf0a1a23f268
2024-03-24 16:12:10 +08:00
marko1616
e21a822744
fix Llama lora merge crash
...
Former-commit-id: 46f7d8e6b85f73fb0c51c8b08bd9955c3b171d93
2024-03-24 03:06:11 +08:00
marko1616
1a6f961ab1
fix Llama lora merge crash
...
Former-commit-id: a8bd8e9149ff79a2707fec9c6d006761cfdd0dee
2024-03-24 02:55:23 +08:00
marko1616
7a5c2f4766
fix Llama lora merge crash
...
Former-commit-id: c29a2893f58cf7a916ff05b2725fadf1ad2c4c9a
2024-03-24 02:44:35 +08:00
hiyouga
5382e30bb7
fix #2936
...
Former-commit-id: 9ae646fbbd809057a9c54fe41e1ae5a07a674556
2024-03-24 00:43:21 +08:00
hiyouga
6d7c325f19
fix #2928
...
Former-commit-id: 9558ee87bc7260a6596385aaa375df544862bfa9
2024-03-24 00:34:54 +08:00
hiyouga
06019b7ee3
fix #2941
...
Former-commit-id: 3775ab52017f0b610ddd8199cccfb8c001eda507
2024-03-24 00:28:44 +08:00
hiyouga
b590e82d41
support fsdp + qlora
...
Former-commit-id: b894bf8e84be689db258021f0638e9ac939abcbc
2024-03-21 00:36:06 +08:00
hiyouga
2d12e88c23
fix #2777 #2895
...
Former-commit-id: 54d5f62d29456a8d9d0c0dd3d0bbfffe48935803
2024-03-20 17:59:45 +08:00
hiyouga
6302cd94c8
fix #2346
...
Former-commit-id: c8888c499b0ac51e2fc86c16e8e91c79400a5993
2024-03-20 17:56:33 +08:00
hiyouga
2f8eb620f6
fix packages
...
Former-commit-id: 2f9f334a123d43267bfb3dd26aaa1ad285ffe7a5
2024-03-17 22:32:03 +08:00
hiyouga
2e81c03f41
fix patcher
...
Former-commit-id: 6a5ad99c8cbf6b7def0a130306d49e7d1eb4e5a5
2024-03-15 19:18:42 +08:00
hoshi-hiyouga
a78d5c8845
Merge pull request #2849 from S3Studio/DockerizeSupport
...
Improve Dockerize support
Former-commit-id: b63cba317266f5ba217de54fda77ec26a4df344d
2024-03-15 19:16:02 +08:00
hiyouga
d3b7e4e433
fix export
...
Former-commit-id: 4e996f194406d7eb27b2bde290a12c82c41219d0
2024-03-15 15:06:30 +08:00
S3Studio
bada9f71a7
Use official Nvidia base image
...
Note that the flash-attn library is installed in this image and the qwen model will use it automatically.
However, if the the host machine's GPU is not compatible with the library, an exception will be raised during the training process as follows:
FlashAttention only supports Ampere GPUs or newer.
So if the --flash_attn flag is not set, an additional patch for the qwen model's config is necessary to set the default value of use_flash_attn from "auto" to False.
Former-commit-id: cd2f5717d676e1a5afd2f4e7a38402d2e55e7479
2024-03-15 08:59:13 +08:00
hiyouga
b83aedd040
tiny fix
...
Former-commit-id: bf8123669be334338b4268d0a8f7703ff2cf6255
2024-03-14 21:19:06 +08:00
hiyouga
0d245f67ea
fix export
...
Former-commit-id: c9b968b84c97c9a00fbb43194c3adc9354d74f3b
2024-03-14 18:17:01 +08:00
hiyouga
e49a9aa457
fix bug
...
Former-commit-id: 38c618b797ec219c2c45de960c9cbe50ec524c94
2024-03-13 23:55:31 +08:00
