hoshi-hiyouga
702ddf0749
fix ds optimizer
...
Former-commit-id: 2675127070a1e7584e71039a11c1ebac54ddd1db
2024-03-26 23:39:56 +08:00
hiyouga
bb5d0954dc
fix #2981
...
Former-commit-id: ede2a913856e52c0a96155705116528d3af15998
2024-03-26 17:53:04 +08:00
hiyouga
8da4e1625f
fix bug
...
Former-commit-id: f513e1415cc3fe87f600318fba855d1286b6d007
2024-03-26 17:30:12 +08:00
hiyouga
52eb06e2ee
fix #2961
...
Former-commit-id: 616917bb3be7f71073b56ad8c7bc4e164b08b9b5
2024-03-26 17:26:14 +08:00
hiyouga
f62e99a71d
release v0.6.0 (real)
...
Former-commit-id: 34e06bf408ccd21e674f896703f1c7b62e97e1ca
2024-03-25 23:37:48 +08:00
hiyouga
247666b665
tiny fix
...
Former-commit-id: bf2455e420cf35c6596528f319c1b18408b5519a
2024-03-25 23:28:52 +08:00
hiyouga
dc8214c01b
update readme
...
Former-commit-id: 32e6a7f10fdc28106e3b086eb79304943c6e8fab
2024-03-25 23:06:13 +08:00
hoshi-hiyouga
8b21475453
Merge pull request #2967 from Tsumugii24/main
...
Update README_zh.md
Former-commit-id: 4c3b8da2caf74e9d6819bdb1a4e30ca3c549a2d8
2024-03-25 23:02:22 +08:00
Tsumugii24
c11a286bb0
Update README.md
...
Former-commit-id: fd28fff2b9dfdb3e59b160c5fcee9cdc69e53564
2024-03-25 22:54:38 +08:00
Tsumugii24
7c2e29ec9e
Update README_zh.md
...
Former-commit-id: 34141ee0515c3e765ca0cb82a0625fb0abfba6f9
2024-03-25 22:54:26 +08:00
hiyouga
83709c5dd2
add arg check
...
Former-commit-id: 86e0d5a5a50ae34307f5176c7c4a6ab9d0c224b9
2024-03-25 22:42:58 +08:00
hiyouga
80861ca109
release v0.6.0
...
Former-commit-id: 51910d5803eb718e4976da0b3bfcdc5eeeea48eb
2024-03-25 22:38:56 +08:00
Tsumugii24
22c44611fe
Update README_zh.md
...
Former-commit-id: deec57ec009ef6c08a90ad8e5800d6d5a936b337
2024-03-25 22:31:03 +08:00
hoshi-hiyouga
55fb3a186b
Merge pull request #2963 from rkinas/patch-1
...
Update requirements.txt
Former-commit-id: 0d56337adabd84aded31dd19f42d8d06ab2d8666
2024-03-25 21:49:34 +08:00
Remek Kinas
54b0d3f4fd
Update requirements.txt
...
Former-commit-id: a640f245ef9cee706c2f982d578f520e6b1eb70b
2024-03-25 14:30:58 +01:00
hiyouga
046bd94ce4
tiny fix
...
Former-commit-id: c39cf3439a3025f703d50ac414c10ef3c8486a1f
2024-03-25 21:18:08 +08:00
hoshi-hiyouga
e7630e3db3
Merge pull request #2945 from marko1616/bugfix/lora-model-merge
...
修复了在 transformers > 4.36.2 版本中部分模型合并 Lora 模型时因生成配置校验而导致的崩溃问题
Former-commit-id: 95afea730e80f58cc2984592fc07e265504c9491
2024-03-25 13:36:08 +08:00
marko1616
e1493e0192
pass ruff check
...
Former-commit-id: 8534b069a05121eb041371a6becccf0a1a23f268
2024-03-24 16:12:10 +08:00
marko1616
e21a822744
fix Llama lora merge crash
...
Former-commit-id: 46f7d8e6b85f73fb0c51c8b08bd9955c3b171d93
2024-03-24 03:06:11 +08:00
marko1616
1a6f961ab1
fix Llama lora merge crash
...
Former-commit-id: a8bd8e9149ff79a2707fec9c6d006761cfdd0dee
2024-03-24 02:55:23 +08:00
marko1616
7a5c2f4766
fix Llama lora merge crash
...
Former-commit-id: c29a2893f58cf7a916ff05b2725fadf1ad2c4c9a
2024-03-24 02:44:35 +08:00
hiyouga
5382e30bb7
fix #2936
...
Former-commit-id: 9ae646fbbd809057a9c54fe41e1ae5a07a674556
2024-03-24 00:43:21 +08:00
hiyouga
6d7c325f19
fix #2928
...
Former-commit-id: 9558ee87bc7260a6596385aaa375df544862bfa9
2024-03-24 00:34:54 +08:00
hiyouga
06019b7ee3
fix #2941
...
Former-commit-id: 3775ab52017f0b610ddd8199cccfb8c001eda507
2024-03-24 00:28:44 +08:00
hoshi-hiyouga
6d5232a6b0
Merge pull request #2919 from 0xez/main
...
Update README.md, fix the release date of the paper
Former-commit-id: e7157cee78688fdd572a873b1e46accc1a32717e
2024-03-22 12:12:24 +08:00
0xez
5f97b01820
Update README_zh.md, fix the release date of the paper
...
Former-commit-id: 6ea16156b6456216cefab59265dae1edc9dc938f
2024-03-22 10:41:17 +08:00
0xez
d048a244b3
Update README.md, fix the release date of the paper
...
Former-commit-id: 4bf9ef3095376f0208f783f180c13bef88581824
2024-03-21 22:14:48 +08:00
hiyouga
2676df3823
move file
...
Former-commit-id: f9017af7fe1dfbe5b799904ca1d900b3051fb719
2024-03-21 17:05:17 +08:00
hiyouga
ec6d4ac99a
add citation
...
Former-commit-id: 54199205f2000c0500d29822387646133e06e8b2
2024-03-21 17:04:10 +08:00
hiyouga
8d77bc35ac
paper release
...
Former-commit-id: 7bd384655244ce6a8c1f34aa6fed54122d0e9da5
2024-03-21 13:49:17 +08:00
hiyouga
85b1fe92ae
update readme
...
Former-commit-id: ab98d4d617b7193c474f58a29ca9475fea7564aa
2024-03-21 00:48:42 +08:00
hiyouga
b590e82d41
support fsdp + qlora
...
Former-commit-id: b894bf8e84be689db258021f0638e9ac939abcbc
2024-03-21 00:36:06 +08:00
hiyouga
5f3f0c53f2
add orca_dpo_pairs dataset
...
Former-commit-id: af683aacbae462a2a37d76d37df583e217664bd5
2024-03-20 20:09:06 +08:00
hoshi-hiyouga
6d7ebe759e
Merge pull request #2905 from SirlyDreamer/main
...
Follow HF_ENDPOINT environment variable
Former-commit-id: fa801ff118433b622f6aa47920c5c93ec9b68414
2024-03-20 18:09:54 +08:00
hiyouga
2d12e88c23
fix #2777 #2895
...
Former-commit-id: 54d5f62d29456a8d9d0c0dd3d0bbfffe48935803
2024-03-20 17:59:45 +08:00
hiyouga
6302cd94c8
fix #2346
...
Former-commit-id: c8888c499b0ac51e2fc86c16e8e91c79400a5993
2024-03-20 17:56:33 +08:00
SirlyDreamer
a7e96bc329
Follow HF_ENDPOINT environment variable
...
Former-commit-id: 22b36a3cfd2909cb624b1bb7385558eda504defe
2024-03-20 08:31:30 +00:00
khazic
2b9dbc70a9
Updated README with new information
...
Former-commit-id: b12f12039ce221decf09a25ec9d64e385d9497c7
2024-03-20 14:38:08 +08:00
khazic
9e99af39c2
Updated README with new information
...
Former-commit-id: 90a81c2e52bd44beb3b7feb5d2517b073f7f6ef9
2024-03-20 14:21:16 +08:00
刘一博
3b80637eb5
Updated README with new information
...
Former-commit-id: fddbc29ca1bd9b13372087e6a349f21240abc013
2024-03-20 14:11:28 +08:00
hiyouga
2f8eb620f6
fix packages
...
Former-commit-id: 2f9f334a123d43267bfb3dd26aaa1ad285ffe7a5
2024-03-17 22:32:03 +08:00
hiyouga
2e81c03f41
fix patcher
...
Former-commit-id: 6a5ad99c8cbf6b7def0a130306d49e7d1eb4e5a5
2024-03-15 19:18:42 +08:00
hoshi-hiyouga
a78d5c8845
Merge pull request #2849 from S3Studio/DockerizeSupport
...
Improve Dockerize support
Former-commit-id: b63cba317266f5ba217de54fda77ec26a4df344d
2024-03-15 19:16:02 +08:00
hiyouga
d3b7e4e433
fix export
...
Former-commit-id: 4e996f194406d7eb27b2bde290a12c82c41219d0
2024-03-15 15:06:30 +08:00
S3Studio
bada9f71a7
Use official Nvidia base image
...
Note that the flash-attn library is installed in this image and the qwen model will use it automatically.
However, if the the host machine's GPU is not compatible with the library, an exception will be raised during the training process as follows:
FlashAttention only supports Ampere GPUs or newer.
So if the --flash_attn flag is not set, an additional patch for the qwen model's config is necessary to set the default value of use_flash_attn from "auto" to False.
Former-commit-id: cd2f5717d676e1a5afd2f4e7a38402d2e55e7479
2024-03-15 08:59:13 +08:00
S3Studio
c55b1e6d97
improve Docker build and runtime parameters
...
Modify installation method of extra python library.
Utilize shared memory of the host machine to increase training performance.
Former-commit-id: 97f9901c2f5c29a6ab517a1f8fa028b8e89edf4e
2024-03-15 08:57:46 +08:00
hiyouga
b83aedd040
tiny fix
...
Former-commit-id: bf8123669be334338b4268d0a8f7703ff2cf6255
2024-03-14 21:19:06 +08:00
hiyouga
0d245f67ea
fix export
...
Former-commit-id: c9b968b84c97c9a00fbb43194c3adc9354d74f3b
2024-03-14 18:17:01 +08:00
hiyouga
e49a9aa457
fix bug
...
Former-commit-id: 38c618b797ec219c2c45de960c9cbe50ec524c94
2024-03-13 23:55:31 +08:00
hiyouga
36bf98aea8
fix bug
...
Former-commit-id: 47ee0276830adbed65bc111d5a83049e77ad360a
2024-03-13 23:43:42 +08:00
