Yaser Afshar
fe4546a7bb
Add trust_remote_code parameter and remove True
...
- Introduced a new model parameter `trust_remote_code`
- Set the default value of `trust_remote_code` to `False`
to enhance security
Former-commit-id: 09437763267bc7081159a6878cee9652a2b1ddac
2024-12-17 12:25:12 +00:00
hoshi-hiyouga
fc18db6290
Merge pull request #6364 from hiyouga/hiyouga/control_reenterent_gc
...
[model] support non-reenterent-gc
Former-commit-id: a665ad6178516faf8aaa628d3b2c672ad831d7b7
2024-12-17 19:58:36 +08:00
hiyouga
64bac4bc7e
support non-reenterent-gc & fix #6358
...
Former-commit-id: f319da6937964c5068864f7815efbf3847bd4ad8
2024-12-17 11:41:59 +00:00
hoshi-hiyouga
002c7d2867
Merge pull request #6363 from hiyouga/hiyouga/control_skip_eos
...
[infer] support control eos
Former-commit-id: 69738283071567eeecb492ac38ce1df3b994d975
2024-12-17 19:35:40 +08:00
hiyouga
a94a1eac67
support control eos, fix #6345
...
Former-commit-id: eda76de32bab103c650f246327d214539ae6f291
2024-12-17 10:42:05 +00:00
hoshi-hiyouga
a8a990a9a7
Merge pull request #6362 from hiyouga/hiyouga/mllm_packing
...
[model] generalized packing
Former-commit-id: 9708a39179d7872ff2039086fcadb021265974cc
2024-12-17 18:41:48 +08:00
hiyouga
bff1b94583
generalized packing & fix #6343
...
Former-commit-id: 2d107d3aefd5af61163056634c8b91fe3cb3e77c
2024-12-17 10:26:19 +00:00
hoshi-hiyouga
4caf043cf8
Merge pull request #6359 from hiyouga/hiyouga/fix_qwen2vl_infer
...
[model] fix qwen2vl infern
Former-commit-id: 81815f053f9eef23fa4906cc47496806cfc1735c
2024-12-17 18:15:23 +08:00
hiyouga
50ca43c3fb
fix #6348
...
Former-commit-id: 142191e4664cb1b920aff2f51d1bac6180f2c24b
2024-12-17 10:06:46 +00:00
hoshi-hiyouga
0f49e9cb07
Merge pull request #6334 from hiyouga/hiyouga/add_examples
...
[assets] update wechat and examples
Former-commit-id: e2fbd07096877f76d6bdd1b0c4b2d42aa10c346f
2024-12-15 01:37:01 +08:00
hiyouga
ba901bc000
update assets
...
Former-commit-id: 7059055e89cc3f8557581d5d5b728a82ab5b0899
2024-12-14 17:36:03 +00:00
hiyouga
6f1e450739
fix mrope
...
Former-commit-id: 2811814fc42fb214b3e8be1055f9f57ffd0ffb12
2024-12-12 15:08:17 +00:00
hoshi-hiyouga
93d1cba06e
Merge pull request #6253 from hiyouga/hiyouga/qwen2vl_mm_proj
...
[model] support qwen2vl train proj only
Former-commit-id: bcb4fb353ed810477b7813c6904b33c51efb7f81
2024-12-05 20:25:33 +08:00
hiyouga
cf8cad8e7e
support qwen2vl train proj only
...
Former-commit-id: 99c62660c67fd2567c094bb49ec7d108e5670730
2024-12-05 10:37:42 +00:00
hoshi-hiyouga
255260cfcb
Merge pull request #6251 from hiyouga/hiyouga/vllm_qwen2vl_infer
...
[infer] support qwen2vl vllm infer
Former-commit-id: 561a8e56d95e31dc9a36da4b369af219d7319688
2024-12-05 18:26:19 +08:00
hiyouga
88b06a0c7f
support qwen2vl vllm infer
...
Former-commit-id: 207f8b069ca35a28de4588b4962e7254f451c52c
2024-12-05 10:17:26 +00:00
hoshi-hiyouga
7f8c59144e
Merge pull request #6246 from hiyouga/hiyouga/update_examples
...
[examples] update examples
Former-commit-id: 967a6c12a72b97104bd262b0f2ed945a8b0cd81d
2024-12-05 16:49:30 +08:00
hiyouga
90fb5605c1
update examples
...
Former-commit-id: e5584dc7ba356264034c14ff7d8efcd3d50fa9a3
2024-12-05 08:48:25 +00:00
hoshi-hiyouga
9f9ad6435d
Merge pull request #6242 from hiyouga/hiyouga/fix_script
...
[script] fix scripts
Former-commit-id: c42890bb2b287ac1fccb369a63a6495e7008b245
2024-12-05 11:54:46 +08:00
hiyouga
819f487c8f
fix scripts
...
Former-commit-id: eb3e147d198a3ecb02c65f7733cec7cd9d3814a3
2024-12-05 03:47:32 +00:00
hoshi-hiyouga
9bbeba6323
Merge pull request #6160 from village-way/pr_dataloader
...
fix:tokenized_path not None and load_from_disk return Dataset Trigger…
Former-commit-id: cf298468309cd923d830dcaf7a1aa837519faf1e
2024-12-04 22:18:19 +08:00
hoshi-hiyouga
92940817e7
lint
...
Former-commit-id: 6a5074e46695378b76d58aac8ad7768b6b034b9c
2024-12-04 22:08:27 +08:00
hoshi-hiyouga
68614f6bc1
Merge pull request #6238 from hiyouga/hiyouga/vllm_batchinfer
...
[infer] feat: support batch infer in vllm
Former-commit-id: 8328bd8fbbb0699bde8269ba13d78e400668dca7
2024-12-04 21:59:13 +08:00
hiyouga
235cdcacee
support batch infer in vllm
...
Former-commit-id: 1324d158f954d777f1fbf09f46149c372704b388
2024-12-04 13:50:00 +00:00
hoshi-hiyouga
b2c67a989a
Merge pull request #6190 from JieShenAI/main
...
add vllm_infer script
Former-commit-id: dc7835500297322c85ba1d69fb12e743a5cd7bfd
2024-12-04 21:19:23 +08:00
hoshi-hiyouga
ed4c4bab49
Merge pull request #6170 from hykilpikonna/main
...
[+] Show the hostname in webui title
Former-commit-id: 263cb82bdb3dfb0c213018f25022686a3dab617e
2024-12-04 18:07:29 +08:00
hoshi-hiyouga
1804e8a491
Merge pull request #6233 from hiyouga/hiyouga/vlm_zero3
...
[data] fix vlm zero3 training
Former-commit-id: 187402203bb0aeada3c97f65fa9d178ff490c261
2024-12-04 17:51:10 +08:00
hiyouga
0ef1dc4dd5
fix vlm zero3 training
...
Former-commit-id: dbb9e5b70efab37ed057b2d5822b9d0d23e99fb1
2024-12-04 09:40:39 +00:00
hoshi-hiyouga
b34c3bb796
Merge pull request #6224 from hiyouga/hiyouga-patch-1
...
[assets] chore: update wechat
Former-commit-id: 7965e9840c18c71028c1a3a04c404e9fae196c0d
2024-12-03 21:25:38 +08:00
hoshi-hiyouga
aa5535c622
update wechat
...
Former-commit-id: 722a396b69ce340609175b0bc1cf118e5593a749
2024-12-03 20:48:48 +08:00
JieShen
d4bf81b36a
add async call api
...
Former-commit-id: 4c61368600a6648ec20753a41536ad3c7986703b
2024-12-01 22:18:05 +08:00
JieShen
99265c7d2f
add vllm_infer script
...
Former-commit-id: 961e8c2d2e5505de14702cf8609d54b4f3a23b1e
2024-11-29 14:22:20 +08:00
Azalea
0efa34c9ef
[U] Compute hostname differently
...
Former-commit-id: 6554cdeedb8ab669ce500be404dab8d42a90779b
2024-11-28 22:23:41 -05:00
hoshi-hiyouga
f4729904f2
Merge pull request #6175 from hiyouga/hiyouga/add_qwq
...
[model] add QwQ
Former-commit-id: f2b2a37f0819bb344998a9059c02b97ca5a5ae74
2024-11-28 17:01:53 +08:00
hiyouga
1c3d86cd65
add qwq
...
Former-commit-id: 68a612115aebba51695d22be4397c16c86f3b40a
2024-11-28 08:50:57 +00:00
Azalea
f5e6e25a1b
[+] Show the hostname
...
Former-commit-id: dfb953b1adb79ed9f886ecac6851a559617aaa5a
2024-11-28 12:25:02 +08:00
wangdepeng
ae09c6c214
fix:tokenized_path not None and load_from_disk return Dataset Trigger stuck
...
Former-commit-id: 4424d4de8aca0e4d3b92672584978f3cc3fc33da
2024-11-27 16:44:42 +08:00
hoshi-hiyouga
265a5821de
Merge pull request #6156 from hiyouga/hiyouga/add_o1
...
[data&model] add marco-o1, skywork-o1 and openo1
Former-commit-id: 86f41513c04424f8f4ccfeca78d28a4cc7ba8728
2024-11-27 14:36:01 +08:00
hiyouga
9822cb7bac
fix dataset
...
Former-commit-id: 046b6fb118e3ea75062c6a759720a1759639e93c
2024-11-27 06:27:44 +00:00
hiyouga
d51d96d594
add skywork o1
...
Former-commit-id: ec9ff8caa2637965d41937cce7de4e4d51d054eb
2024-11-27 05:51:59 +00:00
hiyouga
09a3a59c88
Merge remote-tracking branch 'origin/main' into hiyouga/add_o1
...
Former-commit-id: b7c7f3066f7c33fc6e52d9d1d57b831966e53c7f
2024-11-27 05:36:41 +00:00
hoshi-hiyouga
dfa4e927dd
Merge pull request #6157 from hiyouga/hiyouga/fix_ci
...
[ci] pin tokenizers version
Former-commit-id: 14d0d92bf38ac09f0b7f7f9484345310e32fbbe6
2024-11-27 13:33:04 +08:00
hiyouga
61320965aa
pin tokenizers version
...
Former-commit-id: b7d4cf2caf2d02f7d16bab3f0ec8bf0108b7be75
2024-11-27 05:24:58 +00:00
hiyouga
ab3782b0fa
add marco-o1 and openo1 dataset
...
Former-commit-id: 17afb7d4103499a9a090a6624896cfa123e9e1d6
2024-11-27 04:20:23 +00:00
hoshi-hiyouga
6cd90efb82
Merge pull request #6152 from hiyouga/hiyouga/add_num_proc_in_data_load
...
[data] add num_proc in load_dataset
Former-commit-id: b26c490ac3a0a8a6342f940eb6ccb7b8b6d78f93
2024-11-27 00:16:15 +08:00
hoshi-hiyouga
08ca40876a
Merge pull request #6151 from hiyouga/hiyouga/fix_mllama
...
[model] fix mllama cross mask
Former-commit-id: 88f087c8b9cb22fa4f4e4f867ea3d71dd8606a98
2024-11-27 00:07:54 +08:00
hiyouga
358708ee97
fix #6149
...
Former-commit-id: 362d579ce83e63007e6f89f264d06d2698671cc6
2024-11-26 16:03:02 +00:00
hiyouga
006022cadd
fix mllama cross_mask
...
Former-commit-id: 598c22e43f3f10a335933339cc612744c4835eb0
2024-11-26 15:56:58 +00:00
hoshi-hiyouga
e0325b1123
Merge pull request #6141 from hiyouga/hiyouga-patch-1
...
[misc] chore: lint
Former-commit-id: 00031b1a66ade1c2665ce7a069a756cccbcb07f1
2024-11-25 23:02:11 +08:00
hoshi-hiyouga
118ffe50e3
lint
...
Former-commit-id: da9e4ddd26ebd6e7eb266aa0bef7505465a6b119
2024-11-25 22:55:56 +08:00
