Yaser Afshar
6f1c8dacea
Add missing key to init_kwargs
...
Former-commit-id: 03fc4621dad132164596a58d3e8693787b7e1aca
2024-12-17 12:34:05 +00:00
Yaser Afshar
8881237475
Add trust_remote_code parameter and remove True
...
- Introduced a new model parameter `trust_remote_code`
- Set the default value of `trust_remote_code` to `False`
to enhance security
Former-commit-id: 4bf23f406cf5235c16f9f8139850c53354901814
2024-12-17 12:25:12 +00:00
hoshi-hiyouga
3d3324be5c
Merge pull request #6364 from hiyouga/hiyouga/control_reenterent_gc
...
[model] support non-reenterent-gc
Former-commit-id: a8a13cb360980bb4acd493e33ed405e07460fe73
2024-12-17 19:58:36 +08:00
hiyouga
4196d5b4d6
support non-reenterent-gc & fix #6358
...
Former-commit-id: 20446141e408885eb36d512bfb2dfb62bbc0c20d
2024-12-17 11:41:59 +00:00
hoshi-hiyouga
101c95ce65
Merge pull request #6363 from hiyouga/hiyouga/control_skip_eos
...
[infer] support control eos
Former-commit-id: 963640cff370be9f2fab649c88a120a645e6992e
2024-12-17 19:35:40 +08:00
hiyouga
19ebc0e7a2
support control eos, fix #6345
...
Former-commit-id: cb0f8399356bf372f3b7963f2565c3d504be0923
2024-12-17 10:42:05 +00:00
hiyouga
d670d62a66
generalized packing & fix #6343
...
Former-commit-id: 3b1e4194616cacd5c24f08b328e31a008bddcf29
2024-12-17 10:26:19 +00:00
hiyouga
aacd9642f5
fix #6348
...
Former-commit-id: 83e552320909f4775377889f1512994b7e638a7e
2024-12-17 10:06:46 +00:00
hiyouga
fb22651faf
fix mrope
...
Former-commit-id: 55bee1d333549ca19858b3f5c1b7b86926e5fb09
2024-12-12 15:08:17 +00:00
hiyouga
bac2c64f87
support qwen2vl train proj only
...
Former-commit-id: 0e949ef03455726e907c6f1039e93ebe480c897a
2024-12-05 10:37:42 +00:00
hiyouga
bbd432415d
support qwen2vl vllm infer
...
Former-commit-id: 03ddd2555fb97488cd4daab11e8b672d36150c5a
2024-12-05 10:17:26 +00:00
hiyouga
86e4fab0d5
fix scripts
...
Former-commit-id: f94f55d20283298cb7d90d0573992a62df414a8f
2024-12-05 03:47:32 +00:00
hoshi-hiyouga
ff3e40e4a5
Merge pull request #6160 from village-way/pr_dataloader
...
fix:tokenized_path not None and load_from_disk return Dataset Trigger…
Former-commit-id: 63de20970c8062aeebed5f366f1675beb12e05bf
2024-12-04 22:18:19 +08:00
hoshi-hiyouga
ea830cad0c
lint
...
Former-commit-id: 191ccc585399ad4c6c2c4f280b144b2c0a4869f3
2024-12-04 22:08:27 +08:00
hiyouga
c1768cfb14
support batch infer in vllm
...
Former-commit-id: 3ef5ed3b9a44eed2f7e3ff221dfc343d0a97c0b5
2024-12-04 13:50:00 +00:00
hoshi-hiyouga
41a7e128b6
Merge pull request #6170 from hykilpikonna/main
...
[+] Show the hostname in webui title
Former-commit-id: 1cb2f9da317a8db8f45e887ab57cdfdc0e8b9412
2024-12-04 18:07:29 +08:00
hiyouga
2f09c34980
fix vlm zero3 training
...
Former-commit-id: 86fe7fe71b51077310357b7b1895522258f9bc7a
2024-12-04 09:40:39 +00:00
Azalea
b6bc17f730
[U] Compute hostname differently
...
Former-commit-id: fbc735972af6facdaba169603a4c77e613b2e8d7
2024-11-28 22:23:41 -05:00
hiyouga
ed86f621a0
add qwq
...
Former-commit-id: acad977356a7f2e729eb6f2cb919a416b18f8add
2024-11-28 08:50:57 +00:00
Azalea
c6a3175bbf
[+] Show the hostname
...
Former-commit-id: 410847656a760fe4c2c310b0d770072392d7aefb
2024-11-28 12:25:02 +08:00
wangdepeng
452291417d
fix:tokenized_path not None and load_from_disk return Dataset Trigger stuck
...
Former-commit-id: cbf9da35728daaf98d92e699e891e334c74af1e5
2024-11-27 16:44:42 +08:00
hiyouga
877e2ea791
fix dataset
...
Former-commit-id: d4a2d299414984a4043d30034c5c95e2d717a49e
2024-11-27 06:27:44 +00:00
hiyouga
6ea42d5b63
add skywork o1
...
Former-commit-id: 272a6fe972de926e5841c1570995f4e6fed9f28d
2024-11-27 05:51:59 +00:00
hiyouga
8332c85f37
add marco-o1 and openo1 dataset
...
Former-commit-id: 51d49e075470951f109bcdde136203f972450c2e
2024-11-27 04:20:23 +00:00
hoshi-hiyouga
fcf1a3df62
Merge pull request #6152 from hiyouga/hiyouga/add_num_proc_in_data_load
...
[data] add num_proc in load_dataset
Former-commit-id: d8258ba7e792d5f17ae80d5e8b303e8fa820f162
2024-11-27 00:16:15 +08:00
hiyouga
0b08d5882a
fix #6149
...
Former-commit-id: b581b272793314a9602f4dc2fb646a988a6249df
2024-11-26 16:03:02 +00:00
hiyouga
62eeafaba6
fix mllama cross_mask
...
Former-commit-id: c33967308bebd99489d28bd5a879525cf304c1f9
2024-11-26 15:56:58 +00:00
hoshi-hiyouga
e8083f8f3f
lint
...
Former-commit-id: 57c3cf1f498d5ffafdc8c06e0f8713f8ff77de81
2024-11-25 22:55:56 +08:00
hoshi-hiyouga
c8b01b41ac
fix #6139
...
Former-commit-id: a4e9552b9ade6ebb22d782f0412003279ddca23c
2024-11-25 22:22:06 +08:00
hoshi-hiyouga
e3066d1489
fix visual patch
...
Former-commit-id: ac51fa37cc23518b30a6123e188964dce39be82f
2024-11-25 20:06:06 +08:00
hoshi-hiyouga
487e3f2507
fix #6136
...
Former-commit-id: b84e5d91a070c473ea820c379bf9b5abbca6df2c
2024-11-25 19:43:42 +08:00
hiyouga
5bec82ca9d
set dev version
...
Former-commit-id: a0aea74100a9505664023f6a46fc290e332dfa40
2024-11-25 01:36:49 +08:00
hoshi-hiyouga
57354fc990
Merge pull request #6124 from hiyouga/hiyouga/release
...
[release] release v0.9.1
Former-commit-id: f61cdd99fd282612884c92d36e111ad46b4e0d00
2024-11-25 00:20:02 +08:00
hoshi-hiyouga
89f240805c
Merge pull request #6126 from hiyouga/hiyouga/fix_vllm
...
[inference] fix vllm
Former-commit-id: c5025c3ee6e67e62724cc3f34fbf8aa9968590f5
2024-11-25 00:19:54 +08:00
hiyouga
ae9f338bf7
fix vllm
...
Former-commit-id: 9ce0e4b07e3733c015137bc93c7e6d53bf25b08e
2024-11-25 00:07:24 +08:00
hiyouga
bf44f76dc7
fix cli
...
Former-commit-id: 9338c287cc15c0cad8d5ddbdadfb6f64d383c034
2024-11-24 23:56:21 +08:00
hiyouga
c18581f0a4
release v0.9.1
...
Former-commit-id: a134ad42c65dc4d72e3083c932ddfaaa687c513d
2024-11-24 23:48:41 +08:00
hiyouga
7bc03ac986
fix qwen2vl vllm infer
...
Former-commit-id: 3ac98847fdc23129912c8994ed19a8c66fe00b8c
2024-11-24 23:27:24 +08:00
hiyouga
bc4421eeef
add forbidden modules
...
Former-commit-id: c9f4d051d0eca7515bab201afdef17f1ac1b3cb9
2024-11-23 18:34:15 +00:00
hiyouga
5003820a6a
fix inputs
...
Former-commit-id: 7d535bb8cdf7e81edda81152e63c8cfe6c9dcc9f
2024-11-23 18:26:02 +00:00
marko1616
cd2485f28d
Linter.
...
Former-commit-id: 719d124f65ebb18ba0a1212751da9909160fb6f1
2024-11-23 16:09:04 +00:00
marko1616
918a367378
Tiny fix.
...
Former-commit-id: 4c1cef12d812832eed58b5da562ba083104756d3
2024-11-23 16:09:01 +00:00
marko1616
3d35aeca72
Support llama3.2vl.
...
Former-commit-id: 664229d7d1f7994e1ae68c5d197ab81f081bcd2e
2024-11-23 16:07:35 +00:00
hoshi-hiyouga
b852c895cf
do not split save_cmd ret value
...
Former-commit-id: 1e312072fb4a9f472e2d3fa7e6b4fb0aec00b566
2024-11-21 22:30:23 +08:00
superboy-zjc
aaa7ed8712
[patch] Patch remote OS command injection vulnerability
...
Former-commit-id: 4678ceea4ce334a8289caf87d86047e67c67c603
2024-11-21 01:52:12 -05:00
hoshi-hiyouga
205aca5b03
Merge pull request #6078 from wtmlon/support-efficient-tokens-calculation
...
support effective tokens calculation on sft/dpo
Former-commit-id: d0510e6d49b43c5ffadd8af653c3bdecc1582417
2024-11-20 13:43:15 +08:00
Ting
87b1f851f1
code refactor
...
Former-commit-id: ee3f85aa9677d0aeecb3bc396530d2cd7c50dce5
2024-11-19 20:33:18 +08:00
Ting
fca814b30d
update
...
Former-commit-id: 516ed0ea5fed8c74fe3669a7e85dd89b5a0ec3c2
2024-11-19 19:12:10 +08:00
Ting
a20c2b6ecf
update
...
Former-commit-id: a3e8ca53e654136242197a2da872cc0e5cf67880
2024-11-19 19:10:07 +08:00
Ting
fee94e1c54
support efficient tokens calculation on sft/dpo
...
Former-commit-id: b157d5cccdeb42412b8b440d25d5bdfa8a50be68
2024-11-19 17:15:47 +08:00
