222 Commits

Author SHA1 Message Date
wangdepeng
452291417d fix:tokenized_path not None and load_from_disk return Dataset Trigger stuck
Former-commit-id: cbf9da35728daaf98d92e699e891e334c74af1e5
2024-11-27 16:44:42 +08:00
hiyouga
877e2ea791 fix dataset
Former-commit-id: d4a2d299414984a4043d30034c5c95e2d717a49e
2024-11-27 06:27:44 +00:00
hiyouga
6ea42d5b63 add skywork o1
Former-commit-id: 272a6fe972de926e5841c1570995f4e6fed9f28d
2024-11-27 05:51:59 +00:00
hiyouga
8332c85f37 add marco-o1 and openo1 dataset
Former-commit-id: 51d49e075470951f109bcdde136203f972450c2e
2024-11-27 04:20:23 +00:00
hoshi-hiyouga
fcf1a3df62 Merge pull request #6152 from hiyouga/hiyouga/add_num_proc_in_data_load
[data] add num_proc in load_dataset

Former-commit-id: d8258ba7e792d5f17ae80d5e8b303e8fa820f162
2024-11-27 00:16:15 +08:00
hiyouga
0b08d5882a fix #6149
Former-commit-id: b581b272793314a9602f4dc2fb646a988a6249df
2024-11-26 16:03:02 +00:00
hiyouga
62eeafaba6 fix mllama cross_mask
Former-commit-id: c33967308bebd99489d28bd5a879525cf304c1f9
2024-11-26 15:56:58 +00:00
hoshi-hiyouga
e8083f8f3f lint
Former-commit-id: 57c3cf1f498d5ffafdc8c06e0f8713f8ff77de81
2024-11-25 22:55:56 +08:00
hoshi-hiyouga
c8b01b41ac fix #6139
Former-commit-id: a4e9552b9ade6ebb22d782f0412003279ddca23c
2024-11-25 22:22:06 +08:00
hiyouga
ae9f338bf7 fix vllm
Former-commit-id: 9ce0e4b07e3733c015137bc93c7e6d53bf25b08e
2024-11-25 00:07:24 +08:00
hiyouga
5003820a6a fix inputs
Former-commit-id: 7d535bb8cdf7e81edda81152e63c8cfe6c9dcc9f
2024-11-23 18:26:02 +00:00
marko1616
918a367378 Tiny fix.
Former-commit-id: 4c1cef12d812832eed58b5da562ba083104756d3
2024-11-23 16:09:01 +00:00
marko1616
3d35aeca72 Support llama3.2vl.
Former-commit-id: 664229d7d1f7994e1ae68c5d197ab81f081bcd2e
2024-11-23 16:07:35 +00:00
hiyouga
0a0288a286 add qwen-coder and opencoder
Former-commit-id: 9669a42704cd40bdfc76ca278cc6a562549bc27d
2024-11-15 21:48:38 +08:00
hiyouga
e2fa961302 add image input type
Former-commit-id: 6fe260e35ff12662b72f26ec9df44e87b9693551
2024-11-04 08:27:20 +00:00
hoshi-hiyouga
c7c558562e update template
Former-commit-id: 3559ef6115a831dcd1adf7210995ffd62890cff6
2024-11-02 21:21:22 +08:00
hoshi-hiyouga
c2ea5fb618 Merge branch 'main' into main
Former-commit-id: 154f504fc2cebaae2b58c0121d6d8d8016db1bb2
2024-11-02 21:20:27 +08:00
hoshi-hiyouga
80f634cc95 Merge pull request #5910 from Cuiyn/index
Support Index series models.

Former-commit-id: b74d9fa8efeb4f52ba0e20538ad90c8b40492e29
2024-11-02 20:16:54 +08:00
hiyouga
093eda2ad6 support rank0 logger
Former-commit-id: 84528eabe560091bfd866b6a0ca864085af7529b
2024-11-02 18:31:04 +08:00
Cuiyn
dbaf621f57 Add support for Index
Former-commit-id: 4e6dba16ca1755235d2ae117b53b68c5ae2f239a
2024-11-02 13:45:27 +08:00
hoshi-hiyouga
ceb701c2d4 Merge pull request #5909 from hiyouga/hiyouga/dev2
[data] support auto convert for single image, add image_dir argument

Former-commit-id: ced43fa0c84f7d0792694721d2c5e572c0d0e718
2024-11-02 13:43:04 +08:00
hiyouga
fa2386e73c fix #5904
Former-commit-id: 079ebe038b11f36a11681dc8688f8ea48bccf324
2024-11-02 13:08:15 +08:00
hiyouga
e0045e8386 fix #5883
Former-commit-id: 73b93caa9ac16ffd8d3faae24d16210d85ae9754
2024-11-02 13:06:34 +08:00
hiyouga
8cea5cd967 support multiimage inference
Former-commit-id: 8083e4607549e805eb308c4e93c8aa256202f438
2024-11-01 07:25:20 +00:00
hiyouga
03213984ec tiny fix
Former-commit-id: b8f4b145506851cf5488cd8551a04d1c7603019b
2024-10-30 08:56:29 +00:00
hoshi-hiyouga
f6b06d0c6f Update mm_plugin.py
Former-commit-id: 830315cb438e75b589017fd57f70d0a513780a53
2024-10-29 22:16:22 +08:00
hoshi-hiyouga
1047217f78 Update template.py
Former-commit-id: 99a01547ca31adade1c48feae5796e06b73d387c
2024-10-29 22:11:21 +08:00
hoshi-hiyouga
58fb24ce41 Update collator.py
Former-commit-id: 941fa8a0d9c3a9106ad0af6e776db7e57f69548f
2024-10-29 22:03:42 +08:00
Kingsley
42aa9c65be Merge branch 'hiyouga:main' into pixtral-patch
Former-commit-id: 438302edfdb66b6397266b8b17ac66f60a89300c
2024-10-29 21:01:25 +08:00
hiyouga
248d5daaff use pre-commit
Former-commit-id: 7cfede95df22a9ff236788f04159b6b16b8d04bb
2024-10-29 09:07:46 +00:00
KUANGDD
30d7f6a22e rm comment
Former-commit-id: 80b58eaaec1996571d24b2dc2b73859cc28911a1
2024-10-23 15:50:59 +08:00
KUANGDD
5440ebbae6 rm useless code
Former-commit-id: 2dc337a49a8646ce916981b2914718e7472b5946
2024-10-23 15:38:11 +08:00
KUANGDD
64ac6ca396 rm import torch
Former-commit-id: 561a0f8155afca20ac699e124320b0eaef6dac07
2024-10-23 15:32:33 +08:00
KUANGDD
d0889012c2 modify style & little change
Former-commit-id: c988477d14dc656450d5fec31895781b7f9f7dce
2024-10-23 15:24:07 +08:00
KUANGDD
4845a76535 fix bug for webui infer
Former-commit-id: 17768832908cc59ab64ed72522b2954c575ce21d
2024-10-16 01:09:33 +08:00
Kingsley
67645c0db8 Merge branch 'pixtral-patch' of https://github.com/Kuangdd01/LLaMA-Factory-X into pixtral-patch
Former-commit-id: 995eae4333f4346734d76f7d18cfffb5147e2f7b
2024-10-15 17:09:56 +08:00
Kingsley
f463b3f038 add extra test for pixtral mm_input
Former-commit-id: c706ec8a5dbd3c72ab15a709668624c0c7bbd8ce
2024-10-15 17:09:24 +08:00
BUAADreamer
01defc2779 tiny fix [skip ci]
Former-commit-id: 95f968eec2628cb26b3c4f4d4e81a9536e23cc31
2024-10-15 13:53:33 +08:00
KUANGDD
3693d7b571 plugin test & check
Former-commit-id: 76c7c8c5a729b8b43e3a31efc44f2c9c2678bf3d
2024-10-15 12:12:46 +08:00
KUANGDD
2b3b0473cd required transformers version
Former-commit-id: d9915db327a038c93b5e3421c90b1f218fb23f92
2024-10-14 21:11:09 +08:00
Kingsley
9d929897ce remove bs condition
Former-commit-id: bf3520178ab66058c62a9cf31b42f36a9d88ce20
2024-10-14 16:55:59 +08:00
Kingsley
313a5e1494 Merge branch 'hiyouga:main' into pixtral-patch
Former-commit-id: 28696e2f945a9f55e4ca9e9dc5ebd8af9df45d8b
2024-10-13 17:42:02 +08:00
hiyouga
c7efc7f2ed tiny fix
Former-commit-id: 1fe424323b212094856f423351dc2a15774d39c3
2024-10-11 23:51:54 +08:00
huniu20
a6951db970 bugs fixed
Former-commit-id: 5457ba7512d70564ea784b9ec6bdb86cfd2d7e3d
2024-10-11 19:56:13 +08:00
huniu20
5b15ca0b0b add om_hub_token argument
Former-commit-id: b3214e69d32067a1c22dbd60c2cde1545ba75b19
2024-10-10 17:16:46 +08:00
huniu20
aad79127e6 1. add model and dataset info to support webui
Former-commit-id: 92f6226f3fecbd9af744a7232dda2c68b2bb0d86
2024-10-10 16:46:34 +08:00
huniu20
c42dcab32b 1. add modelers hub support
Former-commit-id: 14678eb444d8181176745d18d4a6865fd6860f58
2024-10-09 17:21:37 +08:00
Kingsley
be519c84d9 Merge branch 'hiyouga:main' into pixtral-patch
Former-commit-id: 2076d00dfbe1279a91207157fd6d9a118427626a
2024-10-08 21:04:08 +08:00
hiyouga
b2dc6dc59a tiny fix
Former-commit-id: d8ddd07c2ed14d871fb25743c20265fc99e3e221
2024-10-08 17:48:56 +08:00
Kingsley
32b5364051 Merge branch 'hiyouga:main' into pixtral-patch
Former-commit-id: df0baeaa3fd093433d92b7921d3a57d88061d6d4
2024-09-30 19:33:29 +08:00