577 Commits

Author SHA1 Message Date
hiyouga
65699c29d4 fix vllm
Former-commit-id: 13ee1f5cec815590c5d290f0aca264e6d16ddd5d
2024-11-25 00:07:24 +08:00
hiyouga
dcc67ac1a5 fix qwen2vl vllm infer
Former-commit-id: fa50fc470e46010214f0e509ef75a6c167caac7f
2024-11-24 23:27:24 +08:00
hiyouga
253752ccca add forbidden modules
Former-commit-id: df477370dc67315effac1a8f48068c4e9c4067a5
2024-11-23 18:34:15 +00:00
hiyouga
e99031daa4 fix inputs
Former-commit-id: 446441fdb020b5a102480251cb8536dd8b3f8f99
2024-11-23 18:26:02 +00:00
marko1616
23fc0c863e Linter.
Former-commit-id: b1e43e56db4d898c0d6d27b703b82a1eb9af70c2
2024-11-23 16:09:04 +00:00
marko1616
3295519099 Tiny fix.
Former-commit-id: 8372c5e3771c42f225d7bd80a758af920f80e893
2024-11-23 16:09:01 +00:00
marko1616
20faaf3418 Support llama3.2vl.
Former-commit-id: 3f2c056253c651e8e614c787e2045f4232e82666
2024-11-23 16:07:35 +00:00
hoshi-hiyouga
1eaaa85980 do not split save_cmd ret value
Former-commit-id: d20b97e7e9514e68c33d0428512fd21b548c1131
2024-11-21 22:30:23 +08:00
superboy-zjc
e272f87849 [patch] Patch remote OS command injection vulnerability
Former-commit-id: aa6a174d6822340022433c5ba38182b4932adecb
2024-11-21 01:52:12 -05:00
hoshi-hiyouga
302e4e22bf Merge pull request #6078 from wtmlon/support-efficient-tokens-calculation
support effective tokens calculation on sft/dpo

Former-commit-id: bd639a137e6f46e1a0005cc91572f5f1ec894f74
2024-11-20 13:43:15 +08:00
Ting
e27a0c3d53 code refactor
Former-commit-id: 40627c601efc9f144a227dded8c6b40babff4e8b
2024-11-19 20:33:18 +08:00
Ting
32656bc50d update
Former-commit-id: f566ecc8d1f04615351acbe4f8480b75b2daed42
2024-11-19 19:12:10 +08:00
Ting
bf2b8df540 update
Former-commit-id: ef6e14550dd76810285cee9c268590d1d9423e54
2024-11-19 19:10:07 +08:00
Ting
7ad5b5c088 support efficient tokens calculation on sft/dpo
Former-commit-id: b9f00286d8a017ed9fd2876986da3b4d7034ef07
2024-11-19 17:15:47 +08:00
hoshi-hiyouga
9815d1712c fix #6050
Former-commit-id: dc828218726704ff0453a2d13535663ac6ad7833
2024-11-16 16:11:16 +08:00
hiyouga
d4e0010027 add qwen-coder and opencoder
Former-commit-id: 431ac4892cdddba802a02b285031a797e278d0eb
2024-11-15 21:48:38 +08:00
hoshi-hiyouga
9c394f11ef fix #5988
Former-commit-id: 8d70edf39bce7cb7be272f0bd2820306adf093ad
2024-11-11 13:57:14 +08:00
hiyouga
0d18cca0db add vllm config
Former-commit-id: 58ab4579dc81a1dcea2bf5938ba3f3116cecfc76
2024-11-10 21:28:18 +08:00
hoshi-hiyouga
1e00de38cb Merge pull request #5971 from hiyouga/hiyouga/fix_webui
[webui] fix extra args

Former-commit-id: adc5849ce75cc8eda800c73c59b1f44d3108e992
2024-11-09 00:25:24 +08:00
hiyouga
d7e35ddcaa fix extra args
Former-commit-id: eb294fed1319e97657b54b9aa479d87734c711dc
2024-11-09 00:24:27 +08:00
hiyouga
97f4451912 fix #5966
Former-commit-id: 8f3a32286ebcfb3234e3981db2292dd165b1568d
2024-11-08 23:49:16 +08:00
hiyouga
1598e5d355 add image input type
Former-commit-id: ffa39ba3db0dbfd375cdf20b9f3cbecd359be1a1
2024-11-04 08:27:20 +00:00
hiyouga
2360d63ebc fix chat engines
Former-commit-id: 8c88065c38d0e7dd60e06e27c7f921f7931471ee
2024-11-04 08:18:12 +00:00
hiyouga
3730fc046f update datasets version
Former-commit-id: c5fae465ec8cbc30f9e91e6c32b88e74c805874a
2024-11-04 07:52:26 +00:00
hoshi-hiyouga
e3fb3c313c Merge pull request #5914 from hiyouga/hiyouga/dev_read
[misc] update readme

Former-commit-id: 04c10d2e80b7f7e516eba67fea420498a1238bb5
2024-11-02 21:44:10 +08:00
hiyouga
feb547aa99 fix phi3 template
Former-commit-id: 2e843d989e946725e2dc096d6540531eaaf5b782
2024-11-02 21:31:23 +08:00
hoshi-hiyouga
8c2b7aa1ab update template
Former-commit-id: 478cbb1aa72f218df37b5a4686db2248ad2605dd
2024-11-02 21:21:22 +08:00
hoshi-hiyouga
d99e164cad Merge branch 'main' into main
Former-commit-id: 5f14910910154ba569435e7e68acbd6c30f79e80
2024-11-02 21:20:27 +08:00
hiyouga
604f4005c9 fix webchat
Former-commit-id: ba10d54bf93c54ab8f691ea53bff850e3595a1a6
2024-11-02 21:04:18 +08:00
hiyouga
c2766af6f4 fix dpo metrics
Former-commit-id: 4270f7dfb9a12471c91f6c03dce7ca6fd88566e1
2024-11-02 20:59:01 +08:00
hoshi-hiyouga
6f79974e8b Merge pull request #5910 from Cuiyn/index
Support Index series models.

Former-commit-id: c58cc22d06eb1a466ad92601ceb74c9bae6abb51
2024-11-02 20:16:54 +08:00
Cuiyn
51177c933a fix: rename to Index-1.9B-Charater-Chat and Index-1.9B-Chat-32K
Former-commit-id: ecca9db66ba1f0fe903a9e02eca317c0bbb28a5f
2024-11-02 20:04:14 +08:00
hiyouga
e83cb17f97 support rank0 logger
Former-commit-id: c38aa29336f286266553da4909a7267d7ef21f37
2024-11-02 18:31:04 +08:00
Cuiyn
7806bde8ad Add support for Index
Former-commit-id: a15a69ab4417c6f3273c874cf7ee2c34a5a64141
2024-11-02 13:45:27 +08:00
hoshi-hiyouga
4b2c47fcae Merge pull request #5909 from hiyouga/hiyouga/dev2
[data] support auto convert for single image, add image_dir argument

Former-commit-id: bd08b8c441c47076faa03cc1efde21b22f14f058
2024-11-02 13:43:04 +08:00
hoshi-hiyouga
33e8bfc3ae Merge pull request #5907 from hiyouga/hiyouga/dev
[data] fix template replace behavior

Former-commit-id: 8408339d8388a610a4e5fe303086d7536d1e72d5
2024-11-02 13:42:53 +08:00
hiyouga
ac677205c9 fix #5904
Former-commit-id: bfe1abd7afe4595135b568783753d064cb6e0b28
2024-11-02 13:08:15 +08:00
hiyouga
7fa46a24df fix #5883
Former-commit-id: 24da9f59b0bf4874506bbf1ec214f3d5ca43d943
2024-11-02 13:06:34 +08:00
hiyouga
3f7c874594 update tests
Former-commit-id: 93d3b8f43faf4a81b809d2f7d897e39bdb5475c3
2024-11-02 12:41:44 +08:00
hiyouga
2eba98e152 add examples
Former-commit-id: e824b715ad4bf885241b245b12d75563adab2e26
2024-11-01 08:41:54 +00:00
hiyouga
8ecc12ee2a support multiimage inference
Former-commit-id: e80a4819274d46ac9e85db7469dc59d7c4e323c7
2024-11-01 07:25:20 +00:00
hiyouga
584ce3a105 fix incorrect loss value for vlms
Former-commit-id: 30567a1487727473950104718e626ff660f10cbb
2024-10-30 08:56:46 +00:00
hiyouga
1b02915d19 tiny fix
Former-commit-id: 0c22da4f1cc710b471f6d511d50ce878521173ca
2024-10-30 08:56:29 +00:00
hiyouga
aba4268607 support extra args in llamaboard
Former-commit-id: 86e5affc0d5f5682a20b28d27f80e0015b36fdd9
2024-10-30 08:55:54 +00:00
hoshi-hiyouga
5142faca8f Merge pull request #5581 from Kuangdd01/pixtral-patch
[WIP] Support Pixtral-12B

Former-commit-id: 9009a467e621a17ad9fa25bb30fb9ac9ee15df97
2024-10-29 22:29:10 +08:00
hoshi-hiyouga
15786539d7 fix bug
Former-commit-id: bb0afda8fbb7d3556c1742bccf0b33ee3bf0f0b6
2024-10-29 22:19:04 +08:00
hoshi-hiyouga
49054329d0 Update mm_plugin.py
Former-commit-id: 049c554aee25cf1e29bee88dfb21381b3a4a2947
2024-10-29 22:16:22 +08:00
hoshi-hiyouga
54961946ac Update template.py
Former-commit-id: c3239ba6b24481b092e86ce94a6337fba18c25c0
2024-10-29 22:11:21 +08:00
hoshi-hiyouga
2179b91acb Update visual.py
Former-commit-id: 0baa7735f64cbef9bd90e1db485c120b4c1c88bd
2024-10-29 22:10:29 +08:00
hoshi-hiyouga
fb8f35558a Update collator.py
Former-commit-id: f745c4b28f532c7084d4b8522c972e735729ecee
2024-10-29 22:03:42 +08:00