Noah
1adb46875f
[data] improve error handling ( #6128 )
...
* sync from upstream
* update
* update
* fix
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 4c7bfebcf1ed90800f5b0de4cf67b3036cb9dc13
2025-02-13 01:39:41 +08:00
hoshi-hiyouga
1679930e00
[breaking change] refactor data pipeline ( #6901 )
...
* refactor data
* rename file
Former-commit-id: 617c8ab467d32be5f7d5c94fa89c0e3d7d1963bc
2025-02-13 00:39:20 +08:00
marko1616
bae934dea3
[trainer] fix llama3.2 vision kto train ( #6904 )
...
Former-commit-id: b7fd1e9c00c77a4c2a0f2f347767d22bd47213f1
2025-02-12 19:09:14 +08:00
hoshi-hiyouga
2e2f6bea07
[data] feat: auto template ( #6905 )
...
* support auto template
* add unittest
Former-commit-id: 2f8b6847f5e199d770e91346dfe205c4b9f1fbb7
2025-02-12 00:22:53 +08:00
hoshi-hiyouga
197aa3baf4
[data] fix ollama template ( #6902 )
...
* fix ollama template
* add meta info
* use half precision
Former-commit-id: e1a7c1242cd1e0a1ca9ee7d04377a53872488126
2025-02-11 22:43:09 +08:00
hoshi-hiyouga
c6be9e242c
[misc] support export ollama modelfile ( #6899 )
...
* support export ollama modelfile
* update config
* add system and num ctx
Former-commit-id: 9184a6e0ed7ff5f632c848f861bfa448c4cd06fc
2025-02-11 19:52:25 +08:00
hoshi-hiyouga
2e954d8fd2
[data] refactor template ( #6896 )
...
Former-commit-id: d1b8aa3835f6e3b2e63cf06e6cadbe760d46f9aa
2025-02-11 17:59:25 +08:00
hoshi-hiyouga
593acca556
[data] refactor mm plugin ( #6895 )
...
* refactor plugin
* lint
Former-commit-id: aca63bfcca02ecd95b57cd8949a50e26a913f716
2025-02-11 16:34:49 +08:00
HJ
188f22d8a7
[data] fix qwen_2_5_vl video processing ( #6868 )
...
* fix qwen_2_5_vl video processing
* Update mm_plugin.py
* Update mm_plugin.py
---------
Co-authored-by: hoshi-hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 9153a7bd832cdae84b63a4d7d1f2b12239e84b61
2025-02-11 16:14:50 +08:00
Zhangchi Feng
5433b318bb
[da'ta] fix minicpmv plugin ( #6890 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
* support inference of minicpmv
* update readme
* support dpo of minicpmv
* update init audio
* update init audio
* [model]fix image process in minicpmo
* fix no mm inputs
Former-commit-id: 764627645abcd353f9130d5dd8c584810b0e0b1b
2025-02-11 13:30:44 +08:00
HJ
fe4f4e9758
[data] fix: sharegpt converter ( #6879 )
...
* fix-sharegpt-format
* fix
---------
Co-authored-by: hoshi-hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 0fb44cb3a5499c8da79e73004adc9d16f792b4b3
2025-02-10 21:59:12 +08:00
hoshi-hiyouga
1bb3d17d9e
[data] fix mllama collator ( #6874 )
...
Former-commit-id: b68199db274a53d5916179e1aaf9722fd94fa2dc
2025-02-09 22:42:25 +08:00
hoshi-hiyouga
b93333685b
[test] align test cases ( #6865 )
...
* align test cases
* fix function formatter
Former-commit-id: f6f3f8d0fc79de6bbad0bf892fc2f6c98c27eb8e
2025-02-09 01:03:49 +08:00
hoshi-hiyouga
fcd0f0480d
[dataset] add openthought ( #6866 )
...
Former-commit-id: 1356f9d8400efaccf677d0b36aaf32a146a09833
2025-02-09 00:53:01 +08:00
hoshi-hiyouga
28037c7834
fix qwen2vl plugin ( #6855 )
...
Former-commit-id: 40048ab77a8b25a91a844800f0f1e880b84548cd
2025-02-08 10:59:10 +08:00
Zhangchi Feng
01915eaf40
[model] support audio ( #6701 )
...
* support qwen2_audio
* improve code
* lint
* fix
* fix
* fix
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 24c78429489809873a1269a735ea5421340b32a2
2025-02-05 04:59:09 +08:00
Yueqi Song
e665e1fed5
[data] allow thought in function call ( #6797 )
...
* Update template.py
* Update template.py
* use formatter
* fix regex
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: a5e943f7bcea6e5840da8570055bf3079a49ae8c
2025-02-05 02:26:23 +08:00
hoshi-hiyouga
1fee69f874
[misc] update license year & fix llama pro ( #6814 )
...
* fix llamapro script
* change year
Former-commit-id: e2dc5b952aa22835d5220ba624f44676138b65ac
2025-02-05 01:53:33 +08:00
Yueqi Song
8504bde893
[data] fix qwen tool template ( #6796 )
...
* Update tool_utils.py
* fix unittest
---------
Co-authored-by: hoshi-hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: dd6b7d203eedbf09458c64654e8d97fec85f08d7
2025-02-05 00:02:00 +08:00
Zhangchi Feng
85f22d01bf
[data] fix minicpmv plugin ( #6801 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
* support inference of minicpmv
* update readme
* support dpo of minicpmv
* update init audio
* update init audio
* [model]fix image process in minicpmo
Former-commit-id: ab9bd068efee861452407cdda08ef014d5ce23d5
2025-02-04 21:20:15 +08:00
hoshi-hiyouga
445d643ef3
[model] add mistral small models ( #6786 )
...
Former-commit-id: 94803d8133fbbadff6d224cb6695feb5434fd4fd
2025-02-01 04:31:38 +08:00
hoshi-hiyouga
e8c1979b79
[model] add qwen2.5 vl models ( #6779 )
...
Former-commit-id: 999c7c8fe0caf6b837a1bdc2c6a24fafec327cd8
2025-01-31 03:00:29 +08:00
hoshi-hiyouga
245de012ca
[webui] improve webui & reasoning mode ( #6778 )
...
Former-commit-id: 45e68b9f092879dda55023ebbcd8cf4660e3045a
2025-01-31 00:09:21 +08:00
hoshi-hiyouga
1efe525df7
[model] support yarn ( #6693 )
...
Former-commit-id: 1f47b6186c267de86cbdbd47ba2adbf1f9db7f39
2025-01-18 13:56:09 +08:00
hoshi-hiyouga
f87c788154
[misc] update mm plugin ( #6691 )
...
Former-commit-id: c0caa7afc60ed3015fe6c263ba3566202ba934f1
2025-01-17 23:04:26 +08:00
Zhangchi Feng
555f17c1ee
[data] Fix minicpmv/o dpo training ( #6657 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
* support inference of minicpmv
* update readme
* support dpo of minicpmv
Former-commit-id: 027942789bf3a28b2506a5730c05c8392ef5c885
2025-01-15 17:30:37 +08:00
hoshi-hiyouga
91433d639c
lint ( #6641 )
...
Former-commit-id: 1278c3e92eeb297e883aab89e2384c1df1d0e910
2025-01-14 18:40:07 +08:00
Haian Huang(深度眸)
864ee06243
Support InternLM3 Dense 8B Model ( #6640 )
...
* support internlm3
* update
* update
* update
* add hint
Former-commit-id: deacc00b1226ca3d53bf7bb1231cf276eaa8296b
2025-01-14 18:07:27 +08:00
hoshi-hiyouga
8f73c75c16
[model] fix mllama any image ( #6637 )
...
* fix mllama any image
* reorder classes
Former-commit-id: 98189c8e4d70bf5f8ee83852a023ed27dfc96900
2025-01-14 16:47:58 +08:00
Zhangchi Feng
201a495154
Support new features of MiniCPM-V ( #6626 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
Former-commit-id: c3fda5046d835ba4542d525b8d89cd12838e9f4c
2025-01-14 00:26:19 +08:00
hoshi-hiyouga
d8cba9464f
[inference] fix stop token for object detection ( #6624 )
...
* fix stop token
* update minicpm data pipeline
* fix npu qlora examples
Former-commit-id: e3e2c8c689c54ebb2af264de808502e5a8ba0f2b
2025-01-13 21:34:20 +08:00
Zhangchi Feng
15bba15725
Fix template name of MiniCPM-V ( #6620 )
...
* fix template name
* tiny fix
Former-commit-id: 3077f20339158564009270edf79c8ef1b10e8b4a
2025-01-13 16:46:48 +08:00
fzc8578
313ce9a576
remove tests
...
Former-commit-id: a019cece8009b0ba8a6b5a309ed5abfe6cb88a75
2025-01-13 15:08:35 +08:00
fzc8578
4741eec2d1
fix style
...
Former-commit-id: 0cc7260a93bf7c65451e376245aa143f9237d7d8
2025-01-13 14:19:38 +08:00
fzc8578
d2afe0c63c
fix system prompt and tests
...
Former-commit-id: cfaa8e4890ad99ec1fb90d9550503d734b5c30b7
2025-01-13 14:18:06 +08:00
fzc8578
bdded9d41a
add some
...
Former-commit-id: 01e9cfd406dc21f387b4f2baa1d61195a841ccb5
2025-01-11 15:03:20 +08:00
fzc8578
e7f928adc4
fix format
...
Former-commit-id: 7b44f3127ef7e91a6bedca0311feb14974914ddf
2025-01-11 01:27:40 +08:00
fzc8578
62c12a133e
add some
...
Former-commit-id: a650e114e907278ece188922467c2514de544eeb
2025-01-11 01:10:24 +08:00
fzc8578
08e8499a98
adapt to new mllm_param
...
Former-commit-id: 291384dea8a5c10f0358a30d124eaf85557548eb
2025-01-11 00:16:34 +08:00
Zhangchi Feng
d5b18ee4a6
Merge branch 'main' into minicpmv
...
Former-commit-id: ed0895a9c13b0ea8a5cace6b060f01d9771816ad
2025-01-11 00:01:36 +08:00
hiyouga
c89d17ab63
refactor mllm param logic
...
Former-commit-id: f6f630a1c96514053176abb12e35a06242e62abd
2025-01-10 15:45:48 +00:00
fzc8578
0fb50f9c88
add some
...
Former-commit-id: 771cc802941cf1953b32e5102c817c6a3090b5ce
2025-01-10 23:29:06 +08:00
fzc8578
bcbe37ff52
add some
...
Former-commit-id: ae1f528df31194fe37a123ba1e5a4cd263a61602
2025-01-10 21:25:32 +08:00
fzc8578
994049380d
fix some
...
Former-commit-id: 15bbcdf8d3265f4154d3937719da5e54a5963355
2025-01-10 20:55:52 +08:00
fzc8578
7138b43873
fix some
...
Former-commit-id: 2ee8ba2f390551af1b865cfa813f5c8b7bbb41c5
2025-01-10 20:27:06 +08:00
Zhangchi Feng
f51ac40f0a
Merge branch 'main' into minicpmv
...
Former-commit-id: fc045d7dd871985d621430b5662cba882188a59c
2025-01-10 20:12:07 +08:00
fzc8578
165fe8e219
add some
...
Former-commit-id: 096a6cb67a7dfd14a6e339d96baab78c12d36a87
2025-01-10 20:01:22 +08:00
hiyouga
b471def13d
improve template, add phi4 model
...
Former-commit-id: ae16ea755d581a5a288fb55f12481215f369b255
2025-01-09 18:27:54 +00:00
hiyouga
da542fad18
imporve log
...
Former-commit-id: 47e17dd689840ca9b3c5f34448e5f80265336cca
2025-01-08 09:56:10 +00:00
fzc8578
b9eeaa9706
add some
...
Former-commit-id: 785cc70ff205f5962c3ca67f453589e4a471ba8c
2025-01-06 19:32:39 +08:00
