hoshi-hiyouga
8fda09c195
do not split save_cmd ret value
...
Former-commit-id: 1e312072fb4a9f472e2d3fa7e6b4fb0aec00b566
2024-11-21 22:30:23 +08:00
superboy-zjc
153bd60d08
[patch] Patch remote OS command injection vulnerability
...
Former-commit-id: 4678ceea4ce334a8289caf87d86047e67c67c603
2024-11-21 01:52:12 -05:00
hoshi-hiyouga
874cdc101b
Merge pull request #6078 from wtmlon/support-efficient-tokens-calculation
...
support effective tokens calculation on sft/dpo
Former-commit-id: d0510e6d49b43c5ffadd8af653c3bdecc1582417
2024-11-20 13:43:15 +08:00
Ting
1c8514dc8d
code refactor
...
Former-commit-id: ee3f85aa9677d0aeecb3bc396530d2cd7c50dce5
2024-11-19 20:33:18 +08:00
Ting
befe4310f0
update
...
Former-commit-id: 516ed0ea5fed8c74fe3669a7e85dd89b5a0ec3c2
2024-11-19 19:12:10 +08:00
Ting
8868bb1ed3
update
...
Former-commit-id: a3e8ca53e654136242197a2da872cc0e5cf67880
2024-11-19 19:10:07 +08:00
Ting
a9ae5fe5e5
support efficient tokens calculation on sft/dpo
...
Former-commit-id: b157d5cccdeb42412b8b440d25d5bdfa8a50be68
2024-11-19 17:15:47 +08:00
hoshi-hiyouga
7a5ca3c120
Merge pull request #6065 from hiyouga/hiyouga-patch-1
...
[misc] fix dep package version
Former-commit-id: 34a09e6cd1a8b1c2acddf837f1c787978bc526f5
2024-11-18 21:13:59 +08:00
hoshi-hiyouga
a86b980dc3
fix #6061
...
Former-commit-id: 4eb0b6763f0a1b3cde89bd5c69760178bb35d303
2024-11-18 20:56:44 +08:00
hoshi-hiyouga
f0c8465299
Merge pull request #6052 from hiyouga/hiyouga-patch-1
...
[trainer] fix DPO metrics
Former-commit-id: 94add263fe874d2be1b37110faf5da7a5096df6d
2024-11-16 16:20:12 +08:00
hoshi-hiyouga
e7b825121e
fix #6050
...
Former-commit-id: 028ea3d9b4fa4ab74a969ac80e61a449d6c15e74
2024-11-16 16:11:16 +08:00
hoshi-hiyouga
3e9b110b24
Merge pull request #6046 from hiyouga/hiyouga/add_code_model
...
[model] add qwen-coder and opencoder
Former-commit-id: 5b485671aee8dd2f775371d0b9ff3d0d043159f3
2024-11-15 21:58:03 +08:00
hiyouga
a08f57cae8
add qwen-coder and opencoder
...
Former-commit-id: 9669a42704cd40bdfc76ca278cc6a562549bc27d
2024-11-15 21:48:38 +08:00
XYZliang
e25015a11e
Increase shm_size to 16GB in docker-compose.yml to optimize shared memory allocation for large-scale model fine-tuning tasks.
...
This pull request increases the shm_size parameter in docker-compose.yml to 16GB. The goal is to enhance the LLaMA-Factory framework’s performance for large model fine-tuning tasks by providing sufficient shared memory for efficient data loading and parallel processing.
This PR also addresses the issues discussed in [this comment](https://github.com/hiyouga/LLaMA-Factory/issues/4316#issuecomment-2466270708 ) regarding Shared Memory Limit error.
Former-commit-id: de2616d103b4bdc2458874068b1a223c7de82b4e
2024-11-13 10:13:59 +08:00
hoshi-hiyouga
f297ab3459
Merge pull request #5990 from hiyouga/hiyouga/dev_vllm
...
[generate] fix vllm config args
Former-commit-id: ee0745022bd7484f4f2e6b183088f55d5e60c085
2024-11-11 14:10:35 +08:00
hoshi-hiyouga
39edbec91a
fix #5988
...
Former-commit-id: 9e08e206a8ea9926768b0f1d5ff9d7e3e216c269
2024-11-11 13:57:14 +08:00
hoshi-hiyouga
24073f5966
Merge pull request #5982 from hiyouga/hiyouga/vllm_args
...
[args] add vllm config
Former-commit-id: 07d3de5c8376d3c4147411ec603da4254885d2d7
2024-11-10 21:37:18 +08:00
hiyouga
85343ddf47
add vllm config
...
Former-commit-id: 95365f0ce4f362bde7de8b679b54b548d7055bfb
2024-11-10 21:28:18 +08:00
hoshi-hiyouga
9220b11fa5
Merge pull request #5973 from JJJJerry/fix_vllm_generate
...
fix VllmEngine: 将inputs参数替换为prompt
Former-commit-id: d3271416a316e6b92aea3026f6941f6967215a7b
2024-11-10 21:04:38 +08:00
hoshi-hiyouga
666f34918c
Update vllm_engine.py
...
Former-commit-id: 5638fae81c180b7d91eb6aebe6629640beb217d8
2024-11-10 20:57:00 +08:00
JJJJerry
35c1b0b399
fix VllmEngine: 将inputs参数替换为prompt
...
Former-commit-id: 5affb1d20921afd3fe48802ff80785e412e2e3aa
2024-11-09 11:45:59 +08:00
hoshi-hiyouga
8e5a677ca2
Merge pull request #5971 from hiyouga/hiyouga/fix_webui
...
[webui] fix extra args
Former-commit-id: d04e21d69e60ab4a350e70da7d1abbf11cfeed0e
2024-11-09 00:25:24 +08:00
hiyouga
7c39b15c7e
fix extra args
...
Former-commit-id: 2c98a1bc3d885170f8298872c2ea2e24427fb447
2024-11-09 00:24:27 +08:00
hoshi-hiyouga
3e439be501
Merge pull request #5970 from hiyouga/hiyouga/fix_beam
...
[generation] fix vllm v0.6.3
Former-commit-id: 571d4538568272fd59cc5621e56113329c857546
2024-11-08 23:58:15 +08:00
hiyouga
91ceece9a6
fix #5966
...
Former-commit-id: a9a99b545609083533cca1fd1e5480c60ea68750
2024-11-08 23:49:16 +08:00
hoshi-hiyouga
e951f641c1
Merge pull request #5927 from hiyouga/hiyouga/dev_fixmmchat
...
[fix] chat engines
Former-commit-id: e9c22e2d089927eee3bce052bbf7d6502d0ac544
2024-11-04 16:36:23 +08:00
hiyouga
d0435b50c9
add image input type
...
Former-commit-id: 6fe260e35ff12662b72f26ec9df44e87b9693551
2024-11-04 08:27:20 +00:00
hiyouga
8f6024b373
fix chat engines
...
Former-commit-id: 3a220b7992d265c77d9a1a406ef86eefbc699cfe
2024-11-04 08:18:12 +00:00
hoshi-hiyouga
007f1a4b85
Merge pull request #5926 from hiyouga/hiyouga/dev_deps
...
[version] update datasets version
Former-commit-id: 4a24e8fc8e1c229ef8751bd7eafe024661d46661
2024-11-04 16:04:00 +08:00
hiyouga
2e0092ed48
update datasets version
...
Former-commit-id: feba2c6418a15715fee77a34428fa3cf47fcee5b
2024-11-04 07:52:26 +00:00
hoshi-hiyouga
2febfeb13e
Merge pull request #5914 from hiyouga/hiyouga/dev_read
...
[misc] update readme
Former-commit-id: 2897696bad6bcc2d826845750c0c913882449829
2024-11-02 21:44:10 +08:00
hoshi-hiyouga
4aee53956c
Merge pull request #5475 from menibrief/main
...
Fix phi-3-small issues
Former-commit-id: c1daf49a967f6c0b641c9639a78971275aaa7cae
2024-11-02 21:31:34 +08:00
hiyouga
5c786684ad
fix phi3 template
...
Former-commit-id: b62131a3c5b4ff6f2969a8041e6e7b9cf2c444ed
2024-11-02 21:31:23 +08:00
hiyouga
5a2ddc60eb
update readme
...
Former-commit-id: 94bae8360b1aa124cc57dca481b9e686ba559f31
2024-11-02 21:28:04 +08:00
hoshi-hiyouga
c7be25fbfb
update template
...
Former-commit-id: 3559ef6115a831dcd1adf7210995ffd62890cff6
2024-11-02 21:21:22 +08:00
hoshi-hiyouga
e02cdaa4aa
Merge branch 'main' into main
...
Former-commit-id: 154f504fc2cebaae2b58c0121d6d8d8016db1bb2
2024-11-02 21:20:27 +08:00
hoshi-hiyouga
4a2bc1e43d
Merge pull request #5913 from hiyouga/hiyouga/dev_metrics
...
[train] support gather DPO metrics, fix return output
Former-commit-id: a17ac67f22c4de7699a8f2c1d4980af4babd2c7e
2024-11-02 21:13:43 +08:00
hiyouga
870ea6a8a4
fix webchat
...
Former-commit-id: 071fe40f209156f994c069507a2d53cc4f586d67
2024-11-02 21:04:18 +08:00
hiyouga
0ce5b84ccd
fix dpo metrics
...
Former-commit-id: 57029280da825a39fbf5a05097921b861f126669
2024-11-02 20:59:01 +08:00
hoshi-hiyouga
174c8fdffd
Merge pull request #5880 from sd3ntato/make-image-parametric
...
make base image parametric.
Former-commit-id: e2ea7c8b67cf598bba2b2b298e638b23712f14b3
2024-11-02 20:26:14 +08:00
hoshi-hiyouga
661ed7a6cf
Update Dockerfile
...
Former-commit-id: 89a1c1eb6d717b20107c06a645652b87fba388e8
2024-11-02 20:20:26 +08:00
hoshi-hiyouga
f2cf5d5b6d
Merge pull request #5910 from Cuiyn/index
...
Support Index series models.
Former-commit-id: b74d9fa8efeb4f52ba0e20538ad90c8b40492e29
2024-11-02 20:16:54 +08:00
Cuiyn
fa9edc7f43
fix: rename to Index-1.9B-Charater-Chat and Index-1.9B-Chat-32K
...
Former-commit-id: 95ab64749155a781ab5e55b989388ccd9e094c8d
2024-11-02 20:04:14 +08:00
hoshi-hiyouga
75c529b425
Merge pull request #5912 from hiyouga/hiyouga/dev_logging
...
[misc] support rank0 logger
Former-commit-id: ed34a6322814f302f050ba8ca4ecc53689f4d646
2024-11-02 18:48:41 +08:00
hiyouga
a117731ecb
support rank0 logger
...
Former-commit-id: 84528eabe560091bfd866b6a0ca864085af7529b
2024-11-02 18:31:04 +08:00
Cuiyn
66d235eacc
Add support for Index
...
Former-commit-id: 4e6dba16ca1755235d2ae117b53b68c5ae2f239a
2024-11-02 13:45:27 +08:00
hoshi-hiyouga
537faeeca5
Merge pull request #5909 from hiyouga/hiyouga/dev2
...
[data] support auto convert for single image, add image_dir argument
Former-commit-id: ced43fa0c84f7d0792694721d2c5e572c0d0e718
2024-11-02 13:43:04 +08:00
hoshi-hiyouga
1531d6f97c
Merge pull request #5907 from hiyouga/hiyouga/dev
...
[data] fix template replace behavior
Former-commit-id: 0a51c0bfdd9b193d2a3ac34a62fe8b073569c41a
2024-11-02 13:42:53 +08:00
hiyouga
2a92af3089
fix #5904
...
Former-commit-id: 079ebe038b11f36a11681dc8688f8ea48bccf324
2024-11-02 13:08:15 +08:00
hiyouga
63cf02b98d
fix #5883
...
Former-commit-id: 73b93caa9ac16ffd8d3faae24d16210d85ae9754
2024-11-02 13:06:34 +08:00
