423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2025-08-02 03:32:50 +08:00
Code Issues Packages Projects Releases Wiki Activity

[script] fix vllm version (#7193)

Browse Source 
Former-commit-id: 313355759dc906d3612364dc6c8f6344afdedb97
This commit is contained in:
hoshi-hiyouga 2025-03-06 17:14:17 +08:00 committed by GitHub
parent eba31ae313
commit b6c0e8608e
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
scripts/vllm_infer.py
View File

@ -55,7 +55,7 @@ def vllm_infer(
Performs batch generation using vLLM engine, which supports tensor parallelism. Performs batch generation using vLLM engine, which supports tensor parallelism.
Usage: python vllm_infer.py --model_name_or_path meta-llama/Llama-2-7b-hf --template llama --dataset alpaca_en_demo Usage: python vllm_infer.py --model_name_or_path meta-llama/Llama-2-7b-hf --template llama --dataset alpaca_en_demo
""" """
check_version("vllm>=0.4.3,<=0.7.2") check_version("vllm>=0.4.3,<=0.7.3")
if pipeline_parallel_size > get_device_count(): if pipeline_parallel_size > get_device_count():
raise ValueError("Pipeline parallel size should be smaller than the number of gpus.") raise ValueError("Pipeline parallel size should be smaller than the number of gpus.")