hiyouga 235cdcacee support batch infer in vllm
Former-commit-id: 1324d158f954d777f1fbf09f46149c372704b388
2024-12-04 13:50:00 +00:00

4 lines
132 B
YAML

model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
template: llama3
infer_backend: huggingface # choices: [huggingface, vllm]