mirror of
https://github.com/hiyouga/LLaMA-Factory.git
synced 2025-12-18 12:50:38 +08:00
[inference] support sglang backend (#7278)
* Mimic SGLang offline Engine * Add more tests and args * Pass all current tests * Clean Code * fix sample_params * clean code * Fix Stream Chat * change sglang from engine mode to server mode * fix * Fix Review Issues * Use SGLang Built-In Utilities * Fix test SGLang * Some Doc Issue * fix sglang engine * add readme --------- Co-authored-by: Jin Pan <jpan236@wisc.edu> Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
This commit is contained in:
@@ -106,6 +106,7 @@ class AttentionFunction(str, Enum):
|
||||
class EngineName(str, Enum):
|
||||
HF = "huggingface"
|
||||
VLLM = "vllm"
|
||||
SGLANG = "sglang"
|
||||
|
||||
|
||||
class DownloadSource(str, Enum):
|
||||
|
||||
Reference in New Issue
Block a user