423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2025-10-14 23:58:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #213 from Ehco1996/patch-1

Browse Source 
feat: support pass args before init web app
Former-commit-id: b0612c05bc10c281c0a95e08c5517c3fb0a72029
This commit is contained in:
hoshi-hiyouga 2023-07-20 22:12:07 +08:00 committed by GitHub
commit 5dcd95645f
1 changed files with 3 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
src/llmtuner/api/app.py
View File

@ -30,9 +30,7 @@ async def lifespan(app: FastAPI): # collects GPU memory
torch_gc()
def create_app():
chat_model = ChatModel(*get_infer_args())
def create_app(chat_model: ChatModel):
app = FastAPI(lifespan=lifespan)
app.add_middleware(
@ -124,5 +122,6 @@ def create_app():
if __name__ == "__main__":
app = create_app()
chat_model = ChatModel(*get_infer_args())
app = create_app(chat_model)
uvicorn.run(app, host="0.0.0.0", port=8000, workers=1)