hiyouga 34533b2f35 support vllm
Former-commit-id: d07ad5cc1cdbc13879afd84f653afdfee03a6933
2024-03-07 20:26:31 +08:00
..
2024-03-07 20:26:31 +08:00
2024-03-07 20:26:31 +08:00
2024-03-07 20:26:31 +08:00

Usage:

  • merge.sh: merge the lora weights
  • quantize.sh: quantize the model with AutoGPTQ (must after merge.sh, optional)