Guo, Quan
b302301005
[misc] fix sglang deps ( #7432 )
...
* feat: Add transformer version requirement for sglang
* feat: add srt to sglang which is required for running sglang
Other options are srt_hip, srt_xpu, srt_npu, srt_hpu, srt_cpu, for different computation architectures.
2025-03-23 14:07:10 +08:00
hoshi-hiyouga
6714adf788
[version] fix minicpmo ( #7378 )
2025-03-20 16:59:31 +08:00
Qiaolin Yu
280d9bda76
[inference] support sglang backend ( #7278 )
...
* Mimic SGLang offline Engine
* Add more tests and args
* Pass all current tests
* Clean Code
* fix sample_params
* clean code
* Fix Stream Chat
* change sglang from engine mode to server mode
* fix
* Fix Review Issues
* Use SGLang Built-In Utilities
* Fix test SGLang
* Some Doc Issue
* fix sglang engine
* add readme
---------
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
2025-03-15 04:37:58 +08:00
hoshi-hiyouga
efa86e730c
[misc] upgrade format to py39 ( #7256 )
2025-03-12 00:08:41 +08:00
hoshi-hiyouga
1942d3b119
[core] release v0.9.2 ( #7254 )
2025-03-11 22:42:23 +08:00
hoshi-hiyouga
c6331546a9
[config] update args ( #7231 )
...
Former-commit-id: f71a901840811bf560df671ec63a146ff99140c6
2025-03-10 23:04:43 +08:00
hoshi-hiyouga
00245a86e6
[deps] upgrade vllm ( #7183 )
...
Former-commit-id: 37678a3d64668c3b4a4bfefc054e3b9b40427c1a
2025-03-06 15:25:08 +08:00
leo-pony
1590c3861c
[npu] update cann base image and torch 2.4 ( #7061 )
...
* Update base npu container image version:The Python version required for Hugging Face Transformers is >= python3.10
* Fix the bug: arg type of INSTALL_DEEPSPEED shoud been string now.
* Update Ascend CANN, CANN-Kernel and corresponding torch and torch-npu version
* Upgrade torch-npu needs packages' version: torch==2.1.0 and torch-npu==2.4.0.post2
Former-commit-id: d6dafada58412b0c801e576ef4d8d96203f792af
2025-02-25 23:32:01 +08:00
hoshi-hiyouga
c322512037
[deps] upgrade vllm ( #6857 )
...
Former-commit-id: 4bd50f65a3d62528768561019fda2723d045c7fd
2025-02-08 15:02:28 +08:00
Zhangchi Feng
46a1786595
[model] support audio ( #6701 )
...
* support qwen2_audio
* improve code
* lint
* fix
* fix
* fix
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 5eacb5629e4d7733cd992a63747a1335f2c6a929
2025-02-05 04:59:09 +08:00
hoshi-hiyouga
40b6e9045d
[misc] update license year & fix llama pro ( #6814 )
...
* fix llamapro script
* change year
Former-commit-id: d9ae594178796994d400a5f207d6499712816f89
2025-02-05 01:53:33 +08:00
hoshi-hiyouga
46068b3324
[breaking] support transformers 4.48 ( #6628 )
...
Former-commit-id: f154ab175c513a4d7bb866bf2cffc34b77b50508
2025-01-31 01:36:33 +08:00
zhuHQ
9b29a431db
[optim] add support to APOLLO ( #6617 )
...
Former-commit-id: 5a252e5a458457adbd19da3b68a3897ad2962824
2025-01-15 00:24:56 +08:00
Zhangchi Feng
068d44b509
Support Inference of MiniCPM-V-2.6 and MiniCPM-o-2.6 ( #6631 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
* support inference of minicpmv
Former-commit-id: 7f3c64e853a7cdd49d02bf85e237611941ac7fa8
2025-01-14 17:34:58 +08:00
hoshi-hiyouga
89b308bf30
pin vllm version to 0.6.5 ( #6629 )
...
Former-commit-id: 26097ca0adf25ebb7d9e8eec2d2cef673c6cfe88
2025-01-14 02:44:02 +08:00
Zhangchi Feng
429a027832
Support new features of MiniCPM-V ( #6626 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
Former-commit-id: 53034a61c7654358f46916cbc370910fb2aeff3b
2025-01-14 00:26:19 +08:00
hiyouga
e49c021e22
refactor mllm param logic
...
Former-commit-id: b895c190945cf5d991cb4e4dea2ae73cc9c8d246
2025-01-10 15:45:48 +00:00
hiyouga
92c6c384cf
fix #6482
...
Former-commit-id: 8577f52b4152efe6cc7a8b5f6d37b4f9ba6684e7
2024-12-30 06:03:07 +00:00
hiyouga
c57fbebd55
support report custom args
...
Former-commit-id: d41254c40a1c5cacf9377096adb27efa9bdb79ea
2024-12-21 21:42:45 +00:00
hiyouga
51b18e565d
support batch infer in vllm
...
Former-commit-id: 3ef5ed3b9a44eed2f7e3ff221dfc343d0a97c0b5
2024-12-04 13:50:00 +00:00
hiyouga
85343ddf47
add vllm config
...
Former-commit-id: 95365f0ce4f362bde7de8b679b54b548d7055bfb
2024-11-10 21:28:18 +08:00
hiyouga
dbbfb5f5dc
use pre-commit
...
Former-commit-id: 7cfede95df22a9ff236788f04159b6b16b8d04bb
2024-10-29 09:07:46 +00:00
hiyouga
916804d11a
tiny fix
...
Former-commit-id: 1fe424323b212094856f423351dc2a15774d39c3
2024-10-11 23:51:54 +08:00
Johnny
30851cd72c
Update setup.py
...
Former-commit-id: f85b756ffafa241304624819b7612603ad5e0ee3
2024-10-11 12:29:09 +02:00
hoshi-hiyouga
e044cc7b97
Update setup.py
...
Former-commit-id: 4c017fe014b708d79c65eff24329b9c324399461
2024-10-07 16:26:50 +08:00
Johnny
ff9d5d2c47
Update setup.py
...
Former-commit-id: 73d3f93496712edace38711613e14768922d6c96
2024-10-07 10:16:53 +02:00
Johnny
e689c63af1
Update setup.py
...
Former-commit-id: b4de2c84b078194bb6358697fd6815d622843f58
2024-10-06 08:53:55 +02:00
hiyouga
be233f1102
release v0.9.0 (real)
...
Former-commit-id: 8ff781c8ae5654680f738f69a6db9d7b95d76baf
2024-09-09 01:00:25 +08:00
hiyouga
01dae0f475
support vllm 0.6.0
...
Former-commit-id: e39470ec51a9c74ad901871eb816df10e851f351
2024-09-08 02:26:20 +08:00
hiyouga
3abacf1145
fix #5252
...
Former-commit-id: 73f30b4dfffb260e24f9e2332617b8ca2c249ed5
2024-09-04 03:17:54 +08:00
hiyouga
57130d032a
add extra requires
...
Former-commit-id: c47511773ae9886aae4e5ea1841866d2125abc34
2024-08-27 12:52:12 +08:00
hiyouga
c7a1c3f43a
add adam_mini to readme
...
Former-commit-id: d610c6bcf8a8ba6f4236f5d11f79571b83f4fb11
2024-08-09 20:02:03 +08:00
hiyouga
28c2c7fba5
support HQQ/EETQ #4113
...
Former-commit-id: b7cb51ddb394f04fe4646b2c297fc8d918c9979e
2024-06-27 00:29:42 +08:00
hiyouga
4041aa024b
improve autogptq integration
...
Former-commit-id: d68408c7b123b8ff92014db35cac0b24b414a6f4
2024-06-26 22:11:44 +08:00
hiyouga
4d2c279083
tiny fix about badam
...
Former-commit-id: 03f49267c7406e36aee35639f86e6e0383897090
2024-06-25 01:54:53 +08:00
hiyouga
acfae2e677
add license
...
Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0
2024-06-15 17:54:33 +08:00
hiyouga
ab4b3931fd
fix #4271
...
Former-commit-id: 03707e78d29bfcf5d395a64bb38632bdb3ff47ce
2024-06-15 05:11:33 +08:00
hiyouga
ec97bee944
fix ci
...
Former-commit-id: 95aceebd61d195be5c980a919c12c59b56722898
2024-06-08 01:57:36 +08:00
hiyouga
a4e1fcc881
update readme
...
Former-commit-id: cc331fa2d28afe081937c50ea83d63add21d4e3a
2024-06-06 16:59:18 +08:00
hiyouga
eff00a8172
fix setup
...
Former-commit-id: b2b80d434fcc0c3838d229098e1c21d26632204c
2024-06-06 01:39:02 +08:00
hiyouga
861d93a06a
fix #4079
...
Former-commit-id: fda732d7f4616373844c97beff416880260f49db
2024-06-05 16:56:54 +08:00
hiyouga
d8d02e65a1
fix #4022
...
Former-commit-id: 9541f2f1f1b7d7877eb734f051048e52003a3430
2024-06-03 18:38:36 +08:00
hiyouga
0c722c879a
update readme
...
Former-commit-id: 440e9de66986ef7736361ce8ec3e23ce68655a56
2024-05-29 18:39:11 +08:00
hoshi-hiyouga
69e7238379
Merge pull request #3930 from MengqingCao/npu
...
Add Ascend npu doc and dependency
Former-commit-id: 7210090e4fc6531b9f6122f104875811a8798185
2024-05-29 18:33:38 +08:00
MengqingCao
c21590ca5e
update torch-npu version
...
Former-commit-id: a70d7fcf2967eb30280a1fb845b39db7878f535c
2024-05-29 10:05:11 +00:00
MengqingCao
ecada19358
update cann kernels url
...
Former-commit-id: 23c65e9d7e8817b5815264e44cbf4a7bcb88d3d7
2024-05-29 09:53:31 +00:00
hiyouga
fdfb5e5485
bump vllm version to 0.4.1
...
Former-commit-id: a00fd39a4c2f270620711f2bfbad8d460fb4aa89
2024-05-28 21:27:27 +08:00
MengqingCao
60f6fc3935
add Ascend npu doc and dependency
...
Former-commit-id: 803d9f142a294f8c1e0b4e2046c214b0857ccfd6
2024-05-28 01:33:54 +00:00
hiyouga
ee759aa0d8
rename package
...
Former-commit-id: a07ff0c083558cfe6f474d13027642d3052fee08
2024-05-16 18:39:08 +08:00
hiyouga
9c2da898a4
release v0.7.1
...
Former-commit-id: a4f8adb021b6218d624303b51cd5e93ffa3111a1
2024-05-16 00:57:16 +08:00
