hoshi-hiyouga
2bfcad2394
[model] fix kv cache ( #7564 )
2025-04-01 23:07:46 +08:00
Xiaosu Zhu
6b3b97c738
[misc] update liger-kernel's monkey patch ( #7453 )
...
* Update liger_kernel.py
* Update setup.py
2025-03-25 11:58:52 +08:00
hoshi-hiyouga
919415dba9
[deps] upgrade vllm to 0.8 ( #7436 )
2025-03-23 14:32:22 +08:00
Guo, Quan
a959c2a509
[misc] fix sglang deps ( #7432 )
...
* feat: Add transformer version requirement for sglang
* feat: add srt to sglang which is required for running sglang
Other options are srt_hip, srt_xpu, srt_npu, srt_hpu, srt_cpu, for different computation architectures.
2025-03-23 14:07:10 +08:00
hoshi-hiyouga
a306f0f5a2
[version] fix minicpmo ( #7378 )
2025-03-20 16:59:31 +08:00
Qiaolin Yu
a44a53ebec
[inference] support sglang backend ( #7278 )
...
* Mimic SGLang offline Engine
* Add more tests and args
* Pass all current tests
* Clean Code
* fix sample_params
* clean code
* Fix Stream Chat
* change sglang from engine mode to server mode
* fix
* Fix Review Issues
* Use SGLang Built-In Utilities
* Fix test SGLang
* Some Doc Issue
* fix sglang engine
* add readme
---------
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
2025-03-15 04:37:58 +08:00
hoshi-hiyouga
264538cb26
[misc] upgrade format to py39 ( #7256 )
2025-03-12 00:08:41 +08:00
hoshi-hiyouga
bf8b483186
[core] release v0.9.2 ( #7254 )
2025-03-11 22:42:23 +08:00
hoshi-hiyouga
71a1c1321a
[config] update args ( #7231 )
...
Former-commit-id: f71a901840811bf560df671ec63a146ff99140c6
2025-03-10 23:04:43 +08:00
hoshi-hiyouga
d2f845d70d
[deps] upgrade vllm ( #7183 )
...
Former-commit-id: 37678a3d64668c3b4a4bfefc054e3b9b40427c1a
2025-03-06 15:25:08 +08:00
leo-pony
b9f84900ee
[npu] update cann base image and torch 2.4 ( #7061 )
...
* Update base npu container image version:The Python version required for Hugging Face Transformers is >= python3.10
* Fix the bug: arg type of INSTALL_DEEPSPEED shoud been string now.
* Update Ascend CANN, CANN-Kernel and corresponding torch and torch-npu version
* Upgrade torch-npu needs packages' version: torch==2.1.0 and torch-npu==2.4.0.post2
Former-commit-id: d6dafada58412b0c801e576ef4d8d96203f792af
2025-02-25 23:32:01 +08:00
hoshi-hiyouga
4d1791e905
[deps] upgrade vllm ( #6857 )
...
Former-commit-id: 4bd50f65a3d62528768561019fda2723d045c7fd
2025-02-08 15:02:28 +08:00
Zhangchi Feng
8f401e37f8
[model] support audio ( #6701 )
...
* support qwen2_audio
* improve code
* lint
* fix
* fix
* fix
---------
Co-authored-by: hiyouga <hiyouga@buaa.edu.cn>
Former-commit-id: 5eacb5629e4d7733cd992a63747a1335f2c6a929
2025-02-05 04:59:09 +08:00
hoshi-hiyouga
c2022431aa
[misc] update license year & fix llama pro ( #6814 )
...
* fix llamapro script
* change year
Former-commit-id: d9ae594178796994d400a5f207d6499712816f89
2025-02-05 01:53:33 +08:00
hoshi-hiyouga
222423bcef
[breaking] support transformers 4.48 ( #6628 )
...
Former-commit-id: f154ab175c513a4d7bb866bf2cffc34b77b50508
2025-01-31 01:36:33 +08:00
zhuHQ
c2120432db
[optim] add support to APOLLO ( #6617 )
...
Former-commit-id: 5a252e5a458457adbd19da3b68a3897ad2962824
2025-01-15 00:24:56 +08:00
Zhangchi Feng
f7857c83e1
Support Inference of MiniCPM-V-2.6 and MiniCPM-o-2.6 ( #6631 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
* support inference of minicpmv
Former-commit-id: 7f3c64e853a7cdd49d02bf85e237611941ac7fa8
2025-01-14 17:34:58 +08:00
hoshi-hiyouga
28d145a066
pin vllm version to 0.6.5 ( #6629 )
...
Former-commit-id: 26097ca0adf25ebb7d9e8eec2d2cef673c6cfe88
2025-01-14 02:44:02 +08:00
Zhangchi Feng
ae32c148d1
Support new features of MiniCPM-V ( #6626 )
...
* fix template name
* tiny fix
* support minicpm-o-2.6
Former-commit-id: 53034a61c7654358f46916cbc370910fb2aeff3b
2025-01-14 00:26:19 +08:00
hiyouga
dc65ecdf09
refactor mllm param logic
...
Former-commit-id: b895c190945cf5d991cb4e4dea2ae73cc9c8d246
2025-01-10 15:45:48 +00:00
hiyouga
f8f05a883b
fix #6482
...
Former-commit-id: 8577f52b4152efe6cc7a8b5f6d37b4f9ba6684e7
2024-12-30 06:03:07 +00:00
hiyouga
a897d46049
support report custom args
...
Former-commit-id: d41254c40a1c5cacf9377096adb27efa9bdb79ea
2024-12-21 21:42:45 +00:00
hiyouga
c1768cfb14
support batch infer in vllm
...
Former-commit-id: 3ef5ed3b9a44eed2f7e3ff221dfc343d0a97c0b5
2024-12-04 13:50:00 +00:00
hiyouga
1e6f96508a
add vllm config
...
Former-commit-id: 95365f0ce4f362bde7de8b679b54b548d7055bfb
2024-11-10 21:28:18 +08:00
hiyouga
248d5daaff
use pre-commit
...
Former-commit-id: 7cfede95df22a9ff236788f04159b6b16b8d04bb
2024-10-29 09:07:46 +00:00
hiyouga
c7efc7f2ed
tiny fix
...
Former-commit-id: 1fe424323b212094856f423351dc2a15774d39c3
2024-10-11 23:51:54 +08:00
Johnny
3b19b6f31b
Update setup.py
...
Former-commit-id: f85b756ffafa241304624819b7612603ad5e0ee3
2024-10-11 12:29:09 +02:00
hoshi-hiyouga
6e111eb29f
Update setup.py
...
Former-commit-id: 4c017fe014b708d79c65eff24329b9c324399461
2024-10-07 16:26:50 +08:00
Johnny
13d73574ef
Update setup.py
...
Former-commit-id: 73d3f93496712edace38711613e14768922d6c96
2024-10-07 10:16:53 +02:00
Johnny
1f58943b32
Update setup.py
...
Former-commit-id: b4de2c84b078194bb6358697fd6815d622843f58
2024-10-06 08:53:55 +02:00
hiyouga
dfff411e1a
release v0.9.0 (real)
...
Former-commit-id: 8ff781c8ae5654680f738f69a6db9d7b95d76baf
2024-09-09 01:00:25 +08:00
hiyouga
eb5af3d90b
support vllm 0.6.0
...
Former-commit-id: e39470ec51a9c74ad901871eb816df10e851f351
2024-09-08 02:26:20 +08:00
hiyouga
41a9c415e1
fix #5252
...
Former-commit-id: 73f30b4dfffb260e24f9e2332617b8ca2c249ed5
2024-09-04 03:17:54 +08:00
hiyouga
4cc8e16595
add extra requires
...
Former-commit-id: c47511773ae9886aae4e5ea1841866d2125abc34
2024-08-27 12:52:12 +08:00
hiyouga
59cbce1a46
add adam_mini to readme
...
Former-commit-id: d610c6bcf8a8ba6f4236f5d11f79571b83f4fb11
2024-08-09 20:02:03 +08:00
hiyouga
8aaf1185a5
support HQQ/EETQ #4113
...
Former-commit-id: b7cb51ddb394f04fe4646b2c297fc8d918c9979e
2024-06-27 00:29:42 +08:00
hiyouga
08fa707085
improve autogptq integration
...
Former-commit-id: d68408c7b123b8ff92014db35cac0b24b414a6f4
2024-06-26 22:11:44 +08:00
hiyouga
9fd7a410bb
tiny fix about badam
...
Former-commit-id: 03f49267c7406e36aee35639f86e6e0383897090
2024-06-25 01:54:53 +08:00
hiyouga
bb88536166
add license
...
Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0
2024-06-15 17:54:33 +08:00
hiyouga
2abfad9c1f
fix #4271
...
Former-commit-id: 03707e78d29bfcf5d395a64bb38632bdb3ff47ce
2024-06-15 05:11:33 +08:00
hiyouga
1364190a66
fix ci
...
Former-commit-id: 95aceebd61d195be5c980a919c12c59b56722898
2024-06-08 01:57:36 +08:00
hiyouga
31feb6e26c
update readme
...
Former-commit-id: cc331fa2d28afe081937c50ea83d63add21d4e3a
2024-06-06 16:59:18 +08:00
hiyouga
d992f5353f
fix setup
...
Former-commit-id: b2b80d434fcc0c3838d229098e1c21d26632204c
2024-06-06 01:39:02 +08:00
hiyouga
b74c229498
fix #4079
...
Former-commit-id: fda732d7f4616373844c97beff416880260f49db
2024-06-05 16:56:54 +08:00
hiyouga
d2ede9d393
fix #4022
...
Former-commit-id: 9541f2f1f1b7d7877eb734f051048e52003a3430
2024-06-03 18:38:36 +08:00
hiyouga
9a65820592
update readme
...
Former-commit-id: 440e9de66986ef7736361ce8ec3e23ce68655a56
2024-05-29 18:39:11 +08:00
hoshi-hiyouga
f4e16ae373
Merge pull request #3930 from MengqingCao/npu
...
Add Ascend npu doc and dependency
Former-commit-id: 7210090e4fc6531b9f6122f104875811a8798185
2024-05-29 18:33:38 +08:00
MengqingCao
e2cfd34da0
update torch-npu version
...
Former-commit-id: a70d7fcf2967eb30280a1fb845b39db7878f535c
2024-05-29 10:05:11 +00:00
MengqingCao
668dea9706
update cann kernels url
...
Former-commit-id: 23c65e9d7e8817b5815264e44cbf4a7bcb88d3d7
2024-05-29 09:53:31 +00:00
hiyouga
bceac4f554
bump vllm version to 0.4.1
...
Former-commit-id: a00fd39a4c2f270620711f2bfbad8d460fb4aa89
2024-05-28 21:27:27 +08:00
