mirror of
https://github.com/hiyouga/LLaMA-Factory.git
synced 2026-08-18 05:55:43 +08:00
Compare commits
2 Commits
c8a082e0e3
...
a61cfa692a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a61cfa692a | ||
|
|
7a83d28ce3 |
19
README.md
19
README.md
@@ -558,7 +558,24 @@ Try `dataloader_num_workers: 0` if you encounter `Can't pickle local object` err
|
||||
|
||||
#### Install BitsAndBytes
|
||||
|
||||
If you want to enable the quantized LoRA (QLoRA) on the Windows platform, you need to install a pre-built version of `bitsandbytes` library, which supports CUDA 11.1 to 12.2, please select the appropriate [release version](https://github.com/jllllll/bitsandbytes-windows-webui/releases/tag/wheels) based on your CUDA version.
|
||||
To enable Quantized LoRA (QLoRA) on Windows, you need to install bitsandbytes.
|
||||
|
||||
For most users, it is recommended to install the latest official release:
|
||||
|
||||
```bash
|
||||
pip install bitsandbytes
|
||||
```
|
||||
|
||||
If you are using uv to manage your virtual environment, it is recommended to install bitsandbytes after installing the GPU-enabled version of PyTorch:
|
||||
|
||||
```bash
|
||||
uv pip install bitsandbytes --no-deps
|
||||
```
|
||||
|
||||
[!IMPORTANT]
|
||||
Pay attention to the CUDA Toolkit version when installing bitsandbytes. Official bitsandbytes releases are built for specific CUDA Toolkit versions. On Windows x86-64, separate builds are currently provided for CUDA 11.8–12.6 and CUDA 12.8–12.9. Support for NVIDIA RTX 50 Series GPUs (e.g., RTX 5060 Ti, sm_120) requires the CUDA 12.8–12.9 builds.
|
||||
|
||||
If your environment uses an older CUDA version, or you need compatibility with older Windows / PyTorch combinations, you can install the third-party precompiled Windows wheel:
|
||||
|
||||
```bash
|
||||
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl
|
||||
|
||||
19
README_zh.md
19
README_zh.md
@@ -559,7 +559,24 @@ python -c "import torch; print(torch.cuda.is_available())"
|
||||
|
||||
#### 安装 BitsAndBytes
|
||||
|
||||
如果要在 Windows 平台上开启量化 LoRA(QLoRA),需要安装预编译的 `bitsandbytes` 库, 支持 CUDA 11.1 到 12.2, 请根据您的 CUDA 版本情况选择适合的[发布版本](https://github.com/jllllll/bitsandbytes-windows-webui/releases/tag/wheels)。
|
||||
如果要在 Windows 平台上开启量化 LoRA(QLoRA),需要安装 bitsandbytes。
|
||||
|
||||
对于大多数用户,建议优先使用官方发布的最新版本:
|
||||
|
||||
```bash
|
||||
pip install bitsandbytes
|
||||
```
|
||||
|
||||
如果使用 uv 管理虚拟环境,建议在安装好 GPU 版本 PyTorch 之后,再执行:
|
||||
|
||||
```bash
|
||||
uv pip install bitsandbytes --no-deps
|
||||
```
|
||||
|
||||
> [!IMPORTANT]
|
||||
> 安装 bitsandbytes 时,请注意 CUDA Toolkit 版本。bitsandbytes 的官方发布包是按不同 CUDA 版本分别构建的,Windows x86-64 目前提供了面向 CUDA 11.8–12.6 和 CUDA 12.8–12.9 的不同构建;其中支持 RTX 50 系列(如 RTX 5060 Ti,sm_120)的构建对应 CUDA 12.8–12.9。
|
||||
|
||||
若当前环境的 CUDA 版本较旧,或者需要兼容较老的 Windows / PyTorch 组合,可以使用第三方预编译版本:
|
||||
|
||||
```bash
|
||||
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl
|
||||
|
||||
Reference in New Issue
Block a user