423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2026-08-17 13:35:44 +08:00
Code Issues Packages Projects Releases Wiki Activity

[docker] upgrade NPU images to CANN 9.1 and PyTorch 2.10 (#10729)

Browse Source
This commit is contained in:
xvxuopop
2026-08-10 11:20:41 +08:00
committed by GitHub
parent 63a89710c7
commit 0bbe481e6e
10 changed files with 297 additions and 384 deletions
Download Patch File Download Diff File Expand all files Collapse all files

191
docker/docker-npu/OVERVIEW.md
View File

@@ -1,6 +1,6 @@
# LLaMA Factory for Ascend NPU
# LlamaFactory Image for Ascend NPU
LLaMA Factory Ascend NPU images provide a ready-to-use environment for fine-tuning, evaluating, and serving large language and multimodal models on Huawei Ascend Atlas NPUs. The images are based on Ascend CANN container images and include LLaMA Factory, Python, PyTorch, torch-npu, Triton Ascend, DeepSpeed, and the metric dependencies used by LLaMA Factory.
LlamaFactory Ascend NPU images are designed for Huawei Ascend Atlas NPUs and provide a ready-to-use LlamaFactory environment. Built on Ascend CANN container images, they include Python, PyTorch, TorchNPU, DeepSpeed, LlamaFactory, and other components.
For installation and troubleshooting details, see the [English NPU installation guide](https://llamafactory.readthedocs.io/en/latest/multibackend/npu/npu_installation.html).
@@ -11,77 +11,68 @@ For installation and troubleshooting details, see the [English NPU installation
- `quay.io/ascend/llamafactory`
- Dockerfile: `docker/docker-npu/Dockerfile`
- Docker Compose file: `docker/docker-npu/docker-compose.yml`
- Default base image: `quay.io/ascend/cann:9.0.0-910b-ubuntu22.04-py3.11`
- Supported accelerators: Ascend A2 and A3
- Supported container operating systems: Ubuntu 22.04 and openEuler 24.03
- Target CPU architectures: `linux/amd64` and `linux/arm64`
- Exposed ports:
- `7860`: LLaMA Board Web UI
- `8000`: API service
- Ascend environment script: `/usr/local/Ascend/ascend-toolkit/set_env.sh`
The current image variants are:
The following `latest` NPU image tags are available:
| Accelerator | Container OS | CANN base image |
| Hardware series | Operating system | Tag |
| --- | --- | --- |
| A2 | Ubuntu 22.04 | `quay.io/ascend/cann:9.0.0-910b-ubuntu22.04-py3.11` |
| A3 | Ubuntu 22.04 | `quay.io/ascend/cann:9.0.0-a3-ubuntu22.04-py3.11` |
| A2 | openEuler 24.03 | `quay.io/ascend/cann:9.0.0-910b-openeuler24.03-py3.11` |
| A3 | openEuler 24.03 | `quay.io/ascend/cann:9.0.0-a3-openeuler24.03-py3.11` |
| A2 | Ubuntu 22.04 | `latest-910b-ubuntu` |
| A3 | Ubuntu 22.04 | `latest-a3-ubuntu` |
| A2 | openEuler 24.03 | `latest-910b-openeuler` |
| A3 | openEuler 24.03 | `latest-a3-openeuler` |
## Image Contents and Intended Use
## Image Overview
The image is intended for Ascend NPU training, fine-tuning, evaluation, Web UI, and API workflows supported by LLaMA Factory. It installs the following core components:
The image includes the following core components:
| Component | Version or source |
| Component | Version |
| --- | --- |
| CANN | Inherited from the selected CANN 9.0.0 base image |
| Python | Python 3.11, inherited from the base image |
| PyTorch | `2.7.1` |
| torch-npu | `2.7.1.post4` |
| torchvision | `0.22.1` |
| torchaudio | `2.7.1` |
| CANN | `9.1.0` |
| Python | `3.12` |
| PyTorch | `2.10.0` |
| TorchNPU | `2.10.0.post2` |
| torchvision / torchaudio | `0.25.0` / `2.10.0` |
| Transformers | Latest compatible version at build time |
| Triton Ascend | `3.2.1` |
| DeepSpeed | `>=0.10.0,<=0.18.4` |
| LLaMA Factory | Installed from the repository build context |
| DeepSpeed | Latest compatible version at build time |
| LlamaFactory | Installed from the repository build context |
The image does not include model weights or datasets. Mount or download them separately and comply with their respective licenses and acceptable-use requirements.
## Image Tags and Dockerfile Archive
## Image Tags
Images use the following tag format:
NPU `latest` and release tags use different formats; the following rules do not apply to CUDA images.
Non-release builds reuse the following short tags. Each scheduled build updates the image referenced by the corresponding tag:
```text
<llamafactory-version>-cann<cann-version>-torch_npu<torch-npu-version>-<accelerator>-<os>-<python-version>
latest-<chip>-<os>
```
| Field | Values | Description |
| --- | --- | --- |
| `chip` | `910b` or `a3` | Ascend chip model supported by the image |
| `os` | `ubuntu` or `openeuler` | Container operating system family |
Release builds use full tags:
```text
<LlamaFactory-version>-cann<CANN-version>-torch_npu<TorchNPU-version>-<chip>-<os>-<Python-version>
```
| Field | Example | Description |
| --- | --- | --- |
| `llamafactory-version` | `latest` or `0.9.6` | Non-release builds use `latest`; release builds use the LLaMA Factory version |
| `cann-version` | `9.0.0` | Parsed from the CANN base image tag |
| `torch-npu-version` | `2.7.1` | Parsed from `requirements/npu.txt`; a suffix such as `.post4` is not included in the image tag |
| `accelerator` | `A2` or `A3` | Ascend hardware generation selected for the image |
| `os` | `ubuntu` or `openeuler` | Container operating system family |
| `python-version` | `py3.11` | Parsed from the CANN base image tag |
| `LlamaFactory-version` | `0.9.5` | LlamaFactory release version |
| `CANN-version` | `9.1.0` | Parsed from the CANN base image tag |
| `TorchNPU-version` | `2.10.0.post2` | Full TorchNPU version used by the image, including suffixes such as `.postN` |
| `chip` | `910b` or `a3` | Ascend chip model supported by the image |
| `os` | `ubuntu22.04` or `openeuler24.03` | Container operating system family and version |
| `Python-version` | `py3.12` | Parsed from the CANN base image tag |
Examples:
For example:
```text
latest-cann9.0.0-torch_npu2.7.1-A2-ubuntu-py3.11
latest-cann9.0.0-torch_npu2.7.1-A3-openeuler-py3.11
0.9.6-cann9.0.0-torch_npu2.7.1-A3-ubuntu-py3.11
```
The CPU architecture is not part of the tag. Published images are configured as multi-platform images, and Docker selects the `linux/amd64` or `linux/arm64` manifest for the host automatically.
The Dockerfile and its distribution overview are archived together at:
```text
docker/docker-npu/
├── Dockerfile
├── OVERVIEW.md
├── OVERVIEW.zh.md
└── docker-compose.yml
0.9.5-cann9.1.0-torch_npu2.10.0.post2-a3-ubuntu22.04-py3.12
```
## Quick Start
@@ -94,33 +85,30 @@ Before starting a container:
2. Verify that `npu-smi info` works on the host.
3. Install Docker with permission to access the required Ascend device nodes and driver files.
Driver, firmware, CANN, torch-npu, and the target Ascend hardware must be mutually compatible.
Driver, firmware, CANN, TorchNPU, and the target Ascend hardware must be mutually compatible.
### Pull and Run
The following example starts the latest A2 Ubuntu image with one NPU. Change the image tag and `/dev/davinci0` as needed.
The following example starts the latest A2 Ubuntu image with one NPU. Adjust `DOCKER_IMAGE` and the `--device` options for your environment.
```bash
export IMAGE=quay.io/ascend/llamafactory:latest-cann9.0.0-torch_npu2.7.1-A2-ubuntu-py3.11
docker pull "$IMAGE"
CONTAINER_NAME=llamafactory-npu
DOCKER_IMAGE=hiyouga/llamafactory:latest-910b-ubuntu
docker run --rm -it \
--name llamafactory-npu \
--ipc=host \
--net=host \
--device=/dev/davinci0 \
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
-p 7860:7860 \
-p 8000:8000 \
"$IMAGE" \
bash
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /data:/data \
--name "$CONTAINER_NAME" \
"$DOCKER_IMAGE" \
/bin/bash
```
The host path for `npu-smi` may be `/usr/local/sbin/npu-smi` on some driver installations. Adjust the mount source when necessary. Add more `--device=/dev/davinci<N>` options to expose additional NPUs.
@@ -134,22 +122,16 @@ python -c "import torch, torch_npu; print(torch.__version__, torch_npu.__version
llamafactory-cli help
```
Start LLaMA Board when needed:
```bash
llamafactory-cli webui
```
### Build Locally
Run the build from the repository root. The following example builds the A3 openEuler variant:
Run the build from the repository root. The following example builds the A2 Ubuntu variant:
```bash
docker build \
-f ./docker/docker-npu/Dockerfile \
--build-arg BASE_IMAGE=quay.io/ascend/cann:9.0.0-a3-openeuler24.03-py3.11 \
--build-arg BASE_IMAGE=quay.io/ascend/cann:9.1.0-910b-ubuntu22.04-py3.12 \
--build-arg PIP_INDEX=https://pypi.org/simple \
-t llamafactory:npu-a3-openeuler \
-t llamafactory:npu-910b-ubuntu \
.
```
@@ -157,76 +139,45 @@ Available build arguments:
| Argument | Default | Purpose |
| --- | --- | --- |
| `BASE_IMAGE` | A2 Ubuntu CANN 9.0.0 image | Selects the accelerator and container OS variant |
| `BASE_IMAGE` | `quay.io/ascend/cann:9.1.0-910b-ubuntu22.04-py3.12` | Selects the base image that matches the device model and container operating system |
| `PIP_INDEX` | `https://pypi.org/simple` | Selects the Python package index |
| `PYTORCH_INDEX` | `https://download.pytorch.org/whl/cpu` | Selects the PyTorch wheel index used with torch-npu |
| `PYTORCH_INDEX` | `https://download.pytorch.org/whl/cpu` | Selects the PyTorch wheel index used with TorchNPU |
| `HTTP_PROXY` | Empty | Provides an optional HTTP/HTTPS proxy during the build |
Docker Compose can build and start each supported variant:
### Start with Docker Compose
The preceding `docker build` command invokes the Dockerfile directly. It builds an image but does not start a container. Docker Compose does not use a separate build implementation: it reads the presets in `docker-compose.yml`, reuses the same Dockerfile, and selects a hardware-series and operating-system combination through a profile. Each `up -d` command below starts the selected container in the background. If the image is not available locally, Docker Compose builds it first:
```bash
cd docker/docker-npu
# A2 with Ubuntu
docker compose up -d llamafactory-a2-ubuntu
docker compose --profile a2-ubuntu up -d
# A3 with Ubuntu
docker compose --profile a3 up -d llamafactory-a3-ubuntu
docker compose --profile a3-ubuntu up -d
# A2 with openEuler
docker compose --profile openeuler up -d llamafactory-a2-openeuler
docker compose --profile a2-openeuler up -d
# A3 with openEuler
docker compose --profile a3-openeuler up -d llamafactory-a3-openeuler
docker compose --profile a3-openeuler up -d
```
### Extend or Develop from the Image
For interactive development, mount a local checkout and reinstall it in editable mode inside the container:
```bash
git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
# Add the same Ascend --device and driver mount options shown above.
docker run --rm -it \
--ipc=host \
-v "$PWD:/workspace/LLaMA-Factory" \
-w /workspace/LLaMA-Factory \
"$IMAGE" \
bash
pip install -e . --no-build-isolation
```
For a reproducible derived image, create a separate Dockerfile:
```dockerfile
FROM quay.io/ascend/llamafactory:latest-cann9.0.0-torch_npu2.7.1-A2-ubuntu-py3.11
COPY requirements-extension.txt /tmp/requirements-extension.txt
RUN pip install --no-cache-dir -r /tmp/requirements-extension.txt
COPY . /workspace/application
WORKDIR /workspace/application
```
Pass Ascend devices and driver mounts when running the derived image; device access should not be embedded in the image itself.
To build an image with Docker Compose without starting a container, use `docker compose --profile <profile> build`.
## Hardware Support and Compatibility Notes
- A2 images use the `910b` CANN base image; A3 images use the `a3` CANN base image.
- The image build targets both x86-64 (`linux/amd64`) and AArch64 (`linux/arm64`) hosts. This CPU architecture is independent of whether the accelerator is A2 or A3.
- The image build targets both x86-64 (`linux/amd64`) and AArch64 (`linux/arm64`) hosts. The CPU architecture is independent of whether the hardware series is A2 or A3.
- Ubuntu 22.04 and openEuler 24.03 refer to the operating system inside the container.
- The current dependency baseline aligns PyTorch `2.7.1` with torch-npu `2.7.1.post4`. Upgrading either package independently may break compatibility.
- Use a fixed release tag for reproducible production deployments. The `latest` tag can change after scheduled builds.
- Legacy short tags such as `latest-npu-a2` do not encode the CANN, torch-npu, operating system, or Python versions. Prefer the full tag format documented above.
- Legacy NPU tags are replaced by the `latest-<910b|a3>-<ubuntu|openeuler>` format.
- Validate the exact driver, firmware, CANN, and SoC combination before production deployment.
## License and Disclaimer
LLaMA Factory is distributed under the [Apache License 2.0](../../LICENSE).
LlamaFactory is distributed under the [Apache License 2.0](../../LICENSE).
Ascend CANN, torch-npu, Triton Ascend, DeepSpeed, base operating-system packages, model weights, datasets, and other third-party components are governed by their respective licenses and terms. The LLaMA Factory license does not replace or override those terms.
Ascend CANN, TorchNPU, Triton Ascend, DeepSpeed, base operating-system packages, model weights, datasets, and other third-party components are governed by their respective licenses and terms. The LlamaFactory license does not replace or override those terms.
The image is provided on an "AS IS" basis, without warranties or conditions of any kind. Users are responsible for validating hardware and software compatibility, securing the container and its runtime configuration, complying with applicable licenses and laws, and reviewing model and dataset terms before training, evaluation, or deployment.