docs: add Qwen3.8 LlamaFactory SFT skill (#10901)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

.agents/skills/llamafactory-qwen38-sft/references/ascend-qlora.md

@@ -0,0 +1,15 @@
+# Ascend 4-bit QLoRA dependency path
+
+Read this reference only for single-device 4-bit QLoRA on Ascend. The LlamaFactory checkout's current README and `examples/train_qlora/qwen3_lora_sft_bnb_npu.yaml` remain authoritative.
+
+Build the NPU backend from the `multi-backend-refactor` branch of `bitsandbytes-foundation/bitsandbytes` with CMake `COMPUTE_BACKEND=npu`; a regular CUDA-oriented bitsandbytes wheel is not an Ascend acceptance result.
+
+Prefer building a wheel in an isolated build environment and installing that wheel into the training environment with `--no-deps`. Record the bitsandbytes commit and wheel checksum so the environment can be reproduced without letting the build toolchain replace the selected Python or PyTorch stack.
+
+Before training:
+
+1. Import bitsandbytes from the selected training environment and confirm that the NPU shared library, rather than a CUDA fallback, is loaded.
+2. Run a small NPU tensor-quantization operation. Import success alone is insufficient.
+3. Preserve `double_quantization: false` from the repository's NPU example unless a separately validated backend revision supports otherwise.
+
+If import or the backend operation fails, preserve the build and runtime logs before troubleshooting. Inspect `libbitsandbytes_npu.so` with `readelf` and `ldd`, confirm that its dependencies resolve, and check whether its RPATH or RUNPATH refers to a removed build environment. Stop without replacing the requested QLoRA path with another tuning method when the backend gate remains unresolved.

.agents/skills/llamafactory-qwen38-sft/references/serving.md

@@ -0,0 +1,137 @@
+# Artifact evaluation and serving
+
+Use this reference only after a training artifact exists and passes the reload requirement in [training.md](training.md).
+
+## Identify the artifact
+
+- A QLoRA run produces an adapter. Keep the original base model and load it with `adapter_name_or_path`.
+- A HyperParallel full-SFT run produces a directly loadable Hugging Face model. Do not confuse a resumable distributed checkpoint with the final Hugging Face artifact.
+
+Confirm the expected model or adapter files, trainer state, metrics, and logs before evaluation or export.
+
+## Evaluate
+
+Use an independent dataset or a validation split fixed before training. Register it as a separate LlamaFactory dataset, retain stable sample IDs, and check that neither the prompts nor their reference answers occur in the training set. A smoke dataset or examples seen during training can validate plumbing but cannot establish a quality gain.
+
+Install the repository's generation-metric dependencies with `python -m pip install -r requirements/metrics.txt` before using `predict_with_generate`; a training-only environment may not contain `jieba`, `nltk`, or `rouge-chinese`. Re-run `python -m pip check` after installation.
+
+For instruction-following tasks, run generation on the held-out set with `do_predict: true` and `predict_with_generate: true`. Use the configuration for the artifact type instead of adapting QLoRA fields implicitly.
+
+### QLoRA base and tuned runs
+
+Use this configuration for both runs. Keep `model_name_or_path` on the same base model and add `adapter_name_or_path` only to the tuned run. Preserve the training run's quantization loading fields in both evaluation configurations; in particular, Ascend QLoRA requires `double_quantization: false`:
+
+```yaml
+model_name_or_path: <base-model-path>
+# adapter_name_or_path: <adapter-path>  # Tuned run only.
+quantization_bit: 4
+quantization_method: bnb
+double_quantization: <same-as-training>  # false for the repository's Ascend QLoRA path.
+trust_remote_code: true
+
+stage: sft
+do_predict: true
+finetuning_type: lora
+
+eval_dataset: <held-out-dataset-name>
+template: qwen3_8
+# enable_thinking: false  # Optional; set deliberately and keep identical in both runs.
+cutoff_len: <evaluation-cutoff>
+overwrite_cache: true
+preprocessing_num_workers: <worker-count>
+dataloader_num_workers: <worker-count>
+
+output_dir: <unique-evaluation-output>
+overwrite_output_dir: true
+report_to: none
+
+per_device_eval_batch_size: 1
+predict_with_generate: true
+do_sample: false
+max_new_tokens: <generation-limit>
+seed: <fixed-seed>
+```
+
+### HyperParallel full-SFT base and tuned runs
+
+Use a full-tuning evaluation configuration with no adapter fields. Point the base run at the original model and the tuned run at the final Hugging Face artifact:
+
+```yaml
+model_name_or_path: <base-model-path-or-tuned-full-model-path>
+trust_remote_code: true
+
+stage: sft
+do_predict: true
+finetuning_type: full
+
+eval_dataset: <held-out-dataset-name>
+template: qwen3_8
+# enable_thinking: false  # Optional; set deliberately and keep identical in both runs.
+cutoff_len: <evaluation-cutoff>
+overwrite_cache: true
+preprocessing_num_workers: <worker-count>
+dataloader_num_workers: <worker-count>
+
+output_dir: <unique-evaluation-output>
+overwrite_output_dir: true
+report_to: none
+
+per_device_eval_batch_size: 1
+predict_with_generate: true
+do_sample: false
+max_new_tokens: <generation-limit>
+seed: <fixed-seed>
+```
+
+Do not carry `adapter_name_or_path`, `finetuning_type: lora`, or QLoRA quantization fields into the full-SFT configuration by accident. Prefer the deployment precision when it fits the evaluation hardware. If both full-model runs require on-the-fly 4-bit loading, add identical quantization fields to both configurations and disclose that their absolute scores may differ from unquantized evaluation.
+
+Run each configuration with `llamafactory-cli train <eval.yaml>`. Despite the command name, `do_predict: true` performs prediction without training and writes `generated_predictions.jsonl` under `output_dir`.
+
+Create one run for the base model and one for the tuned artifact:
+
+- Give every run a separate output directory. Keep the held-out data, template, thinking settings, cutoff, decoding parameters, seed, and hardware policy identical.
+
+Choose metrics before inspecting the outputs. Use deterministic checks for structured tasks, such as exact match, schema validity, required-field recall, tool-call validity, or task-specific unit tests. For open-ended answers, use a fixed rubric and blinded human or judge scoring, and retain representative wins, ties, and regressions. Token-level loss or perplexity can supplement generation metrics but should not replace task-level evaluation.
+
+Report paired base and tuned results, the number of evaluated samples, the evaluation-data version, metric definitions, and decoding settings. Claim improvement only when the predefined acceptance criteria pass without unacceptable regressions. A decreasing training loss alone is not evidence that tuning improved behavior.
+
+## Export a QLoRA adapter
+
+Create the export configuration outside the model and checkpoint directories:
+
+```yaml
+model_name_or_path: <base-model-path>
+adapter_name_or_path: <adapter-checkpoint-path>
+template: qwen3_8
+finetuning_type: lora
+export_dir: <merged-model-output-path>
+export_size: 5
+export_device: cpu
+export_legacy_format: false
+```
+
+Run `llamafactory-cli export <export.yaml>`, then reload the tokenizer, processor, and model from the merged directory. Retain the original adapter until the merged artifact passes this check. Full-SFT artifacts do not require adapter merging; reload their final Hugging Face directory directly.
+
+## Verify direct inference and the local API
+
+Perform direct inference first with an artifact-appropriate YAML. Then start the service locally:
+
+```bash
+API_HOST=127.0.0.1 API_PORT=8000 API_MODEL_NAME=<display-name> \
+  llamafactory-cli api <inference.yaml>
+```
+
+From another terminal, verify discovery and generation:
+
+```bash
+curl -sS http://127.0.0.1:8000/v1/models
+curl -sS http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"<display-name>","messages":[{"role":"user","content":"<held-out-prompt>"}],"max_tokens":128}'
+```
+
+Inspect the response content, not only the HTTP status. If `API_KEY` is set, send a bearer token without printing or storing it. Keep the bind address local unless external exposure has been explicitly authorized and secured.
+
+## Completion record
+
+Report the code and package revisions, hardware, model and data versions, exact configs and commands, training and evaluation results, artifact paths, and API results. Include failures, fixes, and untested limitations. Remove credentials, internal addresses, usernames, absolute private paths, private model or dataset names, and sensitive samples before publishing.

.agents/skills/llamafactory-qwen38-sft/references/setup.md

@@ -0,0 +1,62 @@
+# Setup and compatibility
+
+Run commands from the LlamaFactory repository root. Treat the current checkout's `README`, `pyproject.toml`, `requirements/`, examples, and source as authoritative.
+
+## Record the starting state
+
+Before installing or launching anything, record:
+
+- the LlamaFactory commit and selected Python environment;
+- accelerator type, driver and CUDA or CANN version, and selected device IDs;
+- device utilization, process ownership, and available device memory;
+- free space in the model-cache and training-output filesystems.
+
+On Ascend, source the matching toolkit `set_env.sh` in every shell that invokes LlamaFactory. Confirm that the environment, `llamafactory-cli`, `torchrun`, and `accelerate` resolve from the intended installation.
+
+## Install the selected stack
+
+Use an isolated environment and install in this order:
+
+1. Install the hardware-compatible PyTorch stack. On Ascend, use the torch and torch-npu pair pinned by `requirements/npu.txt` and ensure `pyyaml` is installed before importing torch-npu.
+2. Install the LlamaFactory checkout with `python -m pip install -e .`.
+3. Install only the optional dependencies required by the selected training path.
+4. Install `transformers==5.8.0`, the version validated for this workflow and accepted by this checkout.
+5. For HyperParallel, use the version declared compatible by the current LlamaFactory checkout. Until LlamaFactory declares a fixed version, use the latest `master`, resolve it to an immutable full commit SHA, and record that SHA with the run configuration. Before installing the source checkout, explicitly install its compatible build/runtime prerequisites with `python -m pip install numpy==1.26.4 pybind11`; the NumPy pin satisfies HyperParallel's declared `numpy>=1.26,<2.0` constraint instead of accepting the resolver's latest NumPy. Confirm that `HyperParallelArguments` and `parallelize_model` import, then use the smallest representative training smoke test as the compatibility gate. Install the source checkout with `python -m pip install --no-deps -e <hyperparallel-checkout>`. Do not select accelerator extras, because they can replace the torch stack selected in step 1.
+
+For Ascend QLoRA, continue with [ascend-qlora.md](ascend-qlora.md). Do not load those build instructions for CUDA QLoRA or HyperParallel runs.
+
+Do not disable LlamaFactory's version check. If `pyproject.toml` and `check_dependencies()` declare different bounds, obey the stricter bound. Record declared constraints and installed versions separately from the smoke-test result.
+
+## Dependency gates
+
+Run the gates required by the selected path:
+
+```bash
+python -m pip check
+llamafactory-cli version
+python -c 'import torch, transformers, accelerate, peft; assert transformers.__version__ == "5.8.0"; print(torch.__version__, transformers.__version__, accelerate.__version__, peft.__version__)'
+python -c 'import torch_npu; print(torch_npu.__version__)'  # Ascend only
+python -c 'import bitsandbytes'                            # QLoRA only
+python -c 'import hyper_parallel; from hyper_parallel.integration.llamafactory import HyperParallelArguments, parallelize_model; print(hyper_parallel.__file__, HyperParallelArguments())'  # HyperParallel only
+```
+
+For Ascend QLoRA, complete the backend gate in [ascend-qlora.md](ascend-qlora.md); importing bitsandbytes alone does not prove that its NPU backend works. For HyperParallel, verify the printed checkout and both public integration symbols. When using `master`, an unrecorded moving branch or the development version string alone is not reproducible evidence. An import gate does not replace the training smoke test.
+
+For this workflow, HyperParallel's declared `transformers>=5.11.0` requirement is a known metadata-only conflict with the validated `transformers==5.8.0` stack. Install HyperParallel from the recorded source revision with `--no-deps`, report this exact `pip check` finding, and do not treat this finding alone as a failed gate. Do not ignore any other dependency conflict. The HyperParallel integration imports and the real multi-device smoke test remain mandatory compatibility gates.
+
+## Model compatibility and cache gates
+
+Before a full download or training launch, confirm that the current checkout maps `Qwen3.8-27B` to `Qwen/Qwen3.8-27B` and template `qwen3_8`. Confirm with Transformers 5.8.0 that:
+
+- `AutoConfig.model_type` is `qwen3_5`;
+- the model class is `Qwen3_5ForConditionalGeneration`;
+- the tokenizer, processor, and LlamaFactory template load.
+
+Before declaring a local model complete, require:
+
+- every shard named by `model.safetensors.index.json`, validated against the resolved upstream revision;
+- `config.json`, tokenizer files, `preprocessor_config.json`, and `video_preprocessor_config.json`;
+- no relevant incomplete download files;
+- read access for the user that will run training.
+
+Stop at a failed gate, preserve its log, and report the failure without substituting another model, backend, or training method.

.agents/skills/llamafactory-qwen38-sft/references/training.md

@@ -0,0 +1,54 @@
+# Training workflows
+
+Use this reference after the applicable gates in [setup.md](setup.md) pass. Resolve dataset, sequence length, sample limit, gradient accumulation, duration, and output paths for the actual hardware instead of copying them from an example.
+
+Quote YAML values that YAML 1.1 may coerce to booleans when the downstream option expects a string. In particular, disable checkpoint saves with `save_strategy: "no"`, not an unquoted `save_strategy: no`; PyYAML otherwise resolves `no` to `False`, which Transformers rejects as an invalid `SaveStrategy`.
+
+## Path A: single-device 4-bit QLoRA
+
+On Ascend, start from `examples/train_qlora/qwen3_lora_sft_bnb_npu.yaml`. On CUDA, start from `examples/train_qlora/qwen3_lora_sft_otfq.yaml`. Copy the selected file into the run directory, change the model to `Qwen/Qwen3.8-27B` and the template to `qwen3_8`, then resolve the dataset, output path, and run-size controls for the requested workflow.
+
+Launch with one previously selected device:
+
+```bash
+ASCEND_RT_VISIBLE_DEVICES=<id> llamafactory-cli train <single-device-smoke.yaml>
+```
+
+Use `CUDA_VISIBLE_DEVICES` on CUDA. Accept the path only after measuring its actual loading and training peak. The output is a LoRA adapter and must later be loaded with the same base model.
+
+## Path B: multi-device HyperParallel FSDP2 full SFT
+
+### Ascend smoke-tested path
+
+On Ascend, use `examples/ascend/qwen3_8_full_sft_hyper_fsdp2.yaml` directly as the repository training configuration. It already selects full SFT, HyperParallel, `qwen3_8`, batch size 1, and bf16; do not rebuild those settings from the Qwen3.5 example.
+
+Copy `examples/accelerate/fsdp2_config_qwen38.yaml` into the run directory as well and set `num_processes` to the selected visible-device count. The example defaults to 16 processes; a smaller world size may require additional memory controls for full tuning. Its wrap classes use Qwen3.5 names because Qwen3.8 uses the `Qwen3_5` Transformers implementation.
+
+Launch on the selected devices with the platform visibility variable:
+
+```bash
+ASCEND_RT_VISIBLE_DEVICES=<id0,id1,...> accelerate launch \
+  --config_file <fsdp2-smoke.yaml> \
+  src/train.py <multi-device-smoke.yaml>
+
+```
+
+This path is full tuning and does not use quantization. Begin with the repository example unchanged except for the model, data, output, and hardware-dependent run controls. Tune parallelism or memory settings only after that baseline is understood.
+
+Retain the repository example's `bf16: true` because compute precision materially changes memory and training behavior. Do not repeat stable LlamaFactory defaults merely for visibility; write fields when overriding a default or when the resolved configuration must be portable across framework revisions.
+
+### CUDA validation path
+
+The repository does not currently ship a CUDA-specific Qwen3.8 HyperParallel example, and this workflow has not server-validated that combination. Do not present the Ascend result as CUDA support merely by replacing `ASCEND_RT_VISIBLE_DEVICES` with `CUDA_VISIBLE_DEVICES`.
+
+When CUDA HyperParallel validation is explicitly requested, copy the checked-in training and Accelerate YAMLs into a run-specific directory, change only the model, data, output, device-count, and other demonstrated hardware-dependent controls, and launch with `CUDA_VISIBLE_DEVICES`. Apply the same real-model acceptance gate below and record the CUDA, PyTorch, Transformers, HyperParallel, and LlamaFactory revisions. Mark the result pending until the run completes; only a passing run may be reported as server-validated.
+
+## Performance disclosure
+
+Most Qwen3.8 layers use linear attention. Record whether `flash-linear-attention` and `causal-conv1d` enable the optimized path or Transformers falls back to its torch implementation. Do not compare or promise optimized throughput for a fallback run.
+
+## Training acceptance
+
+Before a formal run, execute a bounded smoke test with the selected training path and the real model. Confirm the intended backend and ranks, model loading, data preprocessing, forward, backward, and at least two consecutive optimizer updates with finite losses. A single step can finish before optimizer-state memory reaches its steady level.
+
+For a requested training artifact, require a clean exit, complete expected files, and successful reload before evaluation, export, or serving. Record the environment, device count, exact command and configuration, duration, peak memory, loss or evaluation metrics, artifact paths, and reload result. Choose checkpoint retention deliberately, verify output space before training, and use a fresh output directory after the smoke test.