From d2a3036a23d4cfef7c6ed0e66bf84e5e1de92161 Mon Sep 17 00:00:00 2001 From: hoshi-hiyouga Date: Wed, 21 May 2025 22:41:18 +0800 Subject: [PATCH] [misc] update data readme (#8128) --- data/README.md | 4 ++-- data/README_zh.md | 4 ++-- requirements.txt | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/data/README.md b/data/README.md index 2ed3782a..4cc92b43 100644 --- a/data/README.md +++ b/data/README.md @@ -2,7 +2,7 @@ The [dataset_info.json](dataset_info.json) contains all available datasets. If y The `dataset_info.json` file should be put in the `dataset_dir` directory. You can change `dataset_dir` to use another directory. The default value is `./data`. -Currently we support datasets in **alpaca** and **sharegpt** format. +Currently we support datasets in **alpaca** and **sharegpt** format. Allowed file types include json, jsonl, csv, parquet, arrow. ```json "dataset_name": { @@ -89,7 +89,7 @@ Regarding the above dataset, the *dataset description* in `dataset_info.json` sh ``` > [!TIP] -> If the model has reasoning capabilities but the dataset does not contain chain-of-thought (CoT), LLaMA-Factory will automatically add empty CoT to the data. When `enable_thinking` is `True` (slow thinking), the empty CoT will be added to the model responses and loss computation will be considered; otherwise (fast thinking), it will be added to the user prompts and loss computation will be ignored. Please keep the `enable_thinking` parameter consistent during training and inference. +> If the model has reasoning capabilities (e.g. Qwen3) but the dataset does not contain chain-of-thought (CoT), LLaMA-Factory will automatically add empty CoT to the data. When `enable_thinking` is `True` (slow thinking, by default), the empty CoT will be added to the model responses and loss computation will be considered; otherwise (fast thinking), it will be added to the user prompts and loss computation will be ignored. Please keep the `enable_thinking` parameter consistent during training and inference. > > If you want to train data containing CoT with slow thinking and data without CoT with fast thinking, you can set `enable_thinking` to `None`. However, this feature is relatively complicated and should be used with caution. diff --git a/data/README_zh.md b/data/README_zh.md index e63eb73c..225993b3 100644 --- a/data/README_zh.md +++ b/data/README_zh.md @@ -2,7 +2,7 @@ 其中 `dataset_info.json` 文件应放置在 `dataset_dir` 目录下。您可以通过修改 `dataset_dir` 参数来使用其他目录。默认值为 `./data`。 -目前我们支持 **alpaca** 格式和 **sharegpt** 格式的数据集。 +目前我们支持 **alpaca** 格式和 **sharegpt** 格式的数据集。允许的文件类型包括 json、jsonl、csv、parquet 和 arrow。 ```json "数据集名称": { @@ -88,7 +88,7 @@ ``` > [!TIP] -> 如果模型本身具备推理能力，而数据集不包含思维链，LLaMA-Factory 会自动为数据添加空思维链。当 `enable_thinking` 为 `True` 时（慢思考），空思维链会添加到模型回答中并且计算损失，否则会添加到用户指令中并且不计算损失（快思考）。请在训练和推理时保持 `enable_thinking` 参数一致。 +> 如果模型本身具备推理能力（如 Qwen3）而数据集不包含思维链，LLaMA-Factory 会自动为数据添加空思维链。当 `enable_thinking` 为 `True` 时（慢思考，默认），空思维链会添加到模型回答中并且计算损失，否则会添加到用户指令中并且不计算损失（快思考）。请在训练和推理时保持 `enable_thinking` 参数一致。 > > 如果您希望训练包含思维链的数据时使用慢思考，训练不包含思维链的数据时使用快思考，可以设置 `enable_thinking` 为 `None`。但该功能较为复杂，请谨慎使用。 diff --git a/requirements.txt b/requirements.txt index 1faf1c07..46d2e5f6 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,7 +4,7 @@ accelerate>=0.34.0,<=1.7.0 peft>=0.14.0,<=0.15.2 trl>=0.8.6,<=0.9.6 tokenizers>=0.19.0,<=0.21.1 -gradio>=4.38.0,<=5.29.1 +gradio>=4.38.0,<=5.30.0 scipy einops sentencepiece