diff --git a/README.md b/README.md index 9c0139af..4b4e6c18 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ [![GitHub contributors](https://img.shields.io/github/contributors/hiyouga/LLaMA-Factory?color=orange)](https://github.com/hiyouga/LLaMA-Factory/graphs/contributors) [![GitHub workflow](https://github.com/hiyouga/LLaMA-Factory/actions/workflows/tests.yml/badge.svg)](https://github.com/hiyouga/LLaMA-Factory/actions/workflows/tests.yml) [![PyPI](https://img.shields.io/pypi/v/llamafactory)](https://pypi.org/project/llamafactory/) -[![Citation](https://img.shields.io/badge/citation-760-green)](https://scholar.google.com/scholar?cites=12620864006390196564) +[![Citation](https://img.shields.io/badge/citation-818-green)](https://scholar.google.com/scholar?cites=12620864006390196564) [![Docker Pulls](https://img.shields.io/docker/pulls/hiyouga/llamafactory)](https://hub.docker.com/r/hiyouga/llamafactory/tags) [![Twitter](https://img.shields.io/twitter/follow/llamafactory_ai)](https://twitter.com/llamafactory_ai) @@ -124,12 +124,12 @@ Choose your path: [25/08/06] We supported fine-tuning the **[GPT-OSS](https://github.com/openai/gpt-oss)** models. See [PR #8826](https://github.com/hiyouga/LLaMA-Factory/pull/8826) to get started. +
Full Changelog + [25/07/02] We supported fine-tuning the **[GLM-4.1V-9B-Thinking](https://github.com/THUDM/GLM-4.1V-Thinking)** model. [25/04/28] We supported fine-tuning the **[Qwen3](https://qwenlm.github.io/blog/qwen3/)** model family. -
Full Changelog - [25/04/21] We supported the **[Muon](https://github.com/KellerJordan/Muon)** optimizer. See [examples](examples/README.md) for usage. Thank [@tianshijing](https://github.com/tianshijing)'s PR. [25/04/16] We supported fine-tuning the **[InternVL3](https://huggingface.co/OpenGVLab/InternVL3-8B)** model. See [PR #7258](https://github.com/hiyouga/LLaMA-Factory/pull/7258) to get started. diff --git a/README_zh.md b/README_zh.md index 8ab37ea9..9ec81bd9 100644 --- a/README_zh.md +++ b/README_zh.md @@ -5,7 +5,7 @@ [![GitHub contributors](https://img.shields.io/github/contributors/hiyouga/LLaMA-Factory?color=orange)](https://github.com/hiyouga/LLaMA-Factory/graphs/contributors) [![GitHub workflow](https://github.com/hiyouga/LLaMA-Factory/actions/workflows/tests.yml/badge.svg)](https://github.com/hiyouga/LLaMA-Factory/actions/workflows/tests.yml) [![PyPI](https://img.shields.io/pypi/v/llamafactory)](https://pypi.org/project/llamafactory/) -[![Citation](https://img.shields.io/badge/citation-760-green)](https://scholar.google.com/scholar?cites=12620864006390196564) +[![Citation](https://img.shields.io/badge/citation-818-green)](https://scholar.google.com/scholar?cites=12620864006390196564) [![Docker Pulls](https://img.shields.io/docker/pulls/hiyouga/llamafactory)](https://hub.docker.com/r/hiyouga/llamafactory/tags) [![Twitter](https://img.shields.io/twitter/follow/llamafactory_ai)](https://twitter.com/llamafactory_ai) @@ -87,7 +87,7 @@ https://github.com/user-attachments/assets/43b700c6-a178-41db-b1f8-8190a5d3fcfc - **多种模型**：LLaMA、LLaVA、Mistral、Mixtral-MoE、Qwen、Qwen2-VL、DeepSeek、Yi、Gemma、ChatGLM、Phi 等等。 - **集成方法**：（增量）预训练、（多模态）指令监督微调、奖励模型训练、PPO 训练、DPO 训练、KTO 训练、ORPO 训练等等。 - **多种精度**：16 比特全参数微调、冻结微调、LoRA 微调和基于 AQLM/AWQ/GPTQ/LLM.int8/HQQ/EETQ 的 2/3/4/5/6/8 比特 QLoRA 微调。 -- **先进算法**：[GaLore](https://github.com/jiaweizzhao/GaLore)、[BAdam](https://github.com/Ledzy/BAdam)、[APOLLO](https://github.com/zhuhanqing/APOLLO)、[Adam-mini](https://github.com/zyushun/Adam-mini)、[Muon](https://github.com/KellerJordan/Muon)、 [OFT](https://github.com/huggingface/peft/tree/main/src/peft/tuners/oft)、DoRA、LongLoRA、LLaMA Pro、Mixture-of-Depths、LoRA+、LoftQ 和 PiSSA。 +- **先进算法**：[GaLore](https://github.com/jiaweizzhao/GaLore)、[BAdam](https://github.com/Ledzy/BAdam)、[APOLLO](https://github.com/zhuhanqing/APOLLO)、[Adam-mini](https://github.com/zyushun/Adam-mini)、[Muon](https://github.com/KellerJordan/Muon)、[OFT](https://github.com/huggingface/peft/tree/main/src/peft/tuners/oft)、DoRA、LongLoRA、LLaMA Pro、Mixture-of-Depths、LoRA+、LoftQ 和 PiSSA。 - **实用技巧**：[FlashAttention-2](https://github.com/Dao-AILab/flash-attention)、[Unsloth](https://github.com/unslothai/unsloth)、[Liger Kernel](https://github.com/linkedin/Liger-Kernel)、RoPE scaling、NEFTune 和 rsLoRA。 - **广泛任务**：多轮对话、工具调用、图像理解、视觉定位、视频识别和语音理解等等。 - **实验监控**：LlamaBoard、TensorBoard、Wandb、MLflow、[SwanLab](https://github.com/SwanHubX/SwanLab) 等等。 @@ -126,12 +126,12 @@ https://github.com/user-attachments/assets/43b700c6-a178-41db-b1f8-8190a5d3fcfc [25/08/06] 我们支持了 **[GPT-OSS](https://github.com/openai/gpt-oss)** 模型的微调。查看 [PR #8826](https://github.com/hiyouga/LLaMA-Factory/pull/8826) 以使用。 +
展开日志 + [25/07/02] 我们支持了 **[GLM-4.1V-9B-Thinking](https://github.com/THUDM/GLM-4.1V-Thinking)** 模型的微调。 [25/04/28] 我们支持了 **[Qwen3](https://qwenlm.github.io/blog/qwen3/)** 系列模型的微调。 -
展开日志 - [25/04/21] 我们支持了 **[Muon](https://github.com/KellerJordan/Muon)** 优化器。详细用法请参照 [examples](examples/README_zh.md)。感谢 [@tianshijing](https://github.com/tianshijing) 的 PR。 [25/04/16] 我们支持了 **[InternVL3](https://huggingface.co/OpenGVLab/InternVL3-8B)** 模型的微调。查看 [PR #7258](https://github.com/hiyouga/LLaMA-Factory/pull/7258) 以使用。 diff --git a/assets/wechat.jpg b/assets/wechat.jpg index cf5ab22f..3a3c5d77 100644 Binary files a/assets/wechat.jpg and b/assets/wechat.jpg differ diff --git a/assets/wechat_npu.jpg b/assets/wechat_npu.jpg index 086992b8..2aa857d8 100644 Binary files a/assets/wechat_npu.jpg and b/assets/wechat_npu.jpg differ diff --git a/src/llamafactory/data/formatter.py b/src/llamafactory/data/formatter.py index 68a7e309..9b527a7e 100644 --- a/src/llamafactory/data/formatter.py +++ b/src/llamafactory/data/formatter.py @@ -98,7 +98,7 @@ class FunctionFormatter(StringFormatter): def apply(self, **kwargs) -> SLOTS: content: str = kwargs.pop("content") thought_words, thought = kwargs.pop("thought_words", None), None - if thought_words and len(thought_words)== 2: + if thought_words and len(thought_words) == 2: regex = re.compile(rf"{re.escape(thought_words[0])}(.*?){re.escape(thought_words[1])}", re.DOTALL) thought = re.search(regex, content) diff --git a/src/llamafactory/data/template.py b/src/llamafactory/data/template.py index f2741446..60650086 100644 --- a/src/llamafactory/data/template.py +++ b/src/llamafactory/data/template.py @@ -1855,19 +1855,17 @@ register_template( ) -#copied from seed_coder +# copied from seed_coder register_template( name="seed_oss", format_user=StringFormatter( slots=[{"bos_token"}, "user\n{{content}}", {"eos_token"}, {"bos_token"}, "assistant\n"] ), format_system=StringFormatter(slots=[{"bos_token"}, "system\n{{content}}", {"eos_token"}]), - format_function=FunctionFormatter( - slots=[{"bos_token"}, "\n{{content}}", {"eos_token"}], tool_format="seed_oss" - ), + format_function=FunctionFormatter(slots=[{"bos_token"}, "\n{{content}}", {"eos_token"}], tool_format="seed_oss"), format_tools=ToolFormatter(tool_format="seed_oss"), template_class=ReasoningTemplate, - thought_words=("", "") + thought_words=("", ""), ) diff --git a/src/llamafactory/data/tool_utils.py b/src/llamafactory/data/tool_utils.py index 6a6fd1dd..29257551 100644 --- a/src/llamafactory/data/tool_utils.py +++ b/src/llamafactory/data/tool_utils.py @@ -355,6 +355,7 @@ class GLM4MOEToolUtils(QwenToolUtils): return "\n".join(function_texts) + class SeedToolUtils(ToolUtils): r"""Seed tool using template.""" @@ -387,8 +388,7 @@ class SeedToolUtils(ToolUtils): def tool_extractor(content: str) -> Union[str, list["FunctionCall"]]: results = [] regex = re.compile( - r"\s*\s*", - re.DOTALL + r"\s*\s*", re.DOTALL ) for func_name, params_block in re.findall(regex, content): args_dict = {} @@ -405,6 +405,7 @@ class SeedToolUtils(ToolUtils): return results + TOOLS = { "default": DefaultToolUtils(), "glm4": GLM4ToolUtils(), diff --git a/src/llamafactory/extras/constants.py b/src/llamafactory/extras/constants.py index 30ed4e02..013b849b 100644 --- a/src/llamafactory/extras/constants.py +++ b/src/llamafactory/extras/constants.py @@ -645,6 +645,7 @@ register_model_group( template="falcon", ) + register_model_group( models={ "Falcon-H1-0.5B-Base": { @@ -1264,6 +1265,7 @@ register_model_group( multimodal=True, ) + register_model_group( models={ "Intern-S1-mini": { @@ -1275,6 +1277,7 @@ register_model_group( multimodal=True, ) + register_model_group( models={ "Jamba-v0.1": { @@ -3039,18 +3042,40 @@ register_model_group( models={ "Seed-Coder-8B-Base": { DownloadSource.DEFAULT: "ByteDance-Seed/Seed-Coder-8B-Base", + DownloadSource.MODELSCOPE: "ByteDance-Seed/Seed-Coder-8B-Base", }, "Seed-Coder-8B-Instruct": { DownloadSource.DEFAULT: "ByteDance-Seed/Seed-Coder-8B-Instruct", + DownloadSource.MODELSCOPE: "ByteDance-Seed/Seed-Coder-8B-Instruct", }, - "Seed-Coder-8B-Instruct-Reasoning": { + "Seed-Coder-8B-Thinking": { DownloadSource.DEFAULT: "ByteDance-Seed/Seed-Coder-8B-Reasoning-bf16", + DownloadSource.MODELSCOPE: "ByteDance-Seed/Seed-Coder-8B-Reasoning-bf16", }, }, template="seed_coder", ) +register_model_group( + models={ + "Seed-OSS-36B-Base": { + DownloadSource.DEFAULT: "ByteDance-Seed/Seed-OSS-36B-Base", + DownloadSource.MODELSCOPE: "ByteDance-Seed/Seed-OSS-36B-Base", + }, + "Seed-OSS-36B-Base-woSyn": { + DownloadSource.DEFAULT: "ByteDance-Seed/Seed-OSS-36B-Base-woSyn", + DownloadSource.MODELSCOPE: "ByteDance-Seed/Seed-OSS-36B-Base-woSyn", + }, + "Seed-OSS-36B-Instruct": { + DownloadSource.DEFAULT: "ByteDance-Seed/Seed-OSS-36B-Instruct", + DownloadSource.MODELSCOPE: "ByteDance-Seed/Seed-OSS-36B-Instruct", + }, + }, + template="seed_oss", +) + + register_model_group( models={ "Skywork-13B-Base": {