423A35C7/LLaMA-Factory
1
0
Fork 0
mirror of https://github.com/hiyouga/LLaMA-Factory.git synced 2025-08-06 13:42:51 +08:00
Code Issues Packages Projects Releases Wiki Activity

[train] fix adjusting logits size after adding special tokens (#8823)

Browse Source
This commit is contained in:
davidlightmysterion 2025-08-05 20:35:07 +08:00 committed by GitHub
parent b3445a75f0
commit 48615ddb07
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
src/llamafactory/model/model_utils/embedding.py
View File

@ -67,4 +67,5 @@ def resize_embedding_layer(model: "PreTrainedModel", tokenizer: "PreTrainedToken
_noisy_mean_initialization(model.get_input_embeddings().weight.data, num_new_tokens)
_noisy_mean_initialization(model.get_output_embeddings().weight.data, num_new_tokens)
model.config.vocab_size = new_embedding_size
logger.info_rank0(f"Resized token embeddings from {current_embedding_size} to {new_embedding_size}.")