improve fix tokenizer

This commit is contained in:
hiyouga
2024-02-09 14:53:14 +08:00
parent d0daaa01f9
commit 54ea9684ed
7 changed files with 105 additions and 71 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 137 KiB

After

Width:  |  Height:  |  Size: 142 KiB