Fallen Angel b4c776fa93 fix eos_token_id=0 bug
when eos_token_id=0, will never add eos_token

Former-commit-id: 3399c0d6459e797946daf966e32134a12f8e48f3
2024-02-02 17:34:48 +08:00
..
2024-01-31 17:23:56 +08:00
2024-01-21 01:47:33 +08:00
2024-02-02 17:34:48 +08:00
2024-01-20 20:15:56 +08:00
2024-01-24 16:19:18 +08:00
2024-01-22 22:27:30 +08:00
2024-01-24 16:19:18 +08:00
2024-01-21 12:34:17 +08:00
2024-01-21 19:15:27 +08:00