ai-models/Kimi-VL-A3B-Thinking-2506
2
0
Fork 0
mirror of https://www.modelscope.cn/moonshotai/Kimi-VL-A3B-Thinking-2506.git synced 2026-04-02 19:13:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

Transformers v5 support (#13)

Browse Source 
- Transformers v5 support (f2ab9d166e00f16237b0e09749df2faacc864d1e)


Co-authored-by: Harry Mellor <hmellor@users.noreply.huggingface.co> (batch 1/1)
This commit is contained in:
systemd
2026-01-30 12:15:23 +00:00
parent 8df1310c81
commit f38d1057ba
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tokenization_moonshot.py
View File

@ -17,7 +17,7 @@ from tiktoken.load import load_tiktoken_bpe
from tokenizers import AddedToken from tokenizers import AddedToken
from transformers.tokenization_utils import PreTrainedTokenizer from transformers.tokenization_utils import PreTrainedTokenizer
from transformers.utils import to_py_obj from transformers.utils import to_py_obj
from transformers.models.gpt2.tokenization_gpt2 import bytes_to_unicode from transformers.convert_slow_tokenizer import bytes_to_unicode
logger = getLogger(__name__) logger = getLogger(__name__)