Safetensors
vllm
mistral3
mistral-common
fp8
juliendenize commited on
Commit
5b4be2c
·
verified ·
1 Parent(s): dcfc440

Replace stray [THINK]/[/THINK] special tokens with reserved <SPECIAL_34>/<SPECIAL_35> slots

Browse files

This is a non-reasoning model. Renaming the unused [THINK]/[/THINK] special tokens to reserved <SPECIAL_34>/<SPECIAL_35> slots prevents tooling (e.g. mistral-common) from treating it as reasoning-capable. Only tokenizer files are changed.

Files changed (3) hide show
  1. tekken.json +2 -2
  2. tokenizer.json +2 -2
  3. tokenizer_config.json +2 -2
tekken.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e29d19ea32eb7e26e6c0572d57cb7f9eca0f4420e0e0fe6ae1cf3be94da1c0d6
3
- size 16753777
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:600bb27946565481ecf51ba8aee252e49b9a68507866080ac9c30185bb312843
3
+ size 16753784
tokenizer.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:286acad9b0e27fce778ac429763536accf618ccb6ed72963b6f94685e531c5c7
3
- size 17077402
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99cf274236c60277fcfad861a5a1007518687ad06ba8938760f50b55ffa0b1ef
3
+ size 17077420
tokenizer_config.json CHANGED
@@ -36,8 +36,8 @@
36
  "<SPECIAL_31>",
37
  "[ARGS]",
38
  "[CALL_ID]",
39
- "[THINK]",
40
- "[/THINK]",
41
  "<SPECIAL_36>",
42
  "<SPECIAL_37>",
43
  "<SPECIAL_38>",
 
36
  "<SPECIAL_31>",
37
  "[ARGS]",
38
  "[CALL_ID]",
39
+ "<SPECIAL_34>",
40
+ "<SPECIAL_35>",
41
  "<SPECIAL_36>",
42
  "<SPECIAL_37>",
43
  "<SPECIAL_38>",