blester125/supervocab-CohereLabs-aya-expanse-8b
Updated
supertoken
The initial checkpoints for the token comparison research.
-
-
blester125/supervocab-common-pile-comma-v0.1
Updated -
blester125/supervocab-facebook-xglm-564M
Updated -
blester125/supervocab-google-bert-bert-base-multilingual-cased
Updated -
blester125/supervocab-bigscience-bloom
Updated -
blester125/supervocab-google-bert-bert-base-uncased
Updated -
blester125/supervocab-google-byt5-small
Updated -
blester125/supervocab-google-gemma-2-2b
Updated -
blester125/supervocab-google-mt5-base
Updated -
blester125/supervocab-gpt2
Updated -
blester125/supervocab-meta-llama-Llama-3.2-1B
Updated -
blester125/supervocab-microsoft-Phi-3-mini-4k-instruct
Updated -
blester125/supervocab-mistralai-Mistral-7B-Instruct-v0.3
Updated -
blester125/supervocab-mistralai-tekken
Updated -
blester125/supervocab-Qwen-Qwen3-8B
Updated -
blester125/supervocab-tiktoken-gpt-4
Updated -
blester125/supervocab-google-t5-t5-base
Updated -
blester125/supervocab-tiktoken-gpt-4o
Updated -
blester125/supervocab-tokenmonster-englishcode-32000-consistent-v1
Updated -
blester125/supervocab-superset-vocabulary
Updated