Upload prefix-none a8m model artifacts for 20260707

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+    "word_embedding_dimension": 384,
+    "pooling_mode_cls_token": false,
+    "pooling_mode_mean_tokens": true,
+    "pooling_mode_max_tokens": false,
+    "pooling_mode_mean_sqrt_len_tokens": false,
+    "pooling_mode_weightedmean_tokens": false,
+    "pooling_mode_lasttoken": false,
+    "include_prompt": true
+}

README.md

@@ -0,0 +1,513 @@
+---
+language:
+- multilingual
+license: mit
+base_model:
+- hotchpotch/mmBERT-L4H384-pruned
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- embeddings
+- feature-extraction
+- modernbert
+- matryoshka
+- multilingual
+- retrieval
+- onnx
+- openvino
+---
+
+<p align="center">
+  <img src="https://storage.googleapis.com/secons-site-images/other/huggingface/bekko/bekko-logo.webp" alt="bekko" width="520">
+</p>
+
+# bekko-embedding-v1-a8m
+
+**bekko-embedding-v1-a8m** is an ultra-small multilingual text-embedding model — just **8M active parameters (AP)** and practical even on low-spec CPUs — whose retrieval quality rivals models with **3–10x more active parameters**.
+
+For higher retrieval quality, see the larger [bekko-embedding-v1-a25m](https://huggingface.co/hotchpotch/bekko-embedding-v1-a25m) (25M active parameters).
+
+> [!NOTE]
+> 🚧 **WIP** — we're writing the technical paper.
+
+## Highlights
+
+- 🎯 Retrieval quality rivals models with **3–10x more active parameters**.
+- 🌍 **Multilingual** — 100+ languages supported.
+- 🪶 Just **8M active parameters**, so inference stays fast and cheap.
+- 📜 Long context up to **8k tokens**.
+- 🪆 Native **384-dim** embeddings, Matryoshka-truncatable to **256 / 128 / 64**.
+- 💻 Runs on modest hardware. The default **OpenVINO** / **ONNX** exports compress the static token embedding table to row-wise int8, about one quarter of fp32 for that table, making the main artifact about **124 MiB** while keeping quality nearly unchanged.
+- ⚡ Fast on GPU, with Flash Attention / SDPA when the runtime supports it.
+- 🔓 **MIT License**.
+
+## Quickstart
+
+Install Sentence Transformers (5.0 or newer):
+
+```bash
+pip install -U "sentence-transformers>=5.0"
+```
+
+Use `encode_query()` and `encode_document()` for retrieval (these convenience methods require sentence-transformers 5.0+). They apply the configured query/document prompts automatically. Normalize embeddings when using cosine similarity or dot-product search on normalized vectors.
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer("hotchpotch/bekko-embedding-v1-a8m")
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+
+query_emb = model.encode_query(query, normalize_embeddings=True)
+doc_emb = model.encode_document(docs, normalize_embeddings=True)
+scores = util.cos_sim(query_emb, doc_emb)[0]
+
+print(scores)
+print("best doc:", docs[int(scores.argmax())])
+```
+
+Example output, with exact scores varying slightly by backend:
+
+```text
+tensor([0.2557, 0.2128, 0.2118, 0.3944])
+best doc: A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.
+```
+
+
+Want it faster? For CPU- and GPU-specific tuning — ONNX, OpenVINO, Flash Attention, and quantization — see [Optimized Inference](#optimized-inference) below.
+
+## Model Details
+
+| Item | Value |
+| --- | --- |
+| Model type | Sentence Transformer dense embedding model |
+| Architecture | mmBERT (ModernBERT-style) encoder, 4 layers, hidden size 384 |
+| Base model | [hotchpotch/mmBERT-L4H384-pruned](https://huggingface.co/hotchpotch/mmBERT-L4H384-pruned), pruned from [mmBERT-small](https://huggingface.co/jhu-clsp/mmBERT-small) |
+| Active parameters | 7,671,168 |
+| Total parameters | 105,975,168 |
+| Embedding dimension | 384 |
+| Supported truncate dimensions | 256, 128, 64 |
+| Max sequence length | 8192 tokens |
+| Pooling | Mean pooling |
+| Similarity | Cosine similarity |
+
+## Benchmark results
+
+These are **retrieval** scores on a suite of multilingual *Nano* benchmarks — compact, fast subsets of widely used retrieval benchmarks — measured with [HAKARI-Bench](https://huggingface.co/spaces/hakari-bench/leaderboard). Higher is better.
+
+What each column means:
+
+- **Overall** — HAKARI-Bench Overall, the **micro-average** across all the sets below
+- **MNanoBEIR** — multilingual NanoBEIR, general-purpose retrieval
+- **NanoMMTEB-v2** — Nano subset of MMTEB v2 (massive multilingual retrieval)
+- **NanoRTEB** — multilingual retrieval benchmark
+- **NanoLongEmbed** — long-document retrieval
+- **NanoCoIR** — code retrieval
+
+| Model | Active Params | Overall | MNanoBEIR | NanoMMTEB-v2 | NanoRTEB | NanoLongEmbed | NanoCoIR |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| [bekko-embedding-v1-a8m](https://huggingface.co/hotchpotch/bekko-embedding-v1-a8m) | 7.7M | 0.545 | 0.528 | 0.478 | 0.543 | 0.681 | 0.741 |
+| [bekko-embedding-v1-a25m](https://huggingface.co/hotchpotch/bekko-embedding-v1-a25m) | 24.9M | 0.576 | 0.555 | 0.506 | 0.590 | 0.708 | 0.780 |
+| [multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) | 21.6M | 0.517 | 0.512 | 0.445 | 0.471 | 0.501 | 0.692 |
+| [granite-97m-multilingual-r2](https://huggingface.co/ibm-granite/granite-embedding-97m-multilingual-r2) | 28.3M | 0.525 | 0.505 | 0.531 | 0.567 | 0.659 | 0.780 |
+| [harrier-oss-v1-270m](https://huggingface.co/microsoft/harrier-oss-v1-270m) | 100.3M | 0.555 | 0.523 | 0.522 | 0.550 | 0.617 | 0.789 |
+| [granite-311m-multilingual-r2](https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2) | 110.3M | 0.569 | 0.543 | 0.577 | 0.606 | 0.695 | 0.814 |
+| [gte-multilingual-base](https://huggingface.co/Alibaba-NLP/gte-multilingual-base) | 113.3M | 0.563 | 0.527 | 0.486 | 0.558 | 0.669 | 0.753 |
+| [multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 303.9M | 0.565 | 0.560 | 0.484 | 0.556 | 0.505 | 0.747 |
+| [bge-m3](https://huggingface.co/BAAI/bge-m3) | 311.8M | 0.577 | 0.557 | 0.485 | 0.536 | 0.653 | 0.692 |
+
+### Score vs active parameters
+
+The best models sit toward the upper-left: higher retrieval score with fewer active parameters. The chart includes more multilingual models than the table above; bekko sits on the efficiency frontier at the smallest sizes.
+
+<p align="center">
+  <img src="https://storage.googleapis.com/secons-site-images/other/huggingface/bekko/hakari_score_vs_active_params_v2.png" alt="HAKARI-Bench overall vs active parameters" width="600">
+</p>
+
+## Why active parameters?
+
+For a transformer encoder, most of the inference cost — and most of the latency — comes from the **non-embedding ("active") parameters**: the attention and feed-forward layers that run on every token. The token embedding table dominates the **total** parameter count, but at inference it is just a lookup, not heavy computation.
+
+So a model can be larger on disk yet still run fast, as long as its active-parameter count is small. bekko-embedding-v1-a8m has only **8M active parameters**, which is why it stays fast on CPU and GPU even though its total size is ~106M parameters — most of which is the multilingual embedding table. The default OpenVINO / ONNX artifacts store that static token embedding table as row-wise int8, so the main artifact is about **124 MiB** instead of ~404 MiB fp32.
+
+### Speed vs other models
+
+> [!NOTE]
+> 🚧 TBD — head-to-head throughput / latency comparisons against other models (CPU and GPU) will be added here.
+
+## Optimized Inference
+
+**For CPU, OpenVINO is the recommended backend.** In our tests (NanoBEIR-en) it is the only backend that meaningfully improves speed — about **3.8x faster on a desktop CPU** and **~1.7x on a Raspberry Pi 5**. The default OpenVINO / ONNX artifacts use row-wise int8 only for the large static token embedding table, shrinking the main file from ~404 MiB fp32 to about **124 MiB** with near-identical retrieval quality. ONNX is the recommended browser artifact; for CPU throughput, prefer OpenVINO. Quantized `qint8` / `quint8` artifacts are different, more aggressive experiments and are not default choices.
+
+<details>
+<summary>NVIDIA GPU</summary>
+
+Modern PyTorch and Transformers can use SDPA efficiently. If Flash Attention 2 is installed and supported by your GPU/runtime, you can request it explicitly.
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer(
+    "hotchpotch/bekko-embedding-v1-a8m",
+    device="cuda",
+    model_kwargs={"attn_implementation": "flash_attention_2"},
+)
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+
+scores = util.cos_sim(
+    model.encode_query(query, normalize_embeddings=True),
+    model.encode_document(docs, normalize_embeddings=True),
+)[0]
+print(scores)
+```
+
+If Flash Attention 2 is unavailable, use `model_kwargs={"attn_implementation": "sdpa"}` or omit the argument and use the framework default.
+
+</details>
+
+<details>
+<summary>Mac (Apple Silicon)</summary>
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer("hotchpotch/bekko-embedding-v1-a8m", device="mps")
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+
+scores = util.cos_sim(
+    model.encode_query(query, normalize_embeddings=True),
+    model.encode_document(docs, normalize_embeddings=True),
+)[0]
+print(scores)
+```
+
+</details>
+
+<details>
+<summary>OpenVINO CPU — recommended for CPU</summary>
+
+OpenVINO is the recommended backend for CPU inference (Intel, AMD, and Arm such as Raspberry Pi). The default OpenVINO IR stores the static token embedding table as row-wise int8 with fp32 scales; the transformer layers remain fp32. The main binary is about **124 MiB** instead of ~404 MiB fp32, load-time CPU memory use is lower in our tests, and NanoMIRACL + NanoCoIR mean nDCG@10 dropped by only about **0.06%**.
+
+```bash
+pip install -U "sentence-transformers[openvino]>=5.0"
+```
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer(
+    "hotchpotch/bekko-embedding-v1-a8m",
+    backend="openvino",
+    device="cpu",
+    model_kwargs={"file_name": "openvino/openvino_model.xml", "device": "CPU"},
+)
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+
+scores = util.cos_sim(
+    model.encode_query(query, normalize_embeddings=True),
+    model.encode_document(docs, normalize_embeddings=True),
+)[0]
+print(scores)
+```
+
+</details>
+
+<details>
+<summary>ONNX Runtime / Browser — recommended for browser</summary>
+
+The default ONNX artifact (`onnx/model.onnx`) is the recommended browser artifact. It keeps the tokenizer and full vocabulary unchanged, while storing the static token embedding table as row-wise int8 with fp32 scales. The file is about **124 MiB** instead of ~404 MiB fp32, and mean nDCG@10 dropped by only about **0.06%** across our Nano retrieval evaluation. Use it for Transformers.js / browser and ONNX Runtime compatibility; for CPU throughput, prefer OpenVINO.
+
+```bash
+npm install @huggingface/transformers
+```
+
+```js
+import { pipeline } from "@huggingface/transformers";
+
+const extractor = await pipeline(
+  "feature-extraction",
+  "hotchpotch/bekko-embedding-v1-a8m",
+  {
+    device: navigator.gpu ? "webgpu" : "wasm",
+    // Transformers.js maps dtype="fp32" to onnx/model.onnx.
+    // In this repo, that file is the compact static-embedding-int8 ONNX model.
+    dtype: "fp32",
+  },
+);
+
+const queryEmbedding = await extractor("What are the characteristics of sushi?", {
+  pooling: "mean",
+  normalize: true,
+});
+
+const documentEmbedding = await extractor(
+  "A Japanese dish made with vinegared rice and seafood.",
+  { pooling: "mean", normalize: true },
+);
+
+console.log(queryEmbedding.tolist()[0].slice(0, 8));
+console.log(documentEmbedding.tolist()[0].slice(0, 8));
+```
+
+```bash
+pip install -U "sentence-transformers[onnx]>=5.0"
+```
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer(
+    "hotchpotch/bekko-embedding-v1-a8m",
+    backend="onnx",
+    device="cpu",
+    model_kwargs={"file_name": "onnx/model.onnx", "provider": "CPUExecutionProvider"},
+)
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+print(util.cos_sim(model.encode_query(query, normalize_embeddings=True), model.encode_document(docs, normalize_embeddings=True))[0])
+```
+
+</details>
+
+<details>
+<summary>Smaller embeddings with Matryoshka (truncate_dim)</summary>
+
+These models are trained with Matryoshka representation learning, so you can shrink the **384-dim** embeddings to **256**, **128**, or **64** dimensions by passing `truncate_dim`. Smaller dimensions reduce index size and speed up similarity search, at a small cost in retrieval quality (see [Truncation and Quantization](#truncation-and-quantization)).
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+# Full embedding is 384-dim; 256 / 128 / 64 are supported.
+model = SentenceTransformer("hotchpotch/bekko-embedding-v1-a8m", truncate_dim=256)
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+emb = model.encode_query(query, normalize_embeddings=True)
+print("embedding dim:", emb.shape[-1])
+print(util.cos_sim(emb, model.encode_document(docs, normalize_embeddings=True))[0])
+```
+
+</details>
+
+<details>
+<summary>OpenVINO qint8 (not recommended)</summary>
+
+This is different from the default static-token-embedding int8 OpenVINO artifact above. The qint8 OpenVINO artifact quantizes more of the model and is kept only for experimentation. On these already-small models, qint8 can noticeably degrade retrieval quality and does not reliably improve latency, so it is not a default choice.
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer(
+    "hotchpotch/bekko-embedding-v1-a8m",
+    backend="openvino",
+    device="cpu",
+    model_kwargs={"file_name": "openvino/openvino_model_qint8_not_recommended.xml", "device": "CPU"},
+)
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+print(util.cos_sim(model.encode_query(query, normalize_embeddings=True), model.encode_document(docs, normalize_embeddings=True))[0])
+```
+
+</details>
+
+<details>
+<summary>ONNX qint8 / quint8 (not recommended)</summary>
+
+These files are different from the default static-token-embedding int8 ONNX artifact above. Quantized ONNX files are platform-specific experiments. They can noticeably degrade retrieval quality on these small models, so validate quality on your target hardware before using them.
+
+```bash
+pip install -U "sentence-transformers[onnx]>=5.0"
+```
+
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer(
+    "hotchpotch/bekko-embedding-v1-a8m",
+    backend="onnx",
+    device="cpu",
+    model_kwargs={
+        "file_name": "onnx/model_qint8_avx512_not_recommended.onnx",
+        "provider": "CPUExecutionProvider",
+    },
+)
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+print(util.cos_sim(model.encode_query(query, normalize_embeddings=True), model.encode_document(docs, normalize_embeddings=True))[0])
+```
+
+</details>
+
+## Other inference methods
+
+Beyond the Sentence Transformers backends above, you can also serve or run the model with:
+
+<details>
+<summary>Text Embeddings Inference (production API)</summary>
+
+Text Embeddings Inference (TEI) is a Rust-based production inference server from Hugging Face, with Docker images, dynamic batching, and production features such as Prometheus metrics and tracing.
+
+Before deploying, confirm your TEI version supports this model's encoder architecture, and pick the image tag that matches your target — a CPU image, or a GPU image for your specific architecture. See the [TEI image list](https://github.com/huggingface/text-embeddings-inference#docker-images) for current tags.
+
+```bash
+model=hotchpotch/bekko-embedding-v1-a8m
+volume=$PWD/tei-data
+# Replace <tag> with the current TEI image for your hardware (CPU, or your GPU arch).
+# Add `--gpus all` when using a GPU image.
+docker run -p 8080:80 -v "$volume:/data" --pull always \
+  ghcr.io/huggingface/text-embeddings-inference:<tag> \
+  --model-id "$model"
+```
+
+Send raw text inputs to the endpoint and normalize embeddings before scoring.
+
+```python
+import requests
+import numpy as np
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+
+q = np.array(requests.post("http://127.0.0.1:8080/embed", json={"inputs": [query]}).json()[0])
+d = np.array(requests.post("http://127.0.0.1:8080/embed", json={"inputs": docs}).json())
+q = q / np.linalg.norm(q)
+d = d / np.linalg.norm(d, axis=1, keepdims=True)
+print(d @ q)
+```
+
+</details>
+
+<details>
+<summary>Transformers library</summary>
+
+Use raw text inputs with pure Transformers.
+
+```python
+import torch
+import torch.nn.functional as F
+from transformers import AutoModel, AutoTokenizer
+
+model_id = "hotchpotch/bekko-embedding-v1-a8m"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModel.from_pretrained(model_id, attn_implementation="sdpa").eval()
+
+def embed(texts):
+    batch = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
+    with torch.no_grad():
+        out = model(**batch).last_hidden_state
+    mask = batch["attention_mask"].unsqueeze(-1)
+    pooled = (out * mask).sum(dim=1) / mask.sum(dim=1).clamp(min=1)
+    return F.normalize(pooled, p=2, dim=1)
+
+query = "What are the characteristics of sushi?"
+docs = [
+    "A warm noodle soup served in broth with sliced toppings.",
+    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
+    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
+    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
+]
+
+scores = embed(docs) @ embed([query]).T
+print(scores.squeeze(-1))
+```
+
+</details>
+
+## Truncation and Quantization
+
+The results below are for `bekko-embedding-v1-a8m`. Use 384 dimensions for maximum quality. Use 256 dimensions when index size matters and quality loss must stay small. Quantized search-only variants lose more quality on these small models; rescoring recovers most of that loss.
+
+| Setting | Dim | Encoding | Rescore | HAKARI overall | Delta vs 384-dim float | Recommended use |
+| --- | ---: | --- | :---: | ---: | ---: | --- |
+| Full quality | 384 | float | No | 0.545 | - | Default choice |
+| Smaller index | 256 | float | No | 0.535 | -1.80% | Good size/quality tradeoff |
+| Compact index | 128 | float | No | 0.507 | -7.10% | Memory-constrained indexes |
+| Very compact index | 64 | float | No | 0.447 | -18.07% | Not for quality-sensitive retrieval |
+| INT8 search | 384 | int8 | No | 0.507 | -7.03% | Benchmark before using |
+| INT8 search + rescore | 384 | int8 | Yes | 0.544 | -0.15% | Best quantized option |
+| Binary search | 384 | binary | No | 0.471 | -13.64% | Not recommended by default |
+| Binary search + rescore | 384 | binary | Yes | 0.542 | -0.54% | Strong compression when rescoring is available |
+
+## The name "bekko"
+
+*bekko* (/ˈbek.koː/) is a coined name that joins two pieces of Japanese tradition:
+
+- **akabeko** (赤べこ) — the red ox that has been cherished in Japan for centuries as a guardian charm, believed to ward off illness and misfortune.
+- **bekko-iro** (鼈甲色) — a beautiful traditional Japanese color: a warm, translucent, amber-like hue.
+
+The name pairs the protective spirit of the red ox with the quiet beauty of this classic amber tone.
+
+## Citation
+
+A BibTeX citation will be added when the technical paper is released. It is currently in preparation.
+
+## Training data
+
+The datasets we built to train bekko-embedding:
+
+- [hotchpotch/bekko-embedding-v1-unsupervised](https://huggingface.co/datasets/hotchpotch/bekko-embedding-v1-unsupervised) — unsupervised pretraining data.
+- [hotchpotch/bekko-embedding-hard-negatives](https://huggingface.co/datasets/hotchpotch/bekko-embedding-hard-negatives) — hard negatives for contrastive fine-tuning.
+
+## License
+
+MIT License.
+
+## Author
+
+Yuichi Tateno [@hotchpotch](https://github.com/hotchpotch)

cli_args.json

@@ -0,0 +1,631 @@
+{
+  "attn_implementation": "flash_attention_2",
+  "batch_sampler": "no_duplicates_fast",
+  "bf16": true,
+  "bf16_full_eval": true,
+  "config_path": "configs/training_args/ablation/a8m-ft-prefix-none-from-full-prefix-none-a25m-ftdata.yaml",
+  "data_config_path": "configs/data_args/ablation/ft-bekko-embedding-v1-a8m-prefix-none-a25m-ftdata.yaml",
+  "data_summary": [
+    "ruri_v3_ft-auto-wiki-qa-nemotron: train=154,956 eval=0",
+    "ruri_v3_ft-jaquad: train=31,253 eval=0",
+    "ruri_v3_ft-jqara: train=19,515 eval=0",
+    "ruri_v3_ft-jsquad: train=61,659 eval=0",
+    "ruri_v3_ft-miracl: train=25,905 eval=0",
+    "ruri_v3_ft-mkqa: train=30,310 eval=0",
+    "ruri_v3_ft-mr-tydi: train=18,200 eval=0",
+    "ruri_v3_ft-nli: train=222,429 eval=0",
+    "ruri_v3_ft-quiz-no-mori: train=37,437 eval=0",
+    "ruri_v3_ft-quiz-works: train=30,631 eval=0",
+    "bge_m3_ft-dureader-len-0-500+len-500-1000+len-1000-2000+len-2000-3000+len-3000-4000+len-4000-5000+len-5000-6000+len-6000-7000+len-7000-inf: train=80,416 eval=0",
+    "bge_m3_ft-hotpotqa-len-0-500+len-500-1000+len-1000-2000+len-2000-3000+len-3000-4000+len-4000-5000+len-5000-6000+len-6000-7000+len-7000-inf: train=84,516 eval=0",
+    "bge_m3_ft-miracl-len-0-500+len-500-1000+len-1000-2000+len-2000-3000+len-3000-4000+len-4000-5000+len-5000-6000+len-6000-7000+len-7000-inf: train=40,203 eval=0",
+    "bge_m3_ft-mr-tydi-len-0-500+len-500-1000+len-1000-2000+len-2000-3000+len-3000-4000+len-4000-5000+len-5000-6000+len-6000-7000+len-7000-inf: train=48,729 eval=0",
+    "bge_m3_ft-squad-len-0-500+len-500-1000+len-1000-2000+len-2000-3000+len-3000-4000+len-4000-5000+len-5000-6000+len-6000-7000+len-7000-inf: train=87,599 eval=0",
+    "bge_m3_ft-other-len-0-500+len-500-1000+len-1000-2000+len-2000-3000+len-3000-4000+len-4000-5000+len-5000-6000+len-6000-7000+len-7000-inf: train=500 eval=0",
+    "bge_m3_ft-msmarco-len-0-500+len-500-1000+len-1000-2000+len-2000-3000+len-3000-4000+len-4000-5000+len-5000-6000+len-6000-7000+len-7000-inf: train=200,000 eval=0",
+    "bge_m3_ft-mmarco_chinese-len-0-500: train=80,000 eval=0",
+    "bge_m3_ft-ATEC-len-0-500: train=11,325 eval=0",
+    "bge_m3_ft-BQ-len-0-500: train=12,518 eval=0",
+    "bge_m3_ft-LCQMC-len-0-500: train=10,000 eval=0",
+    "bge_m3_ft-PAWSX-len-0-500: train=9,900 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_english: train=250,000 eval=0",
+    "bekko_embedding_hard_negatives-additional-query-doc: train=80,000 eval=0",
+    "bekko_embedding_hard_negatives-additional-doc-doc: train=40,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_en: train=12,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_ar: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_de: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_es: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_fr: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_it: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_ja: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_ko: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_pt: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_ru: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-wikipedia_hard_negatives_long_docs_zh: train=2,000 eval=0",
+    "bekko_embedding_hard_negatives-extra-four: train=80,000 eval=0"
+  ],
+  "decay_ratio": null,
+  "decay_steps": null,
+  "eval_steps": 400,
+  "evaluation_strategy": "steps",
+  "fp16": false,
+  "ft_base_model": "output/models/embedding_v1/mmBERT-L4H384-pruned/a8m-pretrain-full-unir-v9-prefix-none-mrl-qat/20260704_041654/final",
+  "global_rope_theta": null,
+  "gradient_accumulation_steps": 1,
+  "learning_rate": 1e-05,
+  "local_rope_theta": null,
+  "logging_steps": 10,
+  "loss_mini_batch_size": 32,
+  "loss_mini_batch_size_by_bs": {
+    "1152": 128,
+    "192": 16,
+    "240": 16,
+    "288": 32,
+    "336": 32,
+    "432": 32,
+    "480": 64,
+    "768": 64
+  },
+  "losses": {
+    "doc_doc": [
+      {
+        "klass": "CachedMultipleNegativesMaskedBidirectionalRankingLoss",
+        "kwargs": {
+          "debug_mask_stats": 10,
+          "gather_across_devices": false,
+          "include_doc_to_query_loss": true,
+          "margin": 0.01,
+          "temperature": 0.03
+        }
+      },
+      {
+        "klass": "QuantizationAwareLoss",
+        "kwargs": {
+          "quantization_precisions": [
+            "float32",
+            "int8",
+            "ubinary"
+          ],
+          "quantization_weights": [
+            1.0,
+            0.1,
+            0.1
+          ]
+        }
+      },
+      {
+        "klass": "MatryoshkaLoss",
+        "kwargs": {
+          "matryoshka_dims": [
+            384,
+            256,
+            128,
+            64
+          ],
+          "matryoshka_weights": [
+            1.0,
+            0.3,
+            0.15,
+            0.1
+          ]
+        }
+      }
+    ],
+    "query_doc": [
+      {
+        "klass": "CachedMultipleNegativesMaskedBidirectionalRankingLoss",
+        "kwargs": {
+          "debug_mask_stats": 10,
+          "gather_across_devices": false,
+          "margin": 0.01,
+          "temperature": 0.03
+        }
+      },
+      {
+        "klass": "QuantizationAwareLoss",
+        "kwargs": {
+          "quantization_precisions": [
+            "float32",
+            "int8",
+            "ubinary"
+          ],
+          "quantization_weights": [
+            1.0,
+            0.1,
+            0.1
+          ]
+        }
+      },
+      {
+        "klass": "MatryoshkaLoss",
+        "kwargs": {
+          "matryoshka_dims": [
+            384,
+            256,
+            128,
+            64
+          ],
+          "matryoshka_weights": [
+            1.0,
+            0.3,
+            0.15,
+            0.1
+          ]
+        }
+      }
+    ]
+  },
+  "lr_scheduler_kwargs": null,
+  "lr_scheduler_type": "cosine",
+  "max_seq_length": 8192,
+  "max_steps": null,
+  "merged_config": {
+    "_config_path": "configs/training_args/ablation/a8m-ft-prefix-none-from-full-prefix-none-a25m-ftdata.yaml",
+    "_data_config_path": "configs/data_args/ablation/ft-bekko-embedding-v1-a8m-prefix-none-a25m-ftdata.yaml",
+    "_training_config_path": "configs/training_args/ablation/a8m-ft-prefix-none-from-full-prefix-none-a25m-ftdata.yaml",
+    "data": {
+      "_config_path": "/home/hotchpotch/src/github.com/hotchpotch/mmBERT-embedding-reranker/configs/data/ft-bekko-embedding-v1-a25m-20260625.yaml",
+      "common": {
+        "eval_limit": 0,
+        "group_small_subsets_label": "other",
+        "group_small_subsets_threshold": 2000,
+        "group_subsets_by_prefix": true,
+        "group_subsets_prefixes": [
+          "__auto__"
+        ],
+        "items": 8,
+        "limit": null,
+        "loader": "bge_m3_ft",
+        "neg_sampling": {
+          "strategy": "mid_tail_random"
+        },
+        "pair_type": "query_doc",
+        "path": "output/bge-m3-data-ft-raw-dataset"
+      },
+      "train_datasets": [
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "items": 8,
+          "limit": null,
+          "loader": "ruri_v3_ft",
+          "max_len": 512,
+          "pair_type": "query_doc",
+          "subsets": [
+            "*"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "group_small_subsets_label": "other",
+          "group_small_subsets_threshold": 2000,
+          "group_subsets_by_prefix": true,
+          "group_subsets_prefixes": [
+            "__auto__"
+          ],
+          "items": 8,
+          "limit": null,
+          "loader": "bge_m3_ft",
+          "max_len": 512,
+          "neg_sampling": {
+            "strategy": "mid_tail_random"
+          },
+          "pair_type": "query_doc",
+          "path": "output/bge-m3-data-ft-raw-dataset",
+          "subset_suffixes": [
+            "len-0-500",
+            "len-500-1000",
+            "len-1000-2000",
+            "len-2000-3000",
+            "len-3000-4000",
+            "len-4000-5000",
+            "len-5000-6000",
+            "len-6000-7000",
+            "len-7000-inf"
+          ],
+          "subsets": [
+            "dureader",
+            "hotpotqa",
+            "miracl",
+            "mr-tydi",
+            "pubmed_qa_labeled",
+            "squad"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "group_small_subsets_label": "other",
+          "group_small_subsets_threshold": 2000,
+          "group_subsets_by_prefix": true,
+          "group_subsets_prefixes": [
+            "__auto__"
+          ],
+          "items": 8,
+          "limit": 200000,
+          "loader": "bge_m3_ft",
+          "max_len": 512,
+          "neg_sampling": {
+            "strategy": "mid_tail_random"
+          },
+          "pair_type": "query_doc",
+          "path": "output/bge-m3-data-ft-raw-dataset",
+          "subset_suffixes": [
+            "len-0-500",
+            "len-500-1000",
+            "len-1000-2000",
+            "len-2000-3000",
+            "len-3000-4000",
+            "len-4000-5000",
+            "len-5000-6000",
+            "len-6000-7000",
+            "len-7000-inf"
+          ],
+          "subsets": [
+            "msmarco"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "group_small_subsets_label": "other",
+          "group_small_subsets_threshold": 2000,
+          "group_subsets_by_prefix": true,
+          "group_subsets_prefixes": [
+            "__auto__"
+          ],
+          "items": 8,
+          "limit": 80000,
+          "loader": "bge_m3_ft",
+          "max_len": 512,
+          "neg_sampling": {
+            "strategy": "mid_tail_random"
+          },
+          "pair_type": "query_doc",
+          "path": "output/bge-m3-data-ft-raw-dataset",
+          "subset_suffixes": [
+            "len-0-500"
+          ],
+          "subsets": [
+            "mmarco_chinese"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "group_small_subsets_label": "other",
+          "group_small_subsets_threshold": 2000,
+          "group_subsets_by_prefix": true,
+          "group_subsets_prefixes": [
+            "__auto__"
+          ],
+          "items": 8,
+          "limit": null,
+          "loader": "bge_m3_ft",
+          "max_len": 512,
+          "neg_sampling": {
+            "strategy": "mid_tail_random"
+          },
+          "pair_type": "doc_doc",
+          "pair_type_by_label": {
+            "ATEC": "doc_doc",
+            "BQ": "doc_doc",
+            "LCQMC": "doc_doc",
+            "PAWSX": "doc_doc"
+          },
+          "path": "output/bge-m3-data-ft-raw-dataset",
+          "subset_suffixes": [
+            "len-0-500"
+          ],
+          "subsets": [
+            "ATEC",
+            "BQ",
+            "LCQMC",
+            "PAWSX"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "items": 8,
+          "limit": 250000,
+          "loader": "bekko_embedding_hard_negatives",
+          "max_len": 512,
+          "name": "hotchpotch/bekko-embedding-hard-negatives",
+          "pair_type": "query_doc",
+          "split": "train",
+          "subset": "wikipedia-hard-negatives-english",
+          "subsets": [
+            "wikipedia_hard_negatives_english"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "items": 8,
+          "limit": null,
+          "loader": "bekko_embedding_hard_negatives",
+          "max_len": 512,
+          "merge_subsets": true,
+          "name": "hotchpotch/bekko-embedding-hard-negatives",
+          "pair_type": "query_doc",
+          "split": "train",
+          "subset": "additional-query-doc",
+          "subsets": [
+            "agnews",
+            "gooaq",
+            "natural_questions",
+            "trivia_qa"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "items": 8,
+          "limit": null,
+          "loader": "bekko_embedding_hard_negatives",
+          "max_len": 512,
+          "merge_subsets": true,
+          "name": "hotchpotch/bekko-embedding-hard-negatives",
+          "pair_type": "doc_doc",
+          "split": "train",
+          "subset": "additional-doc-doc",
+          "subsets": [
+            "all_nli",
+            "coco_captions"
+          ]
+        },
+        {
+          "bs_size": 192,
+          "eval_limit": 0,
+          "items": 8,
+          "limit": 12000,
+          "loader": "bekko_embedding_hard_negatives",
+          "max_len": 8192,
+          "name": "hotchpotch/bekko-embedding-hard-negatives",
+          "neg_sampling": {
+            "head_count": 4,
+            "split_at": 7,
+            "strategy": "split_random"
+          },
+          "pair_type": "query_doc",
+          "split": "train",
+          "subset": "wikipedia-hard-negatives-long-docs-en-rebalanced",
+          "subsets": [
+            "wikipedia_hard_negatives_long_docs_en"
+          ]
+        },
+        {
+          "bs_size": 192,
+          "eval_limit": 0,
+          "items": 8,
+          "limit": 2000,
+          "loader": "bekko_embedding_hard_negatives",
+          "max_len": 8192,
+          "name": "hotchpotch/bekko-embedding-hard-negatives",
+          "neg_sampling": {
+            "head_count": 4,
+            "split_at": 7,
+            "strategy": "split_random"
+          },
+          "pair_type": "query_doc",
+          "split": "train",
+          "subset": "wikipedia-hard-negatives-long-docs-other-rebalanced",
+          "subsets": [
+            "wikipedia_hard_negatives_long_docs_ar",
+            "wikipedia_hard_negatives_long_docs_de",
+            "wikipedia_hard_negatives_long_docs_es",
+            "wikipedia_hard_negatives_long_docs_fr",
+            "wikipedia_hard_negatives_long_docs_it",
+            "wikipedia_hard_negatives_long_docs_ja",
+            "wikipedia_hard_negatives_long_docs_ko",
+            "wikipedia_hard_negatives_long_docs_pt",
+            "wikipedia_hard_negatives_long_docs_ru",
+            "wikipedia_hard_negatives_long_docs_zh"
+          ]
+        },
+        {
+          "bs_size": 1152,
+          "eval_limit": 0,
+          "items": 8,
+          "limit": null,
+          "loader": "bekko_embedding_hard_negatives",
+          "max_len": 512,
+          "merge_subsets": true,
+          "name": "hotchpotch/bekko-embedding-hard-negatives",
+          "pair_type": "query_doc",
+          "split": "train",
+          "subset": "extra-four",
+          "subsets": [
+            "arxiv",
+            "codesearch",
+            "fineweb",
+            "pubmed"
+          ]
+        }
+      ]
+    },
+    "model": {
+      "doc_doc_prefix_fallback_to_query": false,
+      "doc_prefix": "",
+      "ft_base_model": "output/models/embedding_v1/mmBERT-L4H384-pruned/a8m-pretrain-full-unir-v9-prefix-none-mrl-qat/20260704_041654/final",
+      "max_seq_length": 8192,
+      "name": "hotchpotch/mmBERT-L4H384-pruned",
+      "query_prefix": ""
+    },
+    "training": {
+      "batch_sampler": "no_duplicates_fast",
+      "bf16": true,
+      "bf16_full_eval": true,
+      "cleanup_checkpoints": false,
+      "dataloader_drop_last": true,
+      "dataloader_num_workers": 4,
+      "dataloader_persistent_workers": false,
+      "dataloader_prefetch_factor": 1,
+      "eval_before_train": false,
+      "eval_prefix_in_dataset": true,
+      "eval_steps": 400,
+      "fp16": false,
+      "gradient_accumulation_steps": 1,
+      "learning_rate": 1e-05,
+      "logging_steps": 10,
+      "loss_detail_logging": {
+        "enabled": true,
+        "flush_every_records": 1,
+        "include_batch_events": false,
+        "include_optimizer_events": true,
+        "output_dir": "logs/loss_detail"
+      },
+      "loss_mini_batch_size": 32,
+      "loss_mini_batch_size_by_bs": {
+        "1152": 128,
+        "192": 16,
+        "240": 16,
+        "288": 32,
+        "336": 32,
+        "432": 32,
+        "480": 64,
+        "768": 64
+      },
+      "losses": {
+        "doc_doc": [
+          {
+            "klass": "CachedMultipleNegativesMaskedBidirectionalRankingLoss",
+            "kwargs": {
+              "debug_mask_stats": 10,
+              "gather_across_devices": false,
+              "include_doc_to_query_loss": true,
+              "margin": 0.01,
+              "temperature": 0.03
+            }
+          },
+          {
+            "klass": "QuantizationAwareLoss",
+            "kwargs": {
+              "quantization_precisions": [
+                "float32",
+                "int8",
+                "ubinary"
+              ],
+              "quantization_weights": [
+                1.0,
+                0.1,
+                0.1
+              ]
+            }
+          },
+          {
+            "klass": "MatryoshkaLoss",
+            "kwargs": {
+              "matryoshka_dims": [
+                384,
+                256,
+                128,
+                64
+              ],
+              "matryoshka_weights": [
+                1.0,
+                0.3,
+                0.15,
+                0.1
+              ]
+            }
+          }
+        ],
+        "query_doc": [
+          {
+            "klass": "CachedMultipleNegativesMaskedBidirectionalRankingLoss",
+            "kwargs": {
+              "debug_mask_stats": 10,
+              "gather_across_devices": false,
+              "margin": 0.01,
+              "temperature": 0.03
+            }
+          },
+          {
+            "klass": "QuantizationAwareLoss",
+            "kwargs": {
+              "quantization_precisions": [
+                "float32",
+                "int8",
+                "ubinary"
+              ],
+              "quantization_weights": [
+                1.0,
+                0.1,
+                0.1
+              ]
+            }
+          },
+          {
+            "klass": "MatryoshkaLoss",
+            "kwargs": {
+              "matryoshka_dims": [
+                384,
+                256,
+                128,
+                64
+              ],
+              "matryoshka_weights": [
+                1.0,
+                0.3,
+                0.15,
+                0.1
+              ]
+            }
+          }
+        ]
+      },
+      "lr_scheduler_type": "cosine",
+      "num_train_epochs": 1,
+      "optim": "adamw_torch",
+      "output_root": "output/models/embedding_v1",
+      "per_device_eval_batch_size": 64,
+      "per_device_train_batch_size": "DS",
+      "report_to": [
+        "wandb"
+      ],
+      "save_steps": 200,
+      "save_strategy": "steps",
+      "save_total_limit": 5,
+      "seed": 12,
+      "skip_evaluation": false,
+      "use_multi_dataset_sampler": true,
+      "wandb_project": "mmbert-embedding",
+      "warmup_ratio": 0.3,
+      "weight_decay": 0.01
+    }
+  },
+  "model_name": "ft-a8m-ft-prefix-none-from-full-prefix-none-a25m-ftdata__a8m-pretrain-full-unir-v9-prefix-none-mrl-qat",
+  "monolithic_config_path": null,
+  "mrl_loss": null,
+  "multi_dataset_batch_sampler": null,
+  "num_train_epochs": 1,
+  "output_dir": "output/models/embedding_v1/ft-a8m-ft-prefix-none-from-full-prefix-none-a25m-ftdata__a8m-pretrain-full-unir-v9-prefix-none-mrl-qat/ft-a8m-prefix-none-fullpretrain-a25m-ftdata/20260707_092449",
+  "override_paths": [],
+  "overrides": [],
+  "per_device_eval_batch_size": 64,
+  "per_device_train_batch_size": "DS",
+  "raw_cli_args": [
+    "/home/hotchpotch/src/github.com/hotchpotch/mmBERT-embedding-reranker/.venv/bin/trainer",
+    "--training_config",
+    "configs/training_args/ablation/a8m-ft-prefix-none-from-full-prefix-none-a25m-ftdata.yaml",
+    "--data_config",
+    "configs/data_args/ablation/ft-bekko-embedding-v1-a8m-prefix-none-a25m-ftdata.yaml",
+    "--run-name",
+    "ft-a8m-prefix-none-fullpretrain-a25m-ftdata"
+  ],
+  "run_name": "ft-a8m-prefix-none-fullpretrain-a25m-ftdata-20260707_092449",
+  "save_steps": 200,
+  "save_strategy": "steps",
+  "save_total_limit": 5,
+  "seed": 12,
+  "sync_once_per_batch": true,
+  "train_ds_label": "ft-bekko-embedding-v1-a8m-prefix-none-a25m-ftdata",
+  "training_config_path": "configs/training_args/ablation/a8m-ft-prefix-none-from-full-prefix-none-a25m-ftdata.yaml",
+  "use_multi_dataset_sampler": true,
+  "warmup_ratio": 0.3,
+  "warmup_steps": null,
+  "weight_decay": 0.01
+}

config.json

@@ -0,0 +1,46 @@
+{
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 1,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "bfloat16",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 1,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 384,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 160000,
+  "mask_token_id": 4,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 6,
+  "num_hidden_layers": 4,
+  "pad_token_id": 0,
+  "position_embedding_type": "sans_pos",
+  "repad_logits_with_grad": false,
+  "sep_token_id": 1,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "transformers_version": "4.57.6",
+  "vocab_size": 256000
+}

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "model_type": "SentenceTransformer",
+  "__version__": {
+    "sentence_transformers": "5.3.0.dev0",
+    "transformers": "4.57.6",
+    "pytorch": "2.8.0+cu129"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

eval_result.json

@@ -0,0 +1,274 @@
+{
+  "nano_beir": {
+    "NanoArguAna": {
+      "MNanoArguAna-ar_cosine_ndcg@10": 0.47162246746260905,
+      "MNanoArguAna-de_cosine_ndcg@10": 0.5244815937958666,
+      "MNanoArguAna-en_cosine_ndcg@10": 0.5475096777475938,
+      "MNanoArguAna-es_cosine_ndcg@10": 0.5411130875783072,
+      "MNanoArguAna-fr_cosine_ndcg@10": 0.5375623402603463,
+      "MNanoArguAna-it_cosine_ndcg@10": 0.4868980354304658,
+      "MNanoArguAna-mean_cosine_ndcg@10": 0.5181611256161548,
+      "MNanoArguAna-no_cosine_ndcg@10": 0.5032684329614588,
+      "MNanoArguAna-pt_cosine_ndcg@10": 0.5528641027564912,
+      "MNanoArguAna-sv_cosine_ndcg@10": 0.4981303925522546
+    },
+    "NanoClimateFEVER": {
+      "MNanoClimateFEVER-ar_cosine_ndcg@10": 0.2388814653084799,
+      "MNanoClimateFEVER-de_cosine_ndcg@10": 0.3025568513408033,
+      "MNanoClimateFEVER-en_cosine_ndcg@10": 0.311647506437599,
+      "MNanoClimateFEVER-es_cosine_ndcg@10": 0.38262380913979593,
+      "MNanoClimateFEVER-fr_cosine_ndcg@10": 0.28797316267086626,
+      "MNanoClimateFEVER-it_cosine_ndcg@10": 0.31072100361418853,
+      "MNanoClimateFEVER-mean_cosine_ndcg@10": 0.3121963115000014,
+      "MNanoClimateFEVER-no_cosine_ndcg@10": 0.3128726227431737,
+      "MNanoClimateFEVER-pt_cosine_ndcg@10": 0.3369089366136735,
+      "MNanoClimateFEVER-sv_cosine_ndcg@10": 0.3255814456314322
+    },
+    "NanoClimateFEVER-ja": {
+      "MNanoClimateFEVER-ja-mean_cosine_ndcg@10": 0.3544010279234292,
+      "MNanoClimateFEVER-ja_cosine_ndcg@10": 0.3544010279234292
+    },
+    "NanoDBPedia": {
+      "MNanoDBPedia-ar_cosine_ndcg@10": 0.5101660803076424,
+      "MNanoDBPedia-de_cosine_ndcg@10": 0.5818283059326789,
+      "MNanoDBPedia-en_cosine_ndcg@10": 0.6101598991986595,
+      "MNanoDBPedia-es_cosine_ndcg@10": 0.59454992145433,
+      "MNanoDBPedia-fr_cosine_ndcg@10": 0.5681121746683402,
+      "MNanoDBPedia-it_cosine_ndcg@10": 0.6068255600927434,
+      "MNanoDBPedia-mean_cosine_ndcg@10": 0.5764677413740426,
+      "MNanoDBPedia-no_cosine_ndcg@10": 0.5609668125019728,
+      "MNanoDBPedia-pt_cosine_ndcg@10": 0.5801074571081114,
+      "MNanoDBPedia-sv_cosine_ndcg@10": 0.5754934611019041
+    },
+    "NanoDBPedia-ja": {
+      "MNanoDBPedia-ja-mean_cosine_ndcg@10": 0.607793412923947,
+      "MNanoDBPedia-ja_cosine_ndcg@10": 0.607793412923947
+    },
+    "NanoFEVER": {
+      "MNanoFEVER-ar_cosine_ndcg@10": 0.829386110513125,
+      "MNanoFEVER-de_cosine_ndcg@10": 0.823139962575593,
+      "MNanoFEVER-en_cosine_ndcg@10": 0.8724424475263557,
+      "MNanoFEVER-es_cosine_ndcg@10": 0.8828593001713432,
+      "MNanoFEVER-fr_cosine_ndcg@10": 0.7509381407954594,
+      "MNanoFEVER-it_cosine_ndcg@10": 0.8544218767031033,
+      "MNanoFEVER-mean_cosine_ndcg@10": 0.8391111267731001,
+      "MNanoFEVER-no_cosine_ndcg@10": 0.826571455255216,
+      "MNanoFEVER-pt_cosine_ndcg@10": 0.8468866641721634,
+      "MNanoFEVER-sv_cosine_ndcg@10": 0.8653541832455425
+    },
+    "NanoFEVER-ja": {
+      "MNanoFEVER-ja-mean_cosine_ndcg@10": 0.7755503010515641,
+      "MNanoFEVER-ja_cosine_ndcg@10": 0.7755503010515641
+    },
+    "NanoFiQA2018": {
+      "MNanoFiQA2018-ar_cosine_ndcg@10": 0.35878927204201594,
+      "MNanoFiQA2018-de_cosine_ndcg@10": 0.3535250386230549,
+      "MNanoFiQA2018-en_cosine_ndcg@10": 0.5124163673285884,
+      "MNanoFiQA2018-es_cosine_ndcg@10": 0.3805220895648997,
+      "MNanoFiQA2018-fr_cosine_ndcg@10": 0.34979989933435596,
+      "MNanoFiQA2018-it_cosine_ndcg@10": 0.32890955078724277,
+      "MNanoFiQA2018-mean_cosine_ndcg@10": 0.3569773959702302,
+      "MNanoFiQA2018-no_cosine_ndcg@10": 0.3029499692908921,
+      "MNanoFiQA2018-pt_cosine_ndcg@10": 0.3281029714266557,
+      "MNanoFiQA2018-sv_cosine_ndcg@10": 0.297781405334366
+    },
+    "NanoFiQA2018-ja": {
+      "MNanoFiQA2018-ja-mean_cosine_ndcg@10": 0.335525591097055,
+      "MNanoFiQA2018-ja_cosine_ndcg@10": 0.335525591097055
+    },
+    "NanoHotpotQA": {
+      "MNanoHotpotQA-ar_cosine_ndcg@10": 0.728950229115614,
+      "MNanoHotpotQA-de_cosine_ndcg@10": 0.7737285139164686,
+      "MNanoHotpotQA-en_cosine_ndcg@10": 0.7801378951693863,
+      "MNanoHotpotQA-es_cosine_ndcg@10": 0.7388921955611543,
+      "MNanoHotpotQA-fr_cosine_ndcg@10": 0.7571623416912887,
+      "MNanoHotpotQA-it_cosine_ndcg@10": 0.7631236832708947,
+      "MNanoHotpotQA-mean_cosine_ndcg@10": 0.7648584009779095,
+      "MNanoHotpotQA-no_cosine_ndcg@10": 0.7750724657952014,
+      "MNanoHotpotQA-pt_cosine_ndcg@10": 0.7826420888088002,
+      "MNanoHotpotQA-sv_cosine_ndcg@10": 0.7840161954723769
+    },
+    "NanoHotpotQA-ja": {
+      "MNanoHotpotQA-ja-mean_cosine_ndcg@10": 0.7128002111186521,
+      "MNanoHotpotQA-ja_cosine_ndcg@10": 0.7128002111186521
+    },
+    "NanoMSMARCO": {
+      "MNanoMSMARCO-ar_cosine_ndcg@10": 0.4631724639847954,
+      "MNanoMSMARCO-de_cosine_ndcg@10": 0.5098791033462391,
+      "MNanoMSMARCO-en_cosine_ndcg@10": 0.5849804514463571,
+      "MNanoMSMARCO-es_cosine_ndcg@10": 0.49811951886739164,
+      "MNanoMSMARCO-fr_cosine_ndcg@10": 0.5295831079529444,
+      "MNanoMSMARCO-it_cosine_ndcg@10": 0.5257877689930516,
+      "MNanoMSMARCO-mean_cosine_ndcg@10": 0.501135037738672,
+      "MNanoMSMARCO-no_cosine_ndcg@10": 0.4282152602394671,
+      "MNanoMSMARCO-pt_cosine_ndcg@10": 0.5454471629879832,
+      "MNanoMSMARCO-sv_cosine_ndcg@10": 0.4250305018298194
+    },
+    "NanoMSMARCO-ja": {
+      "MNanoMSMARCO-ja-mean_cosine_ndcg@10": 0.495952316384158,
+      "MNanoMSMARCO-ja_cosine_ndcg@10": 0.495952316384158
+    },
+    "NanoNFCorpus": {
+      "MNanoNFCorpus-ar_cosine_ndcg@10": 0.2243357909705374,
+      "MNanoNFCorpus-de_cosine_ndcg@10": 0.25517217570120965,
+      "MNanoNFCorpus-en_cosine_ndcg@10": 0.33823600104476415,
+      "MNanoNFCorpus-es_cosine_ndcg@10": 0.30777975784999784,
+      "MNanoNFCorpus-fr_cosine_ndcg@10": 0.32110819553443876,
+      "MNanoNFCorpus-it_cosine_ndcg@10": 0.2927151378210865,
+      "MNanoNFCorpus-mean_cosine_ndcg@10": 0.2922564958462841,
+      "MNanoNFCorpus-no_cosine_ndcg@10": 0.2825121169241804,
+      "MNanoNFCorpus-pt_cosine_ndcg@10": 0.31250511821097504,
+      "MNanoNFCorpus-sv_cosine_ndcg@10": 0.29594416855936756
+    },
+    "NanoNFCorpus-ja": {
+      "MNanoNFCorpus-ja-mean_cosine_ndcg@10": 0.2796027539405435,
+      "MNanoNFCorpus-ja_cosine_ndcg@10": 0.2796027539405435
+    },
+    "NanoNQ": {
+      "MNanoNQ-ar_cosine_ndcg@10": 0.4738238810099898,
+      "MNanoNQ-de_cosine_ndcg@10": 0.5169034233367948,
+      "MNanoNQ-en_cosine_ndcg@10": 0.6392269298711112,
+      "MNanoNQ-es_cosine_ndcg@10": 0.4884619602865586,
+      "MNanoNQ-fr_cosine_ndcg@10": 0.5204540082984815,
+      "MNanoNQ-it_cosine_ndcg@10": 0.4855978396962014,
+      "MNanoNQ-mean_cosine_ndcg@10": 0.5038628890871119,
+      "MNanoNQ-no_cosine_ndcg@10": 0.4701257167343876,
+      "MNanoNQ-pt_cosine_ndcg@10": 0.5014927378483448,
+      "MNanoNQ-sv_cosine_ndcg@10": 0.4386795047021373
+    },
+    "NanoNQ-ja": {
+      "MNanoNQ-ja-mean_cosine_ndcg@10": 0.5775267128386041,
+      "MNanoNQ-ja_cosine_ndcg@10": 0.5775267128386041
+    },
+    "NanoQuoraRetrieval": {
+      "MNanoQuoraRetrieval-ar_cosine_ndcg@10": 0.8278646215090353,
+      "MNanoQuoraRetrieval-de_cosine_ndcg@10": 0.837186290069087,
+      "MNanoQuoraRetrieval-en_cosine_ndcg@10": 0.9341474296990717,
+      "MNanoQuoraRetrieval-es_cosine_ndcg@10": 0.8820271175676624,
+      "MNanoQuoraRetrieval-fr_cosine_ndcg@10": 0.8201622507152796,
+      "MNanoQuoraRetrieval-it_cosine_ndcg@10": 0.847730219321762,
+      "MNanoQuoraRetrieval-mean_cosine_ndcg@10": 0.8367000757926226,
+      "MNanoQuoraRetrieval-no_cosine_ndcg@10": 0.771947811182082,
+      "MNanoQuoraRetrieval-pt_cosine_ndcg@10": 0.8096693691794988,
+      "MNanoQuoraRetrieval-sv_cosine_ndcg@10": 0.7995655728901241
+    },
+    "NanoQuoraRetrieval-ja": {
+      "MNanoQuoraRetrieval-ja-mean_cosine_ndcg@10": 0.9079719053009623,
+      "MNanoQuoraRetrieval-ja_cosine_ndcg@10": 0.9079719053009623
+    },
+    "NanoSCIDOCS": {
+      "MNanoSCIDOCS-ar_cosine_ndcg@10": 0.28951512418884506,
+      "MNanoSCIDOCS-de_cosine_ndcg@10": 0.31073509056652765,
+      "MNanoSCIDOCS-en_cosine_ndcg@10": 0.39435733197911177,
+      "MNanoSCIDOCS-es_cosine_ndcg@10": 0.35446969096552117,
+      "MNanoSCIDOCS-fr_cosine_ndcg@10": 0.332550621185874,
+      "MNanoSCIDOCS-it_cosine_ndcg@10": 0.3195499535476747,
+      "MNanoSCIDOCS-mean_cosine_ndcg@10": 0.3261106666309668,
+      "MNanoSCIDOCS-no_cosine_ndcg@10": 0.2808745088175287,
+      "MNanoSCIDOCS-pt_cosine_ndcg@10": 0.33883606805555255,
+      "MNanoSCIDOCS-sv_cosine_ndcg@10": 0.31410761037206514
+    },
+    "NanoSCIDOCS-ja": {
+      "MNanoSCIDOCS-ja-mean_cosine_ndcg@10": 0.3381960374851039,
+      "MNanoSCIDOCS-ja_cosine_ndcg@10": 0.3381960374851039
+    },
+    "NanoSciFact": {
+      "MNanoSciFact-ar_cosine_ndcg@10": 0.6158943198952038,
+      "MNanoSciFact-de_cosine_ndcg@10": 0.6936130981052494,
+      "MNanoSciFact-en_cosine_ndcg@10": 0.7666635173772184,
+      "MNanoSciFact-es_cosine_ndcg@10": 0.6618881209052857,
+      "MNanoSciFact-fr_cosine_ndcg@10": 0.7036002103688646,
+      "MNanoSciFact-it_cosine_ndcg@10": 0.6623497889374568,
+      "MNanoSciFact-mean_cosine_ndcg@10": 0.6681959232833932,
+      "MNanoSciFact-no_cosine_ndcg@10": 0.634286391757496,
+      "MNanoSciFact-pt_cosine_ndcg@10": 0.6301575127025395,
+      "MNanoSciFact-sv_cosine_ndcg@10": 0.645310349501225
+    },
+    "NanoTouche2020": {
+      "MNanoTouche2020-ar_cosine_ndcg@10": 0.4649113860517151,
+      "MNanoTouche2020-de_cosine_ndcg@10": 0.5154857305561092,
+      "MNanoTouche2020-en_cosine_ndcg@10": 0.5580849991842177,
+      "MNanoTouche2020-es_cosine_ndcg@10": 0.4781704716606799,
+      "MNanoTouche2020-fr_cosine_ndcg@10": 0.508312219288739,
+      "MNanoTouche2020-it_cosine_ndcg@10": 0.48064364243821456,
+      "MNanoTouche2020-mean_cosine_ndcg@10": 0.4979933275770823,
+      "MNanoTouche2020-no_cosine_ndcg@10": 0.48435621185708866,
+      "MNanoTouche2020-pt_cosine_ndcg@10": 0.4865894112113976,
+      "MNanoTouche2020-sv_cosine_ndcg@10": 0.5053858759455785
+    }
+  },
+  "nano_fineweb2ir": {
+    "NanoFineWeb2IR-arb_Arab_cosine_ndcg@10": 0.9586135311614679,
+    "NanoFineWeb2IR-ces_Latn_cosine_ndcg@10": 0.9426185950714292,
+    "NanoFineWeb2IR-cmn_Hani_cosine_ndcg@10": 0.9283998915977869,
+    "NanoFineWeb2IR-dan_Latn_cosine_ndcg@10": 0.9686135311614679,
+    "NanoFineWeb2IR-deu_Latn_cosine_ndcg@10": 1.0,
+    "NanoFineWeb2IR-ell_Grek_cosine_ndcg@10": 0.9421470803838846,
+    "NanoFineWeb2IR-eng_Latn_cosine_ndcg@10": 0.9871241437421605,
+    "NanoFineWeb2IR-fas_Arab_cosine_ndcg@10": 0.9353355801725679,
+    "NanoFineWeb2IR-fra_Latn_cosine_ndcg@10": 0.8729759642023812,
+    "NanoFineWeb2IR-hun_Latn_cosine_ndcg@10": 0.9338763851275671,
+    "NanoFineWeb2IR-ind_Latn_cosine_ndcg@10": 0.95035565121612,
+    "NanoFineWeb2IR-ita_Latn_cosine_ndcg@10": 0.8764693163757553,
+    "NanoFineWeb2IR-jpn_Jpan_cosine_ndcg@10": 0.9852371901428583,
+    "NanoFineWeb2IR-nld_Latn_cosine_ndcg@10": 0.9526185950714292,
+    "NanoFineWeb2IR-pol_Latn_cosine_ndcg@10": 0.914979928956448,
+    "NanoFineWeb2IR-por_Latn_cosine_ndcg@10": 0.9550828475372232,
+    "NanoFineWeb2IR-rus_Cyrl_cosine_ndcg@10": 0.96,
+    "NanoFineWeb2IR-spa_Latn_cosine_ndcg@10": 0.9489278926071438,
+    "NanoFineWeb2IR-swe_Latn_cosine_ndcg@10": 0.9704743802857166,
+    "NanoFineWeb2IR-tur_Latn_cosine_ndcg@10": 0.9483562699750574,
+    "NanoFineWeb2IR-vie_Latn_cosine_ndcg@10": 0.9378557852142875,
+    "NanoFineWeb2IR_mean_cosine_ndcg@10": 0.9461934552382264
+  },
+  "nano_miracl": {
+    "NanoMIRACL-ar_cosine_ndcg@10": 0.7159813245895191,
+    "NanoMIRACL-bn_cosine_ndcg@10": 0.6557347615638734,
+    "NanoMIRACL-de_cosine_ndcg@10": 0.6142857881341466,
+    "NanoMIRACL-en_cosine_ndcg@10": 0.6451146719526433,
+    "NanoMIRACL-es_cosine_ndcg@10": 0.714196293227969,
+    "NanoMIRACL-fa_cosine_ndcg@10": 0.692255846237923,
+    "NanoMIRACL-fi_cosine_ndcg@10": 0.7590971945501118,
+    "NanoMIRACL-fr_cosine_ndcg@10": 0.603279657690493,
+    "NanoMIRACL-hi_cosine_ndcg@10": 0.65128085385854,
+    "NanoMIRACL-id_cosine_ndcg@10": 0.6458179662981922,
+    "NanoMIRACL-ja_cosine_ndcg@10": 0.673998133845,
+    "NanoMIRACL-ko_cosine_ndcg@10": 0.6209661409618122,
+    "NanoMIRACL-ru_cosine_ndcg@10": 0.7143493917718859,
+    "NanoMIRACL-sw_cosine_ndcg@10": 0.5805953497600546,
+    "NanoMIRACL-te_cosine_ndcg@10": 0.9433298975036692,
+    "NanoMIRACL-th_cosine_ndcg@10": 0.8140644045739682,
+    "NanoMIRACL-yo_cosine_ndcg@10": 0.721939155186745,
+    "NanoMIRACL-zh_cosine_ndcg@10": 0.6723575757189189,
+    "NanoMIRACL_mean_cosine_ndcg@10": 0.6910358004125259
+  },
+  "summary": {
+    "NanoArguAna_mean_ndcg@10": 0.5181611256161548,
+    "NanoClimateFEVER-ja_mean_ndcg@10": 0.3544010279234292,
+    "NanoClimateFEVER_mean_ndcg@10": 0.3121963115000014,
+    "NanoDBPedia-ja_mean_ndcg@10": 0.607793412923947,
+    "NanoDBPedia_mean_ndcg@10": 0.5764677413740426,
+    "NanoFEVER-ja_mean_ndcg@10": 0.7755503010515641,
+    "NanoFEVER_mean_ndcg@10": 0.8391111267731001,
+    "NanoFiQA2018-ja_mean_ndcg@10": 0.335525591097055,
+    "NanoFiQA2018_mean_ndcg@10": 0.3569773959702302,
+    "NanoHotpotQA-ja_mean_ndcg@10": 0.7128002111186521,
+    "NanoHotpotQA_mean_ndcg@10": 0.7648584009779095,
+    "NanoMSMARCO-ja_mean_ndcg@10": 0.495952316384158,
+    "NanoMSMARCO_mean_ndcg@10": 0.501135037738672,
+    "NanoNFCorpus-ja_mean_ndcg@10": 0.2796027539405435,
+    "NanoNFCorpus_mean_ndcg@10": 0.2922564958462841,
+    "NanoNQ-ja_mean_ndcg@10": 0.5775267128386041,
+    "NanoNQ_mean_ndcg@10": 0.5038628890871119,
+    "NanoQuoraRetrieval-ja_mean_ndcg@10": 0.9079719053009623,
+    "NanoQuoraRetrieval_mean_ndcg@10": 0.8367000757926226,
+    "NanoSCIDOCS-ja_mean_ndcg@10": 0.3381960374851039,
+    "NanoSCIDOCS_mean_ndcg@10": 0.3261106666309668,
+    "NanoSciFact_mean_ndcg@10": 0.6681959232833932,
+    "NanoTouche2020_mean_ndcg@10": 0.4979933275770823,
+    "all_mean_ndcg@10": 0.5606630417552937,
+    "nano_fineweb2ir_mean_ndcg@10": 0.9461934552382264,
+    "nano_miracl_mean_ndcg@10": 0.6910358004125259
+  },
+  "timestamp_utc": "2026-07-07T01:52:44.215035+00:00"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5db74812b69eb6c17845f5ecaab533e6e0816877fcf9639a25dd187862d0e885
+size 211952896

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,55 @@
+{
+  "additional_special_tokens": [
+    "<start_of_turn>",
+    "<end_of_turn>"
+  ],
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bd47075711f75a143d1b78e01a41cc65c1c591b00d3cfeffc23db07adce1392
+size 34363442

tokenizer_config.json

@@ -0,0 +1,2024 @@
+{
+  "add_bos_token": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<unused60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<unused61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<unused62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<unused63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁���▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<start_of_turn>",
+    "<end_of_turn>"
+  ],
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<bos>",
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "max_length": 8192,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_to_multiple_of": null,
+  "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "<eos>",
+  "spaces_between_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<unk>"
+}