Hugging Face's logo

EchoLabs33
/
clip-vit-large-patch14-hxq

Zero-Shot Image Classification
Transformers
Safetensors
clip
vision
multimodal
image-text
compressed
hxq
helix-substrate
vector-quantization
helixcode
Eval Results (legacy)
8-bit precision
Model card Files
xet
 Community
voidstream commited on
Commit
0636546
·
verified ·
1 Parent(s): 39b1bcb

Initial upload: CLIP ViT-L/14 HXQ (3.6x compression, CIFAR-100 zero-shot paired eval)

Browse files
Files changed (11) hide show
  1. README.md +159 -0
  2. completeness_gate_receipt.json +386 -0
  3. config.json +413 -0
  4. conversion_receipt.json +17 -0
  5. merges.txt +0 -0
  6. model.safetensors +3 -0
  7. preprocessor_config.json +19 -0
  8. special_tokens_map.json +1 -0
  9. tokenizer.json +0 -0
  10. tokenizer_config.json +34 -0
  11. vocab.json +0 -0
README.md ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: openai/clip-vit-large-patch14
4
+ tags:
5
+ - clip
6
+ - vision
7
+ - multimodal
8
+ - image-text
9
+ - compressed
10
+ - hxq
11
+ - helix-substrate
12
+ - vector-quantization
13
+ - helixcode
14
+ library_name: transformers
15
+ pipeline_tag: zero-shot-image-classification
16
+ model-index:
17
+ - name: clip-vit-large-patch14-helix
18
+ results:
19
+ - task:
20
+ type: zero-shot-image-classification
21
+ name: Zero-Shot Image Classification
22
+ dataset:
23
+ name: CIFAR-100
24
+ type: cifar100
25
+ metrics:
26
+ - type: accuracy
27
+ value: 0.7275
28
+ name: Top-1 Accuracy
29
+ - type: top5_accuracy
30
+ value: 0.9164
31
+ name: Top-5 Accuracy
32
+ ---
33
+
34
+ # CLIP-ViT-L/14-HXQ
35
+
36
+ > **3.6x smaller from FP32. CIFAR-100 Top-1 72.8%. First vision model compressed with HXQ.**
37
+ >
38
+ > CLIP ViT-Large/14 (text + vision dual encoder) compressed from 1.6 GB to 447 MB. Zero-shot classification accuracy matches the dense baseline. No calibration data. Same codec that compresses Transformers, SSMs, Hybrids, and MoEs.
39
+
40
+ ## Install and Run
41
+
42
+ ```bash
43
+ pip install "helix-substrate[hf]"
44
+ ```
45
+
46
+ ```python
47
+ import helix_substrate # registers the HXQ quantizer with HuggingFace
48
+ from transformers import CLIPModel, CLIPProcessor
49
+ from PIL import Image
50
+
51
+ model = CLIPModel.from_pretrained("EchoLabs33/clip-vit-large-patch14-helix")
52
+ processor = CLIPProcessor.from_pretrained("EchoLabs33/clip-vit-large-patch14-helix")
53
+
54
+ image = Image.open("photo.jpg")
55
+ inputs = processor(
56
+ text=["a photo of a cat", "a photo of a dog", "a photo of a car"],
57
+ images=image,
58
+ return_tensors="pt",
59
+ padding=True,
60
+ )
61
+ outputs = model(**inputs)
62
+ probs = outputs.logits_per_image.softmax(dim=-1)
63
+ print(probs) # [cat_prob, dog_prob, car_prob]
64
+ ```
65
+
66
+ ## Downstream Benchmarks
67
+
68
+ Zero-shot CIFAR-100 classification (10,000 test images, 100 classes, prompt: "a photo of a {class}"):
69
+
70
+ | Metric | Dense | HXQ (3.6x) | Delta |
71
+ |--------|-------|-------------|-------|
72
+ | **Top-1 Accuracy** | 72.48% | **72.75%** | **+0.27%** |
73
+ | **Top-5 Accuracy** | 91.41% | **91.64%** | **+0.23%** |
74
+
75
+ All deltas within noise. Task performance preserved after 3.6x compression.
76
+
77
+ ## Compression Benchmark
78
+
79
+ | | Dense (FP32) | HXQ |
80
+ |---|---|---|
81
+ | **Size** | 1.6 GB | **447 MB** |
82
+ | **Compression ratio** | -- | **3.6x** |
83
+ | **VRAM (eval)** | 3,412 MB | **2,266 MB** |
84
+ | **Compressed modules** | -- | 218 HelixLinear layers |
85
+ | **Architecture** | CLIP (ViT-L/14 + Text Transformer) | unchanged |
86
+
87
+ ## Verification Status
88
+
89
+ - **Compression receipt:** PASS -- 218 compressed, 374 exact, mean cosine 0.9997
90
+ - **Conversion receipt:** PASS (Gate 1 + Gate 2)
91
+ - **Downstream eval:** PASS -- paired dense/HXQ on CIFAR-100 zero-shot
92
+
93
+ ## Good to Know
94
+
95
+ - **GPU and CPU supported** -- runs on any CUDA GPU or CPU.
96
+ - **Not fine-tunable** -- compressed weights are read-only (`is_trainable = False`).
97
+ - **Requires `helix-substrate`** -- you need `pip install "helix-substrate[hf]"`.
98
+ - **Embeddings stored exact** -- token, position, and patch embeddings are at full precision. Only the 218 attention + MLP linear layers are compressed.
99
+
100
+ ## What is HelixCode?
101
+
102
+ HelixCode is a universal weight compression codec based on vector quantization:
103
+
104
+ - Each weight matrix is replaced by a **256-entry codebook** (float32) + **uint8 index matrix** + optional **sidecar corrections** for outlier values
105
+ - The compressed form *is* the executable -- no decompression step
106
+ - Works on any `nn.Linear` regardless of architecture
107
+ - **No calibration data required** -- codebooks are fit from the weights alone
108
+
109
+ ## Architecture Details
110
+
111
+ CLIP ViT-Large/14 is a dual-encoder multimodal model:
112
+ - **Vision encoder:** 24-layer ViT-Large, hidden_size=1024, 16 attention heads, patch_size=14
113
+ - **Text encoder:** 12-layer Transformer, hidden_size=768, 12 attention heads
114
+ - **Cross-modal projections:** visual_projection (1024->768) + text_projection (768->768)
115
+
116
+ All 218 linear layers across both encoders are compressed. Embedding layers (token, position, patch), layer norms, and biases are stored at full precision.
117
+
118
+ ## Why This Matters
119
+
120
+ CLIP is the first **vision model** compressed with HXQ. The same codec now covers:
121
+
122
+ | Family | Models | Eval |
123
+ |--------|--------|------|
124
+ | **Transformer** | TinyLlama, Qwen 1.5B-14B | PPL within noise |
125
+ | **Pure SSM** | Mamba 130m, Mamba2 1.3B | PPL receipted |
126
+ | **Hybrid** | Zamba2 1.2B, 2.7B | PPL receipted |
127
+ | **MoE** | OLMoE 1B/7B | HellaSwag -0.16% |
128
+ | **Vision+Text** | **CLIP ViT-L/14** | **Top-1 +0.27%** |
129
+
130
+ Five architecture families. One codec. One `pip install`.
131
+
132
+ ## Companion Models
133
+
134
+ | Model | Architecture | Ratio | Eval Delta |
135
+ |-------|-------------|-------|------------|
136
+ | **clip-vit-large-patch14-helix** | **Vision+Text (CLIP)** | **3.6x** | **+0.27% Top-1** |
137
+ | [olmoe-1b-7b-instruct-helix](https://huggingface.co/EchoLabs33/olmoe-1b-7b-instruct-helix) | MoE (64 experts) | 1.9x | -0.16% HellaSwag |
138
+ | [zamba2-2.7b-instruct-helix](https://huggingface.co/EchoLabs33/zamba2-2.7b-instruct-helix) | Hybrid (Mamba2+Transformer) | 1.8x | +6.59% PPL |
139
+ | [zamba2-1.2b-helix](https://huggingface.co/EchoLabs33/zamba2-1.2b-helix) | Hybrid (Mamba2+Transformer) | 1.7x | +2.90% PPL |
140
+ | [qwen2.5-14b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-14b-instruct-helix) | Transformer | 3.4x | pending |
141
+ | [qwen2.5-3b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-3b-instruct-helix) | Transformer | 1.6x | +0.69% PPL |
142
+ | [tinyllama-1.1b-helix](https://huggingface.co/EchoLabs33/tinyllama-1.1b-helix) | Transformer | 4.0x | +0.78% PPL |
143
+ | [mamba2-1.3b-helix](https://huggingface.co/EchoLabs33/mamba2-1.3b-helix) | Pure SSM (Mamba2) | 2.1x | +8.0% PPL |
144
+ | [mamba-130m-helix](https://huggingface.co/EchoLabs33/mamba-130m-helix) | Pure SSM | 3.8x | +18.4% PPL |
145
+
146
+ ## Citation
147
+
148
+ ```bibtex
149
+ @software{helix_substrate_2026,
150
+ title={Helix Substrate: Universal Weight Compression via HelixCode},
151
+ author={EchoLabs},
152
+ year={2026},
153
+ url={https://github.com/echo313unfolding/helix-substrate}
154
+ }
155
+ ```
156
+
157
+ ## License
158
+
159
+ Apache 2.0 (inherited from [openai/clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14)).
completeness_gate_receipt.json ADDED
@@ -0,0 +1,386 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "verdict": "PASS",
3
+ "summary": {
4
+ "dense_tensors": 592,
5
+ "output_tensors": 1246,
6
+ "compressed_weights": 218,
7
+ "accounted": 592,
8
+ "missing": 0,
9
+ "skip_tensors_found": 363,
10
+ "skip_categories": {
11
+ "bias": "OK (291/291)",
12
+ "layernorm": "OK (72/72)"
13
+ }
14
+ },
15
+ "failures": [],
16
+ "missing_tensors": [],
17
+ "skip_found": {
18
+ "bias": [
19
+ "text_model.encoder.layers.0.layer_norm1.bias",
20
+ "text_model.encoder.layers.0.layer_norm2.bias",
21
+ "text_model.encoder.layers.0.mlp.fc1.bias",
22
+ "text_model.encoder.layers.0.mlp.fc2.bias",
23
+ "text_model.encoder.layers.0.self_attn.k_proj.bias",
24
+ "text_model.encoder.layers.0.self_attn.out_proj.bias",
25
+ "text_model.encoder.layers.0.self_attn.q_proj.bias",
26
+ "text_model.encoder.layers.0.self_attn.v_proj.bias",
27
+ "text_model.encoder.layers.1.layer_norm1.bias",
28
+ "text_model.encoder.layers.1.layer_norm2.bias",
29
+ "text_model.encoder.layers.1.mlp.fc1.bias",
30
+ "text_model.encoder.layers.1.mlp.fc2.bias",
31
+ "text_model.encoder.layers.1.self_attn.k_proj.bias",
32
+ "text_model.encoder.layers.1.self_attn.out_proj.bias",
33
+ "text_model.encoder.layers.1.self_attn.q_proj.bias",
34
+ "text_model.encoder.layers.1.self_attn.v_proj.bias",
35
+ "text_model.encoder.layers.10.layer_norm1.bias",
36
+ "text_model.encoder.layers.10.layer_norm2.bias",
37
+ "text_model.encoder.layers.10.mlp.fc1.bias",
38
+ "text_model.encoder.layers.10.mlp.fc2.bias",
39
+ "text_model.encoder.layers.10.self_attn.k_proj.bias",
40
+ "text_model.encoder.layers.10.self_attn.out_proj.bias",
41
+ "text_model.encoder.layers.10.self_attn.q_proj.bias",
42
+ "text_model.encoder.layers.10.self_attn.v_proj.bias",
43
+ "text_model.encoder.layers.11.layer_norm1.bias",
44
+ "text_model.encoder.layers.11.layer_norm2.bias",
45
+ "text_model.encoder.layers.11.mlp.fc1.bias",
46
+ "text_model.encoder.layers.11.mlp.fc2.bias",
47
+ "text_model.encoder.layers.11.self_attn.k_proj.bias",
48
+ "text_model.encoder.layers.11.self_attn.out_proj.bias",
49
+ "text_model.encoder.layers.11.self_attn.q_proj.bias",
50
+ "text_model.encoder.layers.11.self_attn.v_proj.bias",
51
+ "text_model.encoder.layers.2.layer_norm1.bias",
52
+ "text_model.encoder.layers.2.layer_norm2.bias",
53
+ "text_model.encoder.layers.2.mlp.fc1.bias",
54
+ "text_model.encoder.layers.2.mlp.fc2.bias",
55
+ "text_model.encoder.layers.2.self_attn.k_proj.bias",
56
+ "text_model.encoder.layers.2.self_attn.out_proj.bias",
57
+ "text_model.encoder.layers.2.self_attn.q_proj.bias",
58
+ "text_model.encoder.layers.2.self_attn.v_proj.bias",
59
+ "text_model.encoder.layers.3.layer_norm1.bias",
60
+ "text_model.encoder.layers.3.layer_norm2.bias",
61
+ "text_model.encoder.layers.3.mlp.fc1.bias",
62
+ "text_model.encoder.layers.3.mlp.fc2.bias",
63
+ "text_model.encoder.layers.3.self_attn.k_proj.bias",
64
+ "text_model.encoder.layers.3.self_attn.out_proj.bias",
65
+ "text_model.encoder.layers.3.self_attn.q_proj.bias",
66
+ "text_model.encoder.layers.3.self_attn.v_proj.bias",
67
+ "text_model.encoder.layers.4.layer_norm1.bias",
68
+ "text_model.encoder.layers.4.layer_norm2.bias",
69
+ "text_model.encoder.layers.4.mlp.fc1.bias",
70
+ "text_model.encoder.layers.4.mlp.fc2.bias",
71
+ "text_model.encoder.layers.4.self_attn.k_proj.bias",
72
+ "text_model.encoder.layers.4.self_attn.out_proj.bias",
73
+ "text_model.encoder.layers.4.self_attn.q_proj.bias",
74
+ "text_model.encoder.layers.4.self_attn.v_proj.bias",
75
+ "text_model.encoder.layers.5.layer_norm1.bias",
76
+ "text_model.encoder.layers.5.layer_norm2.bias",
77
+ "text_model.encoder.layers.5.mlp.fc1.bias",
78
+ "text_model.encoder.layers.5.mlp.fc2.bias",
79
+ "text_model.encoder.layers.5.self_attn.k_proj.bias",
80
+ "text_model.encoder.layers.5.self_attn.out_proj.bias",
81
+ "text_model.encoder.layers.5.self_attn.q_proj.bias",
82
+ "text_model.encoder.layers.5.self_attn.v_proj.bias",
83
+ "text_model.encoder.layers.6.layer_norm1.bias",
84
+ "text_model.encoder.layers.6.layer_norm2.bias",
85
+ "text_model.encoder.layers.6.mlp.fc1.bias",
86
+ "text_model.encoder.layers.6.mlp.fc2.bias",
87
+ "text_model.encoder.layers.6.self_attn.k_proj.bias",
88
+ "text_model.encoder.layers.6.self_attn.out_proj.bias",
89
+ "text_model.encoder.layers.6.self_attn.q_proj.bias",
90
+ "text_model.encoder.layers.6.self_attn.v_proj.bias",
91
+ "text_model.encoder.layers.7.layer_norm1.bias",
92
+ "text_model.encoder.layers.7.layer_norm2.bias",
93
+ "text_model.encoder.layers.7.mlp.fc1.bias",
94
+ "text_model.encoder.layers.7.mlp.fc2.bias",
95
+ "text_model.encoder.layers.7.self_attn.k_proj.bias",
96
+ "text_model.encoder.layers.7.self_attn.out_proj.bias",
97
+ "text_model.encoder.layers.7.self_attn.q_proj.bias",
98
+ "text_model.encoder.layers.7.self_attn.v_proj.bias",
99
+ "text_model.encoder.layers.8.layer_norm1.bias",
100
+ "text_model.encoder.layers.8.layer_norm2.bias",
101
+ "text_model.encoder.layers.8.mlp.fc1.bias",
102
+ "text_model.encoder.layers.8.mlp.fc2.bias",
103
+ "text_model.encoder.layers.8.self_attn.k_proj.bias",
104
+ "text_model.encoder.layers.8.self_attn.out_proj.bias",
105
+ "text_model.encoder.layers.8.self_attn.q_proj.bias",
106
+ "text_model.encoder.layers.8.self_attn.v_proj.bias",
107
+ "text_model.encoder.layers.9.layer_norm1.bias",
108
+ "text_model.encoder.layers.9.layer_norm2.bias",
109
+ "text_model.encoder.layers.9.mlp.fc1.bias",
110
+ "text_model.encoder.layers.9.mlp.fc2.bias",
111
+ "text_model.encoder.layers.9.self_attn.k_proj.bias",
112
+ "text_model.encoder.layers.9.self_attn.out_proj.bias",
113
+ "text_model.encoder.layers.9.self_attn.q_proj.bias",
114
+ "text_model.encoder.layers.9.self_attn.v_proj.bias",
115
+ "text_model.final_layer_norm.bias",
116
+ "vision_model.encoder.layers.0.layer_norm1.bias",
117
+ "vision_model.encoder.layers.0.layer_norm2.bias",
118
+ "vision_model.encoder.layers.0.mlp.fc1.bias",
119
+ "vision_model.encoder.layers.0.mlp.fc2.bias",
120
+ "vision_model.encoder.layers.0.self_attn.k_proj.bias",
121
+ "vision_model.encoder.layers.0.self_attn.out_proj.bias",
122
+ "vision_model.encoder.layers.0.self_attn.q_proj.bias",
123
+ "vision_model.encoder.layers.0.self_attn.v_proj.bias",
124
+ "vision_model.encoder.layers.1.layer_norm1.bias",
125
+ "vision_model.encoder.layers.1.layer_norm2.bias",
126
+ "vision_model.encoder.layers.1.mlp.fc1.bias",
127
+ "vision_model.encoder.layers.1.mlp.fc2.bias",
128
+ "vision_model.encoder.layers.1.self_attn.k_proj.bias",
129
+ "vision_model.encoder.layers.1.self_attn.out_proj.bias",
130
+ "vision_model.encoder.layers.1.self_attn.q_proj.bias",
131
+ "vision_model.encoder.layers.1.self_attn.v_proj.bias",
132
+ "vision_model.encoder.layers.10.layer_norm1.bias",
133
+ "vision_model.encoder.layers.10.layer_norm2.bias",
134
+ "vision_model.encoder.layers.10.mlp.fc1.bias",
135
+ "vision_model.encoder.layers.10.mlp.fc2.bias",
136
+ "vision_model.encoder.layers.10.self_attn.k_proj.bias",
137
+ "vision_model.encoder.layers.10.self_attn.out_proj.bias",
138
+ "vision_model.encoder.layers.10.self_attn.q_proj.bias",
139
+ "vision_model.encoder.layers.10.self_attn.v_proj.bias",
140
+ "vision_model.encoder.layers.11.layer_norm1.bias",
141
+ "vision_model.encoder.layers.11.layer_norm2.bias",
142
+ "vision_model.encoder.layers.11.mlp.fc1.bias",
143
+ "vision_model.encoder.layers.11.mlp.fc2.bias",
144
+ "vision_model.encoder.layers.11.self_attn.k_proj.bias",
145
+ "vision_model.encoder.layers.11.self_attn.out_proj.bias",
146
+ "vision_model.encoder.layers.11.self_attn.q_proj.bias",
147
+ "vision_model.encoder.layers.11.self_attn.v_proj.bias",
148
+ "vision_model.encoder.layers.12.layer_norm1.bias",
149
+ "vision_model.encoder.layers.12.layer_norm2.bias",
150
+ "vision_model.encoder.layers.12.mlp.fc1.bias",
151
+ "vision_model.encoder.layers.12.mlp.fc2.bias",
152
+ "vision_model.encoder.layers.12.self_attn.k_proj.bias",
153
+ "vision_model.encoder.layers.12.self_attn.out_proj.bias",
154
+ "vision_model.encoder.layers.12.self_attn.q_proj.bias",
155
+ "vision_model.encoder.layers.12.self_attn.v_proj.bias",
156
+ "vision_model.encoder.layers.13.layer_norm1.bias",
157
+ "vision_model.encoder.layers.13.layer_norm2.bias",
158
+ "vision_model.encoder.layers.13.mlp.fc1.bias",
159
+ "vision_model.encoder.layers.13.mlp.fc2.bias",
160
+ "vision_model.encoder.layers.13.self_attn.k_proj.bias",
161
+ "vision_model.encoder.layers.13.self_attn.out_proj.bias",
162
+ "vision_model.encoder.layers.13.self_attn.q_proj.bias",
163
+ "vision_model.encoder.layers.13.self_attn.v_proj.bias",
164
+ "vision_model.encoder.layers.14.layer_norm1.bias",
165
+ "vision_model.encoder.layers.14.layer_norm2.bias",
166
+ "vision_model.encoder.layers.14.mlp.fc1.bias",
167
+ "vision_model.encoder.layers.14.mlp.fc2.bias",
168
+ "vision_model.encoder.layers.14.self_attn.k_proj.bias",
169
+ "vision_model.encoder.layers.14.self_attn.out_proj.bias",
170
+ "vision_model.encoder.layers.14.self_attn.q_proj.bias",
171
+ "vision_model.encoder.layers.14.self_attn.v_proj.bias",
172
+ "vision_model.encoder.layers.15.layer_norm1.bias",
173
+ "vision_model.encoder.layers.15.layer_norm2.bias",
174
+ "vision_model.encoder.layers.15.mlp.fc1.bias",
175
+ "vision_model.encoder.layers.15.mlp.fc2.bias",
176
+ "vision_model.encoder.layers.15.self_attn.k_proj.bias",
177
+ "vision_model.encoder.layers.15.self_attn.out_proj.bias",
178
+ "vision_model.encoder.layers.15.self_attn.q_proj.bias",
179
+ "vision_model.encoder.layers.15.self_attn.v_proj.bias",
180
+ "vision_model.encoder.layers.16.layer_norm1.bias",
181
+ "vision_model.encoder.layers.16.layer_norm2.bias",
182
+ "vision_model.encoder.layers.16.mlp.fc1.bias",
183
+ "vision_model.encoder.layers.16.mlp.fc2.bias",
184
+ "vision_model.encoder.layers.16.self_attn.k_proj.bias",
185
+ "vision_model.encoder.layers.16.self_attn.out_proj.bias",
186
+ "vision_model.encoder.layers.16.self_attn.q_proj.bias",
187
+ "vision_model.encoder.layers.16.self_attn.v_proj.bias",
188
+ "vision_model.encoder.layers.17.layer_norm1.bias",
189
+ "vision_model.encoder.layers.17.layer_norm2.bias",
190
+ "vision_model.encoder.layers.17.mlp.fc1.bias",
191
+ "vision_model.encoder.layers.17.mlp.fc2.bias",
192
+ "vision_model.encoder.layers.17.self_attn.k_proj.bias",
193
+ "vision_model.encoder.layers.17.self_attn.out_proj.bias",
194
+ "vision_model.encoder.layers.17.self_attn.q_proj.bias",
195
+ "vision_model.encoder.layers.17.self_attn.v_proj.bias",
196
+ "vision_model.encoder.layers.18.layer_norm1.bias",
197
+ "vision_model.encoder.layers.18.layer_norm2.bias",
198
+ "vision_model.encoder.layers.18.mlp.fc1.bias",
199
+ "vision_model.encoder.layers.18.mlp.fc2.bias",
200
+ "vision_model.encoder.layers.18.self_attn.k_proj.bias",
201
+ "vision_model.encoder.layers.18.self_attn.out_proj.bias",
202
+ "vision_model.encoder.layers.18.self_attn.q_proj.bias",
203
+ "vision_model.encoder.layers.18.self_attn.v_proj.bias",
204
+ "vision_model.encoder.layers.19.layer_norm1.bias",
205
+ "vision_model.encoder.layers.19.layer_norm2.bias",
206
+ "vision_model.encoder.layers.19.mlp.fc1.bias",
207
+ "vision_model.encoder.layers.19.mlp.fc2.bias",
208
+ "vision_model.encoder.layers.19.self_attn.k_proj.bias",
209
+ "vision_model.encoder.layers.19.self_attn.out_proj.bias",
210
+ "vision_model.encoder.layers.19.self_attn.q_proj.bias",
211
+ "vision_model.encoder.layers.19.self_attn.v_proj.bias",
212
+ "vision_model.encoder.layers.2.layer_norm1.bias",
213
+ "vision_model.encoder.layers.2.layer_norm2.bias",
214
+ "vision_model.encoder.layers.2.mlp.fc1.bias",
215
+ "vision_model.encoder.layers.2.mlp.fc2.bias",
216
+ "vision_model.encoder.layers.2.self_attn.k_proj.bias",
217
+ "vision_model.encoder.layers.2.self_attn.out_proj.bias",
218
+ "vision_model.encoder.layers.2.self_attn.q_proj.bias",
219
+ "vision_model.encoder.layers.2.self_attn.v_proj.bias",
220
+ "vision_model.encoder.layers.20.layer_norm1.bias",
221
+ "vision_model.encoder.layers.20.layer_norm2.bias",
222
+ "vision_model.encoder.layers.20.mlp.fc1.bias",
223
+ "vision_model.encoder.layers.20.mlp.fc2.bias",
224
+ "vision_model.encoder.layers.20.self_attn.k_proj.bias",
225
+ "vision_model.encoder.layers.20.self_attn.out_proj.bias",
226
+ "vision_model.encoder.layers.20.self_attn.q_proj.bias",
227
+ "vision_model.encoder.layers.20.self_attn.v_proj.bias",
228
+ "vision_model.encoder.layers.21.layer_norm1.bias",
229
+ "vision_model.encoder.layers.21.layer_norm2.bias",
230
+ "vision_model.encoder.layers.21.mlp.fc1.bias",
231
+ "vision_model.encoder.layers.21.mlp.fc2.bias",
232
+ "vision_model.encoder.layers.21.self_attn.k_proj.bias",
233
+ "vision_model.encoder.layers.21.self_attn.out_proj.bias",
234
+ "vision_model.encoder.layers.21.self_attn.q_proj.bias",
235
+ "vision_model.encoder.layers.21.self_attn.v_proj.bias",
236
+ "vision_model.encoder.layers.22.layer_norm1.bias",
237
+ "vision_model.encoder.layers.22.layer_norm2.bias",
238
+ "vision_model.encoder.layers.22.mlp.fc1.bias",
239
+ "vision_model.encoder.layers.22.mlp.fc2.bias",
240
+ "vision_model.encoder.layers.22.self_attn.k_proj.bias",
241
+ "vision_model.encoder.layers.22.self_attn.out_proj.bias",
242
+ "vision_model.encoder.layers.22.self_attn.q_proj.bias",
243
+ "vision_model.encoder.layers.22.self_attn.v_proj.bias",
244
+ "vision_model.encoder.layers.23.layer_norm1.bias",
245
+ "vision_model.encoder.layers.23.layer_norm2.bias",
246
+ "vision_model.encoder.layers.23.mlp.fc1.bias",
247
+ "vision_model.encoder.layers.23.mlp.fc2.bias",
248
+ "vision_model.encoder.layers.23.self_attn.k_proj.bias",
249
+ "vision_model.encoder.layers.23.self_attn.out_proj.bias",
250
+ "vision_model.encoder.layers.23.self_attn.q_proj.bias",
251
+ "vision_model.encoder.layers.23.self_attn.v_proj.bias",
252
+ "vision_model.encoder.layers.3.layer_norm1.bias",
253
+ "vision_model.encoder.layers.3.layer_norm2.bias",
254
+ "vision_model.encoder.layers.3.mlp.fc1.bias",
255
+ "vision_model.encoder.layers.3.mlp.fc2.bias",
256
+ "vision_model.encoder.layers.3.self_attn.k_proj.bias",
257
+ "vision_model.encoder.layers.3.self_attn.out_proj.bias",
258
+ "vision_model.encoder.layers.3.self_attn.q_proj.bias",
259
+ "vision_model.encoder.layers.3.self_attn.v_proj.bias",
260
+ "vision_model.encoder.layers.4.layer_norm1.bias",
261
+ "vision_model.encoder.layers.4.layer_norm2.bias",
262
+ "vision_model.encoder.layers.4.mlp.fc1.bias",
263
+ "vision_model.encoder.layers.4.mlp.fc2.bias",
264
+ "vision_model.encoder.layers.4.self_attn.k_proj.bias",
265
+ "vision_model.encoder.layers.4.self_attn.out_proj.bias",
266
+ "vision_model.encoder.layers.4.self_attn.q_proj.bias",
267
+ "vision_model.encoder.layers.4.self_attn.v_proj.bias",
268
+ "vision_model.encoder.layers.5.layer_norm1.bias",
269
+ "vision_model.encoder.layers.5.layer_norm2.bias",
270
+ "vision_model.encoder.layers.5.mlp.fc1.bias",
271
+ "vision_model.encoder.layers.5.mlp.fc2.bias",
272
+ "vision_model.encoder.layers.5.self_attn.k_proj.bias",
273
+ "vision_model.encoder.layers.5.self_attn.out_proj.bias",
274
+ "vision_model.encoder.layers.5.self_attn.q_proj.bias",
275
+ "vision_model.encoder.layers.5.self_attn.v_proj.bias",
276
+ "vision_model.encoder.layers.6.layer_norm1.bias",
277
+ "vision_model.encoder.layers.6.layer_norm2.bias",
278
+ "vision_model.encoder.layers.6.mlp.fc1.bias",
279
+ "vision_model.encoder.layers.6.mlp.fc2.bias",
280
+ "vision_model.encoder.layers.6.self_attn.k_proj.bias",
281
+ "vision_model.encoder.layers.6.self_attn.out_proj.bias",
282
+ "vision_model.encoder.layers.6.self_attn.q_proj.bias",
283
+ "vision_model.encoder.layers.6.self_attn.v_proj.bias",
284
+ "vision_model.encoder.layers.7.layer_norm1.bias",
285
+ "vision_model.encoder.layers.7.layer_norm2.bias",
286
+ "vision_model.encoder.layers.7.mlp.fc1.bias",
287
+ "vision_model.encoder.layers.7.mlp.fc2.bias",
288
+ "vision_model.encoder.layers.7.self_attn.k_proj.bias",
289
+ "vision_model.encoder.layers.7.self_attn.out_proj.bias",
290
+ "vision_model.encoder.layers.7.self_attn.q_proj.bias",
291
+ "vision_model.encoder.layers.7.self_attn.v_proj.bias",
292
+ "vision_model.encoder.layers.8.layer_norm1.bias",
293
+ "vision_model.encoder.layers.8.layer_norm2.bias",
294
+ "vision_model.encoder.layers.8.mlp.fc1.bias",
295
+ "vision_model.encoder.layers.8.mlp.fc2.bias",
296
+ "vision_model.encoder.layers.8.self_attn.k_proj.bias",
297
+ "vision_model.encoder.layers.8.self_attn.out_proj.bias",
298
+ "vision_model.encoder.layers.8.self_attn.q_proj.bias",
299
+ "vision_model.encoder.layers.8.self_attn.v_proj.bias",
300
+ "vision_model.encoder.layers.9.layer_norm1.bias",
301
+ "vision_model.encoder.layers.9.layer_norm2.bias",
302
+ "vision_model.encoder.layers.9.mlp.fc1.bias",
303
+ "vision_model.encoder.layers.9.mlp.fc2.bias",
304
+ "vision_model.encoder.layers.9.self_attn.k_proj.bias",
305
+ "vision_model.encoder.layers.9.self_attn.out_proj.bias",
306
+ "vision_model.encoder.layers.9.self_attn.q_proj.bias",
307
+ "vision_model.encoder.layers.9.self_attn.v_proj.bias",
308
+ "vision_model.post_layernorm.bias",
309
+ "vision_model.pre_layrnorm.bias"
310
+ ],
311
+ "layernorm": [
312
+ "text_model.encoder.layers.0.layer_norm1.weight",
313
+ "text_model.encoder.layers.0.layer_norm2.weight",
314
+ "text_model.encoder.layers.1.layer_norm1.weight",
315
+ "text_model.encoder.layers.1.layer_norm2.weight",
316
+ "text_model.encoder.layers.10.layer_norm1.weight",
317
+ "text_model.encoder.layers.10.layer_norm2.weight",
318
+ "text_model.encoder.layers.11.layer_norm1.weight",
319
+ "text_model.encoder.layers.11.layer_norm2.weight",
320
+ "text_model.encoder.layers.2.layer_norm1.weight",
321
+ "text_model.encoder.layers.2.layer_norm2.weight",
322
+ "text_model.encoder.layers.3.layer_norm1.weight",
323
+ "text_model.encoder.layers.3.layer_norm2.weight",
324
+ "text_model.encoder.layers.4.layer_norm1.weight",
325
+ "text_model.encoder.layers.4.layer_norm2.weight",
326
+ "text_model.encoder.layers.5.layer_norm1.weight",
327
+ "text_model.encoder.layers.5.layer_norm2.weight",
328
+ "text_model.encoder.layers.6.layer_norm1.weight",
329
+ "text_model.encoder.layers.6.layer_norm2.weight",
330
+ "text_model.encoder.layers.7.layer_norm1.weight",
331
+ "text_model.encoder.layers.7.layer_norm2.weight",
332
+ "text_model.encoder.layers.8.layer_norm1.weight",
333
+ "text_model.encoder.layers.8.layer_norm2.weight",
334
+ "text_model.encoder.layers.9.layer_norm1.weight",
335
+ "text_model.encoder.layers.9.layer_norm2.weight",
336
+ "vision_model.encoder.layers.0.layer_norm1.weight",
337
+ "vision_model.encoder.layers.0.layer_norm2.weight",
338
+ "vision_model.encoder.layers.1.layer_norm1.weight",
339
+ "vision_model.encoder.layers.1.layer_norm2.weight",
340
+ "vision_model.encoder.layers.10.layer_norm1.weight",
341
+ "vision_model.encoder.layers.10.layer_norm2.weight",
342
+ "vision_model.encoder.layers.11.layer_norm1.weight",
343
+ "vision_model.encoder.layers.11.layer_norm2.weight",
344
+ "vision_model.encoder.layers.12.layer_norm1.weight",
345
+ "vision_model.encoder.layers.12.layer_norm2.weight",
346
+ "vision_model.encoder.layers.13.layer_norm1.weight",
347
+ "vision_model.encoder.layers.13.layer_norm2.weight",
348
+ "vision_model.encoder.layers.14.layer_norm1.weight",
349
+ "vision_model.encoder.layers.14.layer_norm2.weight",
350
+ "vision_model.encoder.layers.15.layer_norm1.weight",
351
+ "vision_model.encoder.layers.15.layer_norm2.weight",
352
+ "vision_model.encoder.layers.16.layer_norm1.weight",
353
+ "vision_model.encoder.layers.16.layer_norm2.weight",
354
+ "vision_model.encoder.layers.17.layer_norm1.weight",
355
+ "vision_model.encoder.layers.17.layer_norm2.weight",
356
+ "vision_model.encoder.layers.18.layer_norm1.weight",
357
+ "vision_model.encoder.layers.18.layer_norm2.weight",
358
+ "vision_model.encoder.layers.19.layer_norm1.weight",
359
+ "vision_model.encoder.layers.19.layer_norm2.weight",
360
+ "vision_model.encoder.layers.2.layer_norm1.weight",
361
+ "vision_model.encoder.layers.2.layer_norm2.weight",
362
+ "vision_model.encoder.layers.20.layer_norm1.weight",
363
+ "vision_model.encoder.layers.20.layer_norm2.weight",
364
+ "vision_model.encoder.layers.21.layer_norm1.weight",
365
+ "vision_model.encoder.layers.21.layer_norm2.weight",
366
+ "vision_model.encoder.layers.22.layer_norm1.weight",
367
+ "vision_model.encoder.layers.22.layer_norm2.weight",
368
+ "vision_model.encoder.layers.23.layer_norm1.weight",
369
+ "vision_model.encoder.layers.23.layer_norm2.weight",
370
+ "vision_model.encoder.layers.3.layer_norm1.weight",
371
+ "vision_model.encoder.layers.3.layer_norm2.weight",
372
+ "vision_model.encoder.layers.4.layer_norm1.weight",
373
+ "vision_model.encoder.layers.4.layer_norm2.weight",
374
+ "vision_model.encoder.layers.5.layer_norm1.weight",
375
+ "vision_model.encoder.layers.5.layer_norm2.weight",
376
+ "vision_model.encoder.layers.6.layer_norm1.weight",
377
+ "vision_model.encoder.layers.6.layer_norm2.weight",
378
+ "vision_model.encoder.layers.7.layer_norm1.weight",
379
+ "vision_model.encoder.layers.7.layer_norm2.weight",
380
+ "vision_model.encoder.layers.8.layer_norm1.weight",
381
+ "vision_model.encoder.layers.8.layer_norm2.weight",
382
+ "vision_model.encoder.layers.9.layer_norm1.weight",
383
+ "vision_model.encoder.layers.9.layer_norm2.weight"
384
+ ]
385
+ }
386
+ }
config.json ADDED
@@ -0,0 +1,413 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "clip-vit-large-patch14/",
3
+ "architectures": [
4
+ "CLIPModel"
5
+ ],
6
+ "initializer_factor": 1.0,
7
+ "logit_scale_init_value": 2.6592,
8
+ "model_type": "clip",
9
+ "projection_dim": 768,
10
+ "text_config": {
11
+ "_name_or_path": "",
12
+ "add_cross_attention": false,
13
+ "architectures": null,
14
+ "attention_dropout": 0.0,
15
+ "bad_words_ids": null,
16
+ "bos_token_id": 0,
17
+ "chunk_size_feed_forward": 0,
18
+ "cross_attention_hidden_size": null,
19
+ "decoder_start_token_id": null,
20
+ "diversity_penalty": 0.0,
21
+ "do_sample": false,
22
+ "dropout": 0.0,
23
+ "early_stopping": false,
24
+ "encoder_no_repeat_ngram_size": 0,
25
+ "eos_token_id": 2,
26
+ "finetuning_task": null,
27
+ "forced_bos_token_id": null,
28
+ "forced_eos_token_id": null,
29
+ "hidden_act": "quick_gelu",
30
+ "hidden_size": 768,
31
+ "id2label": {
32
+ "0": "LABEL_0",
33
+ "1": "LABEL_1"
34
+ },
35
+ "initializer_factor": 1.0,
36
+ "initializer_range": 0.02,
37
+ "intermediate_size": 3072,
38
+ "is_decoder": false,
39
+ "is_encoder_decoder": false,
40
+ "label2id": {
41
+ "LABEL_0": 0,
42
+ "LABEL_1": 1
43
+ },
44
+ "layer_norm_eps": 1e-05,
45
+ "length_penalty": 1.0,
46
+ "max_length": 20,
47
+ "max_position_embeddings": 77,
48
+ "min_length": 0,
49
+ "model_type": "clip_text_model",
50
+ "no_repeat_ngram_size": 0,
51
+ "num_attention_heads": 12,
52
+ "num_beam_groups": 1,
53
+ "num_beams": 1,
54
+ "num_hidden_layers": 12,
55
+ "num_return_sequences": 1,
56
+ "output_attentions": false,
57
+ "output_hidden_states": false,
58
+ "output_scores": false,
59
+ "pad_token_id": 1,
60
+ "prefix": null,
61
+ "problem_type": null,
62
+ "projection_dim": 768,
63
+ "pruned_heads": {},
64
+ "remove_invalid_values": false,
65
+ "repetition_penalty": 1.0,
66
+ "return_dict": true,
67
+ "return_dict_in_generate": false,
68
+ "sep_token_id": null,
69
+ "task_specific_params": null,
70
+ "temperature": 1.0,
71
+ "tie_encoder_decoder": false,
72
+ "tie_word_embeddings": true,
73
+ "tokenizer_class": null,
74
+ "top_k": 50,
75
+ "top_p": 1.0,
76
+ "torch_dtype": null,
77
+ "torchscript": false,
78
+ "transformers_version": "4.16.0.dev0",
79
+ "use_bfloat16": false,
80
+ "vocab_size": 49408
81
+ },
82
+ "text_config_dict": {
83
+ "hidden_size": 768,
84
+ "intermediate_size": 3072,
85
+ "num_attention_heads": 12,
86
+ "num_hidden_layers": 12,
87
+ "projection_dim": 768
88
+ },
89
+ "torch_dtype": "float32",
90
+ "transformers_version": null,
91
+ "vision_config": {
92
+ "_name_or_path": "",
93
+ "add_cross_attention": false,
94
+ "architectures": null,
95
+ "attention_dropout": 0.0,
96
+ "bad_words_ids": null,
97
+ "bos_token_id": null,
98
+ "chunk_size_feed_forward": 0,
99
+ "cross_attention_hidden_size": null,
100
+ "decoder_start_token_id": null,
101
+ "diversity_penalty": 0.0,
102
+ "do_sample": false,
103
+ "dropout": 0.0,
104
+ "early_stopping": false,
105
+ "encoder_no_repeat_ngram_size": 0,
106
+ "eos_token_id": null,
107
+ "finetuning_task": null,
108
+ "forced_bos_token_id": null,
109
+ "forced_eos_token_id": null,
110
+ "hidden_act": "quick_gelu",
111
+ "hidden_size": 1024,
112
+ "id2label": {
113
+ "0": "LABEL_0",
114
+ "1": "LABEL_1"
115
+ },
116
+ "image_size": 224,
117
+ "initializer_factor": 1.0,
118
+ "initializer_range": 0.02,
119
+ "intermediate_size": 4096,
120
+ "is_decoder": false,
121
+ "is_encoder_decoder": false,
122
+ "label2id": {
123
+ "LABEL_0": 0,
124
+ "LABEL_1": 1
125
+ },
126
+ "layer_norm_eps": 1e-05,
127
+ "length_penalty": 1.0,
128
+ "max_length": 20,
129
+ "min_length": 0,
130
+ "model_type": "clip_vision_model",
131
+ "no_repeat_ngram_size": 0,
132
+ "num_attention_heads": 16,
133
+ "num_beam_groups": 1,
134
+ "num_beams": 1,
135
+ "num_hidden_layers": 24,
136
+ "num_return_sequences": 1,
137
+ "output_attentions": false,
138
+ "output_hidden_states": false,
139
+ "output_scores": false,
140
+ "pad_token_id": null,
141
+ "patch_size": 14,
142
+ "prefix": null,
143
+ "problem_type": null,
144
+ "projection_dim": 768,
145
+ "pruned_heads": {},
146
+ "remove_invalid_values": false,
147
+ "repetition_penalty": 1.0,
148
+ "return_dict": true,
149
+ "return_dict_in_generate": false,
150
+ "sep_token_id": null,
151
+ "task_specific_params": null,
152
+ "temperature": 1.0,
153
+ "tie_encoder_decoder": false,
154
+ "tie_word_embeddings": true,
155
+ "tokenizer_class": null,
156
+ "top_k": 50,
157
+ "top_p": 1.0,
158
+ "torch_dtype": null,
159
+ "torchscript": false,
160
+ "transformers_version": "4.16.0.dev0",
161
+ "use_bfloat16": false
162
+ },
163
+ "vision_config_dict": {
164
+ "hidden_size": 1024,
165
+ "intermediate_size": 4096,
166
+ "num_attention_heads": 16,
167
+ "num_hidden_layers": 24,
168
+ "patch_size": 14,
169
+ "projection_dim": 768
170
+ },
171
+ "hxq_exact_patterns": [
172
+ "token_embedding",
173
+ "position_embedding",
174
+ "patch_embedding",
175
+ "position_ids"
176
+ ],
177
+ "quantization_config": {
178
+ "quant_method": "hxq",
179
+ "codebook_size": 256,
180
+ "sidecar_enabled": true,
181
+ "exact_patterns": [
182
+ "embed_tokens",
183
+ "embed_positions",
184
+ "wte",
185
+ "wpe",
186
+ "lm_head",
187
+ "layernorm",
188
+ "layer_norm",
189
+ "norm",
190
+ "backbone.embedding"
191
+ ],
192
+ "compressed_modules": [
193
+ "text_model.encoder.layers.0.mlp.fc1",
194
+ "text_model.encoder.layers.0.mlp.fc2",
195
+ "text_model.encoder.layers.0.self_attn.k_proj",
196
+ "text_model.encoder.layers.0.self_attn.out_proj",
197
+ "text_model.encoder.layers.0.self_attn.q_proj",
198
+ "text_model.encoder.layers.0.self_attn.v_proj",
199
+ "text_model.encoder.layers.10.mlp.fc1",
200
+ "text_model.encoder.layers.10.mlp.fc2",
201
+ "text_model.encoder.layers.10.self_attn.k_proj",
202
+ "text_model.encoder.layers.10.self_attn.out_proj",
203
+ "text_model.encoder.layers.10.self_attn.q_proj",
204
+ "text_model.encoder.layers.10.self_attn.v_proj",
205
+ "text_model.encoder.layers.11.mlp.fc1",
206
+ "text_model.encoder.layers.11.mlp.fc2",
207
+ "text_model.encoder.layers.11.self_attn.k_proj",
208
+ "text_model.encoder.layers.11.self_attn.out_proj",
209
+ "text_model.encoder.layers.11.self_attn.q_proj",
210
+ "text_model.encoder.layers.11.self_attn.v_proj",
211
+ "text_model.encoder.layers.1.mlp.fc1",
212
+ "text_model.encoder.layers.1.mlp.fc2",
213
+ "text_model.encoder.layers.1.self_attn.k_proj",
214
+ "text_model.encoder.layers.1.self_attn.out_proj",
215
+ "text_model.encoder.layers.1.self_attn.q_proj",
216
+ "text_model.encoder.layers.1.self_attn.v_proj",
217
+ "text_model.encoder.layers.2.mlp.fc1",
218
+ "text_model.encoder.layers.2.mlp.fc2",
219
+ "text_model.encoder.layers.2.self_attn.k_proj",
220
+ "text_model.encoder.layers.2.self_attn.out_proj",
221
+ "text_model.encoder.layers.2.self_attn.q_proj",
222
+ "text_model.encoder.layers.2.self_attn.v_proj",
223
+ "text_model.encoder.layers.3.mlp.fc1",
224
+ "text_model.encoder.layers.3.mlp.fc2",
225
+ "text_model.encoder.layers.3.self_attn.k_proj",
226
+ "text_model.encoder.layers.3.self_attn.out_proj",
227
+ "text_model.encoder.layers.3.self_attn.q_proj",
228
+ "text_model.encoder.layers.3.self_attn.v_proj",
229
+ "text_model.encoder.layers.4.mlp.fc1",
230
+ "text_model.encoder.layers.4.mlp.fc2",
231
+ "text_model.encoder.layers.4.self_attn.k_proj",
232
+ "text_model.encoder.layers.4.self_attn.out_proj",
233
+ "text_model.encoder.layers.4.self_attn.q_proj",
234
+ "text_model.encoder.layers.4.self_attn.v_proj",
235
+ "text_model.encoder.layers.5.mlp.fc1",
236
+ "text_model.encoder.layers.5.mlp.fc2",
237
+ "text_model.encoder.layers.5.self_attn.k_proj",
238
+ "text_model.encoder.layers.5.self_attn.out_proj",
239
+ "text_model.encoder.layers.5.self_attn.q_proj",
240
+ "text_model.encoder.layers.5.self_attn.v_proj",
241
+ "text_model.encoder.layers.6.mlp.fc1",
242
+ "text_model.encoder.layers.6.mlp.fc2",
243
+ "text_model.encoder.layers.6.self_attn.k_proj",
244
+ "text_model.encoder.layers.6.self_attn.out_proj",
245
+ "text_model.encoder.layers.6.self_attn.q_proj",
246
+ "text_model.encoder.layers.6.self_attn.v_proj",
247
+ "text_model.encoder.layers.7.mlp.fc1",
248
+ "text_model.encoder.layers.7.mlp.fc2",
249
+ "text_model.encoder.layers.7.self_attn.k_proj",
250
+ "text_model.encoder.layers.7.self_attn.out_proj",
251
+ "text_model.encoder.layers.7.self_attn.q_proj",
252
+ "text_model.encoder.layers.7.self_attn.v_proj",
253
+ "text_model.encoder.layers.8.mlp.fc1",
254
+ "text_model.encoder.layers.8.mlp.fc2",
255
+ "text_model.encoder.layers.8.self_attn.k_proj",
256
+ "text_model.encoder.layers.8.self_attn.out_proj",
257
+ "text_model.encoder.layers.8.self_attn.q_proj",
258
+ "text_model.encoder.layers.8.self_attn.v_proj",
259
+ "text_model.encoder.layers.9.mlp.fc1",
260
+ "text_model.encoder.layers.9.mlp.fc2",
261
+ "text_model.encoder.layers.9.self_attn.k_proj",
262
+ "text_model.encoder.layers.9.self_attn.out_proj",
263
+ "text_model.encoder.layers.9.self_attn.q_proj",
264
+ "text_model.encoder.layers.9.self_attn.v_proj",
265
+ "text_projection",
266
+ "vision_model.encoder.layers.0.mlp.fc1",
267
+ "vision_model.encoder.layers.0.mlp.fc2",
268
+ "vision_model.encoder.layers.0.self_attn.k_proj",
269
+ "vision_model.encoder.layers.0.self_attn.out_proj",
270
+ "vision_model.encoder.layers.0.self_attn.q_proj",
271
+ "vision_model.encoder.layers.0.self_attn.v_proj",
272
+ "vision_model.encoder.layers.10.mlp.fc1",
273
+ "vision_model.encoder.layers.10.mlp.fc2",
274
+ "vision_model.encoder.layers.10.self_attn.k_proj",
275
+ "vision_model.encoder.layers.10.self_attn.out_proj",
276
+ "vision_model.encoder.layers.10.self_attn.q_proj",
277
+ "vision_model.encoder.layers.10.self_attn.v_proj",
278
+ "vision_model.encoder.layers.11.mlp.fc1",
279
+ "vision_model.encoder.layers.11.mlp.fc2",
280
+ "vision_model.encoder.layers.11.self_attn.k_proj",
281
+ "vision_model.encoder.layers.11.self_attn.out_proj",
282
+ "vision_model.encoder.layers.11.self_attn.q_proj",
283
+ "vision_model.encoder.layers.11.self_attn.v_proj",
284
+ "vision_model.encoder.layers.12.mlp.fc1",
285
+ "vision_model.encoder.layers.12.mlp.fc2",
286
+ "vision_model.encoder.layers.12.self_attn.k_proj",
287
+ "vision_model.encoder.layers.12.self_attn.out_proj",
288
+ "vision_model.encoder.layers.12.self_attn.q_proj",
289
+ "vision_model.encoder.layers.12.self_attn.v_proj",
290
+ "vision_model.encoder.layers.13.mlp.fc1",
291
+ "vision_model.encoder.layers.13.mlp.fc2",
292
+ "vision_model.encoder.layers.13.self_attn.k_proj",
293
+ "vision_model.encoder.layers.13.self_attn.out_proj",
294
+ "vision_model.encoder.layers.13.self_attn.q_proj",
295
+ "vision_model.encoder.layers.13.self_attn.v_proj",
296
+ "vision_model.encoder.layers.14.mlp.fc1",
297
+ "vision_model.encoder.layers.14.mlp.fc2",
298
+ "vision_model.encoder.layers.14.self_attn.k_proj",
299
+ "vision_model.encoder.layers.14.self_attn.out_proj",
300
+ "vision_model.encoder.layers.14.self_attn.q_proj",
301
+ "vision_model.encoder.layers.14.self_attn.v_proj",
302
+ "vision_model.encoder.layers.15.mlp.fc1",
303
+ "vision_model.encoder.layers.15.mlp.fc2",
304
+ "vision_model.encoder.layers.15.self_attn.k_proj",
305
+ "vision_model.encoder.layers.15.self_attn.out_proj",
306
+ "vision_model.encoder.layers.15.self_attn.q_proj",
307
+ "vision_model.encoder.layers.15.self_attn.v_proj",
308
+ "vision_model.encoder.layers.16.mlp.fc1",
309
+ "vision_model.encoder.layers.16.mlp.fc2",
310
+ "vision_model.encoder.layers.16.self_attn.k_proj",
311
+ "vision_model.encoder.layers.16.self_attn.out_proj",
312
+ "vision_model.encoder.layers.16.self_attn.q_proj",
313
+ "vision_model.encoder.layers.16.self_attn.v_proj",
314
+ "vision_model.encoder.layers.17.mlp.fc1",
315
+ "vision_model.encoder.layers.17.mlp.fc2",
316
+ "vision_model.encoder.layers.17.self_attn.k_proj",
317
+ "vision_model.encoder.layers.17.self_attn.out_proj",
318
+ "vision_model.encoder.layers.17.self_attn.q_proj",
319
+ "vision_model.encoder.layers.17.self_attn.v_proj",
320
+ "vision_model.encoder.layers.18.mlp.fc1",
321
+ "vision_model.encoder.layers.18.mlp.fc2",
322
+ "vision_model.encoder.layers.18.self_attn.k_proj",
323
+ "vision_model.encoder.layers.18.self_attn.out_proj",
324
+ "vision_model.encoder.layers.18.self_attn.q_proj",
325
+ "vision_model.encoder.layers.18.self_attn.v_proj",
326
+ "vision_model.encoder.layers.19.mlp.fc1",
327
+ "vision_model.encoder.layers.19.mlp.fc2",
328
+ "vision_model.encoder.layers.19.self_attn.k_proj",
329
+ "vision_model.encoder.layers.19.self_attn.out_proj",
330
+ "vision_model.encoder.layers.19.self_attn.q_proj",
331
+ "vision_model.encoder.layers.19.self_attn.v_proj",
332
+ "vision_model.encoder.layers.1.mlp.fc1",
333
+ "vision_model.encoder.layers.1.mlp.fc2",
334
+ "vision_model.encoder.layers.1.self_attn.k_proj",
335
+ "vision_model.encoder.layers.1.self_attn.out_proj",
336
+ "vision_model.encoder.layers.1.self_attn.q_proj",
337
+ "vision_model.encoder.layers.1.self_attn.v_proj",
338
+ "vision_model.encoder.layers.20.mlp.fc1",
339
+ "vision_model.encoder.layers.20.mlp.fc2",
340
+ "vision_model.encoder.layers.20.self_attn.k_proj",
341
+ "vision_model.encoder.layers.20.self_attn.out_proj",
342
+ "vision_model.encoder.layers.20.self_attn.q_proj",
343
+ "vision_model.encoder.layers.20.self_attn.v_proj",
344
+ "vision_model.encoder.layers.21.mlp.fc1",
345
+ "vision_model.encoder.layers.21.mlp.fc2",
346
+ "vision_model.encoder.layers.21.self_attn.k_proj",
347
+ "vision_model.encoder.layers.21.self_attn.out_proj",
348
+ "vision_model.encoder.layers.21.self_attn.q_proj",
349
+ "vision_model.encoder.layers.21.self_attn.v_proj",
350
+ "vision_model.encoder.layers.22.mlp.fc1",
351
+ "vision_model.encoder.layers.22.mlp.fc2",
352
+ "vision_model.encoder.layers.22.self_attn.k_proj",
353
+ "vision_model.encoder.layers.22.self_attn.out_proj",
354
+ "vision_model.encoder.layers.22.self_attn.q_proj",
355
+ "vision_model.encoder.layers.22.self_attn.v_proj",
356
+ "vision_model.encoder.layers.23.mlp.fc1",
357
+ "vision_model.encoder.layers.23.mlp.fc2",
358
+ "vision_model.encoder.layers.23.self_attn.k_proj",
359
+ "vision_model.encoder.layers.23.self_attn.out_proj",
360
+ "vision_model.encoder.layers.23.self_attn.q_proj",
361
+ "vision_model.encoder.layers.23.self_attn.v_proj",
362
+ "vision_model.encoder.layers.2.mlp.fc1",
363
+ "vision_model.encoder.layers.2.mlp.fc2",
364
+ "vision_model.encoder.layers.2.self_attn.k_proj",
365
+ "vision_model.encoder.layers.2.self_attn.out_proj",
366
+ "vision_model.encoder.layers.2.self_attn.q_proj",
367
+ "vision_model.encoder.layers.2.self_attn.v_proj",
368
+ "vision_model.encoder.layers.3.mlp.fc1",
369
+ "vision_model.encoder.layers.3.mlp.fc2",
370
+ "vision_model.encoder.layers.3.self_attn.k_proj",
371
+ "vision_model.encoder.layers.3.self_attn.out_proj",
372
+ "vision_model.encoder.layers.3.self_attn.q_proj",
373
+ "vision_model.encoder.layers.3.self_attn.v_proj",
374
+ "vision_model.encoder.layers.4.mlp.fc1",
375
+ "vision_model.encoder.layers.4.mlp.fc2",
376
+ "vision_model.encoder.layers.4.self_attn.k_proj",
377
+ "vision_model.encoder.layers.4.self_attn.out_proj",
378
+ "vision_model.encoder.layers.4.self_attn.q_proj",
379
+ "vision_model.encoder.layers.4.self_attn.v_proj",
380
+ "vision_model.encoder.layers.5.mlp.fc1",
381
+ "vision_model.encoder.layers.5.mlp.fc2",
382
+ "vision_model.encoder.layers.5.self_attn.k_proj",
383
+ "vision_model.encoder.layers.5.self_attn.out_proj",
384
+ "vision_model.encoder.layers.5.self_attn.q_proj",
385
+ "vision_model.encoder.layers.5.self_attn.v_proj",
386
+ "vision_model.encoder.layers.6.mlp.fc1",
387
+ "vision_model.encoder.layers.6.mlp.fc2",
388
+ "vision_model.encoder.layers.6.self_attn.k_proj",
389
+ "vision_model.encoder.layers.6.self_attn.out_proj",
390
+ "vision_model.encoder.layers.6.self_attn.q_proj",
391
+ "vision_model.encoder.layers.6.self_attn.v_proj",
392
+ "vision_model.encoder.layers.7.mlp.fc1",
393
+ "vision_model.encoder.layers.7.mlp.fc2",
394
+ "vision_model.encoder.layers.7.self_attn.k_proj",
395
+ "vision_model.encoder.layers.7.self_attn.out_proj",
396
+ "vision_model.encoder.layers.7.self_attn.q_proj",
397
+ "vision_model.encoder.layers.7.self_attn.v_proj",
398
+ "vision_model.encoder.layers.8.mlp.fc1",
399
+ "vision_model.encoder.layers.8.mlp.fc2",
400
+ "vision_model.encoder.layers.8.self_attn.k_proj",
401
+ "vision_model.encoder.layers.8.self_attn.out_proj",
402
+ "vision_model.encoder.layers.8.self_attn.q_proj",
403
+ "vision_model.encoder.layers.8.self_attn.v_proj",
404
+ "vision_model.encoder.layers.9.mlp.fc1",
405
+ "vision_model.encoder.layers.9.mlp.fc2",
406
+ "vision_model.encoder.layers.9.self_attn.k_proj",
407
+ "vision_model.encoder.layers.9.self_attn.out_proj",
408
+ "vision_model.encoder.layers.9.self_attn.q_proj",
409
+ "vision_model.encoder.layers.9.self_attn.v_proj",
410
+ "visual_projection"
411
+ ]
412
+ }
413
+ }
conversion_receipt.json ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "verdict": "PASS",
3
+ "validation": {
4
+ "checks": {
5
+ "readable": true,
6
+ "compressed_complete": true,
7
+ "indices_in_range": true,
8
+ "exact_no_nan": true,
9
+ "key_count_match": true
10
+ },
11
+ "details": [],
12
+ "total_keys": 1246,
13
+ "compressed_modules": 218,
14
+ "exact_tensors": 374,
15
+ "sha256": "0a2b6cd14d1637388a4e3352fbea3aa856b3ca3cdf946d6d61dd03d9c5ffe6cd"
16
+ }
17
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a2b6cd14d1637388a4e3352fbea3aa856b3ca3cdf946d6d61dd03d9c5ffe6cd
3
+ size 468311596
preprocessor_config.json ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "crop_size": 224,
3
+ "do_center_crop": true,
4
+ "do_normalize": true,
5
+ "do_resize": true,
6
+ "feature_extractor_type": "CLIPFeatureExtractor",
7
+ "image_mean": [
8
+ 0.48145466,
9
+ 0.4578275,
10
+ 0.40821073
11
+ ],
12
+ "image_std": [
13
+ 0.26862954,
14
+ 0.26130258,
15
+ 0.27577711
16
+ ],
17
+ "resample": 3,
18
+ "size": 224
19
+ }
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"bos_token": {"content": "<|startoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "eos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": "<|endoftext|>"}
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "unk_token": {
3
+ "content": "<|endoftext|>",
4
+ "single_word": false,
5
+ "lstrip": false,
6
+ "rstrip": false,
7
+ "normalized": true,
8
+ "__type": "AddedToken"
9
+ },
10
+ "bos_token": {
11
+ "content": "<|startoftext|>",
12
+ "single_word": false,
13
+ "lstrip": false,
14
+ "rstrip": false,
15
+ "normalized": true,
16
+ "__type": "AddedToken"
17
+ },
18
+ "eos_token": {
19
+ "content": "<|endoftext|>",
20
+ "single_word": false,
21
+ "lstrip": false,
22
+ "rstrip": false,
23
+ "normalized": true,
24
+ "__type": "AddedToken"
25
+ },
26
+ "pad_token": "<|endoftext|>",
27
+ "add_prefix_space": false,
28
+ "errors": "replace",
29
+ "do_lower_case": true,
30
+ "name_or_path": "openai/clip-vit-base-patch32",
31
+ "model_max_length": 77,
32
+ "special_tokens_map_file": "./special_tokens_map.json",
33
+ "tokenizer_class": "CLIPTokenizer"
34
+ }
vocab.json ADDED
The diff for this file is too large to render. See raw diff