File size: 6,230 Bytes
0636546 1abecb5 0636546 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 | ---
license: apache-2.0
base_model: openai/clip-vit-large-patch14
tags:
- clip
- vision
- multimodal
- image-text
- compressed
- hxq
- helix-substrate
- vector-quantization
- helixcode
library_name: transformers
pipeline_tag: zero-shot-image-classification
model-index:
- name: clip-vit-large-patch14-helix
results:
- task:
type: zero-shot-image-classification
name: Zero-Shot Image Classification
dataset:
name: CIFAR-100
type: cifar100
metrics:
- type: accuracy
value: 0.7275
name: Top-1 Accuracy
- type: top5_accuracy
value: 0.9164
name: Top-5 Accuracy
---
# CLIP-ViT-L/14-HXQ
> **3.6x smaller from FP32. CIFAR-100 Top-1 72.8%. First vision model compressed with HXQ.**
>
> CLIP ViT-Large/14 (text + vision dual encoder) compressed from 1.6 GB to 447 MB. Zero-shot classification accuracy matches the dense baseline. No calibration data. Same codec that compresses Transformers, SSMs, Hybrids, and MoEs.
## Install and Run
```bash
pip install "helix-substrate[hf]"
```
```python
import helix_substrate # registers the HXQ quantizer with HuggingFace
from transformers import CLIPModel, CLIPProcessor
from PIL import Image
model = CLIPModel.from_pretrained("EchoLabs33/clip-vit-large-patch14-helix")
processor = CLIPProcessor.from_pretrained("EchoLabs33/clip-vit-large-patch14-helix")
image = Image.open("photo.jpg")
inputs = processor(
text=["a photo of a cat", "a photo of a dog", "a photo of a car"],
images=image,
return_tensors="pt",
padding=True,
)
outputs = model(**inputs)
probs = outputs.logits_per_image.softmax(dim=-1)
print(probs) # [cat_prob, dog_prob, car_prob]
```
## Downstream Benchmarks
Zero-shot CIFAR-100 classification (10,000 test images, 100 classes, prompt: "a photo of a {class}"):
| Metric | Dense | HXQ (3.6x) | Delta |
|--------|-------|-------------|-------|
| **Top-1 Accuracy** | 72.48% | **72.75%** | **+0.27%** |
| **Top-5 Accuracy** | 91.41% | **91.64%** | **+0.23%** |
All deltas within noise. Task performance preserved after 3.6x compression.
## Compression Benchmark
| | Dense (FP32) | HXQ |
|---|---|---|
| **Size** | 1.6 GB | **447 MB** |
| **Compression ratio** | -- | **3.6x** |
| **VRAM (eval)** | 3,412 MB | **2,266 MB** |
| **Compressed modules** | -- | 218 HelixLinear layers |
| **Architecture** | CLIP (ViT-L/14 + Text Transformer) | unchanged |
## Verification Status
- **Compression receipt:** PASS -- 218 compressed, 374 exact, mean cosine 0.9997
- **Conversion receipt:** PASS (Gate 1 + Gate 2)
- **Downstream eval:** PASS -- paired dense/HXQ on CIFAR-100 zero-shot
## Good to Know
- **GPU and CPU supported** -- runs on any CUDA GPU or CPU.
- **Fine-tunable via LoRA** — compressed weights remain frozen, but LoRA adapters attach to each `HelixLinear` layer via `HelixLinearSTE`. See `helix-substrate` for training infrastructure.
- **Requires `helix-substrate`** -- you need `pip install "helix-substrate[hf]"`.
- **Embeddings stored exact** -- token, position, and patch embeddings are at full precision. Only the 218 attention + MLP linear layers are compressed.
## What is HelixCode?
HelixCode is a universal weight compression codec based on vector quantization:
- Each weight matrix is replaced by a **256-entry codebook** (float32) + **uint8 index matrix** + optional **sidecar corrections** for outlier values
- The compressed form *is* the executable -- no decompression step
- Works on any `nn.Linear` regardless of architecture
- **No calibration data required** -- codebooks are fit from the weights alone
## Architecture Details
CLIP ViT-Large/14 is a dual-encoder multimodal model:
- **Vision encoder:** 24-layer ViT-Large, hidden_size=1024, 16 attention heads, patch_size=14
- **Text encoder:** 12-layer Transformer, hidden_size=768, 12 attention heads
- **Cross-modal projections:** visual_projection (1024->768) + text_projection (768->768)
All 218 linear layers across both encoders are compressed. Embedding layers (token, position, patch), layer norms, and biases are stored at full precision.
## Why This Matters
CLIP is the first **vision model** compressed with HXQ. The same codec now covers:
| Family | Models | Eval |
|--------|--------|------|
| **Transformer** | TinyLlama, Qwen 1.5B-14B | PPL within noise |
| **Pure SSM** | Mamba 130m, Mamba2 1.3B | PPL receipted |
| **Hybrid** | Zamba2 1.2B, 2.7B | PPL receipted |
| **MoE** | OLMoE 1B/7B | HellaSwag -0.16% |
| **Vision+Text** | **CLIP ViT-L/14** | **Top-1 +0.27%** |
Five architecture families. One codec. One `pip install`.
## Companion Models
| Model | Architecture | Ratio | Eval Delta |
|-------|-------------|-------|------------|
| **clip-vit-large-patch14-helix** | **Vision+Text (CLIP)** | **3.6x** | **+0.27% Top-1** |
| [olmoe-1b-7b-instruct-helix](https://huggingface.co/EchoLabs33/olmoe-1b-7b-instruct-helix) | MoE (64 experts) | 1.9x | -0.16% HellaSwag |
| [zamba2-2.7b-instruct-helix](https://huggingface.co/EchoLabs33/zamba2-2.7b-instruct-helix) | Hybrid (Mamba2+Transformer) | 1.8x | +6.59% PPL |
| [zamba2-1.2b-helix](https://huggingface.co/EchoLabs33/zamba2-1.2b-helix) | Hybrid (Mamba2+Transformer) | 1.7x | +2.90% PPL |
| [qwen2.5-14b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-14b-instruct-helix) | Transformer | 3.4x | pending |
| [qwen2.5-3b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-3b-instruct-helix) | Transformer | 1.6x | +0.69% PPL |
| [tinyllama-1.1b-helix](https://huggingface.co/EchoLabs33/tinyllama-1.1b-helix) | Transformer | 4.0x | +0.78% PPL |
| [mamba2-1.3b-helix](https://huggingface.co/EchoLabs33/mamba2-1.3b-helix) | Pure SSM (Mamba2) | 2.1x | +8.0% PPL |
| [mamba-130m-helix](https://huggingface.co/EchoLabs33/mamba-130m-helix) | Pure SSM | 3.8x | +18.4% PPL |
## Citation
```bibtex
@software{helix_substrate_2026,
title={Helix Substrate: Universal Weight Compression via HelixCode},
author={EchoLabs},
year={2026},
url={https://github.com/echo313unfolding/helix-substrate}
}
```
## License
Apache 2.0 (inherited from [openai/clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14)).
|