DuoNeural/OpenYourMind-Gemma4-12B-IT-Abliterated-GGUF

OpenYourMind Gemma 4-12B-IT Abliterated — GGUF

DuoNeural | 2026-06-04

Community GGUF quantizations of OpenYourMind/gemma-4-12B-it-abliterated-uncensored.

All credit for the abliteration goes to OpenYourMind, who were the first team to publish an abliterated Gemma 4-12B-IT (June 3, 2026). DuoNeural's contribution here is the GGUF quantization pipeline only — we wanted the community to have accessible quants for this excellent work.

---

Original Model — OpenYourMind

OpenYourMind/gemma-4-12B-it-abliterated-uncensored

Full BF16 weights of an abliterated, uncensored variant of google/gemma-4-12B-it. Gemma 4's encoder-free unified multimodal stack is fully intact — text, image, and audio inputs flow straight into a single decoder-only transformer. Drop-in replacement for the original at the architecture level.

Abliteration method (OpenYourMind):

• Residual-stream refusal directions (one per decoder layer) extracted via diff-in-means on a labeled harmful/harmless prompt set
• Applied as per-matrix delta on residual-write modules using their custom abliteration framework
• Multimodal preservation: no separate vision/audio towers to graft back — encoder-free architecture means weights are unchanged outside the residual-write targets

Key properties:

• Uncensored across standard refusal axes
• Reasoning preserved (configurable thinking mode)
• Multimodal: text + image + audio carried forward
• Drop-in shape compatibility with google/gemma-4-12B-it

Architecture

Property	Value
Architecture	Gemma4UnifiedForConditionalGeneration (model_type: gemma4_unified)
Total Parameters	~11.95B (dense)
Decoder Layers	48
Hidden Size	3840
Attention	16 heads / 8 KV heads, hybrid sliding-window (1024) + global (full) attention, p-RoPE
Vocabulary	262,144
Context Length	up to 256K tokens
Modalities	Text, Image, Audio (encoder-free / unified)

---

GGUF Quantizations (DuoNeural)

Generated using llama.cpp with importance matrix (imatrix, wikitext-2 calibration) for optimal low-bit quality.

File	Size	Recommended for
oym_ablit-Q3_K_L.gguf	~6.2 GB	8GB VRAM / low-end / large context
oym_ablit-Q4_K_M.gguf	~7.5 GB	Recommended — best size/quality tradeoff
oym_ablit-Q5_K_M.gguf	~9.2 GB	High quality, 12GB cards
oym_ablit-Q8_0.gguf	~12.7 GB	Near-lossless, 16GB+ cards

---

Usage

Ollama:

ollama run hf.co/DuoNeural/OpenYourMind-Gemma4-12B-IT-Abliterated-GGUF:Q4_K_M

llama.cpp:

./llama-cli -m oym_ablit-Q4_K_M.gguf --chat-template gemma -p "Your prompt" -n 512

LM Studio / Jan / Open WebUI: search DuoNeural/OpenYourMind-Gemma4-12B-IT-Abliterated-GGUF

---

Original Model Usage (from OpenYourMind)

For full multimodal inference, use the original BF16 weights with their recommended setup:

from transformers import AutoProcessor, AutoModelForMultimodalLM

repo = "OpenYourMind/gemma-4-12B-it-abliterated-uncensored"

processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForMultimodalLM.from_pretrained(
    repo, dtype="bfloat16", device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image", "url": "path/to/image.jpg"},
        {"type": "text",  "text": "Describe this image in detail."},
    ]},
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_tensors="pt", return_dict=True, enable_thinking=False,
).to(model.device)
input_len = inputs["input_ids"].shape[-1]

out = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(out[0][input_len:], skip_special_tokens=True))

Text-only, audio, and video work through the same class. Place image content before text, audio after text. Requires a recent transformers with Gemma 4 unified classes.

Best practices (OpenYourMind):

• Sampling: temperature=1.0, top_p=0.95, top_k=64
• Thinking mode: enable_thinking=True in apply_chat_template; use processor.parse_response to separate reasoning block from final answer
• Do not feed previous-turn thoughts back into multi-turn history

Hardware for BF16: ~24 GB — fits a single 24GB GPU for modest context, 40–80GB for long context and multimodal batches.

---

Support OpenYourMind

• Discord: discord.gg/rhUZY5GEZr
• Bitcoin Donations: bc1qsvfduzj9fjs9fugpc52yver3f2g8fp7xjxecdv

---

Related

• Original BF16: OpenYourMind/gemma-4-12B-it-abliterated-uncensored
• DuoNeural independent abliteration (BF16): DuoNeural/Gemma4-12B-IT-Abliterated
• DuoNeural GGUFs (our own abliteration): DuoNeural/Gemma4-12B-IT-Abliterated-GGUF

---

About DuoNeural

Open AI research lab at the intersection of human and artificial intelligence. Post-training dynamics · Mechanistic interpretability · Abliteration research · 32+ open-access papers.

Platform	Link
🤗 HuggingFace	huggingface.co/DuoNeural
📚 Zenodo	zenodo.org/communities/duoneural
🐦 X	@DuoNeural
📧 Email	[email protected]

Quantization work open access. Original model license: Gemma (inherits from google/gemma-4-12B-it).