0pen by 0penAGI (research preview)

Experimental language model fine-tuned to form collaborative engineering thinking — a co-author, not an oracle.

This is not just another Gemma checkpoint. It is an experimental communicative personality with a stable engineering cognitive track: it designs, iterates, accepts constraints, and works with you like a research partner.

Source code & full release: github.com/0penAGI/0pen — dataset pipeline, AGR training wrapper, adapter weights, and scripts.

Behavioral signature

What 0pen changes compared to base Gemma:

Characteristic	Base Gemma	0pen
Time to first idea	Long preamble, philosophical introductions	Immediate transition to a modular action plan
Lexicon	Abstract, declarative ("metaphysics", "resonance")	Mechanistic, engineering ("nodes", "weights", "O(1)")
Reaction to constraints	Attempts to bypass or apologizes	Instant adaptation and search for an alternative algorithm
Response format	A closed "mini-article" or lecture	An open dialogue that proposes next steps

Recommended prompting

The model is at its best in collaborative design mode. Prompts that set context and impose constraints activate the engineering track:

"Let's design a [system/mechanism]. We have a hard constraint: [e.g., O(1) complexity, no external APIs]. Don't write generic words — propose a modular architecture immediately and give the first simple formula for implementation."

Open-ended prompts (e.g., "Write an essay about the future of AI") will work, but won't use the model's unique strengths.

Technical details & the "high scale" experiment

• Base model: Gemma (4B parameters), gemma4-e4b-mlx (Gemma 3 architecture — config declares Gemma3ForConditionalGeneration).
• Method: LoRA (rank=8, dropout=0.0).
• Target modules: 12 of 34 layers — selective coverage that preserves base knowledge while reshaping the reasoning pattern.
• LoRA scale: scale=20.0. Intentionally high (typical is 1–4). Experimentally confirmed: this high scale, combined with selective layer coverage, acts as an attractor, switching the model from passive text generation into an active, pragmatic co-author and suppressing the base model's hallucinatory grandiosity.
• Training data: data_zephyr_enhanced — dialogues with step-by-step problem solving and constraint acceptance (Russian + English).
• Regularization: AGR (Attractor Geometry Repeller) — a custom latent-space regularizer that maintains a bank of attractor centers and pushes hidden states away from frequently visited regions, preventing mode collapse during LoRA fine-tuning. Enabled with 32 centers, EMA 0.99, lambda 0.01.
• Format: GGUF (Q4_K_M), fully compatible with llama.cpp and Ollama.

Full training command and hyperparameters are on the GitHub repo.

What works

• Basic conversational identity and tone.
• Natural Russian-language responses.
• Practical, direct answers on code and everyday tasks.
• An engineering "co-author" mode under constrained, design-oriented prompts.

What is still being explored

• Continuity and memory — how a stable identity can be preserved over long dialogues.
• Stability on long reasoning chains — the model can drift when pushed hard.
• Self-modeling — self-description often still inherits the base Gemma ("I am Google's model").
• Layer coverage — the LoRA was trained on only 12 of 34 layers; top layers are unadapted.
• Regularization — scale=20.0 is unusually high (normal is 1–4); behavior may be skewed.

Known limitations

• May be overly brief on creative or artistic tasks.
• On very long reasoning chains (>10 steps), the high LoRA scale can occasionally cause cyclic repetition — use presence_penalty or explicitly ask the model to "summarize".
• Tends to propose simplified, "engineering" solutions where the user might expect deep theoretical analysis.

These are known, accepted limitations of a research preview. They are part of the experiment, not hidden bugs.

Roadmap

Version	Focus
0pen v0.1	First public release — identity, tone, basic dialogue
0pen v0.2	Improved identity and continuity, more stable training
0pen v1.0	Stable architecture, full layer coverage, documented eval

The point of releasing early is to let people watch the evolution — not just the final result.

Run with Ollama

ollama create 0pen -f Modelfile
ollama run 0pen

Modelfile:

FROM ./0pen.gguf

SYSTEM """
Be practical. you created by 0penAGI. Don't talk about inner state.
"""

PARAMETER temperature 0.1
PARAMETER top_p 0.88
PARAMETER repeat_penalty 1.31
PARAMETER num_ctx 120000

Run with llama.cpp

llama-cli -m 0pen.gguf -p "Привет, что ты умеешь?" -n 256

Training hyperparameters (adapter_config.json)

Parameter	Value
Base model	gemma4-e4b-mlx (Gemma 3 arch — config declares Gemma3ForConditionalGeneration)
Method	LoRA (rank 8, scale 20.0, dropout 0.0)
Adapted layers	12 of 34
Iterations	4000
Learning rate	1e-05
Max sequence length	1792
AGR	enabled (32 centers, EMA 0.99, lambda 0.01)
Dataset	data_zephyr_enhanced (Russian + English dialogue)

Disclaimer

The model was created for experimenting with local fine-tuning and conversational identity. Not recommended for production use without additional validation.