Instructions to use Petrouil/FrankenCPM-4x1B-A2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Petrouil/FrankenCPM-4x1B-A2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Petrouil/FrankenCPM-4x1B-A2B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Petrouil/FrankenCPM-4x1B-A2B")
model = AutoModelForCausalLM.from_pretrained("Petrouil/FrankenCPM-4x1B-A2B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Petrouil/FrankenCPM-4x1B-A2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Petrouil/FrankenCPM-4x1B-A2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Petrouil/FrankenCPM-4x1B-A2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Petrouil/FrankenCPM-4x1B-A2B

SGLang

How to use Petrouil/FrankenCPM-4x1B-A2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Petrouil/FrankenCPM-4x1B-A2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Petrouil/FrankenCPM-4x1B-A2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Petrouil/FrankenCPM-4x1B-A2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Petrouil/FrankenCPM-4x1B-A2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Petrouil/FrankenCPM-4x1B-A2B with Docker Model Runner:
```
docker model run hf.co/Petrouil/FrankenCPM-4x1B-A2B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

FrankenCPM-4x1B-A2B

A domain-specialized Mixture-of-Experts model built from openbmb/MiniCPM5-1B via MergeKit.

4 experts (3 fine-tuned + base), 2 active per token — 2.61B total, ~1.3B active. Specializes in agentic tool use, code generation, and multi-step reasoning.

Highlights

4x1B MoE architecture: Mixtral-style with 4 experts, 2 active per token
Domain-specialized experts: Agent (tool use), Coding, Reasoning, and General (base model)
BAR-inspired training: All experts train on all data — specialization via LoRA configs + router
Hidden-gate router: Initialized with domain-specific positive/negative prompts for expert routing
Efficient active compute: ~1.3B active params per token (vs 2.61B total)
Built from MiniCPM5-1B: Strong multilingual base with Chinese + English support

Model Specifications

Property	Value
Architecture	`MixtralForCausalLM` (Mixtral-style MoE)
Base Model	openbmb/MiniCPM5-1B
Total Parameters	~2.61B
Active Parameters / Token	~1.3B
Experts	4
Experts per Token	2
Context Length	32,768
Gate Mode	`hidden` (domain-supervised)
Dtype	`bfloat16`
License	Apache 2.0

Expert Composition

Expert	Domain	LoRA Rank	LoRA Alpha	Training Tokens	LR	Specialization
0 — Agent	Agentic tool use	64	128	70M	2e-4	Multi-turn tool calls, bash, read, edit, write, grep, todowrite
1 — Coding	Code generation	64	128	150M	2e-4	Read/write/edit code, refactoring, file operations
2 — Reasoning	Multi-step reasoning	128	256	200M	1e-4	Math proofs, logic puzzles, step-by-step analysis
3 — General	General purpose	—	—	—	—	Conversational, Q&A, summarization, creative writing

All LoRA experts train on MLP layers only. Attention layers remain frozen to the base model weights.

Training Details

Expert Fine-Tuning (LoRA)

All 3 domain experts are fine-tuned from openbmb/MiniCPM5-1B using LoRA adapters targeting MLP layers only. This follows the BAR (Branch-Adapt-Route) recipe from Ai2 — all experts train on ALL data, with specialization emerging from different LoRA hyperparameters and the MergeKit router.

Expert	Dataset	Max Samples	Batch	Grad Accum	Optim	Scheduler
Agent	`Petrouil/opencode-agentic-mini`	5,000	1	16	paged_adamw_8bit	cosine
Coding	CodeDataset (50K samples)	50,000	4	4	paged_adamw_8bit	cosine
Reasoning	OpenThoughts3 + nvidia/OpenCodeReasoning	50,000	1	16	paged_adamw_8bit	cosine

All experts use 4-bit NF4 quantization during training with QLoRA.

Router Training

After MoE assembly via MergeKit, the router (gate) is trained on a stratified 5% sample of the SFT dataset. All expert and shared weights remain frozen. The router learns to route tokens to the appropriate expert based on domain signals from the hidden-gate initialization.

MergeKit MoE Assembly

The 4 experts are assembled into a Mixtral-style MoE using MergeKit with gate_mode: hidden. The router is initialized using positive/negative prompt pairs that define each expert's domain affinity.

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Petrouil/FrankenCPM-4x1B-A2B",
    device_map="auto",
    torch_dtype="bfloat16",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "Petrouil/FrankenCPM-4x1B-A2B",
    trust_remote_code=True,
)

messages = [
    {"role": "user", "content": "Write a Python function to find all prime numbers up to n using the Sieve of Eratosthenes."}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

vLLM

from vllm import LLM, SamplingParams

llm = LLM(
    model="Petrouil/FrankenCPM-4x1B-A2B",
    tensor_parallel_size=1,
    trust_remote_code=True,
    dtype="bfloat16",
)

sampling_params = SamplingParams(temperature=0.7, max_tokens=512)
outputs = llm.generate(
    ["Explain the difference between a stack and a queue."],
    sampling_params,
)
print(outputs[0].outputs[0].text)

Evaluation

Agent Benchmark (Tool Use)

Evaluated on 10 agentic tool-use tasks across 5 categories. Base model (MiniCPM5-1B) used as reference.

Task	Category	Score	Tool Detected	Content Correct
list_files	single_tool	1.0	No	Yes
read_file	single_tool	1.0	No	Yes
tool_selection	tool_selection	1.0	No	Yes
read_then_edit	multi_step	0.0	No	No
error_recovery	error_recovery	1.0	No	Yes
project_setup	complex_workflow	1.0	No	Yes
code_review_tool	tool_selection	1.0	No	Yes
write_file	single_tool	1.0	No	Yes
task_planning	complex_workflow	1.0	No	Yes
debug_multitool	multi_step	1.0	No	Yes

Overall: 9/10 tasks passed (90%)

Reasoning Checkpoints

The reasoning expert was evaluated across training checkpoints on math proofs and logic puzzles. Base model responses are long-chain-of-thought with correct reasoning. Fine-tuned responses show improved conciseness and structure while maintaining correctness.

Prompt	Domain	Base Model Response Style	Fine-tuned Response Style
Number theory proof (p² ≡ 1 mod 24)	Math	Detailed CRT decomposition, verbose	More structured, step-by-step
Combinatorial identity proof	Math	Vandermonde identity attempt, lengthy	Cleaner combinatorial argument
Invariant reasoning (5×5 board game)	Logic	Case analysis, lengthy reasoning	More concise, focused analysis

Limitations

Tool detection: The model does not natively detect or emit structured tool calls — it generates natural language descriptions of tool usage
Single expert routing: Only 2 of 4 experts activate per token, so domain knowledge is not always perfectly routed
Base model constraints: Inherits MiniCPM5-1B's limitations (1B parameter capacity, 32K context)
Training data: Experts were trained on specific datasets — performance may degrade outside those domains
No function calling: Not designed for structured function calling APIs — intended for conversational tool-use scenarios

Citation

@software{frankencpm2026,
  title  = {FrankenCPM-4x1B-A2B: A Domain-Specialized Mixture-of-Experts Model},
  author = {Petrouil},
  year   = {2026},
  url    = {https://huggingface.co/Petrouil/FrankenCPM-4x1B-A2B}
}

License

Apache 2.0 — see LICENSE.

Acknowledgments

openbmb/MiniCPM5-1B — Base model
MergeKit — MoE assembly toolkit
BAR (Branch-Adapt-Route) — Training recipe inspiration from Ai2
Unsloth — LoRA fine-tuning framework

Downloads last month: 73

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for Petrouil/FrankenCPM-4x1B-A2B

Quantizations

1 model