Sentient-Memory

Based On Qwen3.6 35B-A3B Native-MTP

Memory Built Into The Model!

This repository contains Qwen3.6-35B-A3B GGUF variants with baked, direct-answer tool instructions for memory_store and memory_recall. The release contains no immutable factual memories. Runtime persistence is intentionally owned by the host application that executes those tools.

Every model is a single-file GGUF loadable by stock, unmodified llama.cpp. No adapter, prompt file, model-specific package, proxy, database, or external embedder is required for normal chat.

Model architecture

• 35B total parameters, approximately 3B active per token.
• 40 target transformer blocks: 30 Gated-DeltaNet linear-attention layers and 10 full GQA layers.
• 256 routed experts per layer, 8 selected per token, plus one shared expert.
• Hidden size 2,048; 16 query heads; 2 KV heads; full-attention head dimension
• Vocabulary size 248,320; native context metadata 262,144 tokens.
• One standard bundled Native-MTP/NextN layer (blk.40.*).
• Text-generation GGUF only; no vision projector is included.

Files

File	Quantization	Memory facts	Status
Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q4_K_M.gguf	Q4_K_M trunk, F16 output	0	production baseline
Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q8_0.gguf	Q8_0 trunk, F16 output	0	production variant
Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q6_K.gguf	Q6_K trunk, F16 output	0	production variant
Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q2_K.gguf	Q2_K mixed trunk, F16 output	0	low-memory variant; quality is expected to degrade
Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q1_0.gguf	Q1_0 trunk, F16 output	0	extreme low-memory experiment; quality is expected to degrade

The output matrix remains F16 in every variant so the standard output path is not further quantized. Q8, Q6, Q2, and Q1 variants are quantized independently from the BF16 memory-tool-bearing intermediate; they are not requantized from Q4.

Exact byte counts and SHA-256 values are recorded in each release manifest and SHA256SUMS after conversion. Verify a downloaded file with:

shasum -a 256 -c SHA256SUMS

Quick start

llama-server \
  -m Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q4_K_M.gguf \
  --host 127.0.0.1 \
  --port 8085

OpenAI-compatible chat:

curl http://127.0.0.1:8085/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "default",
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 0
  }'

The model defaults to direct answers. Deliberate reasoning can be enabled per request with:

{"chat_template_kwargs":{"sentient_enable_thinking":true}}

Memory tools: schemas are baked, execution is not

The GGUF chat template always teaches the model these functions:

memory_store(content: string)
memory_recall(query: string)

No caller system prompt or tool schema is needed for the model to request a function. With no request-side schemas, llama.cpp returns a machine-readable XML-like request in message.content. With a standard request-side tools array, llama.cpp can return structured OpenAI message.tool_calls.

Neither mode executes the function. A host must receive the call, persist or retrieve data, and return a role: "tool" result before asking for the final answer. Without that host loop, this model has no runtime memory state.

This boundary is deliberate:

user -> baked template -> model tool request -> host executor -> tool result -> answer

The GGUF is immutable while served. Adding, updating, or deleting persistent memory requires a host database or a future replacement GGUF; it cannot happen inside stock llama.cpp inference.

OpenAI-compatible tool call

For structured tool calls, provide the normal request-side schemas:

{
  "model": "default",
  "messages": [{"role": "user", "content": "Remember that my dog is named Biscuit."}],
  "tools": [
    {"type":"function","function":{"name":"memory_store","description":"Store an important fact for later recall.","parameters":{"type":"object","properties":{"content":{"type":"string"}},"required":["content"]}}},
    {"type":"function","function":{"name":"memory_recall","description":"Recall stored facts relevant to a query.","parameters":{"type":"object","properties":{"query":{"type":"string"}},"required":["query"]}}}
  ]
}

The schemas are baked for model understanding. Repeating them is only needed for llama.cpp's structured response parser.

Embeddings

llama-server \
  -m Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q4_K_M.gguf \
  --embeddings --pooling last --ubatch-size 512

curl http://127.0.0.1:8085/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model":"default","input":"text to embed"}'

Embeddings return 2,048-dimensional vectors. They do not create a memory database or execute retrieval. Keep each embedding input within one micro-batch on the tested hybrid architecture.

Native MTP

The bundled NextN layer can be enabled with:

llama-server \
  -m Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q4_K_M.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2

Native MTP is an acceleration path, not memory, verification, or safety enforcement. Do not combine it with embeddings mode on the tested llama.cpp revision.

Provenance and contract

• Source: unsloth/Qwen3.6-35B-A3B
• Source revision: 2ab40a9acc6d567889ca4d4e59feb2da56121454
• llama.cpp revision: e9d1b76d0ad84569c889be265dd270d44b54fbc7
• Tested build: b9798-e9d1b76d0
• Architecture: qwen35moe
• Standard tensor count: 753
• Native-MTP tensors: 20
• Unsupported custom tensors: 0
• Baked memory count: 0

Run the release checker with a llama.cpp checkout:

export SENTIENT_LLAMA_CPP=/path/to/llama.cpp
PYTHONPATH="$SENTIENT_LLAMA_CPP/gguf-py" python3 \
  native_mtp_build/check_sentient_memory_contract.py \
  Sentient-Memory-Qwen3.6-Native-MTP.JointMemory.Production.Q4_K_M.gguf \
  --manifest native_mtp_build/memory_manifest.json

Limitations

• There are no immutable facts in these models; all durable memory is host-side.
• Function schemas do not contain implementations.
• Dynamic storage requires persistence, retrieval, isolation, authorization, deletion, conflict handling, and auditing in the host.
• Q2 and Q1 are aggressive quantizations and may have severe quality loss.
• The model has no special safety wrapper when run in plain llama.cpp.
• No general benchmark, privacy, security, or broad safety evaluation is claimed for these edited tool-template artifacts.

See Memory and tools, Runtime and OpenAI, Validation, and Limitations.

License

Apache License 2.0, matching the declared license of the source model. Users remain responsible for complying with the base model's terms and applicable law.