Xnizzorg/pondllm-qwen3.5-0.8b-base-action-sft

PondLLM Qwen3.5-0.8B-Base Action SFT Adapter

This is a small LoRA adapter trained to make Qwen/Qwen3.5-0.8B-Base select structured actions for PondLLM, an experimental deterministic grid ecology. Organisms perceive only their local surroundings and can forage, move, rest, share, signal, reproduce, and die under scarce energy.

This experiment asks whether the same behavior previously trained on Qwen3-0.6B transfers to the newer Qwen3.5-0.8B pretrained-only checkpoint. The dataset, prompt, LoRA settings, seed, optimizer, held-out cases, and live-world seeds were kept fixed.

It is not a standalone language model. The frozen Qwen base model must be loaded and this adapter attached with PEFT. The simulator itself is not included in this repository.

Intended use

The adapter maps a structured local observation to one legal JSON action:

{"action":"move","target":[2,3]}
{"action":"forage"}
{"action":"share","target":"organism-00002","amount":1}
{"action":"signal","message":"short message"}
{"action":"rest"}
{"action":"reproduce","target":[2,3]}

It is intended for reproducing this bounded action-policy experiment and comparing small-model behavior in simulated environments. It is not a general chat model or a policy for real-world autonomous decisions.

Loading

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.5-0.8B-Base"
adapter_id = "Xnizzorg/pondllm-qwen3.5-0.8b-base-action-sft"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

Use the PondLLM system prompt and official chat template. Greedy decoding was used for all reported behavioral evaluations.

Training data

All 14,006 SFT records were generated inside the deterministic PondLLM simulator. No personal, user-conversation, or scraped private data was used.

Action	Records
Forage	2,463
Move	6,495
Reproduce	1,000
Rest	2,048
Share	1,000
Signal	1,000

Training configuration

Parameter	Value
Base model	Qwen/Qwen3.5-0.8B-Base
Method	4-bit NF4 QLoRA SFT
LoRA rank / alpha	8 / 16
LoRA dropout	0.05
Maximum sequence length	1,024
Learning rate	2e-4, cosine
Effective batch size	32
Epochs / optimizer steps	2 / 864
Seed	7
Hardware	NVIDIA GeForce RTX 5070 Ti, 16 GB
Training time	75 minutes 50 seconds

The LoRA covers the language model's Gated DeltaNet, gated-attention, and feed-forward linear projections. The vision encoder and tied token embeddings were not adapted.

Evaluation

Greedy decoding was evaluated on the same deterministic 600-case set used for the prior Qwen3-0.6B adapter, with 100 examples of each action.

Policy	Parseable	Legal	Macro action accuracy	Exact action accuracy
This Qwen3.5 adapter	100.00%	99.83%	70.33%	58.00%
Qwen3-0.6B V2 adapter	100.00%	99.83%	67.33%	53.17%
Unadapted Qwen3.5 base	100.00%	2.00%	16.67%	0.83%

Per-action recall was 100% forage, 62% move, 90% reproduce, 85% rest, 85% share, and 0% signal.

In four fixed-seed, 40-tick worlds, the adapter made 841 decisions, of which 840 were legal. It produced 13 births and 4 deaths, finished with 25 living organisms, and preserved all 16 founder-lineage instances. The unadapted base drove all founders to extinction by tick 10.

These results demonstrate protocol learning and behavioral transplantation, not broad ecological superiority.

Limitations

• Signalling remains completely unlearned under greedy decoding.
• The single illegal held-out action attempted to share the organism's final unit of energy.
• The single illegal live action attempted reproduction one unit below a mutated threshold.
• This is not a pure architecture ablation: the compared checkpoints also differ in tokenizer, vocabulary, pretraining, and post-training stage.
• Evaluation covers a synthetic simulator, a limited seed set, and one hardware/software stack.
• The policy is prompt- and schema-specific.
• It has not undergone preference optimization, reinforcement learning, or evolutionary weight updates.
• Survival or reproduction in a simulator is not evidence of sentience or subjective experience.

Reproducibility files

• adapter_model.safetensors: learned LoRA weights
• adapter_config.json: PEFT configuration and base-model reference
• tokenizer.json, tokenizer_config.json, chat_template.jinja: tokenizer snapshot
• run_manifest.json: exact training configuration, metrics, and package versions

Adapter SHA-256:

90FF3DD4615059E16EED9FC46698F47AEA259A7F97171BCF6A5578B14619160D

License and attribution

This adapter is released under the Apache License 2.0. It is derived from Qwen/Qwen3.5-0.8B-Base, which is also distributed under the Apache License 2.0.

Training used Transformers, PEFT, and TRL.