README.md

metadata

license: mit
language:
  - en
base_model:
  - Qwen/Qwen3-4B-Instruct-2507
base_model_relation: adapter
pipeline_tag: text-generation
tags:
  - text-to-sql
  - spider
  - reinforcement-learning
  - gigpo
  - lora
  - mlx
  - qwen3
  - agent
datasets:
  - xlangai/spider
metrics:
  - exact_match
model-index:
  - name: spider-rl-qwen3-4b
    results:
      - task:
          type: text-to-sql
          name: Text-to-SQL Execution Accuracy
        dataset:
          type: xlangai/spider
          name: Spider 1.0 dev
        metrics:
          - type: execution_accuracy
            value: 51.6
            name: EX (n=1034 greedy)
          - type: answered_rate
            value: 98.6
            name: answered rate
      - task:
          type: text-to-sql
          name: Text-to-SQL Execution Accuracy
        dataset:
          type: xlangai/spider
          name: Spider 1.0 test
        metrics:
          - type: execution_accuracy
            value: 55.5
            name: EX (n=128 greedy)

spider-rl-qwen3-4b

LoRA adapter trained with GiGPO on Yale Spider 1.0 text-to-SQL benchmark, on top of Qwen/Qwen3-4B-Instruct-2507 (4-bit MLX quantization). Trained on a single Apple Silicon Mac in ~10 hours.

This is the v4 checkpoint — recommended overall configuration. Two other variants (v7 with 2× LoRA capacity, v9b with entropy bonus) are documented in the paper.

Results

Greedy decoding (temperature=0), evaluated against gold result-sets via SQLite execution.

Metric	Baseline	This adapter	Δ
Dev EX (n=1034)	44.0%	51.6%	+7.6pp
Test EX (n=128)	50.0%	55.5%	+5.5pp
answered rate	90.3%	98.6%	+8.3pp
avg steps/episode	2.54	1.97	-22%

Per-hardness on dev (n=1034):

Bucket	Baseline	v4
easy (248)	60.1%	70.2%
medium (446)	50.0%	61.7%
hard (174)	31.0%	28.7%
extra (166)	17.5%	21.1%

Usage

This is a custom MLX-format LoRA delta loaded via the mlx-agent-rl framework, not a PEFT adapter. Standard peft.PeftModel.from_pretrained() will not work.

# git clone github.com/johnhaofu/mlx-agent-rl
from mlx_agent_rl.core.policy import Policy
from huggingface_hub import snapshot_download

adapter_path = snapshot_download(repo_id="x32/spider-rl-qwen3-4b")
policy = Policy(
    model_path="Qwen/Qwen3-4B-Instruct-2507",  # or local Qwen3-4B-MLX-4bit
    lora_rank=8,
    lora_layers=4,
)
policy.load_adapters(adapter_path)

The agent expects multi-turn rollouts with <action>sql[…]</action> for exploration and <action>answer[final SELECT]</action> to commit. See the training script for the full prompt template.

Training recipe

Setting	Value
Algorithm	GiGPO (mean-norm advantages)
Base model	Qwen3-4B-Instruct-2507 (MLX 4-bit)
LoRA	rank=8, last 4 transformer layers
Trainable params	~10M (0.25% of base)
Learning rate	5e-5
KL coefficient	0 (no anchor)
Reward	binary 1.0 / 0.0 (no partial credit)
Group size	8 trajectories per prompt
Epochs	2 over n=300 train questions
Wall time	~10h on M-series Mac

Counter-intuitive findings (from the full ablation)

1. Partial-credit reward hurts vs strict binary — agents settle for syntactically valid mediocrity. Removing partial credit gave +9.3pp on medium queries.
2. KL anchor at 0.001 doesn't lift overall EX, only redistributes across hardness buckets.
3. Scaling training data from n=300 to 600 regressed at fixed compute.
4. 87% of GiGPO groups had zero reward variance — the real bottleneck.

Full ablation across 9 experiments: paper on GitHub.

Limitations

• Trained on 4-bit MLX quantized base — direct comparison to fp16/bf16 baselines requires re-quantization
• Single-seed training (no variance estimates)
• Far below GPT-4-class systems (DAIL-SQL+GPT-4 reaches 86% test EX) — this adapter targets the compute-constrained / on-device deployment niche
• extra hardness bucket (multi-table joins, INTERSECT/EXCEPT) sees minimal lift; capacity ceiling

Code

github.com/johnhaofu/mlx-agent-rl — MLX-native multi-turn agent RL framework, MIT-licensed.

License

MIT