---
license: mit
language:
  - en
base_model:
  - Qwen/Qwen3-4B-Instruct-2507
base_model_relation: adapter
pipeline_tag: text-generation
tags:
  - text-to-sql
  - spider
  - reinforcement-learning
  - gigpo
  - lora
  - mlx
  - qwen3
  - agent
datasets:
  - xlangai/spider
metrics:
  - exact_match
model-index:
  - name: spider-rl-qwen3-4b
    results:
      - task:
          type: text-to-sql
          name: Text-to-SQL Execution Accuracy
        dataset:
          type: xlangai/spider
          name: Spider 1.0 dev
        metrics:
          - type: execution_accuracy
            value: 51.6
            name: EX (n=1034 greedy)
          - type: answered_rate
            value: 98.6
            name: answered rate
      - task:
          type: text-to-sql
          name: Text-to-SQL Execution Accuracy
        dataset:
          type: xlangai/spider
          name: Spider 1.0 test
        metrics:
          - type: execution_accuracy
            value: 55.5
            name: EX (n=128 greedy)
---

# spider-rl-qwen3-4b

LoRA adapter trained with **GiGPO** on Yale [Spider 1.0](https://yale-lily.github.io/spider) text-to-SQL benchmark, on top of `Qwen/Qwen3-4B-Instruct-2507` (4-bit MLX quantization). Trained on a single Apple Silicon Mac in ~10 hours.

This is the **v4** checkpoint — recommended overall configuration. Two other variants (v7 with 2× LoRA capacity, v9b with entropy bonus) are documented in the [paper](https://github.com/johnhaofu/mlx-agent-rl/blob/main/papers/spider_rl_findings.md).

## Results

Greedy decoding (temperature=0), evaluated against gold result-sets via SQLite execution.

| Metric            | Baseline | This adapter | Δ          |
|-------------------|----------|--------------|------------|
| Dev EX (n=1034)   | 44.0%    | **51.6%**    | **+7.6pp** |
| Test EX (n=128)   | 50.0%    | 55.5%        | +5.5pp     |
| answered rate     | 90.3%    | 98.6%        | +8.3pp     |
| avg steps/episode | 2.54     | 1.97         | -22%       |

Per-hardness on dev (n=1034):

| Bucket       | Baseline | v4    |
|--------------|----------|-------|
| easy (248)   | 60.1%    | 70.2% |
| medium (446) | 50.0%    | 61.7% |
| hard (174)   | 31.0%    | 28.7% |
| extra (166)  | 17.5%    | 21.1% |

## Usage

This is a custom MLX-format LoRA delta loaded via the `mlx-agent-rl` framework, not a PEFT adapter. Standard `peft.PeftModel.from_pretrained()` will not work.

```python
# git clone github.com/johnhaofu/mlx-agent-rl
from mlx_agent_rl.core.policy import Policy
from huggingface_hub import snapshot_download

adapter_path = snapshot_download(repo_id="x32/spider-rl-qwen3-4b")
policy = Policy(
    model_path="Qwen/Qwen3-4B-Instruct-2507",  # or local Qwen3-4B-MLX-4bit
    lora_rank=8,
    lora_layers=4,
)
policy.load_adapters(adapter_path)
```

The agent expects multi-turn rollouts with `<action>sql[…]</action>` for exploration and `<action>answer[final SELECT]</action>` to commit. See the [training script](https://github.com/johnhaofu/mlx-agent-rl/blob/main/examples/train_spider.py) for the full prompt template.

## Training recipe

| Setting           | Value                              |
|-------------------|------------------------------------|
| Algorithm         | GiGPO (mean-norm advantages)       |
| Base model        | Qwen3-4B-Instruct-2507 (MLX 4-bit) |
| LoRA              | rank=8, last 4 transformer layers  |
| Trainable params  | ~10M (0.25% of base)               |
| Learning rate     | 5e-5                               |
| KL coefficient    | 0 (no anchor)                      |
| Reward            | binary 1.0 / 0.0 (no partial credit) |
| Group size        | 8 trajectories per prompt          |
| Epochs            | 2 over n=300 train questions       |
| Wall time         | ~10h on M-series Mac               |

## Counter-intuitive findings (from the full ablation)

1. **Partial-credit reward hurts** vs strict binary — agents settle for syntactically valid mediocrity. Removing partial credit gave +9.3pp on medium queries.
2. **KL anchor at 0.001 doesn't lift** overall EX, only redistributes across hardness buckets.
3. **Scaling training data** from n=300 to 600 regressed at fixed compute.
4. **87% of GiGPO groups had zero reward variance** — the real bottleneck.

Full ablation across 9 experiments: [paper on GitHub](https://github.com/johnhaofu/mlx-agent-rl/blob/main/papers/spider_rl_findings.md).

## Limitations

- Trained on 4-bit MLX quantized base — direct comparison to fp16/bf16 baselines requires re-quantization
- Single-seed training (no variance estimates)
- Far below GPT-4-class systems (DAIL-SQL+GPT-4 reaches 86% test EX) — this adapter targets the compute-constrained / on-device deployment niche
- `extra` hardness bucket (multi-table joins, INTERSECT/EXCEPT) sees minimal lift; capacity ceiling

## Code

[github.com/johnhaofu/mlx-agent-rl](https://github.com/johnhaofu/mlx-agent-rl) — MLX-native multi-turn agent RL framework, MIT-licensed.

## License

MIT