MariChatmen/qwen-andaluh-0.8b-lora

Qwen-Andaluh 0.8B LoRA

LoRA adapter for Qwen/Qwen3.5-0.8B trained as the non-persona Qwen-Andaluh checkpoint.

The intended behaviour is simple:

System prompt: Eres un asistente
Input: Spanish or Andalûh
Output: Andalûh EPA

This is an adapter release, not a merged model.

Training Stages

• Andalûh continued pretraining / CPT.
• Accent SFT.
• Accent ORPO preference tuning.

The source data was generated from Spanish instruction rows by filtering licence-compatible Spanish examples, protecting fragile spans, converting assistant messages to Andalûh EPA, converting a controlled share of user messages to Andalûh EPA, and restoring URLs, paths, code, package names and model IDs unchanged.

Adapter Details

• Base model: Qwen/Qwen3.5-0.8B
• Adapter type: LoRA / QLoRA
• Rank: 16
• Alpha: 32
• Dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Use

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "MariChatmen/qwen-andaluh-0.8b-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(model_id, device_map="auto")

Limitations

• Experimental research adapter.
• May leak standard Spanish.
• May produce imperfect Andalûh EPA.
• Not an official linguistic standard.

Related

• Persona dataset: https://huggingface.co/datasets/MariChatmen/MariChatmen-Persona
• AndaluGeeks EPA: https://andaluh.es/epa-2/
• andaluh-py: https://github.com/andalugeeks/andaluh-py