Minuri/sinhala-llama-1b-sft-baseline

Sinhala LLaMA 1B - SFT Baseline

An instruction-tuned Sinhala language model fine-tuned directly on the base meta-llama/Llama-3.2-1B without any continual pretraining. This serves as the SFT baseline for comparison against the three CPT+SFT model variants in a diversity-driven Sinhala language model adaptation study.

SFT model variants in this series:

• Minuri/sinhala-llama-1b-sft-baseline - SFT on base LLaMA 3.2 1B - this repo
• Minuri/sinhala-llama-1b-sft-news - SFT on sinhala-llama-1b-corpus-news
• Minuri/sinhala-llama-1b-sft-random - SFT on sinhala-llama-1b-corpus-random
• Minuri/sinhala-llama-1b-sft-diverse - SFT on sinhala-llama-1b-corpus-diverse

Model Description

This model is the result of supervised fine-tuning (SFT) of the base meta-llama/Llama-3.2-1B on the Minuri/sinhala-sft-dataset (~213K Sinhala instruction pairs). No continual pretraining was applied prior to SFT. This allows direct comparison of the effect of CPT on downstream instruction-following performance.

Training Data

Dataset	Description
Minuri/sinhala-sft-dataset	~213K Sinhala instruction pairs merged from three source datasets

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Minuri/sinhala-llama-1b-sft-baseline")
model = AutoModelForCausalLM.from_pretrained("Minuri/sinhala-llama-1b-sft-baseline")

Intended Uses

• Sinhala instruction following
• Baseline comparison for CPT+SFT ablation studies
• Low-resource NLP research

Limitations

• No continual pretraining on Sinhala - limited Sinhala language understanding compared to CPT+SFT variants
• 1B parameter model with limited reasoning capability

Related Repositories

Repo	Description
Minuri/sinhala-sft-dataset	SFT training dataset (~213K pairs)
Minuri/sinhala-llama-1b-sft-news	SFT on sinhala-llama-1b-corpus-news model
Minuri/sinhala-llama-1b-sft-random	SFT on sinhala-llama-1b-corpus-random model
Minuri/sinhala-llama-1b-sft-diverse	SFT on sinhala-llama-1b-corpus-diverse model

License

This model is derived from meta-llama/Llama-3.2-1B and is subject to the LLaMA 3.2 Community License.