Hugging Face's logo

ChamaraVishwajithRajapaksha
/
sinhala-gpt-v.1.0.0

Text Generation
Transformers
PyTorch
Sinhala
English
sinhala_gpt
sinhala
sinhala-gpt
gpt
autoregressive
causal-language-model
slm
sri-lanka
sinhala-language
Model card Files
xet
 Community

Sinhala GPT - Usage Guide Complete guide to download and use your Sinhala GPT model from Hugging Face.

Table of Contents Quick Start Installation Loading the Model Text Generation Advanced Usage Troubleshooting API Reference Quick Start

Install transformers

pip install transformers torch

Use in Python

python

from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0") tokenizer = AutoTokenizer.from_pretrained("ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0")

prompt = "ශ්‍රී ලංකාව" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(**inputs, max_length=100) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) Installation Step 1: Install Python (if not already installed) Ubuntu/Debian:

sudo apt update sudo apt install python3 python3-pip Windows:

Download from python.org Check "Add Python to PATH" during installation Mac:

brew install python3 Step 2: Create Virtual Environment (Recommended)

Create virtual environment

python3 -m venv sinhala_gpt_env

Activate it

Linux/Mac:

source sinhala_gpt_env/bin/activate

Windows:

sinhala_gpt_env\Scripts\activate Step 3: Install Required Packages

Install transformers and PyTorch

pip install transformers torch

Optional: Install additional packages for better performance

pip install accelerate sentencepiece protobuf For GPU Support (NVIDIA):

CUDA 11.8

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

CUDA 12.1

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 Loading the Model Method 1: Basic Loading (CPU) from transformers import AutoModelForCausalLM, AutoTokenizer

Load model and tokenizer

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

print("✓ Model loaded successfully!") Method 2: GPU Loading (CUDA) import torch from transformers import AutoModelForCausalLM, AutoTokenizer

Check if CUDA is available

device = "cuda" if torch.cuda.is_available() else "cpu" print(f"Using device: {device}")

Load model on GPU

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.float16, # Use half precision for faster inference device_map="auto" # Automatically use GPU if available )

print(f"✓ Model loaded on {device}!") Method 3: Low Memory Loading (8-bit Quantization) from transformers import AutoModelForCausalLM, AutoTokenizer

Install bitsandbytes first: pip install bitsandbytes

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained( model_name, load_in_8bit=True, # Load in 8-bit precision device_map="auto" )

print("✓ Model loaded in 8-bit mode (uses ~4x less memory)!") Text Generation Basic Generation from transformers import AutoModelForCausalLM, AutoTokenizer

Load model

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

Generate text

prompt = "ශ්‍රී ලංකාව" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(**inputs, max_length=100)

Decode and print

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True) print(generated_text) Generation with Parameters from transformers import AutoModelForCausalLM, AutoTokenizer import torch

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "ශ්‍රී ලංකාවේ ඉතිහාසය" inputs = tokenizer(prompt, return_tensors="pt")

Generate with custom parameters

outputs = model.generate( **inputs, max_length=200, # Maximum length of generated text min_length=50, # Minimum length temperature=0.8, # Randomness (0.1 = focused, 1.5 = creative) top_k=50, # Consider top 50 tokens top_p=0.95, # Nucleus sampling do_sample=True, # Enable sampling num_return_sequences=1, # Number of sequences to generate repetition_penalty=1.2, # Penalize repetition no_repeat_ngram_size=3, # Don't repeat 3-grams pad_token_id=tokenizer.eos_token_id )

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True) print(generated_text) Batch Generation (Multiple Prompts) from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

Multiple prompts

prompts = [ "කොළඹ නගරය", "ශ්‍රී ලංකාවේ සංස්කෘතිය", "බුදු දහම" ]

Tokenize all prompts

inputs = tokenizer(prompts, return_tensors="pt", padding=True)

Generate

outputs = model.generate( **inputs, max_length=100, temperature=0.8, top_k=50, do_sample=True, pad_token_id=tokenizer.eos_token_id )

Decode all outputs

for i, output in enumerate(outputs): text = tokenizer.decode(output, skip_special_tokens=True) print(f"\nPrompt {i+1}: {prompts[i]}") print(f"Generated: {text}") print("-" * 80) Interactive Generation from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

print("Sinhala GPT Interactive Mode") print("Type your prompt and press Enter. Type 'quit' to exit.") print("-" * 80)

while True: prompt = input("\nPrompt: ")

if prompt.lower() == 'quit':
    break

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_length=150,
    temperature=0.8,
    top_k=50,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"\nGenerated: {generated_text}")
print("-" * 80)

Advanced Usage

  1. Streaming Generation (Token by Token) from transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer from threading import Thread

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "ශ්‍රී ලංකාව" inputs = tokenizer(prompt, return_tensors="pt")

Create streamer

streamer = TextIteratorStreamer(tokenizer, skip_special_tokens=True)

Generate in separate thread

generation_kwargs = dict( **inputs, max_length=200, temperature=0.8, top_k=50, do_sample=True, streamer=streamer, pad_token_id=tokenizer.eos_token_id )

thread = Thread(target=model.generate, kwargs=generation_kwargs) thread.start()

Print tokens as they're generated

print("Generating: ", end="") for new_text in streamer: print(new_text, end="", flush=True) print() 2. Generation with Custom Stopping Criteria from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList import torch

class StopOnTokens(StoppingCriteria): def init(self, stop_token_ids): self.stop_token_ids = stop_token_ids

def __call__(self, input_ids, scores, **kwargs):
    for stop_id in self.stop_token_ids:
        if input_ids[0][-1] == stop_id:
            return True
    return False

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "ශ්‍රී ලංකාව" inputs = tokenizer(prompt, return_tensors="pt")

Stop when certain tokens appear

stop_token_ids = [tokenizer.eos_token_id] stopping_criteria = StoppingCriteriaList([StopOnTokens(stop_token_ids)])

outputs = model.generate( **inputs, max_length=200, stopping_criteria=stopping_criteria, pad_token_id=tokenizer.eos_token_id )

print(tokenizer.decode(outputs[0], skip_special_tokens=True)) 3. Generate with Logit Processors from transformers import AutoModelForCausalLM, AutoTokenizer, LogitsProcessorList, MinLengthLogitsProcessor

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "කොළඹ" inputs = tokenizer(prompt, return_tensors="pt")

Force minimum length

logits_processor = LogitsProcessorList([ MinLengthLogitsProcessor(50, eos_token_id=tokenizer.eos_token_id) ])

outputs = model.generate( **inputs, max_length=200, logits_processor=logits_processor, pad_token_id=tokenizer.eos_token_id )

print(tokenizer.decode(outputs[0], skip_special_tokens=True)) 4. Save and Load Model Locally from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0"

Download and save locally

print("Downloading model...") tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

Save to disk

local_path = "./sinhala_gpt_local" tokenizer.save_pretrained(local_path) model.save_pretrained(local_path) print(f"✓ Model saved to {local_path}")

Load from local path (much faster next time)

print("\nLoading from local path...") tokenizer = AutoTokenizer.from_pretrained(local_path) model = AutoModelForCausalLM.from_pretrained(local_path) print("✓ Model loaded from local path!") 5. Pipeline Interface (Simplest Method) from transformers import pipeline

Create text generation pipeline

generator = pipeline( "text-generation", model="ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0", device=0 # Use GPU (device=0), or -1 for CPU )

Generate text

result = generator( "ශ්‍රී ලංකාව", max_length=100, temperature=0.8, top_k=50, do_sample=True, num_return_sequences=1 )

print(result[0]['generated_text']) Troubleshooting Issue 1: Out of Memory Error Error:

RuntimeError: CUDA out of memory Solutions:

Solution A: Use smaller batch size

outputs = model.generate(**inputs, max_length=50) # Reduce max_length

Solution B: Use 8-bit quantization

model = AutoModelForCausalLM.from_pretrained( model_name, load_in_8bit=True, device_map="auto" )

Solution C: Use CPU instead

model = AutoModelForCausalLM.from_pretrained(model_name) model = model.to("cpu")

Solution D: Clear GPU cache

import torch torch.cuda.empty_cache() Issue 2: Slow Generation on CPU Solutions:

Solution A: Use smaller max_length

outputs = model.generate(**inputs, max_length=50)

Solution B: Disable gradient calculation (already done in generate)

import torch with torch.no_grad(): outputs = model.generate(**inputs, max_length=100)

Solution C: Use GPU if available

device = "cuda" if torch.cuda.is_available() else "cpu" model = model.to(device) inputs = tokenizer(prompt, return_tensors="pt").to(device) Issue 3: Repetitive Text Generation Solution:

outputs = model.generate( **inputs, max_length=200, repetition_penalty=1.5, # Increase penalty no_repeat_ngram_size=3, # Prevent 3-gram repetition temperature=0.9, # Increase randomness top_p=0.95, do_sample=True ) Issue 4: Tokenizer Warnings Warning:

Token indices sequence length is longer than the specified maximum sequence length Solution:

Truncate long inputs

inputs = tokenizer( prompt, return_tensors="pt", max_length=512, truncation=True ) Issue 5: Model Not Found Error:

OSError: ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0 does not appear to be a valid model identifier Solutions:

Solution A: Check internet connection

Solution B: Verify model name is correct

Solution C: Try with use_auth_token if model is private

model = AutoModelForCausalLM.from_pretrained( model_name, use_auth_token="hf_your_token_here" )

Solution D: Load from local path if already downloaded

model = AutoModelForCausalLM.from_pretrained("./sinhala_gpt_local") API Reference Generation Parameters Parameter Type Default Description max_length int 20 Maximum length of generated text min_length int 0 Minimum length of generated text temperature float 1.0 Sampling temperature (0.1-2.0). Lower = more focused top_k int 50 Consider only top-k tokens top_p float 1.0 Nucleus sampling threshold repetition_penalty float 1.0 Penalty for repeating tokens (>1.0 reduces repetition) no_repeat_ngram_size int 0 Prevent repeating n-grams do_sample bool False Enable sampling (True = random, False = greedy) num_return_sequences int 1 Number of sequences to generate early_stopping bool False Stop when EOS token is generated pad_token_id int None Token ID for padding eos_token_id int None Token ID for end of sequence Temperature Guide Temperature Behavior Use Case 0.1 - 0.3 Very focused, deterministic Factual text, code 0.5 - 0.7 Balanced General purpose 0.8 - 1.0 Creative Stories, poetry 1.1 - 1.5 Very creative, random Brainstorming Model Information from transformers import AutoConfig

config = AutoConfig.from_pretrained("ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0")

print(f"Vocabulary size: {config.vocab_size}") print(f"Embedding dimension: {config.n_embed}") print(f"Number of layers: {config.n_layers}") print(f"Number of heads: {config.n_heads}") print(f"Context length: {config.block_size}") Complete Example Scripts Example 1: Simple CLI Text Generator #!/usr/bin/env python3 """Simple Sinhala text generator"""

from transformers import AutoModelForCausalLM, AutoTokenizer import argparse

def generate_text(prompt, max_length=200, temperature=0.8): """Generate Sinhala text from prompt""" model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0"

print("Loading model...")
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

print("Generating text...")
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_length=max_length,
    temperature=temperature,
    top_k=50,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)

return tokenizer.decode(outputs[0], skip_special_tokens=True)

if name == "main": parser = argparse.ArgumentParser(description="Generate Sinhala text") parser.add_argument("prompt", type=str, help="Input prompt in Sinhala") parser.add_argument("--max-length", type=int, default=200, help="Maximum length") parser.add_argument("--temperature", type=float, default=0.8, help="Temperature")

args = parser.parse_args()

result = generate_text(args.prompt, args.max_length, args.temperature)
print("\n" + "="*80)
print(result)
print("="*80)

Usage:

python generate.py "ශ්‍රී ලංකාව" --max-length 150 --temperature 0.8 Example 2: Batch Text Generator #!/usr/bin/env python3 """Batch Sinhala text generator"""

from transformers import AutoModelForCausalLM, AutoTokenizer import json

def batch_generate(prompts, output_file="outputs.json"): """Generate text for multiple prompts""" model_name = "ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0"

print("Loading model...")
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

results = []

for i, prompt in enumerate(prompts, 1):
    print(f"Generating {i}/{len(prompts)}: {prompt}")
    
    inputs = tokenizer(prompt, return_tensors="pt")
    outputs = model.generate(
        **inputs,
        max_length=200,
        temperature=0.8,
        top_k=50,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )
    
    generated = tokenizer.decode(outputs[0], skip_special_tokens=True)
    results.append({
        "prompt": prompt,
        "generated": generated
    })

# Save results
with open(output_file, "w", encoding="utf-8") as f:
    json.dump(results, f, ensure_ascii=False, indent=2)

print(f"\n✓ Results saved to {output_file}")
return results

if name == "main": prompts = [ "ශ්‍රී ලංකාව", "කොළඹ නගරය", "බුදු දහම", "සංස්කෘතිය" ]

batch_generate(prompts)

Performance Benchmarks Device Model Size Loading Time Generation Speed CPU (i7) 110M ~5s ~2 tokens/sec GPU (RTX 3060) 110M ~2s ~50 tokens/sec GPU (RTX 4090) 110M ~1s ~150 tokens/sec GPU (8-bit) 110M ~3s ~40 tokens/sec Additional Resources Hugging Face Model Card: https://huggingface.co/ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0 Transformers Documentation: https://huggingface.co/docs/transformers Text Generation Guide: https://huggingface.co/docs/transformers/main_classes/text_generation License Please check the model card on Hugging Face for licensing information.

Support For issues or questions:

Check the Troubleshooting section Visit the model page on Hugging Face Open an issue on the model's discussion board Last Updated: May 2026

Downloads last month
6
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ChamaraVishwajithRajapaksha/sinhala-gpt-v.1.0.0

Base model

openai-community/gpt2
Finetuned
(2254)
this model