Instructions to use anakin87/gemma-2-2b-neogenesis-ita with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use anakin87/gemma-2-2b-neogenesis-ita with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="anakin87/gemma-2-2b-neogenesis-ita")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("anakin87/gemma-2-2b-neogenesis-ita")
model = AutoModelForCausalLM.from_pretrained("anakin87/gemma-2-2b-neogenesis-ita", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use anakin87/gemma-2-2b-neogenesis-ita with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "anakin87/gemma-2-2b-neogenesis-ita"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anakin87/gemma-2-2b-neogenesis-ita",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/anakin87/gemma-2-2b-neogenesis-ita

SGLang

How to use anakin87/gemma-2-2b-neogenesis-ita with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "anakin87/gemma-2-2b-neogenesis-ita" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anakin87/gemma-2-2b-neogenesis-ita",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "anakin87/gemma-2-2b-neogenesis-ita" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anakin87/gemma-2-2b-neogenesis-ita",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use anakin87/gemma-2-2b-neogenesis-ita with Docker Model Runner:
```
docker model run hf.co/anakin87/gemma-2-2b-neogenesis-ita
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Gemma 2 2B Neogenesis ITA

Fine-tuned version of google/gemma-2-2b-it optimized for better performance in Italian.

Small yet powerful model with 2.6 billion parameters
Supports 8k context length

GGUF quants: static - weighted/imatrix

Need a stronger model? Try gemma-2-9b-neogenesis-ita.

🎮 Usage

💬🇮🇹 Try the model on Hugging Face Spaces

Text generation with Transformers

import torch
from transformers import pipeline

model_id="anakin87/gemma-2-2b-neogenesis-ita"

pipe = pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device="cuda",
)

messages = [{"role": "user", "content": "Cos'è l'interesse composto? Spiega in maniera semplice e chiara."}]
outputs = pipe(messages, max_new_tokens=500)

print(outputs[0]["generated_text"][1]["content"])

>>> Immagina di avere 100 euro e di depositarli in un conto che ti dà un interesse del 5% all'anno....

For more usage examples and applications, refer to the 📓 Kaggle notebook.

🏆 Evaluation Results

The model was submitted and evaluated in the Open Ita LLM Leaderboard, the most popular leaderboard for Italian Language Models.

Model	MMLU_IT	ARC_IT	HELLASWAG_IT	Average
google/gemma-2-2b-it	47.65	40.03	54.69	47.46
anakin87/gemma-2-2b-ita-sft (SFT checkpoint)	47.77	41.15	55.66	48.19
anakin87/gemma-2-2b-neogenesis-ita (DPO)	48.03	40.46	56.97	48.49

Qualitative evaluation across various domains is available here.

🔧 Training details

The model was fine-tuned using Hugging Face TRL.

The training involved Instruction Fine Tuning and Direct Preference Optimization.

I adopted a relatively new technique for parameter-efficient learning: Spectrum. The idea is to train only the layers of the model with high Signal-to-Noise Ratio (SNR) and ❄️ freeze the rest. Specifically, training focused on the top 25% most informative layers.

Batch size: 16; learning rate: 5e-6; epochs: 1 for SFT and 1 for DPO.

Training required about 15 hours on a single NVIDIA A6000 GPU (48GB VRAM).

For comprehensive training code and details, check out the 📓 Kaggle notebook.

🗃️ Training data

The model was trained primarily on Italian data, with a small portion of English data included.

For Instruction Fine Tuning:

Italian data
- efederici/capybara-claude-15k-ita
- anakin87/fine-instructions-ita-70k

For Direct Preference Optimization

Italian data
English data
- mlabonne/orpo-dpo-mix-40k

🙏 Thanks to the authors for providing these datasets.

Usage limitations

Although the model demonstrates solid Italian fluency and good reasoning capabilities for its small size, it is expected to have limited world knowledge due to its restricted number of parameters. This limitation can be mitigated by pairing it with techniques like Retrieval-Augmented Generation. Check out the 📓 Kaggle notebook for an example.

🛡️ Safety

While this model was not specifically fine-tuned for safety, its selective training with the Spectrum technique helps preserve certain safety features from the original model, as emerged in the qualitative evaluation.