Instructions to use mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0 # Run inference directly in the terminal: llama cli -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0 # Run inference directly in the terminal: llama cli -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
Use Docker
docker model run hf.co/mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
- LM Studio
- Jan
- Ollama
How to use mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled with Ollama:
ollama run hf.co/mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
- Unsloth Studio
How to use mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled to start chatting
- Atomic Chat new
- Docker Model Runner
How to use mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled with Docker Model Runner:
docker model run hf.co/mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
- Lemonade
How to use mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull mfielding92/gemma3-12B-claude-3.7-sonnet-reasoning-distilled:Q8_0
Run and chat with the model
lemonade run user.gemma3-12B-claude-3.7-sonnet-reasoning-distilled-Q8_0
List all available models
lemonade list
## Incompatible license: Apache-2.0 License is Incompatible with Gemma License
Hi, I'd like to report a License Conflict in reedmayhew/gemma3-12B-claude-3.7-sonnet-reasoning-distilled. I noticed this model was quantized from unsloth/gemma-3-12b-it, which is released under the Gemma license, which appears to be incompatible with Gemma’s clauses — especially regarding redistribution, sublicensing, and commercial use.
⚠️ Key violations of Gemma license:
Section 3.1 – Distribution and Redistribution:
• Must include a copy of the Gemma License when distributing any derivative
• Must provide a "NOTICE" file with the text:
"Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms"
• Must pass along the use restrictions from Section 3.2 as enforceable terms
Section 3.2 – Use Restrictions:
• Must not use the model for any prohibited purposes
• Must comply with applicable laws and Google’s Prohibited Use Policy
Section 2.2 – Use Terms:
• Usage is only allowed “in accordance with the terms of this Agreement”
Meanwhile, Apache-2.0 allows:
• Commercial use without additional authorization
• Sublicensing and redistribution under permissive terms
• No requirement to propagate upstream non-permissive terms or use restrictions
This creates a conflict because Gemma’s license explicitly prohibits sublicensing under more permissive terms and requires downstream users to inherit specific use restrictions — something Apache-2.0 does not enforce.
So I'm thinking there might be a licensing conflict here that needs to be sorted out.
🔹 Suggestion:
1. To align with Gemma’s terms, it would be great to revise the licensing structure, for example:
Include a full copy of the Gemma License in the repository or model card
Add a required "NOTICE" file with this statement:
“Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms”
Making it clear in the model card that this is a derivative of a Gemma model, and that it inherits the same use restrictions (e.g., no commercial use, no sublicensing, etc.)
2. Maybe we can just drop the Apache-2.0 tag and going with the Gemma License. This approach may help reduce potential confusion about redistribution rights and downstream usage conditions.
Thanks for your attention!
Hi @xixi126 , I need to be direct here.
Your account is only a few days old and has already opened an unusually high volume of license-related issues across dozens of unrelated repositories.
The formatting, use of markdown, and consistent inclusion of emojis across these reports all follow a templated pattern that strongly suggests AI involvement or automation.
This raises serious questions about the authenticity and intent behind your activity.
Are you part of a coordinated effort, using AI to mass-report perceived license conflicts? Are you affiliated with any legal or auditing entity, or is this a personal campaign?
Without transparency, it's difficult to treat these reports as credible or in good faith.
Hi @reedmayhew , thank you for your candid feedback. I apologize if my report caused you any concerns or inconvenience.
I am currently conducting academic research on license compatibility issues in the AI ecosystem, and one of my work is to develop a tool for automatically detecting potential license conflicts in public model repositories. The report I submitted is generated based on the output of the tool and is intended to help improve license transparency and compliance, not to criticize or discredit any project.
I fully understand that the format and length of the report may seem suspicious, and I appreciate your attention. I just hope it can help the developers of hugging face. No reputation or other issues.
That being said, I will make sure to adjust my approach to be more transparent and respectful to maintainers. Thank you very much for your time and patience.
Thanks for the follow up!
I think what might improve your approach would be to change from:
Hi, I'd like to report a License Conflict in <model>. I noticed this model was quantized from <base_model>...
To:
Hello,
This is an automated notification that you may have the wrong license selected on your model <model>.
Your model appears to be quantized from <base_model>....
Since the word "report" can sometimes have negative implications with it, such as being penalized for having the wrong license. The goal should be a helpful notification to make people aware so they can update the license.
Also, specifically mentioning its automated will make it less suspicious that it's a scam, since it's not trying to appear human.
Hope that helps! Thanks.