README.md

---
license: mit
tags:
- llama-cpp
- llama-cpp-python
- gguf
- cuda
- windows
- prebuilt-wheels
- quantization
- local-llm
---

# llama-cpp-python Pre-built Windows Wheels

**Stop fighting with Visual Studio and CUDA Toolkit.** Just download and run.

Pre-compiled `llama-cpp-python` wheels for Windows across CUDA versions and GPU architectures.

## Quick Start

1. **Find your GPU** in the compatibility list below
2. **Download** the wheel for your GPU from [GitHub Releases](https://github.com/dougeeai/llama-cpp-python-wheels/releases) or [find your card on the README table](https://github.com/dougeeai/llama-cpp-python-wheels)
3. **Install**: `pip install <downloaded-wheel-file>.whl`
4. **Run** your GGUF models immediately

> **Platform Support:**  
> ✅ Windows 10/11 64-bit (available now, biggest pain point)  
> 🔜 Linux support coming soon  

## Supported GPUs

### RTX 50 Series (Blackwell - sm_100)
RTX 5090, 5080, 5070 Ti, 5070, 5060 Ti, 5060, RTX PRO 6000 Blackwell, B100, B200, GB200

### RTX 40 Series (Ada Lovelace - sm_89)
RTX 4090, 4080, 4070 Ti, 4070, 4060 Ti, 4060, RTX 6000 Ada, RTX 5000 Ada, L40, L40S

### RTX 30 Series (Ampere - sm_86)
RTX 3090, 3090 Ti, 3080 Ti, 3080, 3070 Ti, 3070, 3060 Ti, 3060, RTX A6000, A5000, A4000

### RTX 20 Series & GTX 16 Series (Turing - sm_75)
RTX 2080 Ti, 2080 Super, 2070 Super, 2060, GTX 1660 Ti, 1660 Super, 1650, Quadro RTX 8000, Tesla T4

[View full compatibility table →](https://github.com/dougeeai/llama-cpp-python-wheels#available-wheels)

## Usage Example
```python
from llama_cpp import Llama

# Load your GGUF model with GPU acceleration
llm = Llama(
    model_path="./models/llama-3-8b.Q4_K_M.gguf",
    n_gpu_layers=-1,  # Offload all layers to GPU
    n_ctx=2048        # Context window
)

# Generate text
response = llm(
    "Write a haiku about artificial intelligence:",
    max_tokens=50,
    temperature=0.7
)

print(response['choices'][0]['text'])
```

## Download Wheels

➡️ **[Download from GitHub Releases](https://github.com/dougeeai/llama-cpp-python-wheels/releases)**

### Available Configurations:
- **CUDA Versions**: 11.8, 12.1, 13.0
- **Python Versions**: 3.10, 3.11, 3.12, 3.13
- **Architectures**: sm_75 (Turing), sm_86 (Ampere), sm_89 (Ada), sm_100 (Blackwell)

## What This Solves

❌ No Visual Studio required  
❌ No CUDA Toolkit installation needed  
❌ No compilation errors  
❌ No "No CUDA toolset found" issues  
✅ Works immediately with GGUF models  
✅ Full GPU acceleration out of the box  

## Installation

Download the wheel matching your configuration and install:
```bash
# Example for RTX 4090 with Python 3.12 and CUDA 13.0
pip install llama_cpp_python-0.3.16+cuda13.0.sm89.ada-cp312-cp312-win_amd64.whl
```

## Build Details

All wheels are built with:
- Visual Studio 2019/2022 Build Tools
- Official NVIDIA CUDA Toolkits (11.8, 12.1, 13.0)
- Optimized CMAKE_CUDA_ARCHITECTURES for each GPU generation
- Built from official [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) source

## Contributing

**Need a different configuration?**

Open an [issue on GitHub](https://github.com/dougeeai/llama-cpp-python-wheels/issues) with:
- OS (Windows/Linux/macOS)
- Python version
- CUDA version
- GPU model

## Resources

- [GitHub Repository](https://github.com/dougeeai/llama-cpp-python-wheels)
- [Report Issues](https://github.com/dougeeai/llama-cpp-python-wheels/issues)
- [llama-cpp-python Documentation](https://github.com/abetlen/llama-cpp-python)
- [llama.cpp Project](https://github.com/ggerganov/llama.cpp)

## License

MIT License - Free to use for any purpose

Wheels are built from [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) (MIT License)