---
base_model: zai-org/GLM-5
library_name: mlx
license: mit
tags:
- mlx
- safetensors
- glm_moe_dsa
- conversational
- text-generation
- mxfp4
- quantized
language:
- en
- zh
---

# mlx-community/GLM-5-MXFP4-Q8

This model was converted to MLX format from [`zai-org/GLM-5`](https://huggingface.co/zai-org/GLM-5) using a custom MXFP4-Q8 quantization scheme.

GLM-5 is a 744B parameter (40B active) Mixture-of-Experts model developed by Z.ai, targeting complex systems engineering and long-horizon agentic tasks. It uses Multi-Head Latent Attention (MLA) with 47 transformer layers, 64 routed experts (4 active per token), and 1 shared expert.

## Quantization

This model uses a mixed-precision quantization.

| Component | Mode | Bits | Group Size |
|---|---|---|---|
| Expert weights (switch_mlp) | MXFP4 | 4 | 32 |
| Attention, embeddings, shared expert, dense MLP, lm_head | Affine | 8 | 64 |

## Use with mlx-lm

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/GLM-5-MXFP4-Q8")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)
```