Instructions to use FancieF/Qwythos-27B-v1-MLX-VLM-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FancieF/Qwythos-27B-v1-MLX-VLM-bf16 with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("FancieF/Qwythos-27B-v1-MLX-VLM-bf16")
config = load_config("FancieF/Qwythos-27B-v1-MLX-VLM-bf16")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use FancieF/Qwythos-27B-v1-MLX-VLM-bf16 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "FancieF/Qwythos-27B-v1-MLX-VLM-bf16"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FancieF/Qwythos-27B-v1-MLX-VLM-bf16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use FancieF/Qwythos-27B-v1-MLX-VLM-bf16 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "FancieF/Qwythos-27B-v1-MLX-VLM-bf16"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FancieF/Qwythos-27B-v1-MLX-VLM-bf16" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Hermes Agent

How to use FancieF/Qwythos-27B-v1-MLX-VLM-bf16 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "FancieF/Qwythos-27B-v1-MLX-VLM-bf16"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FancieF/Qwythos-27B-v1-MLX-VLM-bf16

Run Hermes

hermes

Qwythos-27B-v1 · MLX VLM BF16

Unquantized bfloat16 MLX conversion of empero-ai/Qwythos-27B-v1 for Apple Silicon. Vision tower kept.

Not the everyday download. Prefer 4-bit VLM on ≤32–48GB Macs. This build is ~55GB and needs high unified memory.

Which repo should I download?

Repo	Size	Use when
VLM 4-bit	~15GB	Default chat / vision
This BF16	~55GB	Max fidelity, further quant, Studio-class RAM
MTP drafter	~829MB	Optional `--draft-model` speed-up

Install

pip install -U mlx-vlm

Quick start

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --prompt "Introduce yourself in one sentence." \
  --thinking-mode disabled \
  --max-tokens 256 \
  --temperature 0.6

Vision

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --image ./photo.jpg \
  --prompt "Describe this image." \
  --thinking-mode disabled

+ MTP

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --draft-model FancieF/Qwythos-27B-v1-MLX-MTP \
  --draft-kind mtp \
  --draft-block-size 4 \
  --prompt "..." \
  --thinking-mode disabled

LM Studio / memory

BF16 27B + KV easily exceeds 32GB — expect swap or OOM on common MacBooks.
For LM Studio with MTP, use official GGUF MTP quants instead of this MLX BF16.

What’s inside

MLX safetensors, BF16, no weight quantization
language_model + vision_tower
No baked-in MTP (same as 4-bit; pair with the MTP repo)

Conversion

mlx_vlm.convert \
  --hf-path /path/to/Qwythos-27B-v1 \
  --mlx-path ./Qwythos-27B-MLX-VLM-bf16 \
  --dtype bfloat16 --trust-remote-code

(No -q.)

License & attribution

Derived from empero-ai/Qwythos-27B-v1 (Apache-2.0), based on Qwen3.5-27B. Unofficial MLX conversion.

Links


Recommended 4-bit	FancieF/Qwythos-27B-v1-MLX-VLM-4bit
MTP	FancieF/Qwythos-27B-v1-MLX-MTP
Base	empero-ai/Qwythos-27B-v1
ModelScope (ZH)	Fancie/Qwythos-27B-v1-MLX-VLM-bf16