Instructions to use datalab-to/surya-ocr-2-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use datalab-to/surya-ocr-2-gguf with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="datalab-to/surya-ocr-2-gguf")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("datalab-to/surya-ocr-2-gguf")
model = AutoModelForMultimodalLM.from_pretrained("datalab-to/surya-ocr-2-gguf", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use datalab-to/surya-ocr-2-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf datalab-to/surya-ocr-2-gguf
# Run inference directly in the terminal:
llama cli -hf datalab-to/surya-ocr-2-gguf

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf datalab-to/surya-ocr-2-gguf
# Run inference directly in the terminal:
llama cli -hf datalab-to/surya-ocr-2-gguf

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf datalab-to/surya-ocr-2-gguf
# Run inference directly in the terminal:
./llama-cli -hf datalab-to/surya-ocr-2-gguf

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf datalab-to/surya-ocr-2-gguf
# Run inference directly in the terminal:
./build/bin/llama-cli -hf datalab-to/surya-ocr-2-gguf

Use Docker

docker model run hf.co/datalab-to/surya-ocr-2-gguf

LM Studio
Jan

vLLM

How to use datalab-to/surya-ocr-2-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "datalab-to/surya-ocr-2-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "datalab-to/surya-ocr-2-gguf",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/datalab-to/surya-ocr-2-gguf

SGLang

How to use datalab-to/surya-ocr-2-gguf with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "datalab-to/surya-ocr-2-gguf" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "datalab-to/surya-ocr-2-gguf",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "datalab-to/surya-ocr-2-gguf" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "datalab-to/surya-ocr-2-gguf",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use datalab-to/surya-ocr-2-gguf with Ollama:
```
ollama run hf.co/datalab-to/surya-ocr-2-gguf
```

Unsloth Studio

How to use datalab-to/surya-ocr-2-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for datalab-to/surya-ocr-2-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for datalab-to/surya-ocr-2-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for datalab-to/surya-ocr-2-gguf to start chatting

Atomic Chat new
Docker Model Runner
How to use datalab-to/surya-ocr-2-gguf with Docker Model Runner:
```
docker model run hf.co/datalab-to/surya-ocr-2-gguf
```

Lemonade

How to use datalab-to/surya-ocr-2-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull datalab-to/surya-ocr-2-gguf

Run and chat with the model

lemonade run user.surya-ocr-2-gguf-{{QUANT_TAG}}

List all available models

lemonade list

vikp commited on May 27

Commit

87900a7

verified ·

1 Parent(s): 0ad7426

Move 28 demo images into assets/ subfolder

Browse files

Files changed (29) hide show

README.md +10 -22
corporate.png → assets/corporate.png +0 -0
corporate_layout.png → assets/corporate_layout.png +0 -0
corporate_reading.png → assets/corporate_reading.png +0 -0
corporate_tablerec.png → assets/corporate_tablerec.png +0 -0
corporate_text.png → assets/corporate_text.png +0 -0
excerpt.png → assets/excerpt.png +0 -0
excerpt_layout.png → assets/excerpt_layout.png +0 -0
excerpt_text.png → assets/excerpt_text.png +0 -0
form.png → assets/form.png +0 -0
form_layout.png → assets/form_layout.png +0 -0
form_reading.png → assets/form_reading.png +0 -0
form_tablerec.png → assets/form_tablerec.png +0 -0
form_text.png → assets/form_text.png +0 -0
handwritten.png → assets/handwritten.png +0 -0
handwritten_layout.png → assets/handwritten_layout.png +0 -0
handwritten_reading.png → assets/handwritten_reading.png +0 -0
handwritten_tablerec.png → assets/handwritten_tablerec.png +0 -0
handwritten_text.png → assets/handwritten_text.png +0 -0
newspaper.png → assets/newspaper.png +0 -0
newspaper_layout.png → assets/newspaper_layout.png +0 -0
newspaper_reading.png → assets/newspaper_reading.png +0 -0
newspaper_text.png → assets/newspaper_text.png +0 -0
olmocr_size_chart.png → assets/olmocr_size_chart.png +0 -0
scanned_tablerec.png → assets/scanned_tablerec.png +0 -0
textbook.png → assets/textbook.png +0 -0
textbook_layout.png → assets/textbook_layout.png +0 -0
textbook_reading.png → assets/textbook_reading.png +0 -0
textbook_text.png → assets/textbook_text.png +0 -0

@@ -1,15 +1,3 @@
----
-library_name: transformers
-license: openrail
-license_link: LICENSE
-tags:
-  - ocr
-  - pdf
-  - markdown
-  - layout
----
 <h1 align="center">Datalab</h1>
 <p align="center">
   <strong>State of the Art models for Document Intelligence</strong>
@@ -49,16 +37,16 @@ Commercial self-hosting of the model weights requires a license — see [Commerc
 ## Model Information
-<img src="olmocr_size_chart.png" width="700"/>
 |                            Detection                             |                                   OCR                                   |
 |:----------------------------------------------------------------:|:-----------------------------------------------------------------------:|
-|  <img src="excerpt.png" width="280"/>  |  <img src="excerpt_text.png" width="280"/> |
 |                               Layout                               |                       Table Recognition                       |
 |:------------------------------------------------------------------:|:-------------------------------------------------------------:|
-| <img src="excerpt_layout.png" width="280"/> | <img src="scanned_tablerec.png" width="280"/> |
 Surya is named for the [Hindu sun god](https://en.wikipedia.org/wiki/Surya), who has universal vision.
@@ -69,11 +57,11 @@ Each row links to five annotated views of the same page: text-line detection, OC
 | Name              |              Detection              |                                       OCR |                                       Layout |                                          Order |                                       Table Rec |
 |-------------------|:-----------------------------------:|------------------------------------------:|---------------------------------------------:|------------------------------------------------:|------------------------------------------------:|
-| Newspaper         | [Image](newspaper.png) | [Image](newspaper_text.png) | [Image](newspaper_layout.png) | [Image](newspaper_reading.png) |                                                  |
-| Textbook          | [Image](textbook.png)  | [Image](textbook_text.png)  | [Image](textbook_layout.png)  | [Image](textbook_reading.png)  |                                                  |
-| Tax Form          | [Image](form.png)      | [Image](form_text.png)      | [Image](form_layout.png)      | [Image](form_reading.png)      | [Image](form_tablerec.png)      |
-| Handwritten Notes | [Image](handwritten.png) | [Image](handwritten_text.png) | [Image](handwritten_layout.png) | [Image](handwritten_reading.png) | [Image](handwritten_tablerec.png) |
-| Corporate Doc     | [Image](corporate.png) | [Image](corporate_text.png) | [Image](corporate_layout.png) | [Image](corporate_reading.png) | [Image](corporate_tablerec.png) |
 # Commercial usage
@@ -365,7 +353,7 @@ standard quality benchmark for document parsers.
 ## olmOCR-bench
-Best-in-class accuracy under 1B parameters; pareto-optimal vs every model 3B and below.
 | Model                       | Params    | Score    |
 |-----------------------------|----------:|---------:|
@@ -417,7 +405,7 @@ Top 15 widely-spoken languages:
 | `es` | Spanish     | 90.7% |
 | `vi` | Vietnamese  | 73.2% |
-See [https://github.com/datalab-to/surya/blob/main/static/docs/multilingual.md](https://github.com/datalab-to/surya/blob/main/static/docs/multilingual.md) for the full 91-language table.
 ## Throughput

 <h1 align="center">Datalab</h1>
 <p align="center">
   <strong>State of the Art models for Document Intelligence</strong>
 ## Model Information
+<img src="assets/olmocr_size_chart.png" width="700"/>
 |                            Detection                             |                                   OCR                                   |
 |:----------------------------------------------------------------:|:-----------------------------------------------------------------------:|
+|  <img src="assets/excerpt.png" width="280"/>  |  <img src="assets/excerpt_text.png" width="280"/> |
 |                               Layout                               |                       Table Recognition                       |
 |:------------------------------------------------------------------:|:-------------------------------------------------------------:|
+| <img src="assets/excerpt_layout.png" width="280"/> | <img src="assets/scanned_tablerec.png" width="280"/> |
 Surya is named for the [Hindu sun god](https://en.wikipedia.org/wiki/Surya), who has universal vision.
 | Name              |              Detection              |                                       OCR |                                       Layout |                                          Order |                                       Table Rec |
 |-------------------|:-----------------------------------:|------------------------------------------:|---------------------------------------------:|------------------------------------------------:|------------------------------------------------:|
+| Newspaper         | [Image](assets/newspaper.png) | [Image](assets/newspaper_text.png) | [Image](assets/newspaper_layout.png) | [Image](assets/newspaper_reading.png) |                                                  |
+| Textbook          | [Image](assets/textbook.png)  | [Image](assets/textbook_text.png)  | [Image](assets/textbook_layout.png)  | [Image](assets/textbook_reading.png)  |                                                  |
+| Tax Form          | [Image](assets/form.png)      | [Image](assets/form_text.png)      | [Image](assets/form_layout.png)      | [Image](assets/form_reading.png)      | [Image](assets/form_tablerec.png)      |
+| Handwritten Notes | [Image](assets/handwritten.png) | [Image](assets/handwritten_text.png) | [Image](assets/handwritten_layout.png) | [Image](assets/handwritten_reading.png) | [Image](assets/handwritten_tablerec.png) |
+| Corporate Doc     | [Image](assets/corporate.png) | [Image](assets/corporate_text.png) | [Image](assets/corporate_layout.png) | [Image](assets/corporate_reading.png) | [Image](assets/corporate_tablerec.png) |
 # Commercial usage
 ## olmOCR-bench
+Pareto-optimal on the size-vs-score frontier, and best in class under 3B params.
 | Model                       | Params    | Score    |
 |-----------------------------|----------:|---------:|
 | `es` | Spanish     | 90.7% |
 | `vi` | Vietnamese  | 73.2% |
+See [https://github.com/datalab-to/surya/blob/master/static/docs/multilingual.md](https://github.com/datalab-to/surya/blob/master/static/docs/multilingual.md) for the full 91-language table.
 ## Throughput