Image-Text-to-Text
Transformers
Safetensors
English
idefics3
text-generation
documents
code
formula
chart
ocr
layout
table
document-parse
docling
granite
extraction
math
conversational
Instructions to use docling-project/granite-docling-2stage-258m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use docling-project/granite-docling-2stage-258m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="docling-project/granite-docling-2stage-258m") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("docling-project/granite-docling-2stage-258m") model = AutoModelForMultimodalLM.from_pretrained("docling-project/granite-docling-2stage-258m", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use docling-project/granite-docling-2stage-258m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "docling-project/granite-docling-2stage-258m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docling-project/granite-docling-2stage-258m", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/docling-project/granite-docling-2stage-258m
- SGLang
How to use docling-project/granite-docling-2stage-258m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "docling-project/granite-docling-2stage-258m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docling-project/granite-docling-2stage-258m", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "docling-project/granite-docling-2stage-258m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docling-project/granite-docling-2stage-258m", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use docling-project/granite-docling-2stage-258m with Docker Model Runner:
docker model run hf.co/docling-project/granite-docling-2stage-258m
| { | |
| "<end_of_utterance>": 100854, | |
| "<layout>": 100352, | |
| "<loc_0>": 100353, | |
| "<loc_100>": 100453, | |
| "<loc_101>": 100454, | |
| "<loc_102>": 100455, | |
| "<loc_103>": 100456, | |
| "<loc_104>": 100457, | |
| "<loc_105>": 100458, | |
| "<loc_106>": 100459, | |
| "<loc_107>": 100460, | |
| "<loc_108>": 100461, | |
| "<loc_109>": 100462, | |
| "<loc_10>": 100363, | |
| "<loc_110>": 100463, | |
| "<loc_111>": 100464, | |
| "<loc_112>": 100465, | |
| "<loc_113>": 100466, | |
| "<loc_114>": 100467, | |
| "<loc_115>": 100468, | |
| "<loc_116>": 100469, | |
| "<loc_117>": 100470, | |
| "<loc_118>": 100471, | |
| "<loc_119>": 100472, | |
| "<loc_11>": 100364, | |
| "<loc_120>": 100473, | |
| "<loc_121>": 100474, | |
| "<loc_122>": 100475, | |
| "<loc_123>": 100476, | |
| "<loc_124>": 100477, | |
| "<loc_125>": 100478, | |
| "<loc_126>": 100479, | |
| "<loc_127>": 100480, | |
| "<loc_128>": 100481, | |
| "<loc_129>": 100482, | |
| "<loc_12>": 100365, | |
| "<loc_130>": 100483, | |
| "<loc_131>": 100484, | |
| "<loc_132>": 100485, | |
| "<loc_133>": 100486, | |
| "<loc_134>": 100487, | |
| "<loc_135>": 100488, | |
| "<loc_136>": 100489, | |
| "<loc_137>": 100490, | |
| "<loc_138>": 100491, | |
| "<loc_139>": 100492, | |
| "<loc_13>": 100366, | |
| "<loc_140>": 100493, | |
| "<loc_141>": 100494, | |
| "<loc_142>": 100495, | |
| "<loc_143>": 100496, | |
| "<loc_144>": 100497, | |
| "<loc_145>": 100498, | |
| "<loc_146>": 100499, | |
| "<loc_147>": 100500, | |
| "<loc_148>": 100501, | |
| "<loc_149>": 100502, | |
| "<loc_14>": 100367, | |
| "<loc_150>": 100503, | |
| "<loc_151>": 100504, | |
| "<loc_152>": 100505, | |
| "<loc_153>": 100506, | |
| "<loc_154>": 100507, | |
| "<loc_155>": 100508, | |
| "<loc_156>": 100509, | |
| "<loc_157>": 100510, | |
| "<loc_158>": 100511, | |
| "<loc_159>": 100512, | |
| "<loc_15>": 100368, | |
| "<loc_160>": 100513, | |
| "<loc_161>": 100514, | |
| "<loc_162>": 100515, | |
| "<loc_163>": 100516, | |
| "<loc_164>": 100517, | |
| "<loc_165>": 100518, | |
| "<loc_166>": 100519, | |
| "<loc_167>": 100520, | |
| "<loc_168>": 100521, | |
| "<loc_169>": 100522, | |
| "<loc_16>": 100369, | |
| "<loc_170>": 100523, | |
| "<loc_171>": 100524, | |
| "<loc_172>": 100525, | |
| "<loc_173>": 100526, | |
| "<loc_174>": 100527, | |
| "<loc_175>": 100528, | |
| "<loc_176>": 100529, | |
| "<loc_177>": 100530, | |
| "<loc_178>": 100531, | |
| "<loc_179>": 100532, | |
| "<loc_17>": 100370, | |
| "<loc_180>": 100533, | |
| "<loc_181>": 100534, | |
| "<loc_182>": 100535, | |
| "<loc_183>": 100536, | |
| "<loc_184>": 100537, | |
| "<loc_185>": 100538, | |
| "<loc_186>": 100539, | |
| "<loc_187>": 100540, | |
| "<loc_188>": 100541, | |
| "<loc_189>": 100542, | |
| "<loc_18>": 100371, | |
| "<loc_190>": 100543, | |
| "<loc_191>": 100544, | |
| "<loc_192>": 100545, | |
| "<loc_193>": 100546, | |
| "<loc_194>": 100547, | |
| "<loc_195>": 100548, | |
| "<loc_196>": 100549, | |
| "<loc_197>": 100550, | |
| "<loc_198>": 100551, | |
| "<loc_199>": 100552, | |
| "<loc_19>": 100372, | |
| "<loc_1>": 100354, | |
| "<loc_200>": 100553, | |
| "<loc_201>": 100554, | |
| "<loc_202>": 100555, | |
| "<loc_203>": 100556, | |
| "<loc_204>": 100557, | |
| "<loc_205>": 100558, | |
| "<loc_206>": 100559, | |
| "<loc_207>": 100560, | |
| "<loc_208>": 100561, | |
| "<loc_209>": 100562, | |
| "<loc_20>": 100373, | |
| "<loc_210>": 100563, | |
| "<loc_211>": 100564, | |
| "<loc_212>": 100565, | |
| "<loc_213>": 100566, | |
| "<loc_214>": 100567, | |
| "<loc_215>": 100568, | |
| "<loc_216>": 100569, | |
| "<loc_217>": 100570, | |
| "<loc_218>": 100571, | |
| "<loc_219>": 100572, | |
| "<loc_21>": 100374, | |
| "<loc_220>": 100573, | |
| "<loc_221>": 100574, | |
| "<loc_222>": 100575, | |
| "<loc_223>": 100576, | |
| "<loc_224>": 100577, | |
| "<loc_225>": 100578, | |
| "<loc_226>": 100579, | |
| "<loc_227>": 100580, | |
| "<loc_228>": 100581, | |
| "<loc_229>": 100582, | |
| "<loc_22>": 100375, | |
| "<loc_230>": 100583, | |
| "<loc_231>": 100584, | |
| "<loc_232>": 100585, | |
| "<loc_233>": 100586, | |
| "<loc_234>": 100587, | |
| "<loc_235>": 100588, | |
| "<loc_236>": 100589, | |
| "<loc_237>": 100590, | |
| "<loc_238>": 100591, | |
| "<loc_239>": 100592, | |
| "<loc_23>": 100376, | |
| "<loc_240>": 100593, | |
| "<loc_241>": 100594, | |
| "<loc_242>": 100595, | |
| "<loc_243>": 100596, | |
| "<loc_244>": 100597, | |
| "<loc_245>": 100598, | |
| "<loc_246>": 100599, | |
| "<loc_247>": 100600, | |
| "<loc_248>": 100601, | |
| "<loc_249>": 100602, | |
| "<loc_24>": 100377, | |
| "<loc_250>": 100603, | |
| "<loc_251>": 100604, | |
| "<loc_252>": 100605, | |
| "<loc_253>": 100606, | |
| "<loc_254>": 100607, | |
| "<loc_255>": 100608, | |
| "<loc_256>": 100609, | |
| "<loc_257>": 100610, | |
| "<loc_258>": 100611, | |
| "<loc_259>": 100612, | |
| "<loc_25>": 100378, | |
| "<loc_260>": 100613, | |
| "<loc_261>": 100614, | |
| "<loc_262>": 100615, | |
| "<loc_263>": 100616, | |
| "<loc_264>": 100617, | |
| "<loc_265>": 100618, | |
| "<loc_266>": 100619, | |
| "<loc_267>": 100620, | |
| "<loc_268>": 100621, | |
| "<loc_269>": 100622, | |
| "<loc_26>": 100379, | |
| "<loc_270>": 100623, | |
| "<loc_271>": 100624, | |
| "<loc_272>": 100625, | |
| "<loc_273>": 100626, | |
| "<loc_274>": 100627, | |
| "<loc_275>": 100628, | |
| "<loc_276>": 100629, | |
| "<loc_277>": 100630, | |
| "<loc_278>": 100631, | |
| "<loc_279>": 100632, | |
| "<loc_27>": 100380, | |
| "<loc_280>": 100633, | |
| "<loc_281>": 100634, | |
| "<loc_282>": 100635, | |
| "<loc_283>": 100636, | |
| "<loc_284>": 100637, | |
| "<loc_285>": 100638, | |
| "<loc_286>": 100639, | |
| "<loc_287>": 100640, | |
| "<loc_288>": 100641, | |
| "<loc_289>": 100642, | |
| "<loc_28>": 100381, | |
| "<loc_290>": 100643, | |
| "<loc_291>": 100644, | |
| "<loc_292>": 100645, | |
| "<loc_293>": 100646, | |
| "<loc_294>": 100647, | |
| "<loc_295>": 100648, | |
| "<loc_296>": 100649, | |
| "<loc_297>": 100650, | |
| "<loc_298>": 100651, | |
| "<loc_299>": 100652, | |
| "<loc_29>": 100382, | |
| "<loc_2>": 100355, | |
| "<loc_300>": 100653, | |
| "<loc_301>": 100654, | |
| "<loc_302>": 100655, | |
| "<loc_303>": 100656, | |
| "<loc_304>": 100657, | |
| "<loc_305>": 100658, | |
| "<loc_306>": 100659, | |
| "<loc_307>": 100660, | |
| "<loc_308>": 100661, | |
| "<loc_309>": 100662, | |
| "<loc_30>": 100383, | |
| "<loc_310>": 100663, | |
| "<loc_311>": 100664, | |
| "<loc_312>": 100665, | |
| "<loc_313>": 100666, | |
| "<loc_314>": 100667, | |
| "<loc_315>": 100668, | |
| "<loc_316>": 100669, | |
| "<loc_317>": 100670, | |
| "<loc_318>": 100671, | |
| "<loc_319>": 100672, | |
| "<loc_31>": 100384, | |
| "<loc_320>": 100673, | |
| "<loc_321>": 100674, | |
| "<loc_322>": 100675, | |
| "<loc_323>": 100676, | |
| "<loc_324>": 100677, | |
| "<loc_325>": 100678, | |
| "<loc_326>": 100679, | |
| "<loc_327>": 100680, | |
| "<loc_328>": 100681, | |
| "<loc_329>": 100682, | |
| "<loc_32>": 100385, | |
| "<loc_330>": 100683, | |
| "<loc_331>": 100684, | |
| "<loc_332>": 100685, | |
| "<loc_333>": 100686, | |
| "<loc_334>": 100687, | |
| "<loc_335>": 100688, | |
| "<loc_336>": 100689, | |
| "<loc_337>": 100690, | |
| "<loc_338>": 100691, | |
| "<loc_339>": 100692, | |
| "<loc_33>": 100386, | |
| "<loc_340>": 100693, | |
| "<loc_341>": 100694, | |
| "<loc_342>": 100695, | |
| "<loc_343>": 100696, | |
| "<loc_344>": 100697, | |
| "<loc_345>": 100698, | |
| "<loc_346>": 100699, | |
| "<loc_347>": 100700, | |
| "<loc_348>": 100701, | |
| "<loc_349>": 100702, | |
| "<loc_34>": 100387, | |
| "<loc_350>": 100703, | |
| "<loc_351>": 100704, | |
| "<loc_352>": 100705, | |
| "<loc_353>": 100706, | |
| "<loc_354>": 100707, | |
| "<loc_355>": 100708, | |
| "<loc_356>": 100709, | |
| "<loc_357>": 100710, | |
| "<loc_358>": 100711, | |
| "<loc_359>": 100712, | |
| "<loc_35>": 100388, | |
| "<loc_360>": 100713, | |
| "<loc_361>": 100714, | |
| "<loc_362>": 100715, | |
| "<loc_363>": 100716, | |
| "<loc_364>": 100717, | |
| "<loc_365>": 100718, | |
| "<loc_366>": 100719, | |
| "<loc_367>": 100720, | |
| "<loc_368>": 100721, | |
| "<loc_369>": 100722, | |
| "<loc_36>": 100389, | |
| "<loc_370>": 100723, | |
| "<loc_371>": 100724, | |
| "<loc_372>": 100725, | |
| "<loc_373>": 100726, | |
| "<loc_374>": 100727, | |
| "<loc_375>": 100728, | |
| "<loc_376>": 100729, | |
| "<loc_377>": 100730, | |
| "<loc_378>": 100731, | |
| "<loc_379>": 100732, | |
| "<loc_37>": 100390, | |
| "<loc_380>": 100733, | |
| "<loc_381>": 100734, | |
| "<loc_382>": 100735, | |
| "<loc_383>": 100736, | |
| "<loc_384>": 100737, | |
| "<loc_385>": 100738, | |
| "<loc_386>": 100739, | |
| "<loc_387>": 100740, | |
| "<loc_388>": 100741, | |
| "<loc_389>": 100742, | |
| "<loc_38>": 100391, | |
| "<loc_390>": 100743, | |
| "<loc_391>": 100744, | |
| "<loc_392>": 100745, | |
| "<loc_393>": 100746, | |
| "<loc_394>": 100747, | |
| "<loc_395>": 100748, | |
| "<loc_396>": 100749, | |
| "<loc_397>": 100750, | |
| "<loc_398>": 100751, | |
| "<loc_399>": 100752, | |
| "<loc_39>": 100392, | |
| "<loc_3>": 100356, | |
| "<loc_400>": 100753, | |
| "<loc_401>": 100754, | |
| "<loc_402>": 100755, | |
| "<loc_403>": 100756, | |
| "<loc_404>": 100757, | |
| "<loc_405>": 100758, | |
| "<loc_406>": 100759, | |
| "<loc_407>": 100760, | |
| "<loc_408>": 100761, | |
| "<loc_409>": 100762, | |
| "<loc_40>": 100393, | |
| "<loc_410>": 100763, | |
| "<loc_411>": 100764, | |
| "<loc_412>": 100765, | |
| "<loc_413>": 100766, | |
| "<loc_414>": 100767, | |
| "<loc_415>": 100768, | |
| "<loc_416>": 100769, | |
| "<loc_417>": 100770, | |
| "<loc_418>": 100771, | |
| "<loc_419>": 100772, | |
| "<loc_41>": 100394, | |
| "<loc_420>": 100773, | |
| "<loc_421>": 100774, | |
| "<loc_422>": 100775, | |
| "<loc_423>": 100776, | |
| "<loc_424>": 100777, | |
| "<loc_425>": 100778, | |
| "<loc_426>": 100779, | |
| "<loc_427>": 100780, | |
| "<loc_428>": 100781, | |
| "<loc_429>": 100782, | |
| "<loc_42>": 100395, | |
| "<loc_430>": 100783, | |
| "<loc_431>": 100784, | |
| "<loc_432>": 100785, | |
| "<loc_433>": 100786, | |
| "<loc_434>": 100787, | |
| "<loc_435>": 100788, | |
| "<loc_436>": 100789, | |
| "<loc_437>": 100790, | |
| "<loc_438>": 100791, | |
| "<loc_439>": 100792, | |
| "<loc_43>": 100396, | |
| "<loc_440>": 100793, | |
| "<loc_441>": 100794, | |
| "<loc_442>": 100795, | |
| "<loc_443>": 100796, | |
| "<loc_444>": 100797, | |
| "<loc_445>": 100798, | |
| "<loc_446>": 100799, | |
| "<loc_447>": 100800, | |
| "<loc_448>": 100801, | |
| "<loc_449>": 100802, | |
| "<loc_44>": 100397, | |
| "<loc_450>": 100803, | |
| "<loc_451>": 100804, | |
| "<loc_452>": 100805, | |
| "<loc_453>": 100806, | |
| "<loc_454>": 100807, | |
| "<loc_455>": 100808, | |
| "<loc_456>": 100809, | |
| "<loc_457>": 100810, | |
| "<loc_458>": 100811, | |
| "<loc_459>": 100812, | |
| "<loc_45>": 100398, | |
| "<loc_460>": 100813, | |
| "<loc_461>": 100814, | |
| "<loc_462>": 100815, | |
| "<loc_463>": 100816, | |
| "<loc_464>": 100817, | |
| "<loc_465>": 100818, | |
| "<loc_466>": 100819, | |
| "<loc_467>": 100820, | |
| "<loc_468>": 100821, | |
| "<loc_469>": 100822, | |
| "<loc_46>": 100399, | |
| "<loc_470>": 100823, | |
| "<loc_471>": 100824, | |
| "<loc_472>": 100825, | |
| "<loc_473>": 100826, | |
| "<loc_474>": 100827, | |
| "<loc_475>": 100828, | |
| "<loc_476>": 100829, | |
| "<loc_477>": 100830, | |
| "<loc_478>": 100831, | |
| "<loc_479>": 100832, | |
| "<loc_47>": 100400, | |
| "<loc_480>": 100833, | |
| "<loc_481>": 100834, | |
| "<loc_482>": 100835, | |
| "<loc_483>": 100836, | |
| "<loc_484>": 100837, | |
| "<loc_485>": 100838, | |
| "<loc_486>": 100839, | |
| "<loc_487>": 100840, | |
| "<loc_488>": 100841, | |
| "<loc_489>": 100842, | |
| "<loc_48>": 100401, | |
| "<loc_490>": 100843, | |
| "<loc_491>": 100844, | |
| "<loc_492>": 100845, | |
| "<loc_493>": 100846, | |
| "<loc_494>": 100847, | |
| "<loc_495>": 100848, | |
| "<loc_496>": 100849, | |
| "<loc_497>": 100850, | |
| "<loc_498>": 100851, | |
| "<loc_499>": 100852, | |
| "<loc_49>": 100402, | |
| "<loc_4>": 100357, | |
| "<loc_500>": 100853, | |
| "<loc_50>": 100403, | |
| "<loc_51>": 100404, | |
| "<loc_52>": 100405, | |
| "<loc_53>": 100406, | |
| "<loc_54>": 100407, | |
| "<loc_55>": 100408, | |
| "<loc_56>": 100409, | |
| "<loc_57>": 100410, | |
| "<loc_58>": 100411, | |
| "<loc_59>": 100412, | |
| "<loc_5>": 100358, | |
| "<loc_60>": 100413, | |
| "<loc_61>": 100414, | |
| "<loc_62>": 100415, | |
| "<loc_63>": 100416, | |
| "<loc_64>": 100417, | |
| "<loc_65>": 100418, | |
| "<loc_66>": 100419, | |
| "<loc_67>": 100420, | |
| "<loc_68>": 100421, | |
| "<loc_69>": 100422, | |
| "<loc_6>": 100359, | |
| "<loc_70>": 100423, | |
| "<loc_71>": 100424, | |
| "<loc_72>": 100425, | |
| "<loc_73>": 100426, | |
| "<loc_74>": 100427, | |
| "<loc_75>": 100428, | |
| "<loc_76>": 100429, | |
| "<loc_77>": 100430, | |
| "<loc_78>": 100431, | |
| "<loc_79>": 100432, | |
| "<loc_7>": 100360, | |
| "<loc_80>": 100433, | |
| "<loc_81>": 100434, | |
| "<loc_82>": 100435, | |
| "<loc_83>": 100436, | |
| "<loc_84>": 100437, | |
| "<loc_85>": 100438, | |
| "<loc_86>": 100439, | |
| "<loc_87>": 100440, | |
| "<loc_88>": 100441, | |
| "<loc_89>": 100442, | |
| "<loc_8>": 100361, | |
| "<loc_90>": 100443, | |
| "<loc_91>": 100444, | |
| "<loc_92>": 100445, | |
| "<loc_93>": 100446, | |
| "<loc_94>": 100447, | |
| "<loc_95>": 100448, | |
| "<loc_96>": 100449, | |
| "<loc_97>": 100450, | |
| "<loc_98>": 100451, | |
| "<loc_99>": 100452, | |
| "<loc_9>": 100362 | |
| } | |