Guidance on ONNX / ONNX Runtime GenAI?
#11
by
EAFA0
- opened
Hi there,
I saw the ONNX tag on this model card but missed seeing any guides on how to use it.
Is there a plan for official support regarding ONNX Runtime GenAI? It would be great to finally run this on edge NPUs so they can stop being just glorified heating elements! 😂
Thanks!
Not all of them have been converted to onnx yet. However, I recommend that you use the full cpp reasoning we developed, which can run on the mac, or of course deploy on the user's own gpu machine.
We have compiled the complete documentation, including one-click installation, and how to use it here.
https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/demo/web_demo/WebRTC_Demo/README.md