A question about inference
#20
by
JohnsonWythe
- opened
Thanks for this amazing job!
I got one question for inference. I tried sample code for inference and it has stuck for several minutes. Is it Normal?
It depends on your hardware. If the model is larger than your VRAM, your PC might be swapping to System RAM, which can make it look stuck for several minutes. What are your GPU specs?