Hacker News new | past | comments | ask | show | jobs | submit login

Recompile time is under a second for most models that I tried. Let's say 150 - 700 ms. Once you compile it, you can use it many times.

The diference for the inference time is in the post below (but YMMV).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: