interesting, what are the hardware requirements? does it happen to run on CPU on... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

DoctorOetker on April 1, 2023 | parent | context | favorite | on: Llama.cpp 30B runs with only 6GB of RAM now

interesting, what are the hardware requirements?

does it happen to run on CPU on a server with 96GB RAM?

w4ffl35 on April 2, 2023 [–]

the compiled app is meant for people to install and use with their GPU and runs on as low as a GTX 1080. I haven't tested against CPU only builds.

You can take a look at the source code and see if it would be useful to you.

Consider applying for YC's W25 batch! Applications are open till Nov 12.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact