How powerful of a computer does this need? It would be useful to see, for one th... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

skykooler on March 13, 2023 | parent | context | favorite | on: Dalai: Automatically install, run, and play with L...

How powerful of a computer does this need? It would be useful to see, for one thing, minimum RAM requirements for these models.

spion on March 13, 2023 [–]

llama.cpp needs 40GB for the 65B model (due to int4 quantization)

RamNeeded(other_size) ~= 40GB * other_size/65B

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact