Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
brucethemoose2
on March 31, 2023
|
parent
|
context
|
favorite
| on:
Llama.cpp 30B runs with only 6GB of RAM now
Yeah, or deepspeed presumably. Maybe torch.compile too.
I dunno why I thought llama.
cpp
would support gpus.
shrug
sp332
on April 1, 2023
[–]
Lots of C++ programs use the GPU. It's irrelevant.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
I dunno why I thought llama.cpp would support gpus. shrug