Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
dislikedtom2
on March 31, 2023
|
parent
|
context
|
favorite
| on:
Llama.cpp 30B runs with only 6GB of RAM now
1. turn the swap off or monitor it closely 2. try to load a big model, like 65b-q4 or 30b-f16 3. observe the OOM - It's not so hard to test this.
gg82
on April 1, 2023
[–]
Using a memory mapped file doesn't use swap. The memory is backed by the file that is memory mapped!
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: