> using one buffer for multiple strings is just a type specific arena allocator.
Sure, but there's a lot of advantages of entirely controlling memory allocation in-process.
> this approach is going to be EXTREMELY wasteful.
Could you elaborate why? Sure, you have to do a bit more juggling in-process, but I would imagine this would be potentially more efficient than having conventional mallocs.
In the environment it is designed to operate in, there is usually a fixed memory limit per process, so you might as well grab all that up front.
> I would imagine this would be potentially more efficient than having conventional mallocs.
Yes, in our experience, static allocation means we use sometimes 10x less memory. For example, TigerBeetle's storage engine can theoretically address up to 100 TiB with less than 1 GiB of RAM for the LSM in-memory manifest, which is efficient.
Because we think about memory so much upfront, in the design phase, we tend not to waste it.
Sure, but there's a lot of advantages of entirely controlling memory allocation in-process.
> this approach is going to be EXTREMELY wasteful.
Could you elaborate why? Sure, you have to do a bit more juggling in-process, but I would imagine this would be potentially more efficient than having conventional mallocs.
In the environment it is designed to operate in, there is usually a fixed memory limit per process, so you might as well grab all that up front.