My java/kotlin app needs to keep a big table fully in memory. (~10 million records). And it is reloaded about 3 times a day.
In C, I would just malloc the whole table in one chunk. Perhaps there is a specialized GC for this usage ?
There's no particular limit to JVM, you can use all available memory if you like.
For best performance you can decompose your structure to primitive fields (int, float, char, etc) and create array for every field. So you have, say, 10 arrays with million items each. Instead of creating one array which holds pointers to another 10 million objects on the heap. It gets tricky with strings (you need to flatten all strings into a giant char[] array and keep two arrays with index and length data, but doable.
Though 10 million of records might be OK for JVM. Measure your GC times.
I'd suggest to hide implementation details behind API, start with ArrayList<MyRecord> and refactor it later if needed.
My java/kotlin app needs to keep a big table fully in memory. (~10 million records). And it is reloaded about 3 times a day. In C, I would just malloc the whole table in one chunk. Perhaps there is a specialized GC for this usage ?