The problem was not a few large objects, but large numbers of small objects eg thousands of orders per market. I considered using manual memory management but it would have required rewriting most of the code (it was almost all business logic). I sketched out a prototype in clojure to see whether a better GC would help and found that even with hastily written code and much abuse of persistent data structures, the G1 collector could keep the pauses under 100ms. I expect that carefully tuned code would reduce those much further.
