I do generally agree with your point here that this isn't very revolutionary. However it's worth pointing out that Apple chips have lots of on-package memory but relatively little on-die and they are quite different from this IBM chip because of that.
I know basically zero about chip fabrication but I remember reading somewhere a while (15÷ years) ago that processor-in-memory was always a desirable design objective for obvious reasons, but that there are fundamental differences in the process for fabbing memory versus logic (different regions of Si doping not possible in same wafer, something like that? See this is where I should stay out of these discussions) that haven't been resolved, so the next best thing is on package pairing.
Roughly speaking, CPU wants smaller space and RAM wants bigger space. At a high-level take:
* CPU design is the most expensive space due to it having the greatest quality and capability requirements. RAM is mostly just a very, very large repetitive structure, so more space better.
* A CPU fault can be potentially corrected by microcode changes to route around the damage (ie part binning). RAM cannot generally take faults.
* DRAM is simpler to make, "just" a capacitor, but capacitance leaks over time; which, means generating resistive heat in an area that we want as little heat as possible. You could use SRAM (two transistors) but now you have substantially more complex part to fail.
* DRAM quality requirements are much less stringent if you make just bigger cells.
