The freedom to reference anything easily from any place is a double edge sword. I agree it makes 5% of hard issues go away, but on the flip side it makes the other 95% more complex. Tracing GC is a "goto" of memory management. You may argue goto is a good thing because it offers you freedom to jump from anywhere to anywhere and you're not tied to constraints enforced by loops and functions. We all know this is not the case. Similarly being able to make a reference from anywhere to anywhere leads to programs that are hard to reason about. We should optimize for readability not the ease of writing.
There is no reason why you could not, in principle, have Rust-style compile-time borrow checking in a managed language.
As an extreme example (that I have occasionally thought about doing though probably won't), you could fork TypeScript and add ownership and lifetime and inherited-mutability annotations to it, and have the compiler enforce single-ownership and shared-xor-mutable except in code that has specifically opted out of this. As with existing features of TypeScript's type system, this wouldn't affect the emitted code at all—heap allocations would still be freed nondeterministically by the tracing GC at runtime, not necessarily at the particular point in the program where they stop being used—but you'd get the maintainability benefits of not allowing unrestricted aliasing.
(Since you wouldn't have destructors, you might need to use linear instead of affine types, to ensure that programmers can't forget to call a resource object's cleanup method when they're done with it. Alternatively, you could require https://github.com/tc39/proposal-explicit-resource-managemen... to be used, once that gets added to JavaScript.)