It's not about doing it without a mouse, it's about doing it fast, comfortably, and powerfully. Pointers are a useful addition to graphic capable vi implementations, but if you do everything with mouse or trackpad movements then I don't think you can claim the zen of vi.
My fascination is that the original motions were not invented to escape the imprecision of graphical pointers, even though that's their main selling point today. They were invented to minimize (1) excessive tty printing, because printing is slow (2) keystrokes.
That, in 2020, these optimizations made in 1960 allow you to avoid the imprecision and slowness of a graphic pointer, we get for free :)
