I feel like these names convey a lot, though. When I see something like this, I immediately expect it to not do any of the following: (1) contain core logic or define primary classes/types/records; (2) have dramatic side effects; (3) be hard to test; (4) reference or depend on other parts of the codebase; (5) do anything controversial in general.
I’d say I’d expect a “utility” to adhere to these conditions more strongly than a “helper,” which might be a bit more entangled with application logic.
I'd also consider a Helper to be something that may have application-awareness. Whereas, as you stated, a Utility would have no serious dependencies and is otherwise "dumb".