gRPC connections are persistent, so a single client will always talk to a single backend in that configuration. It's fine if you have lots of clients but can easily lead to load imbalances.
That's why projects such as Envoy exist. I'd link it, but I'm on mobile.