Do all of your platforms have the same word width? The same -fshort-enums settings? Do you know that none of your data structures include pointers? Do all of your systems use the same compiler? Compiler version?
I agree it will usually work, but this becomes an ABI concern, and it's surprisingly common to have ABI mismatches on one platform with the items I've noted above.
I've seen wire protocols that had junk for the alignment buffering in such structs. And I've seen people have to do a whole lot of work to make the wire protocol work on a newer compiler/platform. Also, the whole point of a network protocol being documented is that it decouples the interface (msgs over a network) from the implementation (parsing and acting on msgs). Your v1 server might be able to just copy the read buffer into a struct, but your v2 server won't. And it is possible and desirable to live in a world where you can change your implementation but leave your interface alone (although some parts of the software ecosystem seem to not know this nice fact and implicitly fight against realizing it).
My issue with gRPC is simple, the Go gRPC server code does a lot of allocations. I have a gRPC service where each container does 50-80K/second of incoming calls and I spend a ton of time in GC and in allocating headers for all the msgs. I have a similar REST service where I use fasthttp with 0 allocs (but all the stupidly high number of connections due to the lack of multiplexing thru the connection).
Go's GC wasn't really made with throughput maximization in mind. It's a language that doesn't scale that well to take advantage of beefy nodes and has weak compiler. I suppose the Google's vision for it is to "crank the replica count up". gRPC servers based on top of ASP.NET Core, Java Vert.X and Rust Thruster will provide you with much higher throughput on multi-core nodes.
I agree it will usually work, but this becomes an ABI concern, and it's surprisingly common to have ABI mismatches on one platform with the items I've noted above.