What time sensitive operation requires you to transform byte arrays into objects? I assumed something like reading from disk or network, and then CPU cycles are usually not the bottleneck.
Anyway, no doubt that you can write faster code in C, but I am glad that I did not have to look at pointer arithmetic for the last 8 years. Using your array->struct technique would be a major headache if the struct get's more complex.
There are plenty of other practical use cases for this kind of control. For example, casting an array of byte[]s to an array of int[]s.
It's not hard to convert using a ByteBuffer, but its still annoying. You and up duplicating a ton of memory for no good reason.