The next release of Caddy will support push, yes. But you'll have to configure it (for now - until we can figure out a good way to do it correctly, automatically).
Well, Caddy doesn't even support push until the next release. But even then, the Go standard lib will have to proxy push frames through. We could look into a contribution to Go or a workaround when we or a contributor has time.