Yes, you can access blobs too. I've been using the HTTP API (the one that backs CloudKitJS) to process data in app engine. So far I just used the JS API token for authentication. Records with an associated blob asset provide a temporary accessible URL to the asset which is usually stored on S3.
Not sure about latency of the requests, never had a problem with that. More often the problem was that I had confirmation my client software had written data to CloudKit but it might take a while before the server can actually query for it. (Could be an indexing delay haven't checked if the object is immediately available using the object ID)
Are we talking 15ms range? 150ms range? 600ms-1s range?
Also, can we access binary blobs/files/images via this API? It seems the max file size of an asset field is 15mb, which is smaller than a Raw photo.