This is a great announcement. Preemptible VMs are a great strategy if:
* you don't care too much about when a workload finishes
* it can be chunked into small units of interruptible work
* the cost of resuming interrupted work is zero or small
Examples of workloads that might look like this:
* geocoding large batches of addresses
* map-reducing large event streams
* content scraping/crawling
Notice that you don't need to run all of your cluster as preemptible if you're using bdutil [0], too -- some of the workload can be preemptible, and some not. So you can guarantee a minimum processing throughput, but get extra throughput for very cheap.
I think that's a great way to do it, though I wish it was generalized for all kinds of clusters, not just for Hadoop ones.
Yeah, I added it to bdutil since it was clearly useful and yet well understood that it would do the right thing. I'd love to see more cluster and machine management tools deal with "machines can come and go".
The guys on the go team are using it for their buildbots (http://build.golang.org) which was already prepared for the box to die.
I think that's a great way to do it, though I wish it was generalized for all kinds of clusters, not just for Hadoop ones.
[0] https://github.com/GoogleCloudPlatform/bdutil