Amazon S3 - 905 Billion Objects and 650,000 Requests/Second

[jeffbarr]

http://aws.typepad.com/aws/2012/04/amazon-s3-905-billion-objects-and-650000-requestssecond.html

[cperciva]

Looks like my prediction on S3 hitting a trillion objects in Q2 is about right: http://news.ycombinator.com/item?id=2784320

[ars]

Note that the last column is for 1/4 of a year, not a full year like the other columns.

It initially looked like growth was slowing down. Amazon: You should fix that graph, it makes you look bad.

[jeffbarr → ars]

I will work on this tomorrow. Appreciate all of the feedback.

[ars → jeffbarr]

Hopefully you'll also get some feedback on the content, not just the presentation :)

Although the exponential growth is amazing to think about, so the graph is pretty important.

[Seth_Kriticos → ars]

For the graph: add more data points (each quarter) and make the scale logarithmic with a linear mean overlay trend line.

Also, I can only see indication of growth from that one chart. How about some more data (different charts):

- What size is it in total, average of the objects?

- Bandwidth (transfer, compressed)?

- How much is cachable (repeat and static requests)?

- How much data duplication is there?

- How is the data change rate (by objects, by volume)?

- How are average volumes and bandwidth change by account (behavior trend)?

- etc..

[pitzips → ars]

I think the Q4 columns are simply stating "at the end of Q4 this is what our number was"

Because Q4 of 2012 has not happened yet, we can't mark a number on the graph, so Q1 which has finished is the best we can do for this year.

That's my understanding of it.

[ars → pitzips]

I understand _why_ they did it. But it's still wrong.

The best option would be for them to put in the quarterly numbers for all years, then all bars would represent an equal amount of time.

[NyxWulf → ars]

They are all for 1/4 of the year, it's just that the other segments are for Q4 and that segment is for Q1.

[dpark → NyxWulf]

Not exactly. They are all labeled with a quarter. But the rest of them are spaced a year apart. The last bar is visually skewed because it's not consistent with the X-axis unit established by the previous bars.

[furtivefelon → NyxWulf]

No from one bar to the next for the previous years, the gap is 1 year (ie, Q4 2010, Q4 2011 -> 1 year), but from the second last bar to the last bar, the gap is 1 quarter (Q4 2011, Q1 2012 -> 1 quarter)

[cafeconleche]

Jeff these updates are allways very interesting thanks for this. When will you be sharing how much space all these objects are taking ;)

[ww520]

Do people ever delete objects from S3?

[cperciva → ww520]

I deleted about 750,000 objects from S3 earlier today.

[Terretta → ww520]

We deleted over 130 terabytes in March, so, yes.

[bitanarch → ww520]

It costs money. So I'd say, yes.