Hetzner mail to customers: 1 megawatt more power due to leap second

imaginator · on July 3, 2012

Better translation of their message to English speaking customers:

During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level of IT power usage by approximately one megawatt.

The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers.

According to heise.de, various Linux distributions are affected by this. Further information can be found at: http://www.h-online.com/open/news/item/Leap-second-Linux-can...

In order to reduce CPU load to a normal level again, a restart of the whole system is necessary in many cases. First, a soft reboot via the command line should be attempted. Failing that, you have the option of performing a hardware reset via the Robot administration interface. For this, select menu item "Server" and the "Reset" tab for the respective server in the administration interface.

Please do not hesitate to contact us, should you have any queries.

Kind regards,

Hetzner Online AG Stuttgarter Str. 1 91710 Gunzenhausen / Germany info@hetzner.de http://www.hetzner.com

imaginator · on July 3, 2012

just remember:

date -s "`date`" solves it.

No reboot, no stopping and starting NTPd.

hendi_ · on July 3, 2012

date -s "$(LC_ALL=C date)"

is the better option which also works on system with a locale != C

Uchikoma · on July 3, 2012

Thanks.

stavros · on July 3, 2012

No need for Google translate:

During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level of IT power usage by approximately one megawatt.

The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers.

According to heise.de, various Linux distributions are affected by this. Further information can be found at: http://www.h-online.com/open/news/item/Leap-second-Linux-can...

Please do not hesitate to contact us, should you have any queries.

metabrew · on July 3, 2012

Here's the requisite graph http://i.imgur.com/hsUDE.png

cellularmitosis · on July 3, 2012

/standard rant about the Y-axis not being anchored at zero, which will cause most people to be mislead.

UPDATE: here's an illustration. compare this: http://i.imgur.com/f3m25.png to this: http://i.imgur.com/6z2cS.png (those two graphs represent the same data)

Dylan16807 · on July 4, 2012

Really? Who is both going to understand leap seconds, and also look at that graph and think power consumption went up tenfold.

I'd much rather see the detail that zoom gives.

iand · on July 3, 2012

Look how spiky the power usage is normally. Is that all the cron jobs set to run at the top of the hour?

danielweber · on July 3, 2012

Maybe people should get discounts for running their cron jobs at the 50-minute mark.

dfc · on July 3, 2012

where did you get that data?

atesti · on July 3, 2012

Maybe from here: http://www.heise.de/newsticker/meldung/Schaltsekunden-Bug-in...

Heise.de got it directly from Hetzner, they acknowledge and thank them for providing it in the update of the article

ck2 · on July 3, 2012

If the leap-second does all this, just imagine the 32-bit rollover issue in 2038

forgotusername · on July 3, 2012

I'm imagining a whole bunch of bricked "classic" cars

leoedin · on July 4, 2012

Interestingly, because of significant changes in the way we manufacture cars, I'd imagine there will be far fewer classic cars around in 26 years.

A car today uses a few very large stamped steel components coupled with intricate plastic pieces. Maintaining a car for significant periods of time is far more of an issue than it was with much simpler cars of the past. When a small component in a modern car fails, you need to replace significantly more of the car with a significantly harder to manufacture piece to get it working again.

Perhaps 3D printing will provide us with a way to produce bespoke complicated pieces, but there's a long way for that technology to go before it's usable for steel car components.

Essentially, modern cars are considerably more reliable in the short term, but very, very hard to maintain in the long term. Classic cars in 26 years will primarily be very high end or unusual cars today (I can imagine kit cars, Rolls Royces, Ariel Atoms and other largely custom cars sticking around) and cars that we consider classics today. A car that's made it 30, 40 or 50 years today is likely to make it another 50.

daeken · on July 3, 2012

I really can't imagine why that would ever happen. What car would ever use a clock for anything essential, instead of a timer? Maybe I'm overestimating the intelligence of car developers, but it seems like the obvious, simple solution.

forgotusername · on July 3, 2012

Engine won't start, service due 300 years ago. I'd be willing to bet $10 there's at least a few floating around with real time clocks, no idea what they'd be used for though. If nothing else there's always in-car navigation and entertainment which are definitely candidates, although I suppose less likely to result in a brick.

mattdeboard · on July 3, 2012

Sorry, what?

forgotusername · on July 3, 2012

I'm engaging in what is commonly known as using one's imagination to make educated guesses at the possible electronic devices within an average motor vehicle (you know, that thing with wheels on it) where a real time clock susceptible to wraparound or overflow might be found, thus resulting in potential malfunction, or in plainer words, "the car won't start, the tuner won't play music, the GPS won't get a signal, this thing is about as useful as a brick".

mattdeboard · on July 3, 2012

I see. I have to apologize, it totally went over my head you were talking about cars considered classic in 2038, not the current day. Whoosh.

thwarted · on July 3, 2012

What car would ever use a clock for anything essential, instead of a timer?

You could say the same thing about the futex option that caused this issue in Linux in the first place. If the futex call specified a timeout in duration rather than an absolute time, this bug wouldn't have been hit (I observed the two different kinds of futex use during the leap second event).

etaty · on July 3, 2012

OVH https://twitter.com/olesovhcom/status/220142781895671810

efutch · on July 3, 2012

Perhaps nitpicking, but this is because of a bug in the leap second implementation, not in the leap second per se.

Uchikoma · on July 3, 2012

I'd guess it happens by the concurrence of a lep second and a leap second implementation bug. The bug without leap seconds would also not lead to problems.

imaginator · on July 3, 2012

Nice graph from Hetzner showing the spike:

http://imgur.com/a/ykoup

kodisha · on July 3, 2012

Huh? I received my mail in English :)