What nobody is talking about here is that there is no more power available in the US. All the FAANGS have scooped up the space and power contracts.
You can buy all the GPUs you can possibly find. If you want to deploy 10MW+, it just doesn't exist.
These things need redundant power/cooling, real data centers, and can't just be put into chicken farms. Anything less than 10MW isn't enough compute now either for large scale training and you can't spread it across data centers because all the data needs to be in one place.
There are datacenters that are specializing in this, and they exist today.
I highly recommend Colovore in Santa Clara. They got purchased by DR not too long ago, but are run independently as far as I can tell. Their team is great, and they have the highest power density per rack out of anyone. I had absolutely no problem setting up a DGX cluster there.
With things like solar and wind installs becoming more off-the-shelf, is there any path there? What does 10MW of solar/wind look like? Are we talking the size of a big ranch or the size of a small county?
It doesn't matter, this stuff wants/needs to be deployed yesterday, not in 2-3 years. I'm starting with raw power, but even that isn't the limiting factor... it goes deeper... try to buy a bunch of large transformers, those are year long waitlists.
Texas has a lot of wind. At this scale, it is mostly grid power anyway. Grid is a mixture of everything. Oh and solar has this pesky issue of not working in the evening, so then you have another problem... storage. ;-)
I should add... you want backup generators for your UPS systems? Those are a 4.5 years backlog.
With AI training, unlike a lot of datacenter work, you have the option to "Make models when the sun shines" and just turn things off at night. That'll push you towards cheaper but less power efficient older node compute but I think the business case should work.
Right now in the US there's about as much proposed renewable production planned and awaiting permitting as their is currently installed. It's the grid connections that are the long pole in expanding renewable use right now. And since the voltage that a solar panel outputs is pretty close to the voltage a GPU consumes you've got some more savings there.
There are still a lot of challenges with that but in general I think people should be looking for ways to collocate intermittent production of various things with solar farms right now, from AI models to amonia.
Is battery storage not yet commonplace? There are gobs of options: pump water uphill, spin giant flywheels, etc. Picking a battery with the right tradeoffs for your situation is a crucial consideration, I would think. And I am a subject matter expert here, having played several hours of Cities: Skylines in my day. Which gives me an idea...
Let's click 6 wind turbines down off the coast, shove our H100s underneath them for water cooling, and ah...separate the water/oxygen into tanks for hydrogen power when it ain't blowy no more? Or something? Someone help me out here.
Grid-scale batteries are basically nonexistent in the US, but also aren't particularly common elsewhere. In 2016 there was only 160mW [0] of battery storage available to the grid. Battery prices have come down since then, but not enough for energy storage to make sense for utilities in a lot of cases. If capacity has doubled in the past seven years, the person you're responding to would still be asking for like 3% of available battery capacity nationwide.
As far as other storage methods, they're really cool but water and trains require a lot of space, and flywheels typically aren't well suited for storing energy for long amounts of time. That being said, pumped water is still about 10x more common than batteries right now and flywheels are useful if you want to normalize a peaky supply of electricity.
I'd like to believe we'll see more innovative stuff like you're suggesting, but I think for the time being the regulatory environment is too complicated and the capex is probably too high for anyone outside of the MAMA companies to try something like that right now.
That is a "chicken farm". Nobody is going to deploy tens of millions in GPUs to something like that, let alone run their tens of millions worth of training data on that.
I'm sort of assuming this is like most other booms and will last several years. Sure the best time to be ready to jump in was yesterday, the the second best time is to get the ball rolling now even if your lead time is years.
And yes obviously renewables won't cover 24/7 but if I have a choice between no data center and a 60% time data center.. give me the 60%.
There's still space, but AI startups are doing the scooping. At least when they fail there will be some nice pre-built datacenter cages for people to move into.
I have never had to even think about the steps a firm with massive utility requirements would need to take to secure supply. So assuming you could wave a magic wand and instantly build out a datacenter in northern Virginia right now, the local power utility (Dominion Energy in this case) would not be able to provide power?
It isn't like you can snap your fingers and magically transport 10MW+ of power to your doorstep. Plus, as I said in other threads, it isn't just power... it is everything around supporting that power. Try ordering a transformer. Or getting EPA approval to install backup generators.
You can buy all the GPUs you can possibly find. If you want to deploy 10MW+, it just doesn't exist.
These things need redundant power/cooling, real data centers, and can't just be put into chicken farms. Anything less than 10MW isn't enough compute now either for large scale training and you can't spread it across data centers because all the data needs to be in one place.
So yea... good luck.