What a Data Center Actually Is
AI didn't just make computers hungrier. It made heat the architect — first of the rack, then of the building, then of the map. “AI needs more compute” really means: AI needs more power, plus a way to
A Data Center Is Not an IT Building Anymore
The claim
For forty years, a data center was an IT building that happened to use electricity. That era is ending. A modern AI data center is a power-and-cooling building that happens to contain computers. The software didn’t force that flip. The heat did. Here’s the proof.
The proof
1. The racks got ten times denser. A rack is the cabinet that servers sit in — a metal frame about the size of a large fridge. For decades, a full rack drew 10 to 20 kilowatts. A kilowatt is simply how fast something uses electricity; a typical American home averages a little over one kilowatt across a day. So an old server rack used about as much power as ten to twenty homes.
Nvidia’s current AI rack — the GB200 NVL72 — draws about 120 kilowatts, and low-130s under full load, according to HPE. Same floor footprint as the old cabinet, roughly eight to ten times the power. That is a hundred homes’ worth of electricity flowing into one fridge-sized cabinet. (Nvidia; HPE, 2024–26.)
2. All of that power becomes heat, and air can’t carry it anymore. There is a rule of physics no computer escapes: nearly every watt of electricity that goes in comes back out as heat. For forty years, fans and moving air handled the job, because 10–20 kilowatts per rack is within what air can carry. At 120–140 kilowatts in the same space, it isn’t. Air physically cannot move heat off the chips fast enough. So cold liquid is now piped directly against the chip. Nvidia and the Open Compute Project treat liquid cooling at AI density as a requirement, not an upgrade (Nov 2024).
And once liquid enters, the building changes. A single AI rack weighs about 1.36 tonnes — roughly 3,000 pounds — so floors, doors, and plumbing are re-engineered around it. Berkeley Lab describes a data center as four physical layers: the servers that compute, the storage that remembers, the network that connects, and the infrastructure that powers and cools the other three. In an AI facility, that fourth layer is now the main event. It decides whether the building can host AI at all.
3. The electricity data shows the flip already happened. US data centers used about 176 terawatt-hours in 2023 — 4.4% of all US electricity — up from 58 TWh in 2014 (DOE / Berkeley Lab, Dec 2024). Look at the timing: consumption was flat at roughly 60 TWh through 2016, then turned upward from 2017 — exactly when GPU-accelerated servers entered the fleet. What is known: use tripled in nine years, and the turn matches a change in hardware. What it suggests: this growth is structural. It tracks server architecture, not the economy.
What it means
The limit on AI moved. The chip is no longer the scarce thing — you can order AI hardware and install it within months. The scarce thing is a location that can deliver enormous amounts of power and absorb enormous amounts of heat, at the same spot, at the same time.
In the US, a new power project now waits a median of about five years from grid-connection request to operation (Berkeley Lab, Queued Up: 2025). To be precise: that five years measures power plants joining the grid, not a data center’s own hookup — but if new chips need new power, they are on that clock. That is why “where” now rivals “how much.” The same rack can power up in months in one region and wait years in another. So when you hear “AI needs more compute,” translate it: AI needs more power, in a place that can also take the heat away.
The counterargument
“Efficiency will save us — it did before.” Partly right, and worth taking seriously. From 2010 to about 2016, computing grew enormously while data center electricity stayed nearly flat, because efficiency gains and the shift to hyperscale facilities absorbed the growth. Two honest replies.
First, that flat era ended for a physical reason, not a lack of effort. Efficiency per computation is still improving — but AI raised the power packed into each rack by roughly ten times, and better efficiency now shows up as more computing per building, not less power drawn by it.
Second, keep the scale honest: globally, data centers still use only about 1.5% of the world’s electricity (IEA, Apr 2025). The global total is not the problem. Concentration is. The US carries 45% of that load, clustered in a handful of grid regions, and the strain is local — this substation, this county, this watershed. US data centers directly consumed about 17 billion gallons of water in 2023 (Berkeley Lab). The fact: a small share of world electricity. What it suggests: national averages will keep saying everything is fine while individual grid regions run out of room.
The chip used to decide what a data center computes. Heat now decides what a data center is. So the more interesting map of the next decade isn’t where the best models get trained — it’s which places can hand a building a hundred-plus megawatts and take the heat back. Which places would you put on that map?
Sources
IEA, Energy & AI (Apr 2025) · DOE / Lawrence Berkeley National Lab, 2024 United States Data Center Energy Usage Report (Dec 2024) · LBNL, Queued Up: 2025 Edition (emp.lbl.gov) · Nvidia GB200 NVL72 documentation (developer.nvidia.com) · HPE (2024–26) · Berkeley Lab via Pew Research (water, 2024–25).