It’s Not the GPUs, it’s the Software: A Guide to How AI Data Centers Create Value

In this issue of CIR DeepTech we begin with a story about crucial new technologies relevant data to the AI data centers, then we follow with an extended essay – the core of this iussue -- on the new trend of AI data centers pushing towards smart data centers, a trend that is making the software the main factor driving value in the AI data center. Here we tell this story from a policy perspective Then we conclude with some news deeper plans that CIR is making.

Priority Technologies at AEI. A Short Book Review: Policy analysts are strange beasts. I know this because I used to be one a long time ago, specifically the senior telecom policy guy at the Cato Institute. Policy analysts are part economist, part political scientist. If they are technology policy analyst, they are also part engineer. This breadth of competence is hard to achieve, and I have to say that, knowing all that was expected of me, I often became nervous.

Thus, it was a feeling of curiosity that I telecommuted to the American Enterprise Institute (AEI) last week to listen to the panel on “Priority Technologies: Ensuring US Security and Shared Prosperity.” Later I bought the book of the same name, which I think will be a useful addition to the bookshelves of any US data center manager or engineers. While not a data center book by any means, we think that the contents of this book will be useful to those in the data center biz, covering as it does much of deep tech that goes into today’s data centers. Also in this issue, we take a small break to discuss CIR’s latest report on quantum – specifically using quantum computing to design new and advanced materials.

Where does AI data value center reside? The chapters of Priority Technologies that will be of most interest to the data center community are those on quantum computing, and advanced manufacturing, but especially the chapter on semiconductors. In fact, these days it is tempting to think of the value in AI data centers as residing in chips, processors especially; the GPUs, XPUs, CPUs or whatever Pee-ews! that happen to in fashion this month. Its processors are the electronic brains inside what, after all, is an artificial intelligence platform. Meanwhile, NVIDIA, the most valuable company on the stock exchange, right now stands out as the little green processor giant. So, AI data center = Electronic Brain = Processors implies Value.

This is view from the present. It has not always been thus. Once upon a time, data centers (they weren’t AI data centers back then, of course) were valued in terms of square footage, cooling capacity, and physical security. Industrial age stuff drove value. The past is prolog; the fact that the value proposition of data centers has changed before, means they will – most probably – change in the future. More specifically, CIR thinks that data centers will be more about software than they have in the past.

The Growing Importance of AI Data Center Software

In saying this, I am adopting broad sense of the word software. I would certainly include LLMs and SLMs and applications layer stuff like that. I have said in several issues of this Substack that SLMs and, more generally, other smaller AIs may save energy, extravagances on cooling systems and perhaps even the need for high-speed interconnects.

But the coming software revolution is about more than the applications layer; much more. The value proposition increasingly embedded in the modern AI data center also involves (1)

orchestration software, (2) optimization engines, and (3) intelligence systems, which determine how efficiently every watt, byte, and cycle is used.

The link between value and software in the context of value is easily constructed. Imagine, two AI data centers that are identical in terms of the building and hardware used and that the weather at both locations is identical too. This would take a miracle of engineering of course, but let’s not go into that now.

Under the old rules the two offices would be predicted to have equal ROIs: equal economic outcomes. In practice, that would never happen, even with a miracle. This is because of hardware and building utilization. It is partly a design issue, but also utilization can no longer be considered static. Rather what will matter at AI data centers in the future are (1) how workloads are distributed across servers, (2) which customers get access to compute, (3) how failures are located and dealt with, and (4) how and where energy is consumed.

Today the hardware and, most surely the buildings for data centers, are big or bigger than ever – in the case of the buildings – perhaps as big as Manhattan island -- but they are also brainier, constantly adjusting themselves for survival and efficiency. That’s where the software comes in. This is also, of course, where the AI Luddite meme comes in too. In fact, he or she is already in plain view. For example, in older data centers, optimization depends on human operators. Skilled teams manually tuned systems, balance loads, and respond to anomalies. And real soon now, these skilled teams look like they will be replaced by AI.

Quantum Computers, Drugs, and Rock and Roll (Ok, Not Quite!)

I’ll get back to all this in a page or two below. Meanwhile, we interrupt this program to bring you an important announcement about the release of CIR’s new report Quantum Drug and Materials Design: Use Cases and Market Forecasts: 2026-2035.

Quantum computing is reshaping future pharmaceutical research and advanced materials design. This sector remains at an early stage; but with real-world demonstrations on quantum hardware underway, with leading pharmaceutical, chemical, automotive, and aerospace firms exploring how quantum systems can accelerate discovery, simulation, and optimization. And as mentioned, CIR has just published its latest report on this topic. You can find more about this report and link to get samples at https://www.cir-inc.com/product/quantum-chemistry-use-cases-and-market-forecasts.

In the report, we track the companies, technologies, partnerships, and algorithms driving this sector. Our research examines how both pure quantum and hybrid quantum-classical workflows are being used today for molecular/drug simulation, catalyst development, battery chemistry, computational fluid dynamics, next-generation materials engineering and other strategically vital areas.

Currently the market is evolving around integrated workflows that combine AI, HPC, GPUs, and quantum processors to solve highly complex chemistry and materials problems that challenge classical computing approaches. CIR’s coverage in this report includes:

· Quantum-assisted drug discovery and molecular modeling

· Advanced battery and fuel-cell materials research

· Catalyst simulation and chemical process optimization

· Semiconductor and electronics materials development

· Computational fluid dynamics and multi-physics simulation

· Carbon capture and sustainability applications

· Hybrid quantum-classical software platforms and algorithms

· End-user adoption strategies in the pharma, chemicals, aerospace, and energy sectors

This report also analyzes the competitive landscape of quantum hardware providers, specialist software firms, and enterprise adopters including IBM, Quantinuum, Google, Microsoft, IonQ, AstraZeneca, BASF, Pfizer, Airbus, Mercedes-Benz, and many others. In addition, the report contains a forecast of quantum computing in materials design and drug discovery with breakouts of the number of trials by size and expenditure levels.

Advanced materials and pharma are supposedly destined to be the first big application for quantum computing. This report delivers detailed market analysis, technology assessment, partnership mapping, and ten-year forecasts for the evolving quantum pharma and materials science ecosystem. For questions about the report and samples go to the report Web site or e-mail me at lawrence@cir-inc.com.

Towards Smart AI Data Centers

Now back to smart AI data centers. Here we note that GPU utilization, power efficiency, cooling optimization, network orchestration, and downtime reduction influence billions of dollars in data center infrastructure value. In this era of trillion-parameter models and hyperscale inference, even small operational inefficiencies can cascade toward massive economic consequences. AI data center intelligence platforms are emerging as the solution to this challenge. These platforms combine telemetry, orchestration, predictive analytics, machine learning, automation, digital twins, and realtime infrastructure optimization into operational framework. Increasingly, they function as the cognitive layer of AI infrastructure itself.

As a result of all this, hyperscale operators are investing heavily in intelligent operational infrastructure. Major cloud providers including Microsoft, Google, Amazon, Meta, Oracle, and emerging AI-native infrastructure firms such as CoreWeave and xAI are building increasingly sophisticated intelligence layers into their facilities. The core of AI data center intelligence systems follows a simple principle: continuous situational awareness. These data streams are collected through distributed sensors and fed into centralized analytics platforms. Modern AI data center intelligence systems increasingly rely on predictive analytics engines trained on operational patterns. These systems identify anomalies, forecast thermal conditions, anticipate hardware degradation, optimize workload placement, and correct actions before failures occur.

Predictive maintenance has become one of the most commercially valuable applications. Instead of relying on scheduled maintenance cycles or reactive repair models, intelligent systems monitor subtle changes in vibration, acoustics, power stability, thermal drift, and performance degradation to identify emerging hardware failures before they disrupt operations. Meanwhile, thermal intelligence has become a major focus area. AI clusters generate concentrated heat loads that strain traditional cooling architectures. Intelligence systems increasingly use reinforcement learning and real-time thermal simulation to optimize fan speeds, liquid cooling systems, coolant routing, and workload distribution.

The market remains early enough that no single dominant architectural model has fully emerged. This creates substantial opportunities for innovation.AI data center intelligence systems introduce dense layers of inter-dependency between software, infrastructure, automation, and machine learning systems. Cybersecurity becomes substantially more important. Intelligent infrastructure platforms represent attractive attack surfaces because they increasingly control physical operations, energy systems, cooling infrastructure, and compute orchestration. An attack on an autonomous AI infrastructure platform could have cascading operational consequences. The next frontier will involve self-healing AI clusters, autonomous workload placement, AI-driven thermal optimization, and – amusingly -- AI-managed AI infrastructure.

The explosion of AI workloads has made novel and powerful software not just important, but indispensable. Without advanced software layers, these workloads become inefficient, even unmanageable. But again, we are in a new era. The winners in the AI data center market will no longer be those with the most racks. Instead, they will be the ones whose systems can “think” about how to use those racks.

NVIDIA is number one: In the emerging AI data center software space, there is no doubt that the market leader is NVIDIA, which, of course should be no surprise. NVIDIA, for example, has introduced energy-optimized power profiles for its Blackwell B200 systems to improve efficiency in power-constrained environments. Schneider Electric has also announced AI infrastructure reference designs integrating power management and liquid-cooling. While today, NVIDIA chips are what gets the publicity, its long-term leverage is almost certain to come from software including CUDA, orchestration layers, cluster management.

And while, NVIDIA stands out, don’t forget about the hyperscalers, who looked at in one way can be thought of as turning turned data centers into something more like programmable environments. AWS, Google, Microsoft and the others can be thought of as merely renters of space. But this is a simple-minded way of thinking of them. The hyperscalers may rent space but they bring more intelligence in their wake.

Consider AWS – it builds systems that can spin up entire AI clusters on demand, scale them, monitor them, and dissolve them back into the ether. Google moves workloads across its “AI Hypercomputer.” Microsoft weaves everything into enterprise workflows, where identity, security, and AI infrastructure blur into a continuous fabric. And therefore, the companies we know from other sector and other eras. Schneider Electric and Vertiv are obvious exemplars. These companies don’t write AI models. They don’t train neural networks. They manage power and heat, which is replacing compute as the AI data center’s scarce resource. Software and intelligence once again.

However, a lot has yet to be sorted out here and I don’t want to leave you with the impression that the biggest names already have the lower layer software market, all carved up. In fact, as things stand now. Vendor-neutral orchestration platforms remain important because enterprises fear lock-in. When workloads are this dynamic, the infrastructure is no longer the product. The orchestration is.

Software for AI data center orchestration: Orchestrations platforms coordinate GPU clusters, networking, storage, power, cooling, workload scheduling, and AI lifecycle operations across the data center infrastructure. There has been cloud orchestration tools around for quite a while now, but these can no longer deal with the unique aspects of AI data centers such as multi-cluster GPU scheduling, GPU virtualization and fractionalization, energy aware scheduling, distributed training orchestration and inference optimization.

Many of the software vendors in this space are far from being household names but it should also be no surprise that once again we find NVIDIA as a top name in infrastructure orchestration, NVIDIA claims advanced orchestration can increase GPU availability “up to 10x” and improve workload scaling efficiency significantly. In the somewhat related AI infrastructure sector, we find Vertiv and Schneider as big names. Platforms such as Armada Bridge and Spectro Cloud explicitly position themselves as unified orchestration layers across cloud, edge, and AI deployments.

A major emerging category is orchestration of autonomous AI agents. Research systems like MegaFlow are designed to orchestrate tens of thousands of AI agents, distributed environments, simulation frameworks and agent-service coordination. In other words, orchestration software increasingly points towards a future where orchestration software manages not just GPUs, but fleets of autonomous AI processes.

And in a future era of AI factories, orchestration software will become the core operating system. From a metaphorical perspective, we expect to see orchestration software evolve beyond a niche product into what will be called the central nervous system of modern compute along with memory, networking, power, cooling, ultimately bringing all these areas under one roof.

Top of Form

Optimization engines: If any software is truly the control brain of AI data centers it is optimization engines. This software decides how to squeeze more training/inference functionality out of limited power, cooling, networking, and GPU capacity. High-density GPUs are pushing designs from traditional 10–20 kW racks toward 50–120+ kW, forcing liquid cooling, better power distribution, and tighter operational controls. As the Exhibit below shows optimize take place over several layers and impacts some network segments more than others; in particular there appear to be notable opportunities in the edge network.

Vistors to the new software-defined data center might still ask the old questions. How many megawatts? How many racks? Such questions will. still matter. Meanwhile, beneath it all, the data center is taking on a character of its own. Companies such as Arista, Cisco, and Broadcom build the connective tissue that allows thousands of GPUs behave like a single organism. In AI workloads, network latency and congestion can quietly destroy efficiency. So, software-empowered data centers network starts making decisions too. Even the pathways between machines are now software-defined.

Inside something remarkable has happened. The data center has learned to think and a handful of companies are teaching it how.

Conferences and Things

In other news CIR has decided to become a sponsor for “TEF 2026 – The AI Future: Ethernet.” This will be held October 7-8, 2026, in Mountain View, CA. More details next month. Also, look for announcements from CIR itself on our conference next year on emerging data center technologies.