👾 The Cost of AI: Breakdown of Investments in Training, Infrastructure and More

In recent years, artificial intelligence has not only achieved technological breakthroughs, but has also attracted massive financial investment. The development of powerful AI systems such as ChatGPT, Claude or Gemini has fundamentally changed the public perception of this technology. However, while the discussion often revolves around ethical issues, social impact or technical milestones, one key aspect is often ignored: the enormous costs associated with the development and operation of modern AI systems.

The financial dimensions of AI development are impressive. OpenAI is said to have spent more than 540 million dollars on training GPT-4 alone. Google has invested billions in its AI infrastructure. Anthropic has completed financing rounds worth billions of dollars, a significant proportion of which flows directly into the development and training of its AI models. xAI, on the other hand, has spent 3-4 billion dollars on the hardware for its supercluster alone. Not to mention Project Stargate, whose investment volume is estimated at $500B.

These figures raise important questions: Where exactly is the money going? Which cost factors dominate AI development? And what are the long-term economic implications for the industry, but also for companies and organizations that want to use AI technologies?
The working hypothesis of this article is therefore that the costs of AI development are distributed unevenly across various factors, with the training of models and the infrastructure required for this being the biggest cost drivers, while at the same time new business models are being developed to amortize these enormous investments.

The Cost Factor of Training: Why Teaching AI Models Costs Millions

Training modern AI models, especially large language models, is one of the biggest cost factors in the AI ecosystem This process requires not only enormous computing resources, but also specialized hardware, data and experts In principle, however, it can be assumed that more compute is spent and needed for inference than for training.

Overall, however, many AI companies remain silent when it comes to concrete figures. For example, we know from numerous reports what the training costs of GPT-4 were, but nothing about GPT-4.5 or the o-series. There are official figures about the inference costs of o3, for example, but nothing about how expensive the training of the reasoning model was.

Calculation Costs

The computational costs of training advanced AI models are remarkably high. Sam Altman, CEO of OpenAI, has publicly stated that the training of GPT-4 cost “more than 100 million dollars”. Other estimates go as high as 540 million dollars. In comparison, the training of GPT-3, the predecessor model, was estimated at around 4.6 million dollars - an increase of 20 to 100 times within a few years.

This exponential increase in costs follows a trend that AI researchers are observing: The computing power required to train state-of-the-art AI models doubles approximately every 3.4 months - a rate that is significantly faster than Moore's Law, which traditionally applies to computer hardware.

Concrete figures show the extent:

• GPT-4's training is estimated to have consumed 25,000 NVIDIA A100 GPUs over several months

• At current cloud prices of around 1-3 dollars per GPU hour for A100 GPUs, this equates to several hundred million dollars for GPU usage alone

• Anthropic's latest model Claude 3 took “tens of millions of dollars” to train, according to the company

Hardware costs

A significant part of the training costs is accounted for by specialized hardware, in particular graphics processing units (GPUs). NVIDIA dominates this market with its A100 and H100 models (and now H200), which are specially optimized for AI workloads.

The cost of this hardware is significant:

• A single NVIDIA H100 chip currently costs around 25,000 to 40,000 dollars

• A typical training system for LLMs consists of thousands of these chips

• Google has reported investing over 500 million dollars for its TPU v4 system (Tensor Processing Units)

• The costs for the cooling systems, power supply and other infrastructure must be added to this

This hardware also has a limited lifespan of around 3-5 years before it needs to be replaced by more powerful generations, leading to regular reinvestment cycles.

Data Costs

Although data costs are often lower than computational costs, they are still significant:

• Acquiring high-quality, curated datasets can be expensive (with ScaleAI, for example, we have companies dedicated to curating and creating datasets only)

• Licensing copyrighted content for training costs millions

• Manual annotation and quality assurance of data requires human labor

• Storing and managing petabytes of training data incurs ongoing costs

For example, Anthropic has stated that it cost several million dollars to create a high-quality, filtered dataset for training their Claude model.

Personnel Costs

An often underestimated cost factor is the highly qualified specialists required for the development and training of AI models:

• AI researchers and engineers with PhDs often earn annual salaries of 300,000 to over 1 million dollars

• A typical AI research team at large companies consists of dozens to hundreds of such specialists

• Talent acquisition and retention in this highly competitive field requires additional investment in the form of stock options and other incentives

Personnel costs alone for an AI research team can easily reach 10-20 million dollars per year.

Infrastructure Costs: The Foundation of AI Development

In addition to direct training costs, infrastructure costs are another significant cost factor that often receives less attention.

Data Centers

The physical infrastructure for AI development and operation is expensive:

• Building a modern, AI-optimized data center can cost anywhere from 500 million to several billion dollars

• The energy costs for AI data centers are enormous: a single training session for a large model can consume as much electricity as a small village over several months. Add to that the ongoing electricity costs of inference.

• Meta (formerly Facebook) has announced that it will invest a total of 9 billion dollars in its AI infrastructure by 2024

• Microsoft has made infrastructure investments of over 50 billion dollars for its Azure cloud, which hosts OpenAI's models, among others

Network Infrastructure

The network infrastructure for the distributed training and operation of AI systems requires

• High-speed network connections between GPU clusters

• Global network infrastructure for the delivery of AI services with low latency

• Redundant systems for high availability

Google, for example, invests several billion dollars a year in its global network infrastructure, which also supports its AI services.

Maintenance and Operation

The ongoing costs of maintaining and operating the AI infrastructure are also considerable:

• Energy costs: large AI data centers consume dozens to hundreds of megawatts of electricity

• Cooling systems: Modern GPUs generate enormous amounts of heat that need to be dissipated

• Maintenance staff: Technicians and engineers for 24/7 operation

• Security and compliance costs: Physical and digital security measures and compliance with regulatory requirements

A modern AI data center can incur operational costs of 10-20 million dollars per year, regardless of hardware costs

Research and Development Costs: The Innovative Core

Subscribe to Premium to continue reading.