The AI Stack, Explained Simply
Think of it like building a city. Every city needs foundations, power, roads, buildings, and people using them. AI is the same, just faster and more expensive.
This post is a high-level summary of AI Ecosystem Academy, a new interactive guide I have built that maps where the money in AI actually goes. The full site breaks down ten layers, profiles the key public and private companies in each, includes a glossary and quiz, and goes deep on the questions people actually ask. If this summary whets your appetite, explore the full ecosystem at aiecosystem.academy.
The full picture at a glance
Seven core layers, plus three cross-cutting forces that wrap around everything. The layers depend on and reinforce one another, but the architecture is not perfectly linear: security, governance and demand are not steps in the sequence, they cut across everything.
- The Sand: chips, memory and manufacturing
- The Power: generation, grid, electrical and cooling
- The Land: hyperscale, colocation and AI clouds
- The Roads: moving data between processors
- The Libraries: warehouses, lakehouses, streaming and search
- The Brains: frontier labs and model platforms
- The Shops: products, workflows and agents
Plus the cross-cutting layers: The Guards (security), The Rules (regulation, sovereignty and governance) and The People (who actually pays for all of this).
Read it bottom to top: each layer only exists because the ones beneath it do. A request travels up the stack when you ask an AI a question, and value flows back down as revenue.
Layer 1: The Sand - Semiconductors & Manufacturing
AI can run on conventional processors, but specialised accelerators make large modern workloads dramatically more efficient. Design and manufacturing are separate businesses: NVIDIA and AMD design the chips, TSMC runs the factories, and ASML builds the lithography machines that print the patterns. High bandwidth memory from SK hynix, Micron and Samsung has repeatedly gated how many AI systems can actually be built, because a large model must stream billions of parameters for every token it produces. And the biggest buyers (Google, Amazon, Microsoft, Meta) are now designing their own custom silicon rather than renting someone else's cost structure.
Layer 2: The Power - Energy, Grid & Cooling
AI's infrastructure problem is increasingly not whether GPUs can be bought, but whether they can be powered and cooled. Compute is electricity in disguise: generation feeds the grid, the grid feeds the data centre, and the heat has to be rejected somewhere. In several markets, securing power is now harder than securing chips, and connecting a new gigawatt-scale site to the grid often takes longer than building the site itself.
Layer 3: The Land - Data Centres & Cloud Infrastructure
The buildings, campuses and cloud platforms where AI actually runs. Hyperscalers (AWS, Azure, Google Cloud), neoclouds like CoreWeave, colocation providers and converted crypto-mining sites are four different businesses with very different economics. The miners matter because they already hold the scarce ingredient: power contracts and energised sites that would take years to permit from scratch.
Layer 4: The Roads - Networking & Interconnect
Training and inference mean moving staggering volumes of data between processors, racks and sites. Switches, optical interconnect and networking silicon determine whether thousands of expensive accelerators actually work as one machine or sit idle waiting for data.
Layer 5: The Libraries - Data & Data Infrastructure
Models are only half the system. Warehouses, lakehouses, operational databases and vector search decide whether an answer is grounded in your business or in someone else's. Retrieval turns a general model into one that knows about your company, and freshness matters: agents acting on yesterday's data cause expensive mistakes.
Layer 6: The Brains - Models & AI Platforms
The organisations training foundation models (OpenAI, Anthropic, Google DeepMind, xAI, Meta, Mistral) and the platforms that distribute them. Several of the most important are private companies, which is inconvenient for investors but does not make them less important. Training builds the model once, very expensively; inference is what everyone pays for, billions of times a day, and much of today's chip, power and data centre demand is driven by inference at scale.
Layer 7: The Shops - Applications, Agents & AI Software
Companies that use AI to deliver products people actually buy. The model is the engine; the application is the car: interface, workflow, data and a customer with a budget. This is where agentic AI lives, and it changes the stakes: a chatbot that is wrong wastes your time, but an agent that is wrong can take an action on your behalf, which is a different category of problem.
Across every layer: The Guards, The Rules and The People
Security and governance are not the ninth and tenth steps. More AI means more attack surface, and AI-powered attacks require AI-powered defence, so the security layer grows in direct proportion to everything else. Regulation and sovereign AI shape where compute, models and data may live. And underneath it all sits the commercial question: who actually pays, and is this a bubble? The honest position is that the ecosystem contains both durable businesses and speculative ones, and the market is not currently doing a careful job of distinguishing them. That is a reason to read the economics of each layer rather than treat AI as one trade.
Explore the full ecosystem
That is the map. The full version at aiecosystem.academy goes much deeper: interactive diagrams showing how a request, a dollar and a product flow through the stack, company-by-company breakdowns for each layer, a searchable glossary, a quiz, and straight answers to the questions people actually ask, from "Doesn't NVIDIA manufacture its own chips?" (no, it is fabless) to "Is this just another tech bubble?"