It’s hard to overstate how decisively Nvidia has broken above its historical growth trajectory.
While Amazon grew most recent quarterly revenue by 14% year over year, Apple by 16%, and Microsoft by 18%, Nvidia grew it by 265%. That’s not a typo, and the company’s net income grew 772%, to $12.29B from $1.41B. As the investor relations page puts it, Nvidia’s work “is at the center of the most consequential mega-trends in technology.”
No kidding.
Founder and CEO Jensen Huang said in the quarterly press release: “Accelerated computing and generative AI have hit the tipping point. Demand is surging worldwide across companies, industries, and nations.”
Its Nvidia RTX platform, which brings AI-enhanced graphics to designers and artists, making short work of real-time photorealistic rendering in video and image processing, has reached a user base of 100 million in less than six years. AI and other computing purposes are proliferating rapidly into every corner of life, from the usual data processing of corporations to the high-tech dashboards of automobiles. The latter is a so-called vertical industry, as are financial services and healthcare, and Huang said that the data center business for such customers has reached “a multibillion-dollar level.”
On the earnings call, CFO Colette Kress seconded the tipping point characterization:
The world has reached the tipping point of a new computing era. The $1T installed base of data center infrastructure is rapidly transitioning from general purpose to accelerated computing.
As Moore’s Law [observation made by former Intel CEO Gordon Moore in 1965 that the number of transistors in an integrated circuit doubles about every two years] slows while computing demand continues to skyrocket, companies may accelerate every workload possible to drive future improvement in performance, TCO [total cost of ownership], and energy efficiency.
At the same time, companies have started to build the next generation of modern data centers, what we refer to as AI factories, purpose-built to refine raw data and produce valuable intelligence in the era of generative AI.
Her comment about TCO highlights the importance of optimizing not just for initial purchase price or performance but for the overall cost and efficiency of operating technology plans, especially as demands on computing continue to grow.
Nvidia estimates that 40% of its data center revenue over the past year came from AI inference. This marks a new step in AI’s evolution, as inference moves from merely aggregating and repackaging information on which it was trained, to making predictions and decision based on new, unseen data. It’s a chance for the model to apply what it learned — the patterns, insights, relationships, and so on that inform its independent “thinking” — to fresh information.
Such inference demands enormous computing power, which is why Nvidia and other companies in the semiconductor industry, are benefiting. Whether AI training and inference happen on cloud servers or edge devices, they need chips. (Edge devices used to be called client devices, in the days of client/server architectures, but the new term “edge” includes bits of infrastructure and computing processes located closer to user devices even if not onboard them.)
The efficiency, speed, and cost of inference is a crucial consideration as it grows in importance because AI applications are spreading like wildfire and customers demand real-time interactions. You may have found yourself growing tired of waiting for a response from ChatGPT or DALL-E. If it’s an issue for personal usage, imagine how much bigger an issue it is at larger scales.
Thus companies are fixated on TCO these days, seeking chips that perform faster at lower power consumption. Nvidia and the rest of the chip industry are racing to improve TCO. Just look at Kress’s rundown, from the call:
Building and deploying AI solutions has reached virtually every industry. …
In the fourth quarter, large cloud providers represented more than half of our data center revenue, supporting both internal workloads and external public cloud customers. …
Copilot for Microsoft 365 adoption grew faster in its first two months than the two previous major Microsoft 365 enterprise suite releases did. Consumer internet companies have been early adopters of AI and represent one of our largest customer categories. Companies from search to e-commerce, social media, news and video services, and entertainment are using AI for deep learning-based recommendation systems.
I wish they’d hurry up. Amazon still doesn’t understand what books I like to read, Netflix still hasn’t cracked the code of my movie taste, and Pandora still can’t accept that my liking a specific song does not mean it should play everything ever created in the genre. There are more misses than hits among this bamboozled cohort. Come on, AI!
Kress presented other examples, including Meta using AI to boost the effectiveness of its advertising platform by better predicting user reactions, enterprise software companies relying on it to improve productivity, not to mention the generative AI makers themselves — “Anthropic, Google, Inflection, Microsoft, OpenAI, and xAI are leading with continued amazing breakthroughs in generative AI.”
It’s a thrilling time, a new gold rush of sorts, with chipmakers selling the picks and shovels.
As Nvidia looks to triple production and expand its global footprint by partnering with companies in Vietnam and Malaysia — notably not China — Taiwan Semiconductor (TSMC) continues playing its lead role in manufacturing.
It sits at the forefront of the industry, now cranking out 3-nanometer chips at a gross margin of 53% and a net income margin of 41%. Anything 7-nm and under is considered advanced, and 3-nm is the most advanced at the moment. Japan is working on a new factory to churn out 2-nm chips. The improvement will continue forever, of course.
On TSMC’s call last month, CFO Wendell Huang said that 3-nm chips accounted for just 6% of revenue last year, with 5-nm and 7-nm bringing 35% and 17%, respectively. The new era remains young.
Other trends are emerging.
The internet of things (IoT) declined last year to just 5% of revenue, while automotive usage grew 13%. IoT has been somewhat of a bust, this idea of networking various physical objects embedded with sensors and software for the purpose of connecting and exchanging data with other devices and systems over the internet. People don’t seem all that interested in refrigerators that announce when milk is running low, and so on.
In their cars, however, which serve as homes away from home, everyone wants total connectivity and as much hands-free help as they can get on the way to fully autonomous driving. This is great news for chipmakers, and should continue for a while as the automotive industry transitions to electric vehicles. Transportation will come to resemble internet cafe spaces.
On the call, CEO C.C. Wei noted that 2023 marked the major emergence of generative AI applications “with TSMC as a key enabler.” He expects this year to bring healthy growth to TSMC, driven by strong demand for 5-nm chips and the ramp-up of 3-nm usage from “robust AI-related demand.” The following comment from him will sound familiar:
The surge in AI-related demand in 2023 supports our already strong conviction that the structural demand for energy-efficient computing will accelerate in an intelligent and connected world.
TSMC is a key enabler of AI applications. No matter which approach is taken, AI technology is evolving to use more complex AI models, as the amount of computation required for training and inference is increasing. As a result, AI models need to be supported by more powerful semiconductor hardware, which requires use of the most advanced semiconductor process technologies.
Thus, the value of TSMC’s technology is increasing and we are well-positioned to capture the major portion of the market in terms of semiconductor components in AI. … Almost all the AI innovators are working with TSMC and we are observing a much higher level of customer interest and engagement at 2-nm as compared with 3-nm …
ASML Holding makes photolithography machines used to produce semiconductors. It’s the largest supplier to the industry, and the only supplier of the extreme ultraviolet lithography (EUV) machines needed to make the most advanced chips. It’s a Dutch multinational, the ASML of its name coming from its original monicker, Advanced Semiconductor Materials Lithography.
On ASML’s earnings call last month, CEO Peter Wennink said the company sees this year as one of transition “in preparation for the expected strong demand in 2025.” Growth is being driven by “AI-related demand” and improving inventory levels. “Litho demand is primarily driven by DRAM technology node transitions in support of advanced memories such as DDR5 and HBM in support of AI-related demand,” he said.
This offers an excellent look at how the rising AI tide lifts all boats.
DRAM is dynamic random access memory, a type of semiconductor memory used in computers, smartphones, and servers for temporary data storage. Every one of your devices uses it. It’s called “dynamic” because it needs to be refreshed with electricity thousands of times per second in order to retain the stored information.
DDR5 is double data rate 5 synchronous DRAM, the fifth generation of DDR memory. It’s speedier and more efficient than its predecessor, DDR4. It meaningfully improves device performance, and is therefore of particular interest to developers of voracious AI applications and machine learning tools, both of which require rapid data processing.
HBM is high bandwidth memory, even faster than traditional DDR. It’s stacked vertically, which enables greater speeds at lower cost, a theme you’ll recognize from above as being at the heart of total cost of ownership. No surprise as well, HBM is particularly beneficial where high-speed data access is necessary, such as graphics processing and — wait for it — AI computations.
The world is building more semiconductor fabrication plants, partly to reduce economic impact in the event of a Chinese invasion of Taiwan, and partly to outfit the AI gold rush. According to electronic supply chain research firm Z2Data, there are currently 73 new chip fabs in construction around the world. Of these, 23 are expansions of existing fabs, most for the purpose of making more low-end 28-nm chips, but four of the expansions will make advanced chips:
Samsung in South Korea will make more 4-nm and 5-nm chips, in two expansions,
TSMC in Arizona will make more 4-nm and 5-nm chips, in one expansion,
Intel in Ireland will make more 7-nm chips, in one expansion.
The other 50 fabs will be brand new, 21 of them in the United States. Others are being built in China, Japan, South Korea, and Taiwan. Three-quarters of the plants will come online in the next three years. The advanced-chip leaders remain Samsung and TSMC, but Japan’s new Rapidus plant will make 2-nm chips starting in 2027, and 1-nm chips after that. Intel, too, is coming on strong.
Through these efforts, semiconductor manufacturing capacity will get an enormous boost this decade, and sources will expand, reducing dependence on Taiwan. Even TSMC is expanding capacity beyond its home shores, to Japan and the United States.
AI took the spotlight in reports from many industry participants, if not every one. AMD, Intel, Lam Research, Qualcomm, Texas Instruments, and others are moving quickly to meet AI-driven demand.
Intel’s efforts are particularly notable. The company was already in the midst of an impressive turnaround plan under CEO Pat Gelsinger’s stewardship, starting from when he took the reins in February 2021, then AI showed up. Overdrive shifted into hyperdrive.
On last month’s earnings call, Gelsinger was proud of Intel becoming in 2023 “the world’s first high-volume manufacturer of logic devices using EUV, both in the US and Europe, as we aggressively ramped Core Ultra on Intel 4 in both Oregon and Ireland.”
The company’s “five nodes in four years” plan, or 5N4Y, aims to catch up with TSMC. So far, the company has delivered two of the five nodes, Intel 7 (10-nm) and Intel 4 (7-nm). The company defines industry leadership as the highest performance per watt of power consumption, harkening back to the push for a lower total cost of ownership. Intel 3 will remain 7-nm, but with an 18% improvement in performance per watt. It will target enterprise users, not consumer devices.
Later this year, the company will roll out Intel 20A and Intel 18A chips, which will improve performance per watt over Intel 3 by 15% and 10%, respectively.
Gelsinger trumpeted Intel being the “first in the industry to have incorporated both gate all around and backside power delivery in a single process node, the latter unexpected two years ahead of our competition.”
Gate all around refers to a Gate All Around Field-Effect Transistor, or GAAFET, something that TSMC is moving to as well. It’s an advanced design using a gate material, such as a metal or metal compound that offers good conductivity, to surround the chip’s channel region on all sides and allow for better control of the current flow through the transistor, which improves efficiency and reduces current leakage. It furthers the cause of endless miniaturization and the all-important total cost of ownership.
Backside power delivery, which Intel dubs PowerVia, supplies power to the chip through connections on the back side of the silicon wafer, instead of along the traditional route through the top side, where the transistors are located. As with every other recent innovation, backside power delivery is more efficient. It reduces electrical resistance.
Gelsinger said AI is driving growth at Intel Foundry Services (IFS):
The rapid adoption of AI by all industries is proving to be a significant tailwind for IFS, as high-performance compute — an area where we have considerable wafer and packaging know-how, and IP — is now one of the largest, fastest-growing segments of the semiconductor market. …
The momentum in advanced packaging is very strong and is another facet of our foundry strategy, which is clearly benefiting from the surge of interest in AI. With leadership technology and available capacity, our opportunity set continues to grow. …
Intel continues its mission to bring AI everywhere. We see the AI workload as a key driver of the $1T semiconductor TAM [total addressable market, or maximum available revenue opportunity] by 2030. And given our foundry and product offerings, we’re the only company able to participate in 100% of the TAM for AI Silicon logic. …
Within our product portfolio, we are the only company with the products, IP, and ecosystem reach to empower customers to seamlessly integrate and effectively run AI in all their applications from the cloud through the network, into the enterprise client and edge. … As AI proliferates and the world moves towards more AI integrated applications, there’s a market shift toward local inferencing and smaller, more nimble models.
Last week, Intel announced that it will make chips for Microsoft in a bid to supplant Samsung and TSMC in the next era of producing silicon for AI. Gelsinger said Intel is ready to “rebuild Western manufacturing at scale,” with a nod toward helping Washington address concerns about China invading Taiwan and the desire to reshore cutting-edge manufacturing in the US. Intel wants to see half of the world’s semiconductors made in the US and Europe within a decade, up from 20% today. Microsoft’s first chip will use Intel’s 18A node.
Gelsinger makes no effort to hide his ambition: “Through our foundry, I want to manufacture every AI chip in the industry.”
AI is bursting every seam, and the semiconductor industry is racing to keep up.
At an Intel company event last week, US Commerce Secretary Gina Raimondo said via video feed that companies building generative AI will need a “mind-boggling” volume of semiconductors in the years ahead, and that demand for the most advanced chips “is just going to explode.” She called Intel America’s “champion chip company” — not to be confused with Hershey.
Coming out of a funk, semiconductor stocks were primed for recovery. Here’s how they’ve done over the past twelve months, along with sector ETFs:
12-Mo Price Change (%)
– – – – – – – – – – – – – – –
235 NVDA
124 AMD
74 SMH (VanEck Semi)
73 INTC
57 SOXX (iShares Semi)
56 SOXQ (Invesco Semi)
50 ASML
48 TSM
12 XSD (SPDR Semi)
Nvidia leads the AI chip charge, but is not alone in this gold rush. These are early days in AI’s tectonic shift of the landscape, which unearthed plenty of riches to go around.
Thank you for reading!
SOURCES
The Kelly Letter
Current Performance
Nvidia
Investor Relations
Taiwan Semiconductor
Investor Relations
ASML Holding
Investor Relations
Z2Data
9 Key Statistics on New Semiconductor Fabs Being Built Around the World
Intel
Investor Relations
Financial Times
Intel to manufacture chips for Microsoft as AI drives demand