Amazon has produced a Chip for the server market and one for the AI market. Got this from Seeking Alpha

 

Intel's And Nvidia's Margin Is Amazon's Opportunity

 

Dec. 5, 2019 4:04 PM ET 

 

|

 

 About: Amazon.com, Inc. (AMZN), Includes: AMD, GOOG, GOOGL, INTC, NVDA, TSM

 

 

 

Summary

 

We discuss the significance of two recent Amazon announcements; about Graviton2 and Inferentia.

 

Graviton2 is a significant negative to Intel and has long-term margin implications for the server business.

 

Inferentia is a significant negative to Nvidia and has long-term implications for machine learning semiconductor market.

 

This idea was discussed in more depth with members of my private investing community, Beyond The Hype. Get started today »

 

Amazon (NASDAQ:AMZN), at its Re:Invent event, made two major announcements of consequence for the semiconductor industry. In one announcement, Amazon unveiled instances based on its new ARM-based Graviton2 chip - an alternative to x86 server chips from Intel (NASDAQ:INTC) and Advanced Micro Devices (NASDAQ:AMD). In another announcement, the company launched Inf1 instances based on its new AWS Inferentia chip – an alternative to the popular T4 inference chip from Nvidia (NASDAQ:NVDA).

 

The announcements are notable because Amazon, in a way, is announcing an open war against its suppliers by building its own alternative ICs to compete with its suppliers.

 

In a sense, it has always been this way. Amazon CEO Jeff Bezos has systematically gone after big margins wherever he sees them as is evident by his oft-quoted favorite aphorism: “Your margin is my opportunity”.

 

Amazon, which has internalized the CEO’s “Your margin is my opportunity” message, is constantly looking for opportunities in margins of vendors like Intel and Nvidia. Amazon AWS operations team led by Andy Jassy has found billions of dollars of opportunity by going after these vendors.

 

Sizing The Opportunity

 

It is no secret that Intel has historically commanded high margins in its x86 processor business and even higher margins in the x86 server processor business. While Intel does not disclose gross margins by product line, we estimate that Intel server CPU gross margins are around 80%. Note that, in the most recent quarter, Intel Datacenter business, which primarily revolves around the sale of the company’s server processors, had a *NET* margin of $3.1B on revenues of $6.4B. From Amazon's perspective, this staggering margin provides an opportunity.

 

How big is this opportunity?

 

Unfortunately, it is not possible to estimate accurately as the number of servers procured by Amazon is a well-kept secret. It is not just Amazon; even other cloud providers are reticent to provide much information on the installed base of servers or annual unit purchases. Over the years, estimates of the server market and Amazon share of the market have varied widely depending on the assumptions and methodologies used by various market research houses.

 

For the purposes of this article, as a first order estimation, we approximate the overall server market to be about 25M CPUs per year and Amazon share of it to be about a tenth of it at 2.5M CPUs. For the sake of discussion, assume that Intel sells 2.5M server chips a year on average to Amazon at $1,000 a pop. i.e. Amazon may be spending $2.5B a year on Intel processors. This, in turn, implies that Amazon may be providing $2B of gross margin to Intel.

 

The math on the inference side is also attractive to Amazon, given the high margins that Nvidia commands. However, the size of the inference opportunity is an order of magnitude lower, given the relatively small size of the AI/ML market compared to the CPU market.

 

Graviton2

 

In spite of the large CPU opportunity, until recently, customers like Amazon had no choice but to buy from Intel. Intel had a strong process and design leadership in CPUs and had kept its x86 solutions ahead of rest of the industry including a bevy of ARM based alternatives. For high-performance cutting-edge solutions, customers had little choice but to stay with Intel.

 

The dynamic has changed in the last few years as Intel fell behind process technology and has also fallen behind building innovative x86 designs. The poor execution from Intel has left the door open for the competition. AMD, as a merchant supplier competitor, was the first to exploit the opportunity with its EPYC class of solutions. And, now, AWS is following a similar path with Graviton2.

 

Amazon’s Graviton2 is a 64 core ARM Neoverse design with special instructions for machine learning. As with AMD, Amazon is very likely using TSMC (NYSE:TSM) to fabricate the silicon. Amazon claims that its Graviton2 based instances deliver about a 20% to 50% performance advantage over Intel Xeon Platinum 8000 series (Skylake) based instances. While this level of performance makes Graviton2 competitive with Intel offerings, it falls short of top of the line AMD EPYC2 equivalents. Thus, it is no surprise that Amazon did not offer any comparisons with AMD EPYC2 based instances.

 

Amazon also touted these instances as having 20% lower cost compared to Intel alternative. Once again, no such comparison was made against AMD alternatives.

 

Amazon did not publish critical information about Graviton2 but given the historical strength of ARM, it is likely that the ARM solution uses significantly lower energy than Intel equivalents for a comparable unit of work. This is likely to be a big part why AWS claims cost savings on this solution.

 

Amazon is keenly aware that customers who run x86 applications are unlikely to easily shift to Graviton2 ARM based solution. However, there are many open source applications that can run on ARM, and these are the applications that Amazon will be looking to migrate to its Graviton2 instances. The immediate opportunity for Graviton2 is Amazon’s internal use servers. As long as the servers can support whatever software that Amazon is running on its servers, Amazon could migrate.

 

While it is unclear what percentage of Amazon workloads can benefit from Graviton2, Amazon now has a strong internal solution that it can use as a leverage against Intel and AMD.

 

This is a major setback for Intel as one of its highest volume server customers now has an internal alternative. Amazon is likely to deploy its internal solution wherever it perceives it has a better TCO compared to Intel and AMD equivalents. Between EPYC and Gravitron2, we can now expect that Amazon will reap a windfall reduction in server costs.

 

Over time, to the chagrin of Intel investors, this dynamic will help drive x86 CPU margins down as the server market prices on the x86 CPUs are likely to turn lower on a secular basis. Concurrently, this dynamic improves economics of Amazon solutions.

 

Inferentia

 

The story with Inferentia is similar to that of Graviton2. With Inferentia, Amazon is after Nvidia margins.

 

Amazon provided very little information about the AWS Inferentia chip itself, but Amazon claims that, when compared to top of the line Nvidia T4 inferencing solution based G4 instances, Inf1 instances offer up to 3x the inferencing throughput, and up to 40% lower cost per inference. In other words, Amazon is telling customers that it now has a better inference solution than its current supplier Nvidia. Amazon claims that, according to customer feedback, inferencing can account for up to 90% of the cost of customers' machine learning work. By offering a promise of big performance boost at a lower price, Amazon is clearly looking to incentivize customers to try and benchmark its solution against Nvidia alternatives. If Amazon’s claims stand to scrutiny in customer applications, and we have little reason to be skeptical, Amazon will cause customers using Nvidia T4 solutions to switch to AWS Inferentia solutions.

 

In addition, for all of its internal applications, which is probably a big volume driver, we can now expect Amazon to use AWS Inferentia instead of Nvidia alternatives. This is a big setback to Nvidia which has already lost much of the internal Google (NASDAQ:GOOG) (NASDAQ:GOOGL) inference market.

 

While the opportunity size of AWS Inferentia is small compares to Graviton2, and probably closer to $100M, Amazon expects a large growth in this segment as machine learning applications multiply.

 

Sustainability Of Custom Silicon Cannot Be Taken For Granted

 

While Amazon has undeniably scored a coup on Intel and Nvidia with Graviton2 and Inferentia, this does not automatically imply that Amazon will be successful with its internal CPUs and AI chips in the long term.

 

Amazon, compared to merchant vendors like Intel, suffers from diseconomies of scale. The total annual volume of servers at Amazon for ARM, considering it is only applicable for a subset of its needs, is likely far less than its server CPU consumption of 2.5M units per year. It is unlikely that Graviton2 will get to a level higher than a million units per year during the initial years. This volume pales in comparison to the 25M per year x86 server market. Given the smaller TAM, it is not possible for Amazon to invest in server solutions at a rate merchant suppliers like Intel or AMD can. On a long-term basis, it is also difficult for players like Amazon to keep up with vendors like Intel who specialize in building CPUs.

 

The reason that Graviton2 even exists is because of the obscenely high margins Intel commands in the server business. Whether Amazon can keep innovating on the Graviton front depends on Intel and AMD, and how high a margin they would like to see for their server chips. The higher the margin these players want, the higher the incentive for Amazon to keep investing in its internal solution.

 

To be sure, Graviton2 may have arrived into the market a bit late. AMD is already significantly cutting down margins in the x86 business through EPYC2 products that offer higher performance and lower cost than Intel alternatives. As an example, at the top end of the range, AMD claims to offer 4 times the price performance ratio of Intel.

 

Prior to 2019, Graviton2 would have been far more attractive to Amazon but now, with AMD dramatically improving price performance ratio, it is far less so. Thanks to AMD’s aggressive price performance, we expect that Graviton2 is far less attractive in some of the niches where Amazon may have been considering the chip. As such, we see Graviton gaining volume very slowly.

 

Chip suppliers like Intel, AMD, and Nvidia have a complex dynamic that they need to overcome. The challenge is that many of their customers, unlike Amazon, do not have sufficient volumes, incentives, or capabilities to initiate internal CPU efforts. From a supplier perspective, profit maximization may dictate that, rather than collapse ASPs across the board, it is better to offer steeper discounts exclusively to Amazon and other capable parties or to lose some share of business to internal silicon. Suppliers may also offer lower margin products to Amazon and other players who need such an option.

 

These dynamics suggest that Arm based products, whether internal to large customers or from AMD and Intel, may see an uptick in adoption in the server market.

 

It is difficult to predict how this battle plays out in the near term but, in the long term, it is highly likely that merchant silicon vendors will lose a part of the TAM to players like Amazon who develop their own solutions. It is also likely that the margins in the industry will shrink in areas where there is a threat of substitution.

 

Prognosis

 

Out of nowhere, margins in the x86 business are under assault with Amazon building a capable competitor in server CPU business. Graviton2 has a limited applicability in the near term and is a threat to a subset of use cases which can be supported by ARM. Most of the customer loads run on x86, and customers are unlikely to migrate to ARM for a long time to come. However, Graviton2 will help drive down Intel and AMD server business margins and create a tremendous opportunity for Amazon.

 

A similar dynamic is playing in the GPU space as Amazon, Google, and other customers are looking at Nvidia’s high margins as their opportunity. Google’s TPU and Amazon’s Inferentia chips show that, while the machine learning market is growing, a high margin product line such as the one offered by Nvidia may not be attractive to hyperscalers who have the capability to build their own internal solutions. Unfortunately, for Nvidia investors, hyperscalers are a large part of the ML TAM. In the past, we have questioned why Nvidia is not seeing meaningful growth in a market with torrid growth. This dynamic, we believe, is one of the answers.

 

While there is a strong appeal on the part of companies and investors to chase high margin opportunities, in a world where customers can create competitive products, suppliers of high margin products may find their valuations crumbling.

 

This does not mean high margins are not possible. Companies can command high margins if they have unique solutions that are difficult to emulate. Intel was in the enviable position for decades. However, with its process advantage now gone and design leadership in question, maintaining high margins would be difficult. Nvidia is in a tougher position given that its solutions are more easily substitutable.

 

Subscribers to Beyond The Hype have access to all the linked articles that may otherwise be inaccessible. For timely, cutting-edge insights, analysis and investing ideas in technology, semiconductor, solar, battery, autonomous vehicles, and other emerging technology stocks, check out Beyond the Hype. This Marketplace service gives you early access to my best investing ideas, along with event driven and arbitrage opportunities when they are most edgy and actionable.