HPC Market Update

HPC Advisory Council, February 2018

Addison Snell
Christopher Willard, Ph.D.
Laura Segervall
Intersect360 Research in 2017

10 year history of HPC analyst business

Covering “high performance data center” markets, including

traditional HPC and supercomputing, high-performance enterprise,
cloud, big data, and hyperscale

New hyperscale advisory service

Weekly podcast, “This Week in HPC”

Market research partners of TOP500.org, contributing editors to

The Next Platform, media partnerships with HPCwire, insideHPC

Sponsors of HPC Advisory Council

 HPC Market, 2016 - 2021
• Total worldwide HPC market (servers, storage, software,
etc.) reached $35.6 billion in 2016, up 3.5% from 2015.
• Servers were the largest component, reaching $11.5
billion (3.3% growth over 2015).
• Total market is forecast to grow to $43.9 billion in 2021,
4.3% CAGR.
• Growth has been driven significantly by commercial HPC
markets; this continues over the next five years.
 A Caution About Forecasting

“When you create a forecast, it has to

be bound by what is realistic, whereas
reality itself has no such limitations.”

– Chris Willard
Chief Research Officer
Intersect360 Research
HPC Market Revenue by Product Class ($K)
 Total HPC Revenue by Product Category

• Servers will lead the market in terms of absolute revenue growth,

adding about $2.9 billion to the market by 2021
• Cloud (aka utility computing) growth is based on a low starting point;
by 2021 the segment will capture only about 3% of total revenue
• Software revenue is challenged by a combination of: growth in open
source options, consolidation, bundling with hardware, and using
software as a loss leader to promote other products
• Storage is the highest-growth segment other than cloud and will add
roughly $2 billion in growth in the next five years
 Total HPC Market Revenue
High Performance Computing Market by Economic Sectors -- Total Market Forecast by Economic Sectors ($M)
2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 CAGR
Government 7,396 8,453 8,425 8,418 8,936 9,217 9,363 9,517 9,702 9,920 10,194 2.0%
Academia 4,888 5,768 6,028 5,976 6,470 6,619 6,612 6,609 6,612 6,629 6,667 0.1%
Industry 13,385 15,478 16,694 17,142 18,978 19,751 20,949 22,274 23,727 25,322 27,083 6.5%
Total 25,669 29,698 31,147 31,536 34,384 35,587 36,923 38,400 40,042 41,871 43,944 4.3%

• 2011-16 actual, 2017-21 forecast

• The HPC market will grow at a 4.3% CAGR between 2016 and 2021
• Revenues in HPC will grow by $8.3 billion to about $43.9 billion
• Industrial HPC is the driving engine of the market
 HPC 2016 Revenue by Vertical

Academic,
18.6%

Commercial, Government,
55.5% 25.9%
 HPC 2016 Revenue by Vertical

Academic,
18.6%

Commercial, Government,
55.5% 25.9%
 Commercial HPC Growth
Five-Year Forecast Revenue Growth in HPC, over 2016 Base
Intersect360 Research, 2017
 Potential Five-Year Growth of HPC
“Solve”: U.S. Council on Competitiveness, with Intersect360 Research, 2014
Free Download: http://www.compete.org/reports/all/2695-solve
 Best Metric to Justify HPC Investments
“Solve”: U.S. Council on Competitiveness, with Intersect360 Research, 2014
Free Download: http://www.compete.org/reports/all/2695-solve
HPC System Market Share
 HPC Storage Revenue Share Model

• Includes disk (NAS, SAN, scratch, etc.), flash, tape

• Revenue assigned to the final seller (may be OEM from other vendor)
 The Hyperscale Market
• Included since our 2007 HPC methodology as
“ultrascale internet”
• In 2015, recognized “Hyperscale” as distinct
market in proximity to HPC; began process of
separating it from HPC
• Defined hyperscale methodology with market
research and feedback from interested clients
• Published definitions and methodology report in
April 2016
 Hyperscale Market Definition

The hyperscale market

consists of arbitrarily
scalable, web-facing
application infrastructure
that is distinct from general
IT investment
 HPC vs. Hyperscale
Similarities
Application infrastructure that is distinct from general enterprise
Applications are arbitrarily scalable
Top tier users push boundaries and influence technology throughout industry

Differences
HPC Hyperscale
• Primary focus is performance • Primary focus is scalability
• Large jobs • Many jobs
• Is still “HPC” on only a few nodes • At lower levels it is just enterprise
– there is entry-level HPC computing – no entry-level
• Largest supercomputers are • Largest hyperscale installations
>$100 million are >$1 billion
 Hyperscale Market Size and Tiers
• Intersect360 Research has a tiered model for
the hyperscale market, as follows:

2016 estimates
Intersect360 Research, 2017
 AI / Deep Learning
• In 2016, Intersect360 Research placed AI in the
hyperscale market
– Similar to but distinct from HPC
– Low precision, intensely parallel, strong affinity to
public cloud
• DL is not a vertical market. It is an algorithmic or
computational approach
• We are monitoring spending patterns
– Are budgets increasing?
– What is the effect on configurations?
– On premise versus cloud
 Sizing Deep Learning
• We have looked at the size of the deep learning
market opportunity from several angles:
– Supplier-side information on market expansion
– Demand-side market analysis on acquisitions
– Top-down analysis of the hyperscale market
– Analyst opinion
• Our independent methods have produced
consistent views of the current opportunity in
deep learning
 Deep Learning Size and Growth
• According to our analysis, the deep learning
market opportunity is $2.0B to $2.5B for 2016
– Represents about 6.5% of the hyperscale market
– Data presented one-on-one to clients of Hyperscale
advisory service
• We expect growth rate to be very high for the
next two years, 2017-18, close to 100% Y/Y
• After that, growth will moderate, and possibly
even contract by end of five-year period
The Top of All Things in HPC, 2017
Site Census Snapshot Analysis
January 2018

Christopher Willard, Ph.D., Addison Snell,

Laura Segervall
 Introduction
• The purpose of this report is to provide the top suppliers, packages,
or categories within specific segments.
– Shares reported are based on Site Census survey results, not total sales.
– Top is defined as companies accounting for 50% market share or the top five,
whichever comes first.
• Our goals for collecting and maintaining site census data include:
– Developing demand-side data for tracking and modeling fundamental
characteristics of the HPC market in such areas as computer room contents,
vendor positioning, technology adoption, user purchase patterns, etc.
– Collecting and comparing demand-side data to supply-side data in order to
accurately frame the entire HPC ecosystem.
– Tracking trends in these characteristics over time.
– Supporting our HPC market-sizing and forecasting models.
• Survey was conducted in the third quarter of 2017.
• Responses were received from 454 sites which responded to at
least one section of our last three site census surveys. Entries
reported in two or more surveys were counted once.
 What is “Top”
• Given the wide range of product categories and technologies
combined with the variable number of vendors selling into any
category a strict count base definition of “top” (e.g. top 10, top 3) is
not meaningful in all cases.
• We define “top” as the fewest companies necessary to account for
50% share in a given category or the top five, whichever comes first.
• We ignore this rule for some software categories because of either
too few category options or too many companies needed to get to
50%
• Note: These charts indicate responses to our surveys of the HPC
user community. Survey responses are not always indicative of
market share by revenue. For inquiries on revenue share, contact
Intersect360 Research.
 Top System Suppliers

• HPE, Dell EMC, and Lenovo remained the top three HPC
system suppliers, accounting for 45% share of all mentions.
• At commercial sites, HPE accounted for about 36% of the
reported installations, compared to Dell, the next closest
supplier, with about 13% of reported installations.
 Top Accelerator* Supplier
* Accelerators includes GPUs, co-processors, and FPGAs

Top Suppliers by Share of Systems Top Suppliers by Share of Total Accelerators

(N=310 systems) (N=84,597 accelerators)

• 40% of all systems reported have accelerators installed.

• NVidia was installed in more than three-fourths of the systems
with accelerators.
• Intel Xeon Phi was installed in about 15% of systems yet
accounted for 43% of the total number of accelerators
reported, due to several high density system installations.
• FPGAs are installed in less than 5% of the systems.
 Top Storage System Suppliers

• DDN’s share of reported storage systems in our Site Census survey

increased by 2% between 2016 and 2017.
• IBM’s share of mentions also grew by about 1%, while Dell EMC’s share
remained flat.
• Amazon Web Services continues to be the top named supplier of cloud
storage. Cloud storage accounted for about 4% of the mentions.
 Top File Systems
reported with storage systems

Top Storage Management Software Suppliers Top Storage Management Software Packages
Storage Management No. of Storage Share of Storage Management Software No. of Storage Share of
Software Supplier Systems Systems Package Systems Systems
IBM 102 25% Lustre and derivatives 76 24%
DDN 38 9% Spectrum Scale and derivatives 75 24%
Dell EMC 28 7% Panfs 16 5%
Intel 28 7% Data OnTap 11 3%
Lustre.org 26 6% NFS 11 3%
Other 181 45% Other 128 40%
Total 403 100% Total 317 100%
Source: Intersect360 Research, 2017 Source: Intersect360 Research, 2017

• IBM was the leading supplier with its multiple storage management
packages (Spectrum Scale, Spectrum Protect, HPSS). Spectrum Scale
accounts for about 70% of IBM’s mentions.
• Including all derivatives of Lustre and Spectrum Scale, the share for
both is almost the same.
 Top Interconnects/Network Suppliers
Top System Interconnect Suppliers Top Storage Interconnect Suppliers Top LAN Suppliers

• In all locales, two vendors account for over 50% of the mentions, with
Mellanox being the only supplier in all three locales. Entries not specifying
supplier were not included in total.
• Mellanox gained share in every segment, with the largest increase (5%)
being for storage networks.
• Intel’s Omni-Path increased its presence for both interconnects and storage,
accounting for roughly 2% of all mentions in both locales.
 Source of HPC Middleware
Source of Primary Middleware by Sector
Source of Primary Middleware Reported

• ISV and Open Source account for about 96% of all primary
middleware reported.
• Commercial sites have a preference for ISV supplied middleware.
• Top ISVs were Intel with its compilers and programming
environment software, MathWorks with Matlab, and IBM with
primarily its job and storage management software.
 Top Primary Middleware Categories

• Programming Environment and Compilers account for 56% of all

mentions. Programming Environment includes mathematical applications,
parallel tools, and other tools.
• Top package and share of mentions for each of the top three usage
categories were:
• Programming Environments: MathWorks Matlab, 14.8%
• Job Management: SchedMD Slurm, 21.2%
• Compiler: GNU Compiler Collection, 19.8%
• Each top package accounted for double the mentions of the second most
mentioned package in these three categories.
Top Generally-Available Applications

• About 76% of applications reported were Open Source or ISV

software (generally-available applications).
• Top applications are primarily used for traditional HPC application –
chemistry, fluid dynamics, and structural analysis -- with the
exception of WRF, a weather forecasting application.
• ISV and in-house packages are still the majority at commercial
sites, representing about 85% of the packages reported.
• Open source accounts for more than 50% of the packages
reported at academic and government sites.
 Summary
• HPE became the largest HPC vendor reported in our Site Census
surveys, accounting for roughly 21% of all mentions. Much of this
increase in mentions is due to the acquisition of SGI systems.
• NVidia continues to dominate the accelerator market in system
installations. Their lead in total processors installed shrank to less
than 5% with installation of several large Intel Phi-based systems.
• Mellanox gained share of mentions in every locale, with storage
showing the largest gain.
• Middleware profile remained consistent with the prior year with
programming environment, job management, and compilers being
the top three subcategories.
• Top ten generally-available application packages continue to
primarily be in traditional segments – chemistry, fluid dynamics, and
structural analysis.
HPC Market Update
SC17 – November 2017

Addison Snell
Christopher Willard, Ph.D.
Laura Segervall