TOP500 50th List Anniversary: A Visualized Breakdown

This week TOP500.org celebrates its 50th list anniversary. But what does this list actually looks like? We’ve compiled some statistics to give you an insight into the list.

Changes Since June

The 49th TOP500 list was published less than six months ago in June. Since then, a large number of changes took place to the list including Intel’s introduction of their new Skylake-SP server processors and a number of new novel chips such as PEZY’s SC2. All in all, the entire TOP500 computing power was increased by 96.75 petaFLOPS.

To put that number into perspective, it took until June of 2012 for the entire TOP500 list to break the petaFLOPS barrier. It’s worth pointing out that 96.75 PFLOPS is only the third-highest increase. In both June and November 2016 the list saw an increase of over 100 PFLOPS each time.

In The Lead

In total, 28 countries have a presence in this iteration of the TOP500.

76% Is The Top Three

It might come as no surprise that only a handful of countries operate the majority of supercomputers. Just 3 of the 28 countries on the list operate 76% of all the systems: China, the United States, and Japan.

China Rises, While U.S. Declines

Historically, the United States has held the highest number of supercomputers on the list since the list’s inception in June of 1993. Over the last couple of years China has been steadily building their presence. Currently, the United States holds 143 spots in the top 500. While that might sound like a large portion, it actually happens to be the lowest amount in TOP500’s history. In fact, this is also the first time the United States doesn’t represent the majority of supercomputers on the list. With the introduction of the 50th list, China managed to edge out the US with 202 systems.

Other than China and the United States, the rest of the world maintained a relatively constant ranking throughout the decade.

Control Of The FLOPS

With 202 supercomputers in the TOP500, China has 299 PFLOPS at their disposal. The United States is 49.05 PFLOPS behind or 19% less; this is despite having 30% less systems in the list. In other words, while China has more systems ranked in the TOP500, the United States has fewer, but higher performing systems.

The graph below better shows the distribution of supercomputers by country by processing power (in TFLOPS). Other than a handful of top positions, most of the Chinese systems tend to be at the lower end of the performance. This is in contrast to the United States and Japan which have a large amount of systems toward the middle of the list.

Barrier To Entry

When most people think about the TOP500, they think about the 93 PFLOPS Chinese supercomputer Sunway TaihuLight or the 20 PFLOPS Swiss supercomputer Piz Daint. Those systems are actually not representative of the majority of computers on the list. In fact, just 10 computers have over 10 PFLOPS of computing power and just 20 have over 5. 318 systems have yet to break the petaFLOPS barrier and the minimum requirement to enter the current list would have been 550 TFLOPS.

Market Segment

Supercomputers fall into one of the six categories: industry, research, academic, government, vendor, and classified. While the majority of the Chinese supercomputers are considered industry, in the United States there is almost an even split between industry and research. In Germany and Japan there are almost no industry supercomputers, instead most are either academic or research.

When compared to the world’s distribution of supercomputers, the United States is more balanced all-around.

Computing Power By Segment

Despite having more supercomputers in industry, research actually leads in absolute computing power in the United States. The situation is very similar in China where 11 research supercomputers have almost the same computing power as the other 169 industry systems.

Collectively, 109 research supercomputers in the TOP500 have almost twice as much computing power as the other 275 industry systems. Another way of looking at it is this: research systems account for 21.8% of the TOP500 list while accounting for 50% of the computing power.

System Designers

Just five manufacturers built 73% of all the systems in the TOP500: HPE, Lenovo, Inspur, Cray, and Sugon. (Note: A small number of joint collaborations were not taken into account.)

When comparing manufacturers based on the computing power of their systems, Cray wins with less than half the number of systems in the list as HPE. In other words, Cray and IBM build fewer but more powerful systems whereas HPE, Lenovo, and Inspur build many more smaller and less powerful systems.

The Chips

When it comes to supercomputers, one name is ubiquitous: Intel. Intel powers 94.2% of the entire TOP500.

There are no systems based on Qualcomm’s recently launched Centriq server processors. It’s also too early to see any new POWER-based systems. And in case you’re wondering wondering, those AMD systems are not EPYC, but rather older Opteron parts.

Intel

Intel represents such a big portion of the TOP500 list that we had to compile some Intel-specific statistics. With 471 supercomputers powered by Intel, only 18 of them are powered by Intel’s recently launched Skylake server microarchitecture. Historically, there has always been a delay before systems started upgrading to Intel’s latest chips. Haswell still holds the second largest share after Broadwell. The list still includes a couple of Nehalem-based systems from 2010.

Skylake

Launched earlier this year, Skylake-SP is powering 18 supercomputers. The majority of those systems use Xeon Gold.

Seven of the systems are powered by the 20-core Xeon Gold 6148.

A similar picture emerges with older chips. A handful of chips dominate the list. In fact, just 30 SKUs power 413 of those supercomputers.

Process Technology

A slightly different way of plotting Intel-based supercomputers is based on the technology node those chips are manufactured on. Thanks to Skylake, the majority of Intel processors in the TOP500 are now based on their latest 14 nm process.

Core Count

While the new non-Phi Xeons get up to 28 cores, the highest core processor on the list is still a 22-core part. However, it is actually the 12-core parts that has the largest share of the list with close to a third of the systems.

Accelerators

Accelerators are becoming a growing part of the supercomputing ecosystem. 102 supercomputers make use of accelerators. 85% of those accelerators are GPUs designed by NVIDIA.

Note that two systems have both Xeon Phis and NVIDIA GPUs. A special mention should be given to PEZY, a small Japanese design firm that designs many-core processor accelerators. Four of the five PEZY-based systems are ranked 1st, 2nd, 3rd, and 5th place in the current Green500 list of most power-efficient supercomputers.

NVIDIA

As with Intel, NVIDIA dominates the accelerators used in the TOP500. The most widely used GPU is the Tesla P100 with 54% of the systems.

Fabric

There is no clear winner when it comes to the fabrics used by the TOP500 systems. The two leading solutions is 10G Ethernet and Infiniband (FDR/EDR/QDR). Additionally, a slowly growing solution is Intel’s own Omni-Path.

The Operating System

An often asked question is which operating systems run on those supercomputers. It turns out the answer is really simple – Linux! ALL 500 supercomputers in the TOP500 run Linux. There are no exceptions.

Distro

Although the information offered by TOP500 regarding the individual Linux distributions is largely incomplete, CentOS appears to be the most widely deployed Linux distro.

If you enjoyed this article, we recommend you check out the TOP500.org own 50th List Anniversary article which features an interactive historical TOP500 timeline.

Source: Data for this article was compiled using the freely available data from top500.org. If you’re looking to analyze the data like we did, top500.org provides a parsable XML and Excel spreadsheet of their lists.