At the system level, IBM continues to use central processor complex (CPC) drawers which incorporate the processors, memory, and I/O interconnects. A drawer on the z15 comprises two logical CP clusters and one SC. The split cluster design is intended to improve memory traffic. This is the same design as the z14. However, the similarities stop here. On the z14, a single drawer comprises six central processors and a single system controller chip. Within a drawer, everything is fully connected via the X-Bus.
On the z15 IBM scaled back on the number of central processors. There are now four processors and a single SC chip per drawer. We will get as to why we believe IBM made this change later on. Within a drawer, everything is still fully connected via the X-Bus. Each processor is connected to the other processor within a cluster as well as to the SC chip.
The change from six to four central processors is an interesting one. We will touch on the z15 processor in more detail late in this article but for now, we want to point out that the z15 has twelve cores per processor, two more than the z14. This means that in total, the z15 drawer has 48 cores while the z14 had 60 cores. As far as memory channels are concerned, this has not changed from the z14. There are five channels per CP for a total of 20 channels per CPC Drawer.
Taking another step back, the full z15 system configuration supports one additional drawer compared to the z14. With up to five CPC Drawers, IBM can compensate for the reduced processor count per drawer and offer up to 190 cores or 380 threads (note that the system is overprovisioned for redundancy). Note that the max z14 configuration was capped at 170 cores so you are looking at roughly 12% more cores on the new system. By the way, the usable memory has also increased from 32 TiB to 40 TiB. Communication inter-drawer is done over the A-Bus just like in the prior architectures. With four drawers, the SC on the z14 only needed three A-Bus links. On the z15, IBM added a fourth A-Bus link in order to fully connect every drawer to every other drawer.
z15 Central Processor
The overall architecture of the chip itself is very similar to the z14. Each core has a private L1 and L2. There is a massive L3 cache that is shared by all the cores. IBM increased the core count by 2 from 10 to 12. The L3 cache has been doubled to a whopping 256 MiB.
|L3 Cache Comparison|
|Capacity||48 MiB||64 MiB||128 MiB||256 MiB|
|Line Size||256 B||256 B||256 B||256 B|
Another way to compare the cache size is by ratio to core count. The z15 nearly doubles the effective per-core L3 cache.
|L3 Cache Comparison|
|Capacity||24 MiB||48 MiB||64 MiB||128 MiB||256 MiB|
|Effective per-core capacity||6 MiB||8 MiB||8 MiB||12.8 MiB||21.33 MiB|
Back in July 2013, IBM introduced the IBM zEnterprise Data Compression (zEDC) card. With the large L3 cache, IBM decided to move the Nest Acceleration Unit (NXU) on-chip. Because the NXU is on-chip, it can perform compression directly in the L3 cache, removing the I/O bottleneck. IBM is claiming up to 17 times the throughput compared to the old zEDC card with up to 260 GB/sec of compression throughput.
z15 Core – Wider, IPC Uplift
The z15 CPU improves on the prior z14. They are still manufactured on GlobalFoundries 14-nanometer using 17 metal layers. This is still 14HP, a special FinFET on SOI process that was co-designed by IBM and GlobalFoundries. Interestingly, although the z15 die is identical in size to the z14, at almost 700 mm², IBM has managed to pack 3 billion more transistors – 9.1 billion transistors in total. Late in this article, we will explain the magic.
Like its predecessor, the z15 operates at up to 5.2 GHz – but now with two more cores. Compared to the z14 at 5.2 GHz, IBM claims an average 10-13% uplift in single-thread performance.
Like the z14, the z15 features 128 KiB private L1 instruction cache.
|L1I Cache Comparison|
|uArch||z12||z13||z14||z15||Sunny Cove||Zen 2|
|Capacity||64 KiB||96 KiB||128 KiB||128 KiB||32 KiB||32 KiB|
|Line Size||256 B||256 B||256 B||256 B||64 B||64 B|
As with its predecessors, the z15 continues to use a split-cache hierarchy within the core itself. On the z15, IBM doubled the size of the level 2 instruction cache to 4 MiB, reaching parity with the data cache size. Unlike any other non-IBM processor, those caches are not implemented in SRAM but, instead, use their highly dense eDRAM cell.
|L2I Cache Comparison|
|Capacity||1 MiB||2 MiB||2 MiB||4 MiB|
|Line Size||256 B||256 B||256 B||256 B|
Improved BPU – New TAGE Predictor
Asynchronously of instruction processing, the branch predictor operates ahead of the instruction stream, locating and predicting branch instructions. On the z14, IBM used a number of different algorithms. The z14 has a first level BTB and history table (BHT) as well as a second level BTB. The z14 also has a BTB pre-buffer (BTBp) which was essentially an L1 BTB1 victim cache. Once used, it passes it on to the L1 BTB. On the z15, the BTBp has been removed. Instead, it was replaced by a simpler write buffer with the two independent read ports being replaced by single double-bandwidth port. Additionally, the L1 BTB has increased from 4 sets of 2K rows to 8 sets of 2K rows. The L2 BTB is unchanged.
With the introduction of z14, IBM added two new predictors – a Hashed Perceptron and a simple call-return stack for target prediction. With the z15, IBM made a number of enhancements. First, the simple call-return stack can now return to branches located up to 8 bytes past the next sequential instruction. Previously, the predictor was limited to returning to the next sequential instruction. The most major addition to the z15 is the introduction of a new TAGE predictor. AMD has recently added the same predictor to their Zen 2 core. If you are interested, we have covered the behavior of the TAGE predictor in this article.
- TSMC Details 5 nm
- IBM Doubles Its 14nm eDRAM Density, Adds Hundreds of Megabytes of Cache
- TSMC Announces 2x Reticle CoWoS For Next-Gen 5nm HPC Applications
- CEA-Leti Demos a 6-Chiplet 96-Core 3D-Stacked MIPS Processor
- Intel Refreshes 2nd Gen Xeon Scalable, Slashes Prices
- Radeon RX 5700: Navi and the RDNA Architecture
- 7nm Boosted Zen 2 Capabilities but Doubled the Challenges
- Arm Launches the Cortex-M55 and Its MicroNPU Companion, the Ethos-U55
- Inside Rosetta: The Engine Behind Cray’s Slingshot Exascale-Era Interconnect
- Arm Ethos is for Ubiquitous AI At the Edge