Intel Labs Builds A Neuromorphic System With 64 To 768 Loihi Chips: 8 Million To 100 Million Neurons

One of the major features of Loihi is that it integrates neuromorphic cores on a low-overhead scalable NoC fabric. This fabric exposes a parallel I/O interface on the east, west, north, and south sides of the chip in order to allow glueless connection the chip to its neighbor Loihi chips, enabling the formation of larger systems with dozens and even hundreds of chips.

Full details of Loihi can be found here: Loihi – Intel – WikiChip

Large-Scale Novel Neural-Inspired Algorithms

The ability to scale the mesh to multiple chips is a fundamental design choice made by Intel Labs in order to build large systems and experiment with new neural-inspired algorithms that were not possible or were rather difficult on a relatively small number of neurons. Mike Davies, Intel’s neuromorphic computing program leader talked about graph search as one of those interesting algorithms they were experimenting with. With graph search, it’s possible to explorers all the different permutations of the spikes propagating through a network. By following the effects of Spike-Timing-Dependent-Plasticity (STDP), it’s possible to discover the shortest gradient path to a destination. This allows you to trace all paths from a source to a destination. Graph search on a neuromorphic chip tends to scale much better than conventional systems. Other algorithms include spatiotemporal attractors which rely on attractions being associated with certain temporal spikes. This is done through iterative cycles from rhythmic synchronization generally in the gamma frequency range (30 iterations per second etc…) which results in very distinct spiking firing patterns. Lastly, various they are also exploring algorithms that deal with stochastic-spiking neuronal networks intended for solving constraint satisfaction/Markov chain Monte Carlo (MCMC) related problems.

When Loihi was first introduced in 2017, they made it available to researchers through Wolf Mountain. Wolf Mountain is a board with 4 Loihi chips. This gave researchers access to a system with roughly 500K neurons and 500M synapses.

Late last year Intel followed up with Nahuku. Nahuku is an Arria10 FPGA expansion board with up to 32 Loihi chips. At its biggest configuration, the board has 16 chips on each side organized in a mesh of 4×4.

With 32 chips on each Nahuku board, you are now looking at close to 4.2 million neurons and 4.2 billion synapses. Intel designed the Nahuku board as the foundation for constructing larger systems.

Introducing Pohoiki Beach And Springs

Pohoiki Beach And Pohoiki Springs are the next two evolutionary steps. Pohoiki Beach is a system the comprises two Nahuku board, doubling chips to 64 with more than eight million neurons. Intel says that the boards are now available to the broader research community. Pohoiki Springs is one step further. This a system with 24 cards organized in 8 rows and 3 slots per row. This means a fully populated system will incorporate 768 Loihi chips. Pohoiki Springs will be made available later this year.

Neuromorphic Landscape

When launched, Pohoiki Springs will be one of the largest neuromorphic systems available to researchers but it’s not the only system. There are currently a number of other interesting research systems being studied. Perhaps the most notable one is IBM’s TrueNorth. Like Intel, in 2016 IBM developed the NS16e board which featured 16 TrueNorth chips and, in theory, could scale up to multiple of such boards. In mid-2017, Air Force Research Lab (AFRL) in collaboration with IBM announced the largest system, the NS16e-4, which incorporated 64 chips for a total of 64 million neurons and 16 billion synapses. One of the strengths of IBM TrueNorth was its high neuron density. Unfortunately, this comes with the disadvantage of no in situ learning, requiring networks to be trained offline. More recently, in the last year, a second-generation, SpiNNaker 2, was announced which is planned for GlobalFoundries 22FDX and will feature 144 processing elements made of Cortex-M4F cores which emulate neurons along with communication hardware and the SpiNNaker spiking network-on-chip handling.