23 Nov Computers Inspired by the Human Brain
Hello all! My name is Jon, for those of you who don’t know me, I am an Embedded Systems Engineer at FYELABS.
Here at FYELABS we are constantly working with computers, whether it’s a linux server in the cloud, a microcontroller in an embedded device, or anything in between. But the world of computers is much larger than that, and it’s constantly expanding in new directions. One such new and exciting direction goes by the name of neuromorphic computing.
In a nutshell, neuromorphic computing aims to build computers inspired by the human brain. Neuromorphic computing builds on the success of machine learning techniques like neural networks, which themselves are loosely brain inspired, but aims for better scalability and power efficiency than what we can get today. Nvidia’s A100 AI focused server GPU can pull up to 400W, where the human brain averages around 20W. That’s a 20x increase in power consumption! Now, how does neuromorphic computing fare in this comparison? If a 2019 paper[1] is to be believed, pretty well. They demonstrated 20x better power efficiency than Nvidia’s Jetson TX1, which looks like human brain levels of efficiency. Looks like is key here though, all these comparisons I’m drawing between GPUs, neuromorphic and the human brain are really hand wavy, the general trend is correct but the numbers shouldn’t be taken too seriously.
Ok, so GPUs (and CPUs in fact) are really power hungry relative to the brain, and neuromorphic computing is much closer to brain-like power efficiency, but how? Let’s get into the architecture of one of these neuromorphic processors. For this example I’ll be focusing on Intel’s research neuromorphic chip, named Loihi.
Traditional processors (like CPUs and GPUs) are variations of a von Neumann machine, meaning they have some processing units that do the actual computations, some memory that stores the data it’s operating on and some interconnect to tie everything together.
On the other hand, Loihi is built out of many small units connected together via directed links, analogous to neurons and synapses in the brain. The digital neurons communicate by sending spikes, similar to neuron impulses in the brain. There’s no centralized memory, no instructions, there isn’t even a centralized clock! Loihi is just a big grid of digital neurons, connected by digital synapses, sending digital neuron impulses. Loihi also scales all the way from a little USB stick with 262 thousand neurons, codename Kapoho Bay, to a massive enclosure with 100 million neurons, codename Pohoiki Springs.
A die shot of Intel’s Loihi chip, the outlined block contains 1024 digital neurons.
Neuromorphic computing is certainly an interesting direction to keep an eye on going forwards. Although it may be a while before they match the brain’s 100 billion neurons.
__
[1] Blouw, P., Choo, X., Hunsberger, E., & Eliasmith, C. (2019, April 2). Benchmarking keyword spotting efficiency on neuromorphic hardware. arXiv:1812.01739v2 [cs.LG]
Pingback:Meet our Embedded Systems Engineer, Jonathan Casella - FYELABS
Posted at 14:10h, 28 January[…] and learn throughout the process. Don’t forget to read Jon’s recent blog article, “Computers Inspired by the Human Brain,” and let us know your thoughts in the […]