Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks

Achievement

A team of researchers from Oak Ridge National Laboratory (ORNL) and a graduate student intern from the University of Minnesota developed an approach for training spiking neural networks for neuromorphic deployment that are resilient to common device errors, including small changes to synaptic weight values, and that optimize for small networks that are more energy efficient. Researchers studied spiking neural networks in two neuromorphic architecture implementations with the goal of decreasing their size, while at the same time increasing their resiliency to hardware faults. They leveraged an evolutionary algorithm to train the SNNs and proposed a multi-objective fitness function to optimize the size and resiliency of the SNN. They demonstrated that this strategy leads to well-performing, small-sized networks that are more resilient to hardware faults.

Significance and Impact

Neuromorphic architectures are designed and developed with the goal of having small, low power chips that can perform control and machine learning tasks. However, the power consumption of the developed hardware can greatly depend on the size of the network that is being evaluated on the chip. Furthermore, the accuracy of a trained SNN that is evaluated on chip can change due to voltage and current variations in the hardware that perturb the learned weights of the network. While efforts are made on the hardware side to minimize those perturbations, a software-based strategy to make the deployed networks more resilient can help further alleviate that issue. This approach allows for the development of small, energy efficient, resilient spiking neural networks for deployment in neuromorphic architectures.

Research Details

Researchers proposed and implemented a multi-objective fitness function for the Evolutionary Optimization for Neuromorphic Systems (EONS) neuromorphic training algorithm that incorporates a penalty for the number of neurons in a network as a way to generate smaller sized networks.
Researchers further included the performance of several variations of the network to produce networks that are resilient to some particular kind of perturbations that are possible to be encountered in the hardware.
They showed that this is a flexible approach for generating well performing, small sized SNNs that are more resilient to hardware faults for two different neuromorphic computing architectures.

Publication

Mihaela Dimovska, J. Travis Johnston, Catherine D. Schuman, J. Parker Mitchell, and Thomas E. Potok. “Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks.” IEEE Annual Ubiquitous Computing, Electronics, and Mobile Communication Conference.

Overview

Neuromorphic architectures are designed and developed with the goal of having small, low power chips that can perform control and machine learning tasks. However, the power consumption of the developed hardware can greatly depend on the size of the network that is being evaluated on the chip. Furthermore, the accuracy of a trained SNN that is evaluated on chip can change due to voltage and current variations in the hardware that perturb the learned weights of the network. While efforts are made on the hardware side to minimize those perturbations, a software-based strategy to make the deployed networks more resilient can help further alleviate that issue. In this work, researchers studied spiking neural networks in two neuromorphic architecture implementations with the goal of decreasing their size, while at the same time increasing their resiliency to hardware faults. They leveraged an evolutionary algorithm to train the SNNs and propose a multi-objective fitness function to optimize the size and resiliency of the SNN. They demonstrated that this strategy leads to well-performing, small-sized networks that are more resilient to hardware faults.