Highlight

Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems

Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems - CDA ORNL
Area under curve (AUC) resiliency metric for 30% failure rate for the original EONS algorithm (EONS-1.0), SLAYER, and Whetstone, as well as the updated EONS approach (EONS-0.1), which trains to be more resilient. This figure shows that the EONS-0.1 approach produces significantly more resilient networks than the other training approaches.

Achievement

A team of researchers at Oak Ridge National Laboratory (ORNL) and Purdue University has studied the effect of synapse failures on the performance of four different training algorithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS).  The researchers showed that these four different approaches have very different resilience characteristics with respect to simulated hardware failures.  They analyzed an approach for training more resilient spiking neural networks using the evolutionary optimization approach and showed how this approach produces more resilient networks.

Significance and Impact

This work demonstrates that though robustness and resilience are commonly quoted as features of neuromorphic computing systems, these characteristics vary significantly across different training algorithms, indicating that some algorithms are more robust than others and some algorithms are very susceptible to a large decrease in performance due to a small numbers of failures.  The researchers presented an approach for training more resilient spiking neural networks for neuromorphic deployment using an evolutionary optimization approach and discussed how this approach can be extended for other training approaches.  Using this new approach, spiking neural networks can be trained that are more resilient to hardware failures, allowing for neuromorphic systems to be more performant in hazardous conditions that may induce hardware failures.

Research Details

  • Researchers studied the effect of failures on the performance of four different training algorithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS). 
  • Researchers showed that these four different approaches have very different resilience characteristics with respect to simulated hardware failures. 
  • Researchers analyzed an approach for training more resilient spiking neural networks using the evolutionary optimization approach and showed how this approach produces more resilient networks.
  • Researchers discussed how this approach can be extended to other spiking neural network training algorithms as well.

Overview

Though robustness and resilience are commonly quoted as features of neuromorphic computing systems, the expected performance of neuromorphic systems in the face of hardware failures is not clear.  In this work, researchers from ORNL and Purdue University studied the effect of failures on the performance of four different training algorithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS).  They showed that these four different approaches have very different resilience characteristics with respect to simulated hardware failures.  They then analyzed an approach for training more resilient spiking neural networks using the evolutionary optimization approach.  They showed how this approach produces more resilient networks and discussed how it can be extended to other spiking neural network training approaches as well.

Last Updated: January 15, 2021 - 10:03 am