Deep Neural Networks motivated by PDEs

One of the most promising areas in artificial intelligence is deep learning, a form of machine learning that uses neural networks containing many hidden layers. Recent success has led to breakthroughs in applications such as speech and image recognition. However, more theoretical insight is needed to create a rigorous scientific basis for designing and training deep neural networks, increasing their scalability, and providing insight into their reasoning.

This talk bridges the gap between partial differential equations (PDEs) and neural networks and presents a new mathematical paradigm that simplifies designing, training, and analyzing deep neural networks. It shows that training deep neural networks can be cast as a dynamic optimal control problem similar to path-planning and optimal mass transport. The talk outlines how this interpretation can improve the effectiveness of deep neural networks. First, the talk introduces new types of neural networks inspired by to parabolic, hyperbolic, and reaction-diffusion PDEs. Second, the talk outlines how to accelerate training by exploiting multi-scale structures or reversibility properties of the underlying PDEs.

The talk is joint work with the groups of Eldad Haber and Eran Treister. The main reference is https://arxiv.org/abs/1804.04272

Biography:  Lars is an assistant professor at Emory University in Atlanta, Georgia. He received his diploma and his Ph.D. in mathematics from the University of Münster in 2010 and 2012, respectively. Prior to joining Emory, he was a postdoctoral research fellow at the University of British Columbia and during his PhD, he held positions at the Universities of Lübeck and Münster. His research interests include numerical analysis (particularly numerical methods for optimization, linear algebra, and partial differential equations) and scientific computing with applications in medical and geophysical imaging and machine learning. His research is supported by the National Science Foundation, the Centers for Disease Control and Prevention, and NVIDIA. He am also a senior consultant at Xtract Technology.