Abstract: We develop a new scalable evolution strategy (ES) for high-dimensional blackbox optimization with application to reinforcement learning. Standard ES has been proved to suffer from the curse of dimensionality, due to the random directional search and low accuracy of Monte Carlo sampling. The key idea of this work is to develop a directional Gaussian smoothing approach which only averages the original objective function along orthogonal directions and forms the full gradient by assembling their partial derivatives. This strategy requires the computation of multiple one-dimensional integrals for a search direction, as opposed to a single high-dimensional integral as in the standard ES methods. In approximating the averaged partial derivatives, we use the Gauss-Hermite quadrature rule, instead of Monte Carlo, which significantly improves the accuracy of the averaged gradients. Further, by choosing a moderately large smoothing parameter, our smoothing technique reduces the barrier of local minima, such that global minima become easier to achieve. We provide three sets of examples, including benchmark functions for global optimization, a rocket shell design problem, and reinforcement learning, to demonstrate the performance of our method. A thorough comparison with other state-of-the-art approaches is given.
Speaker Bio: Hoang Tran received his Ph.D. in Applied Mathematics from University of Pittsburgh in 2013, under the supervision of Catalin Trenchea and William Layton. He is currently a Research Scientist at Computational and Applied Mathematics Group at Oak Ridge National Laboratory. His research interests include numerical methods for PDEs, approximation theory, uncertainty quantification and computational fluid dynamics.
Host: Eirik Endeve, endevee@ornl.gov
About the Seminar: The Computational and Applied Math Seminar features talks by invited speakers, local mathematicians, and domain scientists working on problems of mathematical interest. The seminar is held weekly, every Thursday from 3:00pm-4:00pm. If you are interested in giving a seminar, please contact Eirik Endeve, endevee@ornl.gov. To subscribe to the CAM Seminar mailing list, please contact Kasi Arnold, arnoldkl@ornl.gov. To see the full list of previous and upcoming seminars, go to https://csmd.ornl.gov/events/9/seminars.
To join the meeting on a computer or mobile phone: https://bluejeans.com/943905068?src=calendarLink
Connecting directly from a room system?
1) Dial: 199.48.152.152 or bjn.vc
2) Enter Meeting ID: 943905068 -or- use the pairing code
Just want to dial in on your phone?
1) +1.888.240.2560 (US Toll Free)
+1.408.317.9253 (US (Primary))
+1.408.740.7256 (US (San Jose))
Global Numbers: https://www.bluejeans.com/premium-numbers
2) Enter Meeting ID: 943905068
3) Press #
Want to test your video connection?
https://bluejeans.com/111
Last Updated: May 28, 2020 - 4:01 pm