Project Status: Active
Project Summary: We propose to develop a scalable black-box training framework for scientific machine learning (SciML) models that are non-trainable with existing automatic differentiation-based algorithms. Our particular interest to this effort is to study how to train data-driven SciML models to learn missing physics of a complex system for advancing forward simulations. Specifically, this effort aims at achieving the following objectives: (1) develop a novel non-local gradient with structured sampling to enable non-local exploration for escaping from local minima and to achieve sufficient accuracy in gradient estimation; (2) advance theoretical analyses for a class of non-convex training problems to help domain scientists tune the hyper-parameters of the proposed training framework; and (3) exploit high-performance computing to accelerate the time to solution for black-box training problems for which loss functions involve computationally expensive black-box simulators. The proposed framework will be demonstrated on two distinct applications. The first is to train machine learning-based constitutive models to predict mercury dynamics in the Spallation Neutron Source mercury target, and the second is to train heat-source models to predict time-dependent laser scan paths that yield desirable micro-structures in three-dimensional printed metal components. This effort will advance the state of the art of several machine learning areas such as reinforcement learning and variational inference.
Principal Investigator: Guannan Zhang (CSMD, ORNL)
Senior Investigators: , Jiaxin Zhang (CSMD, ORNL), Hoang Tran (CSMD, ORNL), Dan Lu (CSED, ORNL), Matthew Bement (CSED, ORNL), Yousub Lee (CSED, ORNL), Bejamin Stump (MSTD, ORNL), Sirui Bi (CSED, ORNL)
Funding Period: Sept. 2020 to Aug. 2022
- Hoang Tran, and Guannan Zhang, AdaDGS: An adaptive black-box optimization method with a nonlocal directional Gaussian smoothing gradient, submitted. (https://arxiv.org/abs/2011.02009).
- Sirui Bi, Benjamin Stump, Yousub Lee, John Coleman, Matt Bement, Guannan Zhang, Blackbox Optimization for High-fidelity Heat Transfer Calculations in Metal Additive Manufacturing, Results in Materials, 12, pp. 100258, 2022.
- Jiaxin. Zhang, Hoang. Tran, and Guannan. Zhang, Accelerating Reinforcement Learning with a Directional-Gaussian-Smoothing Evolution Strategy, Electronic Research Archive (doi:10.3934/era.2021075), 2021.
- Jiaxin. Zhang, Hoang. Tran, Dan. Lu, and Guannan. Zhang, Enabling long-range exploration in minimization of multimodal functions, Proceedings of 37th Conference on Uncertainty in Artificial Intelligence (UAI), 2021. (https://arxiv.org/abs/2002.03001)
- Hoang Tran, Dan Lu and Guannan Zhang, Boosting black-box adversarial attack via exploiting loss smoothness, Proceedings of ICLR Workshop on Security and Safety in Machine Learning Systems, 2021.
- Jiaxin Zhang, Sirui Bi, and Guannan Zhang, A directional Gaussian smoothing optimization method for computational inverse design in nanophotonics, Materials & Design, 197 (1), pp. 109213, 2021.
- Jiaxin Zhang, Sirui Bi, and Guannan Zhang, A nonlocal-gradient descent method for inverse design in nanophotonics, Proceedings of NeurIPS Workshop on Machine Learning for Engineering Modeling, Simulation and Design, Dec. 2020.
- In July 2021, Sirui Bi gave a presentation on "A hybrid blackbox optimization for efficient calibration of heat conduction models in additive manufacturing" at the U.S. National Congress on Computational Mechanics.
- In July 2021, Guannan Zhang gave a presentation on "Enabling long-range exploration in minimization of multimodal functions" at the 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021).
- In March 2021, Guannan Zhang gave a presentation on "A nonlocal gradient for high-dimensional blackbox optimization in scientific machine learning", at the SIAM Conference on CS&E.
- In May 2021, Hoang Tran gave a presentation on "Boosting black-box adversarial attack via exploiting loss smoothness" at the ICLR 2021 Workshop on Security and Safety in Machine Learning Systems.
- In December 2020, Sirui Bi gave a presentation on "Directional Gaussian Smoothing Optimization for Inverse Design in Nanophotonics" at The Conference on Machine Learning in Science and Engineering (MLSE 2020).
- In December 2020, Jiaxin Zhang gave a presentation on "A nonlocal-gradient descent method for inverse design in nanophotonics" at NeurIPS 2020 Workshop on Machine Learning for Engineering Modeling, Simulation and Design. [Download our poster] [A short video presentation]
Last Updated: January 29, 2022 - 2:18 pm