**Project Status:** Active

**Project Summary**: We propose to develop a scalable black-box training framework for scientific machine learning (SciML) models that are non-trainable with existing automatic differentiation-based algorithms. Our particular interest to this effort is to study how to train data-driven SciML models to learn missing physics of a complex system for advancing forward simulations. Specifically, this effort aims at achieving the following objectives: (1) develop a novel non-local gradient with structured sampling to enable non-local exploration for escaping from local minima and to achieve sufficient accuracy in gradient estimation; (2) advance theoretical analyses for a class of non-convex training problems to help domain scientists tune the hyper-parameters of the proposed training framework; and (3) exploit high-performance computing to accelerate the time to solution for black-box training problems for which loss functions involve computationally expensive black-box simulators. The proposed framework will be demonstrated on two distinct applications. The first is to train machine learning-based constitutive models to predict mercury dynamics in the Spallation Neutron Source mercury target, and the second is to train heat-source models to predict time-dependent laser scan paths that yield desirable micro-structures in three-dimensional printed metal components. This effort will advance the state of the art of several machine learning areas such as reinforcement learning and variational inference.

**Principal Investigator**: Guannan Zhang (CSMD, ORNL)

**Senior Investigators: **, Jiaxin Zhang (CSMD, ORNL), Hoang Tran (CSMD, ORNL), Dan Lu (CSED, ORNL), Matthew Bement (CSED, ORNL), Yousub Lee (CSED, ORNL), Bejamin Stump (MSTD, ORNL), Sirui Bi (CSED, ORNL)

**Funding Period**: Sept. 2020 to Aug. 2022

**Publications: **

- H. Tran and G. Zhang, An adaptive nonlocal gradient descent method for high-dimensional black-box optimization,
**SIAM Journal on Scientific Computing**, under review. - H. Tran, D. Lu, and G. Zhang, Exploiting the local parabolic landscapes of adversarial losses to accelerate black-box adversarial attack,
**Proceedings of 17th European Conference on Computer Vision (ECCV 2022)**, pp. 317–334, 2022. - M. Radaideh, H. Tran, L. Lin, H. Jiang, D. Winder, S. Gorti, G. Zhang, J. Mach, S. Cousineau, Model Calibration of the Liquid Mercury Spallation Target using Evolutionary Neural Networks and Sparse Polynomial Expansions,
**Nuclear Inst. and Methods in Physics Research B**, 525(15), pp. 41-54, 2022. - Sirui Bi, Benjamin Stump, Yousub Lee, John Coleman, Matt Bement, Guannan Zhang, Blackbox Optimization for High-fidelity Heat Transfer Calculations in Metal Additive Manufacturing,
, 12, pp. 100258, 2022.**Results in Materials** - Jiaxin. Zhang, Hoang. Tran, and Guannan. Zhang, Accelerating Reinforcement Learning with a Directional-Gaussian-Smoothing Evolution Strategy,
**Electronic Research Archive**(doi:10.3934/era.2021075), 2021. - Jiaxin. Zhang, Hoang. Tran, Dan. Lu, and Guannan. Zhang, Enabling long-range exploration in minimization of multimodal functions,
, 2021. (https://arxiv.org/abs/2002.03001)**Proceedings of 37th Conference on Uncertainty in Artificial Intelligence (UAI)** - Hoang Tran, Dan Lu and Guannan Zhang, Boosting black-box adversarial attack via exploiting loss smoothness,
, 2021.**Proceedings of ICLR Workshop on Security and Safety in Machine Learning Systems** - Jiaxin Zhang, Sirui Bi, and Guannan Zhang, A directional Gaussian smoothing optimization method for computational inverse design in nanophotonics,
, 197 (1), pp. 109213, 2021.**Materials & Design** - Sirui Bi, Jiaxin Zhang and Guannan Zhang, Towards efficient uncertainty estimation in deep learning for robust energy prediction in materials chemistry,
**Proceedings of ICLR Workshop on Deep Learning for Simulation**, 2021. - Jiaxin Zhang, Sirui Bi, and Guannan Zhang, A nonlocal-gradient descent method for inverse design in nanophotonics,
**Proceedings of NeurIPS Workshop on Machine Learning for Engineering Modeling, Simulation and Design**, Dec. 2020.

**Activities:**

- In October 2022, H. Tran presented our work on “Exploiting the local parabolic landscapes of adversarial losses to accelerate black-box adversarial attack” at the
**17th European Conference on Computer Vision (ECCV 2022)**. - In July 2022, G. Zhang presented our work on “a nonlocal gradient for high-dimensional black-box optimization” at the
**SIAM Annual Meeting**. - In April 2022, H.Tran presented our work on“a nonlocal gradient descent method for high-dimensional black-box optimization” at the
**SIAM conference on UQ**. - In July 2021, Sirui Bi gave a presentation on "A hybrid blackbox optimization for efficient calibration of heat conduction models in additive manufacturing" at the
**U.S. National Congress on Computational Mechanics**. - In July 2021, Guannan Zhang gave a presentation on "Enabling long-range exploration in minimization of multimodal functions" at the
**37th Conference on Uncertainty in Artificial Intelligence (UAI 2021).** - In March 2021, Guannan Zhang gave a presentation on "A nonlocal gradient for high-dimensional blackbox optimization in scientific machine learning", at the
**SIAM Conference on CS&E**. - In May 2021, Hoang Tran gave a presentation on "Boosting black-box adversarial attack via exploiting loss smoothness" at the
**ICLR 2021 Workshop on Security and Safety in Machine Learning Systems.** - In December 2020, Sirui Bi gave a presentation on "Directional Gaussian Smoothing Optimization for Inverse Design in Nanophotonics" at
**The Conference on Machine Learning in Science and Engineering (MLSE 2020).** - In December 2020, Jiaxin Zhang gave a presentation on "A nonlocal-gradient descent method for inverse design in nanophotonics" at
**NeurIPS 2020 Workshop on Machine Learning for Engineering Modeling, Simulation and Design**. [Download our poster] [A short video presentation]

Last Updated: November 29, 2022 - 12:45 pm