Robust Control of Scientific Simulations with Deep Reinforcement Learning

Daniel Faissol | 20-ERD-033

Project Overview

Deep reinforcement learning (DRL) is revolutionizing artificial intelligence by demonstrating the ability to learn how to make smart and strategic decisions in the context of a highly dynamic and complex system. This project focused on exploring DRL approaches for performing adaptive mesh refinement (AMR) in the context of finite-element modeling. We demonstrated the feasibility of the concept and gained understanding of the challenges that must be addressed for it to become practical for real-world problems.

Specifically, we created a testbed for developing DRL algorithms for AMR and applied DRL algorithms to simple AMR problems of both smooth and non-smooth analytical functions. Our DRL solutions outperformed the Zienkiewicz-Zhu (ZZ) error estimator, a widely used heuristic, on this testbed for meshes up to 256 x 256 elements. We also demonstrated that the learned policies are somewhat generalizable. For example, policies trained on simple, non-smooth functions still outperform ZZ when deployed on a hydrodynamic code solution of three interacting Sedov blast waves, resulting in 45 percent more error reduction, or 30 percent fewer added degrees of freedom.

These results demonstrate the feasibility of the approach. This project has, for the first time, enabled the use of DRL to accelerate finite element modeling, potentially leading to order-of-magnitude performance improvements in these types of large-scale simulations and more accurate computational predictions.

Mission Impact

Large-scale, finite-element modeling on high-performance computing platforms supports many Lawrence Livermore National Laboratory and NNSA/DOE missions, including stockpile stewardship and multi-domain deterrence. This research enabled development of robust methods for performing AMR with DRL on more challenging and impactful science and engineering problems. Such a capability could result in significant advances in the efficiency of finite-element-based simulations, which are widely used in applications ranging from supporting Livermore's National Ignition Facility to designing biomedical devices.