Accelerating Feedstock Optimization Using Computer Vision, Machine Learning, and Data Analysis Techniques

Project Overview

Feedstock materials are the building blocks used throughout Lawrence Livermore National Laboratory to create components, prototypes, and products for its missions. These materials include metal powders, ceramic particles, optical materials, polymers, nanomaterials, and high explosive crystals. Often, these building blocks are studied, optimized, engineered, and customized to meet performance requirements before becoming critical components integrated at the systems level. However, a persistent obstacle to developing and deploying materials in a timely manner is the optimization and qualification cycle. Fundamentally, macroscopic component performances are largely dependent on the micro- and mesoscale properties of the starting feedstock materials, but the relationships between these disparate length scales are often unclear and difficult to establish. In part, not knowing what feedstock attributes to optimize, tune, and synthesize are some of the reasons that materials scientists are in a perpetual optimization cycle, taking years to create optimized custom feedstock materials for specific applications with stringent requirements.

In this project, we use computer vision, machine learning, and data analysis approaches to (1) identify critical feedstock attributes obtained from experimental data and databases; (2) correlate feedstock material attributes with component performance; and (3) predict overall performance based on these attributes. We also addressed computational challenges posed by "small data" (i.e., prohibitively high data-acquisition costs resulting in limited information) and developed computer-generated models that are easier to understand and explain. To demonstrate this capability, we focused our efforts on high-explosives (HE) feedstock development, an important class of materials for the Laboratory and for the nation. We explored the application of computer vision and deep learning (DL) to predict material properties based on scanning electron microscopy (SEM) images. We showed that it is possible to train machine learning (ML) models to predict materials performance based on SEM images alone, demonstrating this capability on the mission-relevant problem of predicting uniaxially compressed peak stress. Our image-based approach reduces error by an average of 24% over baselines representative of the current state-of-the-practice (i.e., domain-expert's analysis and correlation). Furthermore, we demonstrated that models trained using ML techniques can discover and utilize informative material attributes previously underutilized by domain experts.

Mission Impact

Successful completion of the project has provided fundamental understanding of how HE material attributes including particle size, voids, surface textures, and sizes contribute to the mechanical and physics performance. These insights will significantly accelerate future HE development for multiple programs. More importantly, this project has generated and demonstrated utilities of computer vision, deep learning, and data analytic tools in accelerating custom feedstock optimization, qualification, and deployment. Development of such tools will fundamentally change how we approach materials discovery, optimization, and certification and open up new opportunities to tackle grand challenges in materials science. In addition, the project has significantly enhanced the science and technology base of the Laboratory, while strengthening our core competencies in advanced materials and manufacturing, high-performance computing, simulation, and data science and nuclear weapons science.

Publications, Presentations, and Patents

Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi, and T. Yong-Jin Han. "Deep probabilistic kernels for sample-efficient learning," arXiv preprint arXiv:1910.05858 (2019).

Bhavya Kailkhura, Brian Gallagher, Sookyung Kim, Anna Hiszpanski, and T. Yong-Jin Han. "Reliable and explainable machine-learning methods for accelerated material discovery," npj Computational Materials 5, no. 1 (2019): 1-9.

Youngwoo Cho, Sookyung Kim, Peggy Pk Li, Michael P. Surh, T. Yong-Jin Han and Jaegul Choo. "Physics-guided Reinforcement Learning for 3D Molecular Structures," (2019).

Elsa A. Olivetti, Jacqueline M. Cole, Edward Kim, Olga Kononova, Gerbrand Ceder, T. Yong-Jin Han, and Anna M. Hiszpanski. "Data-driven materials research enabled by natural language processing and information extraction," Applied Physics Reviews 7, no. 4 (2020): 041317.

Shusen Liu, Bhavya Kailkhura, Jize Zhang, Anna M. Hiszpanski, Emily Robertson, Donald Loveland, and T. Yong-Jin Han. "Explainable Deep Learning for Uncovering Actionable Scientific Insights for Materials Discovery and Design," arXiv preprint arXiv:2007.08631 (2020).

Donald Loveland, Bhavya Kailkhura, Piyush Karande, Anna M. Hiszpanski, and T. Yong-Jin Han. "Automated Identification of Molecular Crystals' Packing Motifs," Journal of Chemical Information and Modeling 60, no. 12 (2020): 6147-6154.

Jize Zhang, Bhavya Kailkhura, and T. Yong-Jin Han. "Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning," In International Conference on Machine Learning, pp. 11117-11128. PMLR, 2020.

Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi, and T. Yong-Jin Han. "Probabilistic neighbourhood component analysis: Sample efficient uncertainty estimation in deep learning," arXiv preprint arXiv:2007.10800 (2020).

Shusen Liu, Bhavya Kailkhura, Jize Zhang, Anna M. Hiszpanski, Emily Robertson, Donald Loveland, and T. Yong-Jin Han. "Actionable attribution maps for scientific machine learning," arXiv preprint arXiv:2006.16533 (2020).

Brian Gallagher, Matthew Rever, Donald Loveland, T. Nathan Mundhenk, Brock Beauchamp, Emily Robertson, Golam G. Jaman, Anna M. Hiszpanski, and T. Yong-Jin Han. "Predicting compressive strength of consolidated molecular solids using computer vision and deep learning," Materials & Design 190 (2020): 108541.

Anna M. Hiszpanski, Brian Gallagher, Karthik Chellappan, Peggy Li, Shusen Liu, Hyojin Kim, Jinkyu Han, Bhavya Kailkhura, David J. Buttler, and T. Yong-Jin Han. "Nanomaterial synthesis insights from machine learning of scientific articles by extracting, structuring, and visualizing knowledge," Journal of chemical information and modeling 60, no. 6 (2020): 2876-2887.

Hyojin Kim, Jinkyu Han, and T. Yong-Jin Han. "Machine vision-driven automatic recognition of particle size and morphology in SEM images," Nanoscale 12, no. 37 (2020): 19461-19469.

Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi, and T. Yong-Jin Han. "Can Your AI Differentiate Cats from Covid-19? Sample Efficient Uncertainty Estimation for Deep Learning Safety," in International Conference on Machine Learning (ICML). 2020.

Qunwei Li, Bhavya Kailkhura, Rushil Anirudh, Jize Zhang, Yi Zhou, Yingbin Liang, T. Yong-Jin Han, and Pramod K. Varshney. "MR-GAN: Manifold Regularized Generative Adversarial Networks for Scientific Data," Proceedings of Machine Learning Research vol 107 (2020): 1-27.

Imanuel Bier, Dana O'Connor, Yun-Ting Hsieh, Wen Wen, Anna M. Hiszpanski, T. Yong-Jin Han, and Noa Marom. "Crystal structure prediction of energetic materials and a twisted arene with Genarris and GAtor," CrystEngComm (2021).

Phan Nguyen, Donald Loveland, Joanne T. Kim, Piyush Karande, Anna M. Hiszpanski, and T. Yong-Jin Han. "Predicting Energetics Materials' Crystalline Density from Chemical Structure by Machine Learning," Journal of Chemical Information and Modeling 61, no. 5 (2021): 2147-2158.

Jize Zhang, Bhavya Kailkhura, and T. Yong-Jin Han. "Leveraging Uncertainty from Deep Learning for Trustworthy Material Discovery Workflows," ACS omega 6, no. 19 (2021): 12711-12721.

T. Nathan Mundhenk, Ian. A. Palmer, Brian J. Gallagher, and T. Yong-Jin Han. Explaining neural network predictions of material strength," No. LLNL-CONF-825111. Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States), 2021.

Xiaoting Zhong, Brian Gallagher, Keenan Eves, Emily Robertson, T. Nathan Mundhenk, T. Yong-Jin Han, "A study of real-world micrograph data quality and machine learning model robustness," njp Computational Materials, ASAP, (2021).