Methods for Explainable Artificial Intelligence

Terrell Mundhenk | 18-ERD-021

Project Overview

The overarching aim of this project was to find novel ways to explain the behavior of neural networks. We explored ways of quantifying information in a neural network, which can be used to determine the right size of a network or to infer the way in which a network is processing information. During this study, we developed a new way of computing explainable artificial intelligence saliency maps, which attempt to tell us which parts of an image are important to the decisions made by neural networks. We called the technique we developed a Fast Class Activation Map (CAM) or FastCAM, which is several orders of magnitude faster than methods with similar fidelity. The FastCAM method works by combining a CAM method such as GradCAM with a forward-activation map computed with a statistic that we call the Saliency Map Order Equivalence (SMOE) scale. The addition of the forward-activation maps to CAM methods improves their fidelity, and computational overhead is not substantially increased. The FastCAM source code has been released to the public, allowing its use in popular deep learning tools such as PyTorch.

Mission Impact

Our novel method supports Lawrence Livermore National Laboratory's core competency in high-performance computing, simulation, and data science. Applications of this capability support multiple missions at the Laboratory, including efforts to enhance the capabilities of the Optics Mitigation Facility in Livermore's National Ignition Facility, where neural networks track the outcomes of optic repairs and enable operators to zero in on the parts of the image effecting the network decision so they can more easily second guess a false positive. 

Publications, Presentations, and Patents

Choi, J., et al. 2019. "From Intra-Modal to Inter-Modal Space: Multi-Task Learning of Shared Representations for Cross-Modal Retrieval." IEEE BigMM 2019: 5th IEEE International Conference on Multimedia Big Data, Singapore, September 2019. LLNL-CONF-788499

Friedland, G. 2018. "Predicting the Predictor: Engineering Laws for Sizing Neural Networks." ICSI Presentation, University of California Berkeley, Berkeley, CA. LLNL-PRES-758923

Friedland, G., and M. Krell. 2018. "Capacity Scaling Law for Artificial Neural Networks." arXiv:1708.06019. LLNL-TR-236950

Friedland, G. and A. Metere. 2018. "Machine Learning for Science: Occam's Razor, Accountability and Explainability." SIAM-USC UQ SciML Workshop. Los Angeles, CA, June 2018. LLNL-PROP-744327

Friedland, G., et al. 2018a. "The Helmholtz Method: Using Perceptual Compression to Reduce Machine Learning Complexity." arXiv:1807:10569. LLNL-CONF-751806

——— 2018b. "A Practical Approach to Sizing Neural Networks." arXiv:1810.02328. LLNL-TR-758456

Mundhenk, T. N., et al. 2019. "Efficient Saliency maps for Explainable AI." MLCon National Reconnaissance Office, Washington DC. LLNL-PRES-791936

Palmer, I. A., and T. N. Mundhenk. 2020. "Rapid Materials Engineering via Explainable AI." CED Summer Internship Presentation. LLNL-PRES-813230

Wang, J., et al. 2018. "One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy." arXiv:1810. LLNL-CONF-752187