Next Generation Machine Learning

Project Overview

Despite the explosive growth and several incredible successes in machine learning, existing techniques rely heavily on human intuition and complex heuristics derived from extensive empirical studies. At its most fundamental level, the underlying learning rules responsible for the current state of the art are variants of gradient descent based statistical learning methods. However, these methods are limited by a set of engineering challenges inherent to statistical learning -- mainly the need for large amounts of training data, mercurial optimizations, and opaque solutions. This suggests that a better understanding of how and why these techniques work, as well as different ideas on how to create, for example, better architectures have the potential to significantly advance the field. This project set out to build a unique collaboration between computational and experimental neuroscientists, and machine learning across three sites (Lawrence Livermore National Laboratory [LLNL], Georgetown University and UC San Diego) to obtain insights into when and how learning occurs in both biological and artificial neural networks. The teams at Georgetown and UC San Diego focused on computational and experimental studies of the human brain to characterize learning, and ultimately develop biologically inspired dynamic networks for learning. LLNL led the effort to develop new interpretability approaches for deep networks, and new ML algorithms to analyze EEG datasets commonly used in neuroscience.

Georgetown's experiments reveal exciting new results to pinpoint the "when" and "where" learning occurs using a new task of made-up word categorization among several human subjects. LLNL has developed sophisticated new deep learning tools to analyze noisy EEG datasets [1]; to obtain remarkable accuracy, that have helped reinforce some of the hypotheses identified by the neuroscientists. LLNL has made significant advances in designing interpretable models for modern deep learning systems - from intuitive and explainable visualizations for high dimensional scientific datasets [2]; exploiting graph structure for easy interpretability on any kind of deep network [3]; robust explanations and interpretability with noisy and unreliable datasets [4]. Together, the project has advanced our understanding of learning in human brains, as well as deep neural networks. It has also laid the foundation for several potential research projects and collaborations.

Mission Impact

This project has fundamentally advanced LLNL's core competency in data science and machine learning, particularly in the areas of robustness and interpretability. Furthermore, the team is in contact with multiple LDRD and programmatic efforts that have expressed significant interest in and could benefit directly from results of this project. Tools developed within this project (for e.g., function-preserving linear projections (FPP)) have benefited other projects due to their general applicability. Finally, applying advanced machine learning techniques to the cognitive science experiments at Georgetown have led to new insights from a neuroscience perspective, and will lead to high impact publications in cognitive neuroscience and involve LLNL in a new area of computationally intensive science. We expect our results to be integrated with programmatic efforts and the new approaches to lead to future proposals to Advanced Scientific Computing Research (ASCR), i.e., in the area of scientific machine learning, or Defense Advanced Research Projects Agency (DARPA), i.e. as part of the explainable AI efforts.

Publications, Presentations, and Patents

Thiagarajan, Jayaraman J., Deepta Rajan, Sameeksha Katoch, and Andreas Spanias. "DDxNet: a deep learning model for automatic interpretation of electronic health records, electrocardiograms and electroencephalograms," Scientific Reports 10, no. 1: 1-11. 2020.

Liu, Shusen, Rushil Anirudh, Jayaraman J. Thiagarajan, and Peer-Timo Bremer. "Uncovering interpretable relationships in high-dimensional scientific data through function preserving projections," Machine Learning: Science and Technology 1, no. 4: 045016. 2020.

Anirudh, Rushil, Jayaraman J. Thiagarajan, Rahul Sridhar, and Peer-Timo Bremer. "MARGIN: Uncovering Deep Neural Networks Using Graph Signal Analysis," Frontiers in Big Data 4:3. 2021.

Thiagarajan, Jayaraman J., Vivek Narayanaswamy, Rushil Anirudh, Peer-Timo Bremer, and Andreas Spanias. "Accurate and Robust Feature Importance Estimation under Distribution Shifts." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 9, pp. 7891-7898. 2021.