MADSTARE: Modeling and Analysis for Data-Starved or Ambiguous Environments

Project Overview

The Modeling and Analysis for Data-Starved or Ambiguous Environments (MADSTARE) project is focused on developing data analytics capabilities for space situational awareness (SSA). SSA has many challenging data problems because the space domain is remote, which leads to observations that are sparse in time or information and often indeterminate about the mission objectives. These challenges have been exacerbated in the past five years by the transition of U.S. government SSA objectives from passive catalog maintenance to dynamic and short time-scale support to operations.

A major focus of MADSTARE research has been in developing the modeling, simulation, and analysis tools to support this new shift in SSA priorities and objectives. We developed methods for data exploitation that can be reliably applied to critical security missions when only small or incomplete training data is available. To enable mission assurance, we focus on interpretability through uncertainty quantification (UQ) and regularization through incorporation of physics knowledge. We developed and validated a new machine learning code based on Gaussian processes (GPs) that provides meaningful UQ and state-of-the-art performance in limited data or limited training time regimes. We demonstrated how these machine learning advances help connect deep learning to Bayesian modeling methods, how they can be implemented on near-term quantum computers, and how they can be applied to SSA domain problems within a high-fidelity modeling and simulation framework.

Mission Impact

The technologies developed under this project address fundamental limitations of deep learning and are broadly applicable to many core Lawrence Livermore National Laboratory (LLNL) missions and broader scientific investigations and national security domains. Other research fields that can readily use the technologies developed herein include: quantum computing, collaborative autonomy, computational workflows, and simulation surrogates. The chief intellectual merit of the proposal is in the enhanced understanding of how and why deep learning works, the mitigation of several deep learning shortfalls, and the implications for the reliability of scientific analyses and critical operations that rely on machine learning methods. This project supports the LLNL Mission Research Area in Space Security and has impacted modeling, simulation, and analysis capabilities for space situational awareness. The project also supports LLNL Core Competencies in High-Performance Computing, Simulation, and Data Science.

Publications, Presentations, and Patents

Bernstein, Jason. "Probabilistic Data Association for Orbital-Element Estimation Using Multistage Expectation-Maximization," Journal of Aerospace Information Systems 18, no. 5 (2021): 250-268.

Otten, Matthew, Imène R. Goumiri, Benjamin W. Priest, George F. Chapline, and Michael D. Schneider. "Quantum machine learning using gaussian processes with performant quantum kernels," arXiv preprint arXiv:2004.11280 (2020).

Goumiri, Imene R., Benjamin W. Priest, and Michael D. Schneider. "Reinforcement Learning via Gaussian Processes with Neural Network Dual Kernels." In 2020 IEEE Conference on Games (CoG), pp. 1-8. IEEE, 2020.

Goumiri, Imène R., Amanda L. Muyskens, Michael D. Schneider, Benjamin W. Priest, and Robert E. Armstrong. "Star-Galaxy Separation via Gaussian Processes with Model Reduction," arXiv preprint arXiv:2010.06094 (2020).

Miller, Caleb, Michael D. Schneider, Jem N. Corcoran, and Jason Bernstein. "Bayesian Fusion of Data Partitioned Particle Estimates," arXiv preprint arXiv:2010.13921 (2020).

Muyskens, Amanda, Benjamin Priest, Imène Goumiri, and Michael Schneider. "MuyGPs: Scalable Gaussian Process Hyperparameter Estimation Using Local Cross-Validation," arXiv preprint arXiv:2104.14581 (2021).

Muyskens, Amanda L., Imène R. Goumiri, Benjamin W. Priest, Michael D. Schneider, Robert E. Armstrong, Jason M. Bernstein, and Ryan Dana. "Star-Galaxy Image Separation with Computationally Efficient Gaussian Process Classification," Accepted in the Astronomical Journal. arXiv:2105.01106 (2021).

Buchanan, James J., Michael D. Schneider, Robert E. Armstrong, Amanda L. Muyskens, Benjamin W. Priest, and Ryan J. Dana. "Gaussian Process Classification for Galaxy Blend Identification in LSST," Accepted in the Astrophysical Journal. arXiv:2107.09246 (2021).

Muyskens, Amanda, Benjamin Priest, Imène Goumiri, and Michael Schneider. "Sensitivity Analysis of Kriging Weights," in review, arXiv (2021).

Muyskens, Amanda, Benjamin Priest, Imène Goumiri, and Michael Schneider. "On the Correspondence of the NNGP Kernel and the Matérn Kernel," in review, arXiv (2021).

Bernstein, Jason, Andrey Fillipov, Michael Schneider, Caleb Miller, Jem Corcoran. "Quantifying Uncertainty in All-to-All Estimates of Space Object Conjunction Probabilities using U-Statistics," in review, arXiv (2021).

Chapline, George, Michael D. Schneider, Matthew Otten, Caleb Miller. "Quantum Solutions to Classical Optimal Control," in review, arXiv (2021).

Golovich, Nathan, Noah Lifset, Robert Armstrong, Eric Green, Michael D. Schneider, and Roger Pearce. "A New Blind Asteroid Detection Scheme," arXiv preprint: 2104.03411(2021).

Otten, Matthew, Iméne Goumiri, George Chapline, Michael D. Schneider. "Quantum Kernel Based Machine Learning Using Gaussian Processes." In Quantum Machine Learning and Data Analytics Workshop, Purdue University (September 2019).

Schneider, Michael, Matt Otten, George Chapline, Jonathan Dubois, Iméne Goumiri, Benjamin Priest. "Quantum Machine Learning with Gaussian Processes." In Workshop on Machine Learning, Quantum Acceleration and Robust Quantum Systems, UCLA CQSE. (November 18, 2019).

Golovich, Nathan. "Significant Detections are Hiding in the Noise." In Space Situational Awareness Data Science Working Group, Washington, D.C. (2020).

Goumiri, Iméne, Amanda Muyskens, Michael Schneider, Benjamin Priest, Robert Armstrong. "Star-Galaxy Separation via Gaussian Processes with Model Reduction." In Advanced Maui Optical and Space Surveillance Technologies Conference (September, 2020).

Schneider, Michael D., Matthew Otten, George Chapline, Iméne Goumiri, Benjamin Priest. "Quantum Machine Learning using Gaussian Processes." In University of Colorado, Boulder Applied Mathematics Department Colloquium (September 2020).

Bernstein, Jason, Michael Schneider, Andrey Filippov, Caleb Miller, Jem Corcoran. "Estimating Space Object Conjunction Probabilities using U-Statistics." In Space Situational Awareness Data Science Working Group (2021).

Muyskens, Amanda. "MuyGPs: Scalable Gaussian Process Hyperparameter Estimation Using Local Cross-Validation." In LLNL Data Science Summer Institute (August 2021).

Priest, Benjamin, Amanda Muyskens. "MuyGPyS: Fast Implementation of the MuyGPs Gaussian Process Hyperparameter Estimation Algorithm." Github repository: https://github.com/LLNL/MuyGPyS (2021).

Meyers, Joshua, Edward Schlafly, Michael Schneider. "SSAPy: Space Situational Awareness Modeling in Python." LLNL release under government purpose rights (2021).

Pruett, Kerianne, Ryan Dana, Amanda Muyskens, Iméne Goumiri, Benjamin Priest, Michael Schneider. "Zwicky Transient Facility (ZTF) Image Classification using MuyGPyS." LLNL Data Science Institute Open Data Initiation: https://data-science.llnl.gov/open-data-initiative (2021).