A Deep Bayesian Active Learning Framework for Temporal Multimodal Data

Project Overview

The need for efficient learning from temporal multimodal information sources, a scenario that often arises when inferring about complex dynamical systems, is pervasive to many strategic missions of Lawrence Livermore National Laboratory (LLNL), such as biosecurity and power grid security. There is a dire capability gap, as current machine learning models are inadequate for addressing temporal data fusion challenges. This Laboratory Directed Research and Development (LDRD) project addresses this capability gap by developing statistical machine learning techniques to integrate heterogenous timeseries data. The LDRD demonstrates that incorporating temporal dynamics when fusing information from disparate sources significantly improves predictive performance and/or provides new insights over techniques that disregard temporal dynamics, in multiple healthcare and energy applications. This project produced numerous academic contributions including multiple publications in high-impact journals and conferences and directly contributed to three new externally funded proposals. In addition, this LDRD supported the recruitment and training of a post-doc, three summer students and three full-time staff members, and established strategic collaborations with multiple academic, government and industry partners, such as University of California San Francisco (UCSF), University of California Merced, Stanford University, University of Toledo, Veterans Affairs, Kaiser Permanente Research and ProMedica Health System. Overall, this LDRD has directly contributed towards broadening and enhancing LLNL's growing leadership in biomedical research.

Mission Impact

This LDRD has resulted in the development of a suite of general methods and technical capabilities for learning from temporal multimodal data, that are immediately transferable to LLNL's growing portfolio of biomedical research and LLNL's core competency in Bioscience and Bioengineering. The techniques developed in this LDRD will directly support LLNL's ongoing biomedical research efforts (e.g., Track-TBI, NCI pilot series) as well as future research efforts (e.g., Million Veterans Program from US Department of Veterans Affairs) in modeling EHR records of patients as well as detecting unforeseen, and potentially actionable, patient health changes. The techniques developed in this LDRD will also directly support LLNL's ongoing research in smart grid systems. In the longer term, the techniques developed in this LDRD can support other biomedical/energy/non-proliferation applications at LLNL: by 1) providing a platform to assess the relative efficacy of candidate compounds in affecting system states (e.g., tumor growth) over time, as part of the ATOM initiative; 2) expanding to address largely disparate equipment types and distributed energy resources in the smart grid; 3) supplementing our current non-proliferation threat detection capabilities with temporal inferences.

Publications, Presentations, and Patents

Cadena, Jose, Priyadip Ray, and Emma Stewart. "Fingerprint discovery for transformer health prognostics from micro-phasor measurements." In ICML 2019, Time Series Workshop. 2019. LLNL-CONF-774229.

Nguyen, Sam, Brenda Ng, Alan D. Kaplan, and Priyadip Ray. "Attend and Decode: 4D fMRI Task State Decoding Using Attention Models." In Machine Learning for Health, pp. 267-279. PMLR, 2020. LLNL-CONF-788817.

Nguyen, Sam, Brenda Ng, Alan D. Kaplan, and Priyadip Ray. "Task State Classification of Four-Dimensional fMRI Time Series Using Deep Residual Convolutional Neural Networks." In International Conference on Medical Imaging and Case Reports (MICR-2019). LLNL-POST-796919.

Bhat, Harish S., Majerle Reeves, and Ramin Raziperchikolaei. "Estimating Vector Fields from Noisy Time Series." In 2020 54th Asilomar Conference on Signals, Systems, and Computers, pp. 599-606. IEEE, 2020. LLNL-CONF-800042.

Meng, Rui, Braden Soper, Herbert KH Lee, Vincent X. Liu, John D. Greene, and Priyadip Ray. "Nonstationary multivariate Gaussian processes for electronic health records," Journal of Biomedical Informatics 117 (2021): 103698. LLNL-JRNL-810575.

Soper, Braden, Jose Cadena, Sam Nguyen, Kwan Ho Ryan Chan, Paul Kiszka, Lucas Womack, Mark Work et al. "Dynamic Stratification of Disease Severity and Prognosis of Hospitalized Covid-19 Patients Using Hidden Markov Model," Journal of Investigative Medicine (2021): 1111-1112. LLNL-ABS-818779.

Nguyen, Sam, Kwan Ho Ryan Chan, Jose Cadena, Braden Soper, Paul Kiszka, Lucas Womack, Mark Work et al. "Early Prediction Of Adverse Outcomes For Covid-19 Patients Under Cost Constraints," Journal of Investigative Medicine (2021): 1112-1114. LLNL-ABS-818777.

Vilson, Fernandino, Boya Zhang, James Brooks, Thomas Osborne, Priyadip Ray, and John Leppert. " Psa Variability And The Diagnosis Of Clinically Significant Prostate Cancer," The Journal of Urology 206, no. Supplement 3 (2021): e501-e501. LLNL-ABS-820491.

Nguyen, Sam, Ryan Chan, Jose Cadena, Braden Soper, Paul Kiszka, Lucas Womack, Mark Work et al. "Budget Constrained Machine Learning for Early Prediction of Adverse Outcomes for COVID-19 Patients." Nature Scientific Reports (Accepted). LLNL-JRNL-821075.

Kwon, Daniel H., JE Cadena Pico, Sam Nguyen, Kwan Ho Ryan Chan, Brian Soper, A. L. Gryshuk, P. Ray, and Franklin W. Huang. "COVID-19 Outcomes in Patients with Cancer: Findings from the University of California Health System Database." medRxiv (2021). LLNL-JRNL-823909.