Safe and Trustworthy Machine Learning

Project Overview

Systems based on machine learning (ML) often face a major challenge when applied "in the wild": The conditions under which the system was deployed can differ from those under which it was developed. These distribution shifts can occur either due to intentionally crafted perturbations or benign natural corruptions. In such scenarios, ML models are known to behave in a highly unpredictable manner. Many of the mission-critical applications at LLNL/DOE are considered high regret and require predictable behavior and strong assurance to achieve safe and correct operation to obtain actionable inferences. Unfortunately, we currently lack the necessary mathematical framework to provide guarantees on correctness and safety of modern ML methods such as deep neural nets (DNNs).

To solve the grand challenge of safe and trustworthy ML for mission-critical applications, we made fundamental advances in two key research directions: 1) Certification: developing mathematical techniques to verify the robustness properties of a pretrained model; and 2) Assured Design: designing training schemes to create ML models that are provably robust to a wide range of distribution shifts. Our project has resulted in several new algorithmic tools and theoretical techniques, thereby, making fundamental advances in trustworthy ML. For example, our proposed innovations have paved the path to designing ML models that are foolproof to intentional perturbations and robust to natural corruption, thereby, facilitating their successful adoption in mission-critical applications.

Mission Impact

Due to its fundamental nature and the widespread impact of ML, our results impact all four of the Lawrence Livermore National Laboratory's focus areas. Theoretical and algorithmic capabilities developed in the project made fundamental advances in Machine Learning. Science and technology tools designed in this project will help in meeting future national security challenges. Specifically, provably robust predictive models will help establish trust in applying machine learning to high regret application areas, such as, automated threat recognition, asteroid detection, critical infrastructure security. The software libraries developed for this project enables a wider adoption of these tools in several mission-critical applications of interest to NNSA.

Publications, Presentations, and Patents

Xu, Kaidi, Sijia Liu, Pin-Yu Chen, Mengshu Sun, Caiwen Ding, Bhavya Kailkhura, and Xue Lin. "Towards an Efficient and General Framework of Robust Training for Graph Neural Networks." In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8479-8483. IEEE, Virtual, May 4-8, 2020.

Diffenderfer, James, and Bhavya Kailkhura. "Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network." International Conference on Learning Representations. Virtual. April 26-30, 2020.

Pan, Boyuan, Yazheng Yang, Kaizhao Liang, Bhavya Kailkhura, Zhongming Jin, Xian-Sheng Hua, Deng Cai, and Bo Li. "Adversarial Mutual Information for Text Generation." In International Conference on Machine Learning, pp. 7476-7486. PMLR, Virtual. July 12-18, 2020.

Gokhale, Tejas, Rushil Anirudh, Bhavya Kailkhura, Jayaraman J. Thiagarajan, Chitta Baral, and Yezhou Yang. "Attribute-Guided Adversarial Training for Robustness to Natural Perturbations." Thirty-Fourth AAAI Conference on Artificial Intelligence, Virtual. February 7-12, 2020.

Xu, Kaidi, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. "Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond." Advances in Neural Information Processing Systems 33, Virtual. December 2020.

Bulusu, Saikiran, Bhavya Kailkhura, Bo Li, Pramod K. Varshney, and Dawn Song. 2020. "Anomalous Example Detection in Deep Learning: A Survey." IEEE Access 8, 132330-132347.

Liu, Sijia, Pin-Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O. Hero III, and Pramod K. Varshney. 2020. "A Primer on Zeroth-order Optimization in Signal Processing and Machine Learning: Principals, Recent Advances, and Applications." IEEE Signal Processing Magazine 37, no. 5, 43-54.

Mehra, Akshay, Bhavya Kailkhura, Pin-Yu Chen, and Jihun Hamm. "How Robust are Randomized Smoothing based Defenses to Data Poisoning?" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13244-13253. Virtual. June 19-25, 2021.

Jia, Ruoxi, Fan Wu, Xuehui Sun, Jiacen Xu, David Dao, Bhavya Kailkhura, Ce Zhang, Bo Li, and Dawn Song. "Scalability vs. Utility: Do We Have To Sacrifice One for the Other in Data Importance Quantification?" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8239-8247. Virtual. June 19-25, 2021.

Sun, Mingjie, Zichao Li, Chaowei Xiao, Haonan Qiu, Bhavya Kailkhura, Mingyan Liu, and Bo Li. "Can Shape Structure Features Improve Model Robustness under Diverse Adversarial Settings?." In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7526-7535. Virtual. June 19-25, 2021.

Mehra, Akshay, Bhavya Kailkhura, Pin-Yu Chen, and Jihun Hamm. "On the Effectiveness of Poisoning against Unsupervised Domain Adaptation." In ICML 2021 Workshop on Adversarial Machine Learning. Virtual. July 24, 2021.

Yang, Zhuolin, Linyi Li, Xiaojun Xu, Bhavya Kailkhura, Tao Xie, and Bo Li. "On the Certified Robustness for Ensemble Models and Beyond." In International Conference on Learning Representations. Virtual. May 3-7, 2021.

Diffenderfer, James, Brian Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura. "A Winning Hand: Compressing Deep Networks can Improve Out-of-Distribution Robustness." Advances in Neural Information Processing Systems 34. 664-676. Virtual. December 6-14, 2021.

Long, Yunhui, Boxin Wang, Zhuolin Yang, Bhavya Kailkhura, Aston Zhang, Carl Gunter, and Bo Li. "G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators." Advances in Neural Information Processing Systems 34, 2965-2977. Virtual. December 6-14, 2021.

Wu, Fan, Linyi Li, Huan Zhang, Bhavya Kailkhura, Krishnaram Kenthapadi, Ding Zhao, and Bo Li. "COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks." In International Conference on Learning Representations. Virtual. May 3-7, 2021.

Bhardwaj, Kshitij, James Diffenderfer, Bhavya Kailkhura, and Maya Gokhale. "Unsupervised Test-Time Adaptation of Deep Neural Networks at the Edge: a Case Study." In 2022 Design, Automation & Test in Europe Conference & Exhibition, pp. 412-417. IEEE, Virtual. March 14-23, 2022.