Large-Scale Multimodal Deep Learning for Nuclear Nonproliferation Analysis

Barry Chen | 17-SI-003


This project developed artificial intelligence algorithms to help nuclear nonproliferation analysts sort through vast amounts of unlabeled multimodal data in the form of text, images, and videos and quickly find data that pertain to the equipment, technologies, materials, and processes used in the development of nuclear fuels. These algorithms make it possible for analysts to process much larger quantities of data, enabling them to find evidence of proliferation processes earlier and more thoroughly.

Impact on Mission

This project supports Lawrence Livermore National Laboratory's core competencies in high-performance computing, simulation, and data sciences and advances Livermore's nonproliferation mission. New capabilities developed as a result of our work helped Livermore establish three new programs in Global Security, of which the NA-22 Advanced Data Analytics for Proliferation Detection (ADAPD) Venture Project is the most significant. ADAPD is a five-lab, five-year research program developing novel data science algorithms to help analysts detect and characterize nuclear proliferation activities. This work also supports the biosecurity mission of the Laboratory and is generating interest in cooperative research and business ventures from industry partners.

Publications, Presentations, Etc.

Chen, B. 2018. "Deep Learning: A Guide for Practitioners in the Physical Sciences." Phys. Plasmas. 25. doi: 10.1063/1.5020791. LLNL-JRNL-743970.

Chen, B., et al. 2019. "Toward a Deep Learning System for Making Sense of Unlabeled Multimodal Data." The Next Wave . 2019. LLNL-JRNL-761289.

Choi, J., et al. 2017. "The Geo-Privacy Bonus of Popular Photo Enhancements." International Conference on Multimedia Retrieval. 2017. LLNL-CONF-739933.

Cong, G., et al. 2018. "Accelerating Deep Neural Network Training for Action Recognition on a Cluster of GPUs." International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD). 2018. LLNL-CONF-760618.

––– . 2019. "From Intra-Modal to Inter-Modal Space: Multi-Task Learning of Shared Representations for Cross-Modal Retrieval." IEEE International Conference on Multimedia Big Data. 2019. LLNL-CONF-788499.

––– . 2019. "Video Action Recognition with An Additional End-To-end Trained Temporal Stream." Winter Conference on Applications of Computer Vision (WACV19). 2019. LLNL-PRES-764547.

Dryden, N., et al. 2018. "Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems." Workshop on Machine Learning in HPC Environments (MLHPC'18). 2018. LLNL-CONF-757866.

––– . 2019. "Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism." International Parallel and Distributed Processing Symposium (IPDPS'19). 2019. LLNL-CONF-759919.

––– . 2019. "Channel and Filter Parallelism for Large-Scale CNN Training." International Conference for High Performance Computing, Networking, Storage, and Analysis 2019. LLNL-CONF-771796.

Feldman, Y., et al. 2018. "Toward a Multimodal-Deep Learning Retrieval System for Monitoring Nuclear Proliferation Activities." Journal of Nuclear Materials Management. 2018. LLNL-JRNL-751737.

Jacobs, S., et al. 2017. "Towards Scalable Parallel Training of Deep Neural Networks." Workshop on Machine Learning in HPC Environments (MLHPC'17) 2017. LLNL-CONF-737759.

––– . 2019. "Parallelizing Training of Deep Generative Models on Massive Scientific Datasets." IEEE Cluster Conference. 2019. LLNL-CONF-776577.

Jing, L., et al. 2017. "DCAR: A Discriminative and Compact Audio Representation for Audio Processing." IEEE Transactions on Multimedia 2017. LLNL-JRNL-739946.

Mundhenk, T., et al. 2018. "Improvements to Context Based Self-Supervised Learning." Conference on Computer Vision and Pattern Recognition (CVPR) 2018. LLNL-CONF-741553.

Ni., K., et al. 2017. "Sampled Image Tagging and Retrieval Methods on User Generated Content." British Machine Vision Conference 2017. LLNL-CONF-712477.

Spears, B., et al. 2018. "Deep Learning: A Guide for Practitioners in the Physical Sciences." Physics of Plasmas 2018. LLNL-JRNL-743970.