Lawrence Livermore National Laboratory

Gerald Friedland (17-ERD-096)

Executive Summary

We are developing robust models for graphical analytics that work in conjunction with swarm-sensor architectures to maximize the benefit of distributed intelligence sensor readings. This research advances the state of the art in collaborative autonomy relevant to nonproliferation and defense-related situational awareness and response needs.

Project Description

Intelligence is often gathered from diverse sources, including in swarms of autonomous gathering entities. The question is how best to exploit and combine the available information to maximize its benefit. One approach is to capture the sources and their statistical dependencies using graphical models. These generative graphical models—belief networks, Bayesian networks, or Bayesian graphical models—represent a set of random variables and their conditional dependencies via a directed acyclic graph. For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms. Given symptoms, the network can be used to compute the probabilities of the presence of various diseases. When mapped to a swarm intelligence-gathering scenario, it could represent the values or states of (possibly multi-dimensional) sensor measurements related to the surveilled environment. While graphics processing units may speed up belief networks, a host of relevant swarm applications involve small-footprint and low-power devices that cannot be fitted with power-hungry graphics processing units. We are developing a prototype and evaluating real-time, swarm decision-making by posing representative challenges in a Bayesian graphical model framework optimized for modeling, analysis, interpretability, and formulation of provably efficient, probabilistic inference algorithms. We are also exploring specialized hardware for running these algorithms on a power-efficient platform. We will explore efficient swarm architectures that enable real-time collaboration and re-configurability based on detected changes in the sensing environment. While swarm intelligence has been explored, hardware-accelerated belief networks have not, and if proved successful will represent a unique Livermore capability.

Belief propagation in graphical Bayesian models is a powerful statistical tool enabling self-healing in the presence of noisy or corrupted data, adaptive reconfiguration when new data are presented, and the ability to reconfigure when confronted with unknown scenarios. We expect to contribute new theoretical foundations to improve computational complexity of belief propagation, to create models that address realistic swarm scenarios, and to develop hardware for real-time evaluation with dynamic data streams. We intend to (1) create a demonstrator to show the power of belief networks for swarm intelligence along with the impact of various parameter variations in decision-making based on social-media data, (2) show the impact of various parameter variations in decision-making using hardware simulation, and (3) evaluate the decision-making impact of the demonstrator based on hardware architecture, configuration decisions, and environmental conditions. We expect to improve performance over existing software by an order of magnitude through improved algorithms, models, and field-programmable gate-array execution. Our research will result in advances in the fields of machine learning, information theory, and scientific computing.

Mission Relevance

Our work supports the NNSA’s goal of strengthening the science, technology, and engineering base by expanding and applying capabilities to deal with broader national security challenges. Creating models for swarm-situation intelligence gathering enhances the Laboratory’s high-performance computing, simulation, and data science core competency.

FY17 Accomplishments and Results

In FY17 we (1) began an external collaboration with the U.S. Army Research Laboratory regarding collaborative and explainable artificial intelligence; (2) implemented a belief network for location estimation in Python and C++ computer programming languages, and have determined that C++ is superior for hardware experiments; (3) began initial experiments in data stream sampling and computational requirements; and (4) developed a task-independent framework.

Figure 1.
This graph illustrates the upper limit of the discriminatory power for any combination of artificial neurons, where the x-axis represents the number of samples to be trained divided by the number of parameters in the network, the y-axis represents the maximum achievable training accuracy (1.0=100%), and the various colored lines represent different input dimensions (block lengths). As our research revealed, collaborative intelligence implemented with Artificial Neural Networks (including Deep Learning) is limited by this curve independent of the type of threshold activation function or the concrete architecture of the network because not even artificial intelligence can break the limits given by information theory. The only relevant parameter is the number of weights.

Publications and Presentations

Friedland, G., and M. M. Krell. 2017. Capacity Scaling Law for Artificial Neural Networks. LLNL-TR-736950.

Friedland, G., and A. Metere. 2017. An Isomorphism Between Lyapunov Exponents and Shannon's Channel Capacity. LLNL-TR-733786.