1. Home
  2. Report Archives
  3. Project Highlights - 2021
  4. High-Performance Computing Simulation and Data Science - 2021
  5. Scalable Motif-Driven Parallel Generative Adversarial Net for Community Detection

Scalable Motif-Driven Parallel Generative Adversarial Net for Community Detection

Byung Yoo | 21-FS-029

Project Overview

Community detection refers to the problem of finding groups of objects that are closely related to each other (called communities) from a given graph data set and has been widely used in many key national security applications. However, accurately detecting communities embedded in very large datasets in a scalable way is a non-trivial task. This research addresses this challenge by developing a scalable and efficient parallel community detection framework that is based on machine learning.

We achieve this goal by using state-of-art machine learning (ML) technique called Generative Adversarial Net (GAN) model, where the training is driven by simple yet effective patterns that constitute the core of underlying community structure. This model is implemented on Livermore Big Artificial Neural Network (LBANN), a scalable parallel machine learning engine. We identified technical challenges in realizing scalable community detection and viable solutions for these challenges and optimized our framework by applying these findings. It is demonstrated that the developed system achieves excellent scalability on large graphs with hundreds of millions of edges and community detection accuracy comparable to that of the current state-of-art.

Mission Impact

Community detection is widely used in critical national security applications. The Global Security (GS) directorate has been actively engaged in this research area. This work offers a new scalable and efficient community detection capability by combining a state-of-art ML technique and a parallel learning engine on a large high performance computer (HPC). Furthermore, this work identifies technical challenges and proposes viable solutions for realizing scalable and effective community detection, which defines directions for future research. The results from this research will benefit current and future efforts in national security applications within the GS directorate and external sponsors.