Exploration of In-Memory Analytics

Project Overview

To present a robust cyber defense in today's threat landscape, a defender must have an in-depth understanding of memory usage. The prevalence of "fileless" or "memory-based" attacks has increased more than 900% since 2020. Since these types of attacks are resistant to traditional cyber defense strategies such as scanning the hard drive for known-bad files, the ability to characterize normal memory behavior is essential to developing modern detectors of abnormal and malicious activity.

For this effort, we investigated the typical memory usage of common Windows processes and how in-memory behavior changed when maliciousness was introduced to the system. We took snapshots of memory and used a customized Volatility plugin to extract static memory-based features such as sizes of loaded executables and program images. We also implemented an intercept library based on the Microsoft Detours framework to extract dynamic features such as memory allocation API calls and a WinDbg wrapper to gather virtual-address descriptor (VAD) tree information from running processes. We used the Hollows Hunter open-source benignware to simulate malicious behaviors such as process hollowing and dynamic linked library (DLL) or portable executable (PE) injection.

We demonstrated that many processes have characteristic memory footprints, and that machine-learning models are able to flag simulated malicious behavior as anomalous. We also demonstrated techniques for using memory-related features to potentially detect this type of malicious behavior at scale.

Mission Impact

Cyber security is a central focus of many Laboratory research efforts. Investigations into state-of-the-art methods and the development of new techniques are critical to the security of LLNL and other government entities and assets. This project provided the groundwork for research into a relatively new and critical area of cyber security.

Two of our papers have appeared in peer-reviewed publications, and two more papers will be finalized and submitted for publication under other efforts. Our research has been briefed to potential sponsors and presented at multiple conferences. We curated a data set which was made available to researchers at the LLNL LLAMA Workshop. We developed several tools for extracting memory-based features and made contributions to existing open-source software. We also developed a remote-access laboratory for investigating malware as it executes. These have the potential to be valuable resources for future research efforts.

Several government agencies have expressed interest in the work. The research will be continued under existing programmatic funding, with the potential of additional funding being supplied to support some of these new directions.

Publications, Presentations, and Patents

Lyles, S. "Open-Source Code Contribution to Volatility Project." September 2020; https://github.com/volatilityfoundation/volatility.

Battisti, C. et al. "Seeing the Cyber-Defense Forest Through the Process Trees." Government Conference. November 2020. LLNL-PRES-816122.

Monteith, K. "Embedding Virtual Address Descriptor (VAD) Trees." Government Workshop Outbrief, November 2020. LLNL-PRES-826876.

Monteith, K. et al. "MNEMOSYNE: Exploration of Memory-BasedAnalytics for Cyber Threats." Classified Briefing to Potential Government Sponsor. December 2020.

DeSantis, M. et al. "Curated VAD Tree Dataset." LLNL LLAMA Workshop, Livermore, CA. January 2021.

Monteith, K. et al. "MNEMOSYNE: Exploration of Memory-Based Analytics for Cyber Threats." Briefing to Potential Government Sponsor. February 2021. LLNL-PRES-826875.

Monteith, K, et al. "Embedding Virtual Address Descriptor (VAD) Trees." LLNL LLAMA Workshop Classified Outbrief. Livermore, CA. February 2021.

Lyles, S. "Open-Source Code Contribution to Minidump Project." March 2021; https://github.com/skelsec/minidump.

Monteith, K., et al. "MNEMOSYNE: Exploration of Memory-Based Analytics for Cyber Threats." Classified Briefing to Potential Government Sponsor. May 2021.

Nyholm, H. "Volatile Memory Forensics: A Survey." Malware Technical Exchange Meeting. July 2021. LLNL-CONF-824108.

Monteith, K. "Exploration of Memory-Based Analytics for Cyber Threats." Malware Technical Exchange Meeting, July 2021. LLNL-POST-826888.

Monteith, K. et al. "MNEMOSYNE: Exploration of Memory-Based Analytics for Cyber Threats." Classified Briefing to Potential Government Sponsor. September 2021.

Monteith, K. et al. "MNEMOSYNE: Exploration of Memory-Based Analytics for Cyber Threats." Briefing to LLNL Science and Technology Board of Governors. September 2021. LLNL-PRES-826881.

Monteith,K. et al. "MNEMOSYNE: Exploration of Memory-Based Analytics for Cyber Threats." Classified Conference. November 2021.

Nyholm, H. et al. 2022. "The Evolution of Volatile Memory Forensics." Journal of Cybersecurity and Privacy 2/3: 556-572 (2022); doi.org/10.3390/jcp2030028.

Lyles, S. et al. "Machine Learning Analysis of Memory Images for Process Characterization and Malware Detection." Proceedings of the Digital Networking Security Conference, Artificial Intelligence to Security Workshop. June 2022; doi: 10.1109/DSN-W54100.2022.00035.

Lyles, S. et al. "Machine Learning Analysis of Memory Images for Process Characterization and Malware Detection." Presentation, Digital Networking Security Conference, Artificial Intelligence to Security Workshop. June 2022. LLNL-PRES-836778.

Monteith, K. et al. "MNEMOSYNE: Exploration of Memory-Based Analytics for Cyber Threats." Classified Briefing to Potential Government Sponsor. September 2022.