Persistent Concurrent Data Structures

Peter Pirkelbauer | 20-FS-040

Mission Overview

Byte-addressable non-volatile random access memory (NVRAM) is an emerging memory technology that provides data persistence and higher data densities than conventional dynamic random access memory. Storing application data directly in NVRAM avoids the need for data serialization and data movement to and from persistent storage. NVRAM also adds crash resilience against system failures, such as power shutdown and process failures. To avoid the costly overhead of logging and redo functionality, this work applies non-blocking programming techniques for developing persistent and concurrent data structures for NVRAM. This project serves as a feasibility study by developing a hash table for NVRAM systems. In our performance tests, we found that our design outperformed state-of-the-art alternatives by as much as 6.3 times on Intel Optane DC persistent memory for synthetic tests. Further tests demonstrated the ability to recover from catastrophic system failures. The experience gained in this feasibility study will guide continued work on concurrent containers and the development of other on-node parallel abstractions for NVRAM.

Mission Impact

This project strengthens Lawrence Livermore National Laboratory's core competency in high-performance computing, simulation, and data science. The study produced new insights into how to utilize an emerging memory technology. Exploring concurrent data structures for NVRAM will inform future designs of on-node parallel abstractions. The prototyped containers may serve as a foundation for implementing distributed data structures for interactive exploratory data analytics, an emerging need from the data science community. Persistent data structures as prototyped by this project could provide a key technology for enabling such interactive analytics on modern high-performance computing systems containing NVRAM.