Approximate High-Performance Computing: A Fast and Energy-Efficient Computing Paradigm in the Post-Moore Era

Project Overview

As we reach the limits of Moore's law (doubling the number of transistors per square inch every 18 months), and the end of Dennard scaling (power use proportional to area), it has become necessary to explore new paradigms to shape future computer architectures and software systems. One promising approach is approximate computing, where the computation accuracy is traded for better performance and energy efficiency. Approximate computing relies on the ability of the applications to tolerate some loss of accuracy in the output results to trade off for better performance and energy efficiency. To enable wider acceptance of Approximate Computing from the HPC community we needed to address three concerns: ensure accuracy, ensure efficiency, and ensure ease of use. An overarching goal was to establish Lawrence Livermore National Laboratory (LLNL) in a leadership role in what we anticipate to be an important direction in High Performance Computing (HPC).

To make it feasible and effective to use Approximate Computing in HPC, we pursued two integrated thrusts that were focused on algorithms and programming language support to reason about approximate algorithms and enable efficient use of existing and emerging approximate techniques. Specifically, our plan was to evaluate the applicability of approximate computing techniques on a subset of HPC workloads and to develop new approximate algorithms suitable for HPC applications where we pay close attention to estimating the loss in accuracy while satisfying a user-provided error threshold. Since applying approximate computing strategies uniformly throughout the code could result in a violation of important correctness properties, we planned to develop tools and techniques to identify approximable code regions that can be relaxed without impacting the final accuracy of the quantities of interest. To enable ease of use, we sought to develop an Approximate Computing framework aimed towards exposing approximate algorithms to developers and facilitate ease of use within HPC codes.

We made significant progress on both thrusts. We developed an easy-to-use Approximate Computing framework that supports using AC techniques on HPC applications. We developed a benchmark suite for AC, a framework to design error-bounded approximate algorithms, an approximate quantized dot-product algorithm, and several analysis tools to identify approximable regions of HPC code. Our work on the Approximate Computing framework received a lot of interest from the HPC community due to its efficiency and ease of use. The project produced numerous papers and presentations in important venues. The software artifacts from this project are being provided to the HPC community for their use and further development as open source software.

High Performance Computing (HPC) is central to solving large problems in science and engineering and how the Laboratory meets its missions. Therefore, new paradigms, such as Approximate Computing, that can change the way we compute can have significant impact, especially in the post-moore era. In this way, the ApproxHPC Project has directly advanced LLNL's core competencies in High-Performance Computing. Because our work involved developing Approximate Computing framework, error-bounded approximate algorithms, and tools to identify approximable regions to make it easy and efficient to use Approximate Computing in HPC, it is well-suited to benefit different codes at the laboratory. More importantly, as demonstrated by the numerous publications and invited talks, the work developed in this project have firmly established LLNL as the leader in this new paradigm of computing, just as we are reaching the end of Moore's law. The ApproxHPC project has also enriched the Laboratory's research environment. We have maintained a strong collaboration with Prof. Michael Lam's research group at James Madison University and forged new collaborations with Dr. Vassil Vassilev at CERN and Prof. David Lange's research group at Princeton University.

Publications, Presentations, and Patents

Konstantinos Parasyris, Ignacio Laguna, Harshitha Menon, Markus Schordan, Daniel OseiKuffuor, Giorgis Georgakoudis, Michael O. Lam, and Tristan Vanderbruggen. HPC-MixPBench: An HPC benchmark suite for mixed-precision analysis. 2020 IEEE International Symposium on Workload Characterization (IISWC), pages 25-36. 2020. LLNL-CONF-809027.

James Diffenderfer, Daniel Osei-Kuffuor, and Harshitha Menon. A framework for error-bounded approximate computing, with an application to dot products. SIAM Journal on Scientific Computing, 44(3):A1290-A1314, 2022. LLNL-JRNL-820357.

Konstantinos Parasyris, Giorgis Georgakoudis, Harshitha Menon, James Diffenderfer, Ignacio Laguna, Daniel Osei-Kuffuor, and Markus Schordan. HPAC: Evaluating approximate computing techniques on HPC OpenMP applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '21, St. Louis, MO, Nov. 2021. LLNL-CONF-821216. Deputy Director for Science and Technology Excellence in Publication Award and SC Best Reproducibility Advancement Award.

James Diffenderfer*, Konstantinos Parasyris*, Harshitha Menon, Ignacio Laguna, Jackson Vanover, Ryan Vogt, and Daniel Osei-Kuffuor. Approximate computing through the lens of uncertainty quantification. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '22, Dallas, TX, Nov. 2022. LLNL-CONF-829328.

Ignacio Laguna, Harshitha Menon, James Diffenderfer, Giorgis Georgakoudis, Daniel OseiKuffuor, Konstantinos Parasyris, and Markus Schordan. "Challenges and Opportunities to Codesign Approximate Computing Scientific Applications." Reimagining Codesign DOE ASCR Workshop. Virtual. March 2021. LLNL-TR-819497.

Ignacio Laguna, Giorgis Georgakoudis, Harshitha Menon, and Konstantinos Parasyris. "Compiler Analysis for Early Detection of Software Defects: Challenges and Opportunities." Science of Scientific Software Development and Use DOE ASCR Workshop, Virtual, December 2021. LLNL-ABS-829147.

Harshitha Menon and Zhimin Li. "Visualization System for In-depth Analysis of Scientific Applications." Visualization for Scientific Discovery, Decision-Making, & Communication DOE ASCR Workshop, Virtual, January 2022. LLNL-PROP-829943.

Harshitha Menon. "Automatic Mixed Precision Analysis and Transformation Tool." ICERM Workshop on Variable Precision in Mathematical and Scientific Computing, 2020.

Harshitha Menon. "Approximate High Performance Computing : A Fast and Energy Efficient Computing Paradigm in the Post-Moore Era". LANS seminar series at Argonne National Laboratory, 2020. LLNL-PRES-822149.

Harshitha Menon. "Approximate High Performance Computing Guided by Automatic Differentiation." Differentiable Programming Workshop, NeurIPS, 2021. LLNL-PRES-829944.

James Diffenderfer. "Qdot: Quantized Dot Product for Approximate Computing." ECP xSDK Multiprecision Project Pitch Talk, July 29, 2021. LLNL-PRES-824978.

K. Parasyris. "HPAC: Evaluating Approximate Computing Techniques on HPC OpenMP Applications." Society for Industrial and Applied Mathematics, Approximate Computing for Scientific Applications, 2021. LLNL-PRES-827975.

K. Parasyris. "Challenges, Solutions and Limitations of Approximate Computing." University of California Merced, CA. 2022. LLNL-PRES-840060.

Harshitha Menon. "Numerical Reproducibility in Scientific Codes." UC Santa Cruz Open Source Symposium 2022, 2022. LLNL-PRES-840579.

Ignacio Laguna, David Bailey, Osman Unsal, Harshitha Menon, Hartwig Anzt, and Radha Venkatagiri. "Relaxing Correctness via Approximate Computing: Is it Feasible in HPC?" In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '22, 2022.

K. Parasyris. "An HPC Benchmark Suite for Mixed-Precision Analysis." IEEE International Symposium on Workload Characterization, 2020. LLNL-PRES-814856.

K. Parasyris. "HPAC: Evaluating Approximate Computing Techniques on HPC OpenMP Applications." The International Conference for High Performance Computing, Networking, Storage, and Analysis,. St. Louis, MO. November 2021. LLNL-PRES-827975.

K. Parasyris. "Approximate Computing through the Lens of Uncertainty Quantification." The International Conference for High Performance Computing, Networking, Storage, and Analysis. Dallas, TX. November 2022.

HPAC: High Performance Approximate Computing Framework. https://github.com/koparasy/HPAC. August 2021.

Puppeteer: Framework to Rank Code Regions for Approximation.https://github.com/koparasy/HPAC/releases/tag/v0.0.0-Puppeteer. July 2022.