1. Home
  2. Report Archives
  3. LDRD Annual Report FY2019
  4. High-Performance Computing, Simulation, and Data Science
  5. Variable Precision Computing

Variable Precision Computing

Jeffrey Hittinger | 17-SI-004

Overview

Our project aimed to initiate and promote a new paradigm in high-performance computing (HPC) that would fundamentally change how we represent and make use of representations of real numbers in finite precision. We pursued three integrated and concurrent research thrusts: 1) new, more dynamic data representations to address data motion and capacity limitations, 2) improved mixed-precision algorithms to accelerate computation and to leverage new hardware, and 3) new tools to help developers reason about precision and to automate code transformations.

Within these areas, we made significant advances in 1) the use of compressed array data types in numerical calculations, 2) the theoretical justification, 3) improved performance, 4) the prospects of hardware implementation, 5) the development of metrics to significantly reduce the storage needs of molecular dynamics simulations, 6) the development and analysis of efficient mixed-precision algorithms for time-dependent partial differential equations and for orthogonal factorization, and 7) the capability of tools that can analyze sensitivity of computed results to choices in precision and that can make automated code transformations based on such analysis. We established that opportunities exist, using the technologies we developed, to realize significant improvements in HPC above and beyond the 2x limits of traditional mixed precision.

Impact on Mission

Our research leveraged and advanced Lawrence Livermore National Laboratory's core competencies in HPC, simulation, and data science to elevate the Laboratory as a leader in precision-related research for HPC. Two members of the project team have been invited to organize a 2020 workshop on variable precision computing at the Institute for Computational and Experimental Research in Mathematics. Improvements in HPC yield positive impacts in the Laboratory's work to meet all mission focus areas and other core competencies.

Publications, Presentations, Etc.

Diffenderfer, J. 2017. "Implementing Anderson Acceleration in Mixed Precision." LLNL Summer Student Poster Symposium, Livermore, CA, August 2017. LLNL-POST-735360.

––– . 2019. "Error Analysis of ZFP Compression for Floating-Point Data." SIAM Conference on Computational Science and Engineering, Spokane, WA, February 2019. LLNL-PRES-768145.

Diffenderfer, J., et al. 2019. "Error Analysis of ZFP Compression for Floating-point Data," SIAM Journal on Scientific Computing 41(3): A1867-A1898. doi:10.1137/18M1168832. LLNL-JRNL-744818.

Fox, A. 2018. "Propagation of Compression Error of ZFP with Fixed Precision for Floating-Point Data in Iterative Methods." CM2018: Fifteenth Copper Mountain Conference on Iterative Methods, Copper Mountain, CO, March 2018. LLNL-PRES-757672.

——2019. "Stability Analysis of Inline ZFP Compression for Floating-point Data in Iterative Methods." SIAM Conference on Computational Science and Engineering, Spokane, WA. February 2019. LLNL-PRES-768209.

––– . 2019. "Error Analysis of Inline ZFP Compression for Multigrid Methods." Copper Mountain Conference Iterative Methods, Copper Mountain, CO, March 2019. LLNL-PRES-770240.

Fox, A., et al. 2018. "Stability Analysis of inline ZFP Compression for Floating-Point Data in Iterative Methods." 25th International Domain Decomposition Conference, Newfoundland, Canada. July 27, 2018. LLNL-POST-752539.

––– . 2018. "Investigation of Spectral Clustering for Signed Graph Matrix Representation." IEEE HPEC Conference Proceedings , September 2018. doi:10.1109/HPEC.2018.8547575. LLNL-CONF-754816.

——2018. "Investigation of Spectral Clustering for Signed Graph Matrix Representation." IEEE HPEC Conference, September 25, 2018. LLNL-POST-758625.

Gordon, J. 2017. "Analysis of Mixed Precision Iterative Refinement Method." LLNL Summer Student Poster Symposium, Livermore, CA, August 2017. LLNL-POST-735423.

Herring, J. 2017. "Variable Precision: Toward a GPU-Enabled LOBPCG in HYPRE." LLNL Summer Student Poster Symposium, Livermore, CA, August 2017. LLNL-POST-735534.

Hittinger, J. 2017. "Making Every Bit Count: Variable Precision?" Big Data Meets Computation Workshop, Institute for Pure and Applied Mathematics, UCLA, Los Angeles, CA, Feb 3, 2017. LLNL-PRES-722917.

––– . 2017. "Variable Precision: Making Every Bit Count?" Computer Science and Engineering Department Seminar, Lehigh University, Bethlehem, PA, October 2017. LLNL-PRES-722917.

––– . 2018. "Variable Precision: Making Every Bit Count." Université Nice Sophia Antipolis, Nice, France, June 2018. LLNL-PRES-753450.

––– . 2018. "Variable Precision: Making Every Bit Count." PASC18, Basel, Switzerland, July 2018. LLNL-PRES-753451.

––– . 2018. "Variable Precision Computing Research in the Center for Applied Scientific Computing." Technische Universität München, Garching, Germany, July 2018. LLNL-PRES-753448.

––– . 2018. "Rethinking Finite Precision." Science at Extreme Scales Workshop IV. Institute for Pure and Applied Mathematics, UCLA, Los Angeles, November 2018. LLNL-PRES- 767245.

––– . 2019. "Variable Precision Computing Research in the Center for Applied Scientific Computing." University of Illinois Urbana-Champaign Computational Science and Engineering Seminar, Urbana, IL, April 2019. LLNL-PRES-753448.

––– . 2019. "Variable Precision Computing Strategies." ISC High Performance 2019, Frankfurt, Germany, June 2019. LLNL-PRES-779040.

––– . 2019. "Variable Precision Computing Improves Simulation Speed and Efficiency." Computing Report, Lawrence Livermore National Laboratory, p. 42-45. LLNL-TR-787575.

Hittinger, J., et al. 2017. "Universal Coding of the Reals: Alternatives to IEEE Floating Point -An Example Application." SC17 BoF: Improving Numerical Computation with Practical Tools and Novel Computer Arithmetic, Denver, CO, November 2017. LLNL-PRES-741490.

––– . 2018. "Adapting Precision in Scientific Simulations." 13th World Congress on Computational Mechanics, New York, NY, July 2018. LLNL-PRES-756553.

Hoang, D., and P. Lindstrom. 2018. "Compression of Particle Simulation Data Using Space Partitioning and Binomial Coding.." LLNL Student Poster Symposium, Livermore, CA, August 2018. LLNL-POST-755761.

Hoang, D., et al. 2019. "A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and Visualization." IEEE Transactions on Visualization and Computer Graphics , 25(1):1193-1203. doi:10.1109/TVCG.2018.2864853. LLNL-CONF-755005.

Kolasinksi, A. 2019. "Error Analysis of Inline ZFP Compression for Multigrid Methods." SIAM Conference on Computational Science and Engineering, Spokane, WA, February 2019. LLNL-PRES-770240.

Kolasinksi, A. and A. Fox. 2018. "Error Analysis of Inline ZFP Compression for Multigrid Methods." LLNL Student Poster Symposium, Livermore, CA, August 2018. LLNL-POST-755648.

Lindstrom, P. 2017. "Reducing Data Movement Using Adaptive-Rate Computing." Computation Directorate External Review Committee Meeting, LLNL, Livermore, CA, May 2017. LLNL-POST-728998.

––– . 2017. "Lossy Compression Algorithms for Floating-Point Data." Joint Statistical Meetings, Baltimore, MD, July/August 2017. LLNL-PRES-736379.

––– . 2017. "Error Distributions of Lossy Floating-Point Compressors." Proceedings of 2017 Joint Statistical Meetings, Baltimore, MD, 29 July/August 2017. LLNL-CONF-740547.

––– . 2017. "Lossy Compression of Floating-Point Data." International Computing for the Atmospheric Sciences Symposium, Annecy, France, September 2017. LLNL-PRES-739189.

––– . 2017. "Universal Coding of the Reals Using Bisection." SC17 BoF: Improving Numerical Computation with Practical Tools and Novel Computer Arithmetic, Denver, CO, November 2017. LLNL-PRES-731413.

––– . 2018. "Universal Coding of the Reals: Alternatives to IEEE Floating Point." Keynote, Conference for Next Generation Arithmetic 2018, Singapore. March 2018. LLNL-PRES-748385.

––– . 2018. "Alternatives to IEEE: NextGen Number Formats for Scientific Computing." Science at Extreme Scales Workshop. Institute for Pure and Applied Mathematics, UCLA, Los Angeles, October 2018. LLNL-PRES-759606.

––– . 2019. "Beyond IEEE: Next-Generation Floating-Point Formats." SIAM Conference on Computational Science and Engineering, Spokane, WA, February 2019. LLNL-PRES-768285.

––– . 2019. "Universal Coding of the Reals Using Bisection." Proceedings of the Conference for Next Generation Arithmetic 2019, In cooperation with ACM, Singapore, March 2019. doi:10.1145/3316279.3316286. LLNL-CONF-764361

Lindstrom, P., et al. 2018. "Universal Coding of the Reals: Alternatives to IEEE Floating Point." Proceedings of the Conference for Next Generation Arithmetic 2018, in cooperation with ACM, Singapore, March 2018. doi:10.1145/3190339.3190344. LLNL-CONF-743265.

Menon, H. 2018. "ADAPT: Algorithmic Differentiation Applied to Floating-Point Precision Tuning." International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018, Dallas, TX, November 2018. LLNL-PRES-759649.

––– . 2018. "Error Analysis in HPC Applications Using Algorithmic Differentiation." Ninth International Women in HPC Workshop, in conjunction with SC18, Dallas, TX, November 2018. LLNL-PRES-759413.

––– . 2019. "Error Analysis in HPC Applications Using Algorithmic Differentiation." James Madison University, Harrisonburg, VA, April 2019. LLNL-PRES-759649.

Metere, A. and G. Friedland. 2018. "Isomorphism between the Maximum Lyapunov Exponent and the Shannon's Channel Capacity." LLNL-TR-733786.

Menon, H., et al. 2018. "ADAPT: Algorithmic Differentiation Applied to Floating-Point Precision Tuning." Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, Paper 48:1-13, SC 2018, Dallas, TX, November 2018, doi:10.1109/SC.2018.00051. LLNL-CONF-748742.

Moody, L., et al. 2018."Automatic Generation of Mixed-Precision Programs." International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018, Dallas, TX, November 2018. LLNL-POST-756004.

––– . 2019. "Viability of Hardware Accelerated Floating-Point Compression." LLNL Summer Student Poster Symposium, Livermore, CA, August 2019. LLNL-POST-783177.

Osei-Kuffuor, D. 2019. "Variable Precision Computing for Scientific Applications." Smoky Mountains Conference on Computational Science and Engineering, Kingsport, TN, August 2019. LLNL-PRES-788465.

Sanders, G., et al. 2018. "Floating Point Analysis for Half-Precision Eigensolvers." CM2018: Fifteenth Copper Mountain Conference on Iterative Methods, Copper Mountain, CO, March 2018. LLNL-PRES-748762.

Schordan, M. 2017. "Verifying the Floating-Point Computation Equivalence of Manually and Automatically Differentiated Code." Correctness'17: First International Workshop on Software Correctness for HPC Applications, Denver, CO, November 2017. LLNL-PRES-741134.

Schordan, M., et al. 2017. "Verifying the Floating-Point Computation Equivalence of Manually and Automatically Differentiated Code." Correctness'17: First International Workshop on Software Correctness for HPC Applications, in conjunction with SC17, Denver, Colorado, 12 November 2017. In Proceedings of the First International Workshop on Software Correctness for HPC Applications (Correctness'17), New York, NY: ACM, 34-41. doi:10.1145/3145344.3145489. LLNL-CONF-737605.

Usher, W., et al. 2018. "Exploring ZFP for Compression of Particle Data." LLNL Student Poster Symposium, Livermore, CA, August 2018. LLNL-POST-755563.

Yang, M. 2019. "Half-Precision Block QR Factorization." SIAM Conference on Computational Science and Engineering, Spokane, WA, February 2019. LLNL-PRES-768669.

Yang, L. and G. Sanders. 2018. "Simulated Half-Precision Implementation of Blocked QR Factorization and Graph Clustering Applications." LLNL Student Poster Symposium, Livermore, CA, August 2018. LLNL-TR-756282.

Open Source Software