Validating Large Fluid-Dynamics Simulations of Complex Geometries with Three-Dimensional Printing

Maxim Shusteff (15-LW-029)

Project Description

Computational fluid dynamics plays an increasingly important role in numerous fields, including microfluidics, hydraulic fracturing, biosecurity, and circulatory disease modeling. Advances in high-performance computing and multiphysics algorithms have made it possible to model fluid flow through a variety of complex geometries with unprecedented fidelity. We intend to combine the Laboratory's expertise in additive manufacturing and high-performance computing to create a new, tightly coupled validation technique. By carrying out experiments on three-dimensional printed geometries generated from the same input meshes used in fluid dynamics simulations, we will be able to directly compare experiment and theory. We will apply this technique of additive manufacturing to validate current research into high-resolution simulation of the human circulatory system with explicit red blood cells, which is being conducted on Livermore's Blue Gene/Q supercomputer. With this project we will, for the first time, investigate the cell- and patient-specific parameters that are most effective in metastatic cancer site prediction by creating a novel technique for profiling the likelihood of adhesion in the circulatory system for secondary tumor site formation based on a representative range of patient characteristics and cancer cell types. The rapid feedback loop enabled by this research will be used to direct large-scale simulations of cancer metastasis with explicit red blood cells and circulating tumor cells.

This image shows the computer-simulated pressures in a physiologically accurate human arterial system, including all arteries with diameters greater than 1 mm. the simulation was run on llnl's ibm bluegene/q system on all 1.57-million cores.
This image shows the computer-simulated pressures in a physiologically accurate human arterial system, including all arteries with diameters greater than 1 mm. The simulation was run on LLNL's IBM BlueGene/Q system on all 1.57-million cores.

We expect to directly validate fluid dynamics simulations in complex geometries to quantify and improve the accuracy of multiphysics simulations for biological applications. Such a study requires a combination of both high-performance computing and additive manufacturing expertise. Additive manufacturing, or three-dimensional printing, enables fabrication of highly complex, three-dimensional structures with embedded hollow channels. This is a sound approach for creating replicas of complex biological structures that would be very difficult or impossible to create in any other way. We are able to use three-dimensional reconstructions of actual human vasculature, modify the design to add connections for fluidic flow control and diagnostic instruments, and then print it to create a basically identical match to the design file. We are using this technique to inform the accuracy of multiphysics terms needed to model circulatory disease, which will be used in large-scale simulations of cancer metastasis. These results will be of broad interest to the fluid dynamics and biology communities, and clinical collaborators are helping to develop risk-assessment tools.

Mission Relevance

With simulation playing an increasingly important role in NNSA and Laboratory missions, the use of additive manufacturing to directly quantify individual terms in a multiphysics calculation would be a powerful tool for verification and validation. This research has specific applications to microfluidics, bioscience, and bioengineering, but the lessons learned from closely coupling theory and experiment will be useful for directly validating other applications as well. This work supports the Laboratory's core competencies in high-performance computing, simulation, and data science, as well as advanced materials and manufacturing.

FY16 Accomplishments and Results

In FY16 we (1) developed simulations of blood flow in patient-specific geometries using supercomputers and demonstrated the first three-dimensional simulation of flow in the full human arterial network (see figure); (2) experimentally recapitulated large-artery fluid flow in physical three-dimensional printed structures with the same geometry used in the simulation, which was validated for steady and pulsatile flow; (3) designed a method to track cell trajectories and velocities in three dimensions throughout large blood vessel models; (4) added a model of deformable cells to the supercomputer simulation (to represent both blood and cancer cells); (5) validated the model by tracking rigid beads and deformable gel particles in the three-dimensional physical model; and (6) tested flows with whole blood and fluorescing cancer cells.

Publications and Presentations

  • Randles, A., et al., Massively parallel models of the human circulatory system. Intl. Conf. High Performance Computing, Networking, Storage and Analysis. Salt Lake City, UT, Nov. 13–18, 2016. LLNL-CONF-670030.
  • Randles, A., et al., "Massively parallel simulations of hemodynamics in the primary large arteries of the human vasculature." J. Comput. Sci. 9, 70 (2016). LLNL-CONF-666133.