This project, developing DISCI, an all-around computing instrument that compensates the limitations of existing computing-centric HPC instruments toward data-intensive applications, supports five large research projects in HPC system design, computational chemistry, biotechnology, and atmospheric science. Based on research introducing the application-aware and decoupled-execution paradigm concept, the project addresses the big gap between research prototypes and the engineering solution. Impact on future applications, algorithms, and instruments design is expected since the instrument could open up new research areas in supporting data-intensive sciences and possibly reshape HPC instruments adopted in National Computing Facilities and some institutions. In addition to the conventional HPC compute nodes, DISCI has a set of specially designed data nodes. The data nodes offer in-situ data processing to reduce data movement and data-access delay and dynamic provisioning as 'fat' compute nodes when necessary, while the functionality of compute nodes remains the same as in conventional instrumentation. These data nodes work with compute nodes in concert and together they provide an optimum system performance for data-intensive HPC. Based on a hardware-software co-development principle, the instrument consists of two components: the DISCI system architecture and the DISCI runtime software. The system architecture builds an HPC instrument with a data-centric view. The runtime software extends the MPI (Message Passing Interface) and MPI-IO library to support data nodes and their associated in-situ processing. The instrument will enable and foster research activities in the areas of chemical dynamics simulation, simulations of turbulent flows, atmospheric data assimilation and weather forecasting, computational biology, and computer systems that PIs and senior personnel conduct.
This project is funded by the National Science Foundation under grant CNS-1338078.
ACKNOWLEDGMENT: We are grateful to the National Science Foundation for the sponsorship of this project.