Programming the Linpack benchmark for Roadrunner

We describe the challenges and opportunities we encountered when developing a hybrid version of the Linpack benchmark for the Los Alamos National Laboratory Roadrunner supercomputing system, which combines traditional x86-64 host processors with IBM PowerXCelle™ 8i accelerator processors. The challenges included determining the proper division of the host and accelerator roles in the computation, transfer of data between the host and accelerator memory domains, alignment of data for communication and computation, and data format differences between the two processors. We also describe our approach to modeling the performance of the hybrid system and compare our performance estimates to witnessed performance on the system at different scales and levels of memory consumption. Through careful attention to these issues, we have produced a hybrid version of the Linpack benchmark for the Roadrunner system that achieves 77.8% of peak performance on a single compute node and 74.6% of peak performance over the entire system, making this system the first to achieve a Linpack result exceeding one petaflops (1015 floating-point operations per second). © 2009 IBM.