HPC benchmarking: scaling right and looking beyond the average

The paper “HPC benchmarking: scaling right and looking beyond the average” by Milan Radulovic, Kazi Asifuzzaman, Paul Carpenter, Petar Radojković and Eduard Ayguadé from ExaNode Partner BSC will be presented at EuroPar 2018 in Turin, August 2018.

Abstract:
Designing a balanced HPC system requires an understanding of the dominant performance bottlenecks. There is as yet no well established methodology for a unified evaluation of HPC systems and workloads, and which quantifies the main performance bottlenecks.
In this paper, we execute seven production HPC applications on a production HPC platform,
and analyse key performance bottlenecks: FLOPS performance and memory bandwidth congestion, with scaling-out implications. We show that the results depend significantly on the number of execution processes and granularity of measurements. Therefore, we advocate for guidance in the application suites, on selecting the representative scale of the experiments. Also, we propose a representation of the results as a proportion of time spent in low, moderate and severe utilization of specific metric, comparing to sustained maximum, which gives a much more precise picture than the average.