A Technique To Evaluate Benchmarks: a Case Study Using the Livermore Loops

This paper is devoted to an analysis of the data from the Livermore kernels benchmark. We will show that in the sense of least squares prediction the dimension of these data is rather small; a reduction of the data to dimen sion four has about the same predictive power as the original data. Two techniques are used that reduce the 72 kernel timings for each machine to a few scores by which the machine is characterized. The first is based on a principal component analysis, the second on a cluster analysis of the kernels. The validity of the reduction to lower dimension is checked by various means. The pos sible use of the Livermore data to predict the running time of larger codes is demonstrated.