How Does Miss Clustering Affect The Cost of A Cache Miss

It has long been observed that misses cluster and the amount of time a processor loses due to a cache miss can vary greatly, depending on the cluster size. In this talk we describe a new technique, called pipeline spectroscopy, and use it to measure the cost of each cache miss within a cluster. The cost of a miss is displayed (graphed) as a histogram, which represents a precise readout showing a detailed visualization of the cost of each cache miss within the cluster and throughout all levels of the memory hierarchy. We call the graphs 'spectrograms' because they reveal certain signature features of the processor's memory hierarchy, the pipeline, miss parallelism (overlap), and the miss pattern itself (cluster size). Next, a theory is presented that describes a miss cluster and shape of a spectrogram. Finally, we use this theory to characterize clusters of two misses and use it to predict the shape and cost for larger miss clusters. Detailed analysis of a spectrogram leads to much greater insight in pipeline dynamics, programming structures, prefetching, and miss queueing delays.
Thomas R. Puzak received a B. S. in Mathematics and M. S. in Computer Science from the University of Pittsburgh and a Ph. D. in Electrical and Computer Engineering from the University of Massachusetts. Since joining IBM in 1970, he has spent over thirty years working in IBM Research. While at IBM he received Technical Achievement, Outstanding Contribution, and Innovation Awards, served as Chairman of the Computer Architecture Special Interest Group at the T. J. Watson Research Center and holds more than 30 patents, on topics concerning processor and memory design.