Performance metrics for access pattern-aware analysis of heterogeneous memory power consumption in HPC

  • Metriken zur Analyse des Energieverbrauchs von heterogenem Arbeitsspeicher im Hochleistungsrechnen unter Berücksichtigung der Zugriffsmuster

Alt, Lukas; Müller, Matthias S. (Thesis advisor); Lankes, Stefan (Thesis advisor); Kozhokanova, Anara (Consultant)

Aachen : RWTH Aachen University (2023)
Master Thesis

Masterarbeit, RWTH Aachen University, 2023


In response to the rising demand for memory performance and capacity, memory heterogeneity in HPC systems increased. In particular, technologies such as high-bandwidth memory (HBM) and high-capacity memory (HCM) are employed in addition to DRAM. While the power consumption of the memory subsystem was often neglected in node-level power optimizations in the past, the increased power consumption by HCM motivates studying the energy consumption of heterogeneous memory on the latest architectures under workloads with different memory access patterns. A method for measuring the memory energy consumption using hardware instrumentation of memory slots is described and implemented. Measurements using this approach are compared to RAPL, a software interface for limiting and controlling power consumption on Intel systems. Results show that RAPL energy measurements for the memory domain can differ significantly - up to 120% - from reference measurements on Intel Ice Lake-SP systems. A discussion of possible reasons yields that the RAPL memory domain may include losses at the voltage regulator level. The accuracy of the reference measurements was validated by comparing the results from the literature to results obtained from a similar architecture (Broadwell-EP). This thesis presents the new metrics DEL and DES for heterogeneous memory energy evaluation using different memory access patterns. Additionally, the BpW metric was utilized for memory energy efficiency characterization. The metrics are based on the instrumented energy measurements conducted on the Ice Lake architecture equipped with DRAM and Intel Optane Persistent Memory (PMem). The results demonstrate that the memory access pattern and the concurrency in memory accesses significantly impact the memory’s dynamic energy consumption. Furthermore, it shows that PMem is more energy efficient per capacity than DRAM at idle and is better suited for storing rarely accessed data. When PMem is under load, DRAM is more energy efficient. The proposed metrics are then used to estimate the energy consumption of real-world applications, followed by a discussion on the applicability of this approach and potential improvements.


  • IT Center [022000]
  • Department of Computer Science [120000]
  • Chair of Computer Science 12 (High Performance Computing) [123010]