Towards CPU Performance Prediction: New Challenge Benchmark Dataset and Novel Approach
Xiaoman Liu
xiaoman.liu@intel.com
Model Structure
NCPP Architecture: Feature Extraction from Character and Numerical Inputs Using a Deep Learning Model with Convolution and Multiple Attention Mechanisms
Abstract
CPU performance prediction, which involves forecasting the performance scores of a CPU based on its hardware characteristics during its operation, is a critical technology for computational system design and resource management in the big data era. However, this research field currently faces two significant challenges. First, collecting real-world data is challenging due to the wide variety of CPU products on the market and the highly specialized nature of relevant hardware characteristics. In the research process, this field lacks a standard dataset with unified hardware characteristics, wide data coverage, and comprehensive benchmarks. Second, existing methods based on hardware simulation models or machine learning exhibit notable shortcomings, such as lengthy simulation test cycles and low prediction accuracy. To bridge these gaps, we first collect, preprocess, and standardize historical data from the 4th Generation Intel Xeon Scalable Processors across multiple benchmark suites to create a new dataset, named PerfCastDB. Subsequently, we design a deep learning based model called Nova CPU Performance Predictor (NCPP) as the baseline for this new dataset. The NCPP network is designed based on group attention mechanism. It effectively quantifies the implicit relationships between hardware characteristics within and across groups and comprehensively models the impact of various hardware characteristics on CPU performance prediction. We conduct comparative experiments using the proposed PerfCastDB dataset. Compared to existing approaches, NCPP achieves superior evaluation results, demonstrating its effectiveness. Furthermore, we have open-sourced part of the dataset and the NCPP network code to facilitate subsequent research. The resources can be accessed at https://github.com/xiaoman-liu/NCPP.
Keywords： CPU Performance Prediction, Group Attention Mechanism, New Dataset, Deep Learning
Data Distribution
- Bubble Chart Comparison of Core Count and Base Frequency Distribution for Various Intel Xeon Server CPU Models, with Bubble Size Representing Sample Size.
- Multivariate Violin Plot Displaying the Distribution of Various CPU Characteristics: From Cache Size to Frequency, as well as Multicore Processing Capabilities and Memory Configuration SPECrate2017_int_base
- Horizontal Error Bar Chart of Benchmark Performance Measuring Throughput in SPECrate2017_int_base
Comparison of Performace
The performance prediction results of NCPP and the comparison apoproaches on the MAE, MAPE, and MSE metrics under different benchmark suites.
BibTeX
@article{Liu2024TowardsCP,
title={Towards CPU Performance Prediction: New Challenge Benchmark Dataset and Novel Approach},
author={Xiaoman Liu},
journal={ArXiv},
year={2024},
volume={abs/2407.03385},
url={https://api.semanticscholar.org/CorpusID:271039362}
}
Acknowledgements
I gratefully acknowledge Intel for in-house data and financial support.
