Autotuning exascale applications with Gaussian process regression

Abstract: Significant effort has been invested to develop highly scalable numerical libraries and high-fidelity modeling and simulation for the upcoming exascale computers. These codes typically involve many parameters which need to be selected properly to optimize performance on the underlying parallel machine. They are also expensive to run and thus have limited "function evaluation" values, which post significant challenges to efficient performance tuning on diverse architectures.

Bayesian optimization with Gaussian process regression is an attractive machine learning framework to build surrogate models with limited function evaluation points. In order to fully utilize all the available data, we leverage multitask learning and multi-armed bandit strategies to build a more advanced Bayesian optimization framework.

We have developed an open-source software tool, called GPTune, for optimizing expensive large-scale HPC codes. We will show several features of GPTune, e.g., incorporation of coarse performance models to improve the Bayesian model, multi-objective tuning such as tuning a hybrid of time, memory and accuracy, and reuse of historical data base for model portability.

We will demonstrate the efficiency and effectiveness of GPTune when it is applied to numerical linear algebra libraries, such as ScaLAPACK, SuperLU and Hypre, as well as fusion simulation codes M3D-C1 and NIMROD.

This talk describes joint work with James Demmel, Yang Liu, Osni Marques, Wissam Sid-Lakhdar and Xianran Zhu

computational engineering, finance, and sciencenumerical analysis

Audience: researchers in the topic

---

E-NLA - Online seminar series on numerical linear algebra

Series comments: E-NLA is an online seminar series dedicated to topics in Numerical Linear Algebra. Talks take place on Wednesdays at 4pm (Central European Time) via Zoom and are initially scheduled on a weekly basis.

To join the seminar, please complete the sign up form at the bottom of the webpage. Information about how to connect to the conference call will be circulated via email to all registered attendees.

---