Direct-Search for Min-Max Derivative-Free Optimization

Abstract: Recent applications in machine learning have renewed the community’s interest in min-max optimization problems. While gradient-based optimization methods are widely used to solve these problems, there exist many scenarios where such techniques are not well suited, or even not applicable, particularly when gradients are not accessible. In this talk, we will investigate the use of direct-search methods, which belong to a class of derivative-free techniques that only require access to the objective function through an oracle. We will present a novel direct-search method for min-max saddle-point problems, where the min and max players are updated sequentially. The convergence of this algorithm will be discussed in both deterministic and stochastic settings. Finally, experimental results related to robust optimization and Generative Adversarial Networks will be presented to illustrate how the proposed method can outperform commonly used optimization schemes.

Mathematics

Audience: researchers in the topic

PIMS-CORDS SFU Operations Research Seminar