Benefits of Randomization on First Order Algorithms for Min-Max Optimization

Ahmet Alacaoglu, Univ. of Wisconsin Madison
January 30, 2023 3:00 pm MATX 1100

 

Reception preceding the talk in the IAM Lounge (LSK 306), 2:15

Modern data science applications require solving high dimensional optimization problems with large number of data points. Min-max optimization provides a unified framework for many problems in this context ranging from empirical risk minimization and distributionally robust optimization in machine learning to medical imaging and nonlinear programming. This talk will present two approaches for using randomization to design simple, practical and adaptive optimization algorithms that improve the complexity guarantees for convex-concave optimization. I will describe first order primal-dual algorithms with random coordinate updates and discuss their complexity guarantees as well as practical adaptivity properties. I will then present an algorithm with stochastic variance reduction that harnesses the finite-sum min-max structure to obtain sharp complexity bounds.