I will present secant penalized BFGS (SP-BFGS), a noise robust variant of the well-known BFGS optimization technique (see https://doi.org/10.1007/s10589-022-00448-x). SP-BFGS is designed to perform well when gradient measurements are corrupted by noise. I will demonstrate how treating the secant condition with a penalty method approach motivated by regularized least squares estimation generates a parametric family with the original BFGS update at one extreme and not updating the inverse Hessian approximation at the other extreme. SP-BFGS allows one to relax the secant condition based on the amount of noise in the gradient measurements, which provides a means of incrementally updating the new inverse Hessian approximation with a controlled amount of bias towards the previous inverse Hessian approximation. The controlled bias allows one to replace the overwriting nature of the original BFGS update with an averaging nature that resists the destructive effects of noise and can cope with negative curvature measurements. I will present an overview of the theoretical properties of SP-BFGS, including convergence when minimizing strongly convex functions in the presence of uniformly bounded noise, and the results of extensive numerical experiments that demonstrate the superior performance of SP-BFGS compared to BFGS in the presence of both noisy function and gradient evaluations.

Pizza lunch will be served.

We gratefully acknowledge generous financial support by the Pacific Institute for the Mathematical Sciences (PIMS) and the Institute of Applied Mathematics (IAM).