Courtney Paquette, University of Waterloo

Abstract

We will present a very general framework for unconstrained stochastic optimization which encompasses standard frameworks such as line search and trust region using random models. In particular this framework retains the desirable practical features such step acceptance criterion, trust region adjustment and ability to utilize of second order models. The framework is based on bounding the expected stopping time of a stochastic process, which satisfies certain assumptions. Then the convergence rates are derived for each method by ensuring that the stochastic processes generated by the method satisfies these assumptions. The methods include a version of a stochastic trust-region method and a stochastic line-search methods and provide strong convergence analysis under weaker conditions than alternative approaches in the literature.