On the Convergence Time of Dual Subgradient Methods for Strongly Convex Programs

Abstract

This paper studies the convergence time of dual gradient methods for general (possibly non-differentiable) strongly convex programs. For general convex programs, the convergence time of dual subgradient/gradient methods with simple running averages (running averages started from iteration $0$) is known to be $O(\frac{1}{ε^{2}})$. This paper shows that the convergence time for general strongly convex programs is $O(\frac{1}ε)$. This paper also considers a variation of the average scheme, called the sliding running averages, and shows that if the dual function of the strongly convex program is locally quadratic (Note that the locally quadratic property is implied by the locally strongly concave property.) then the convergence time of the dual gradient method with sliding running averages is $O(\log(\frac{1}ε))$. The convergence time analysis is further verified by numerical experiments.

A short version is accepted to IEEE Transactions on Automatic Control. The current version extends the previous (CDC conference) version as follows: (1) simplify proofs; (2) relax the smooth dual function assumption in Sec IV by showing it always holds; (3) improve convergence time from $O(1/ε^{2/3})$ to $O(\log(1/ε))$ when dual function is locally quadratic