Proximal gradient method
This article includes a list of general references, but it lacks sufficient corresponding inline citations. (November 2013) |
Proximal gradient methods are a generalized form of projection used to solve non-differentiable convex optimization problems.
Many interesting problems can be formulated as convex optimization problems of the form
where are possibly non-differentiable
Proximal gradient methods starts by a splitting step, in which the functions are used individually so as to yield an easily
For the theory of proximal gradient methods from the perspective of and with applications to statistical learning theory, see proximal gradient methods for learning.
Projection onto convex sets (POCS)
One of the widely used convex optimization algorithms is projections onto convex sets (POCS). This algorithm is employed to recover/synthesize a signal satisfying simultaneously several convex constraints. Let be the indicator function of non-empty closed convex set modeling a constraint. This reduces to convex feasibility problem, which require us to find a solution such that it lies in the intersection of all convex sets . In POCS method each set is incorporated by its
However beyond such problems
Examples
Special instances of Proximal Gradient Methods are
- Projected Landweber
- Alternating projection
- Alternating-direction method of multipliers
See also
Notes
- ^ Daubechies, I; Defrise, M; .
- ].
References
- Rockafellar, R. T. (1970). Convex analysis. Princeton: Princeton University Press.
- Combettes, Patrick L.; Pesquet, Jean-Christophe (2011). Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Vol. 49. pp. 185–212.
External links
- Stephen Boyd and Lieven Vandenberghe Book, Convex optimization
- EE364a: Convex Optimization I and EE364b: Convex Optimization II, Stanford course homepages
- EE227A: Lieven Vandenberghe Notes Lecture 18
- ProximalOperators.jl: a Julia package implementing proximal operators.
- ProximalAlgorithms.jl: a Julia package implementing algorithms based on the proximal operator, including the proximal gradient method.
- Proximity Operator repository: a collection of proximity operators implemented in Matlab and Python.