go home Home | Main Page | Modules | Namespace List | Class Hierarchy | Alphabetical List | Data Structures | File List | Namespace Members | Data Fields | Globals | Related Pages
Public Types | Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | Private Attributes
itk::StandardStochasticVarianceReducedGradientOptimizer Class Reference

#include <itkStandardStochasticVarianceReducedGradientDescentOptimizer.h>

Detailed Description

This class implements a gradient descent optimizer with a decaying gain.

If $C(x)$ is a cost function that has to be minimised, the following iterative algorithm is used to find the optimal parameters $x$:

\[ x(k+1) = x(k) - a(k) dC/dx \]

The gain $a(k)$ at each iteration $k$ is defined by:

\[ a(k) =  a / (A + k + 1)^alpha \]


It is very suitable to be used in combination with a stochastic estimate of the gradient $dC/dx$. For example, in image registration problems it is often advantageous to compute the metric derivative ( $dC/dx$) on a new set of randomly selected image samples in each iteration. You may set the parameter NewSamplesEveryIteration to "true" to achieve this effect. For more information on this strategy, you may have a look at:

S. Klein, M. Staring, J.P.W. Pluim, "Evaluation of Optimization Methods for Nonrigid Medical Image Registration using Mutual Information and B-Splines" IEEE Transactions on Image Processing, 2007, nr. 16(12), December.

This class also serves as a base class for other StochasticVarianceReducedGradient type algorithms, like the AcceleratedStochasticVarianceReducedGradientOptimizer.

See also
StandardStochasticVarianceReducedGradient, AcceleratedStochasticVarianceReducedGradientOptimizer

Definition at line 56 of file itkStandardStochasticVarianceReducedGradientDescentOptimizer.h.

Inheritance diagram for itk::StandardStochasticVarianceReducedGradientOptimizer:
Inheritance graph

Public Types

using ConstPointer = SmartPointer< const Self >
using Pointer = SmartPointer< Self >
using Self = StandardStochasticVarianceReducedGradientOptimizer
enum  StopConditionType
using Superclass = StochasticVarianceReducedGradientDescentOptimizer
- Public Types inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
using ConstPointer = SmartPointer< const Self >
using Pointer = SmartPointer< Self >
using ScaledCostFunctionPointer = ScaledCostFunctionType::Pointer
using ScaledCostFunctionType = ScaledSingleValuedCostFunction
using ScalesType = NonLinearOptimizer::ScalesType
using Self = StochasticVarianceReducedGradientDescentOptimizer
enum  StopConditionType {
  MaximumNumberOfIterations , MetricError , MinimumStepSize , InvalidDiagonalMatrix ,
  GradientMagnitudeTolerance , LineSearchError
using Superclass = ScaledSingleValuedNonLinearOptimizer
- Public Types inherited from itk::ScaledSingleValuedNonLinearOptimizer
using ConstPointer = SmartPointer< const Self >
using Pointer = SmartPointer< Self >
using ScaledCostFunctionPointer = ScaledCostFunctionType::Pointer
using ScaledCostFunctionType = ScaledSingleValuedCostFunction
using ScalesType = NonLinearOptimizer::ScalesType
using Self = ScaledSingleValuedNonLinearOptimizer
using Superclass = SingleValuedNonLinearOptimizer

Public Member Functions

void AdvanceOneStep () override
virtual const char * GetClassName () const
virtual double GetCurrentTime () const
virtual double GetInitialTime () const
virtual double GetParam_a () const
virtual double GetParam_A () const
virtual double GetParam_alpha () const
virtual double GetParam_beta () const
 ITK_DISALLOW_COPY_AND_MOVE (StandardStochasticVarianceReducedGradientOptimizer)
virtual void ResetCurrentTimeToInitialTime ()
virtual void SetInitialTime (double _arg)
virtual void SetParam_a (double _arg)
virtual void SetParam_A (double _arg)
virtual void SetParam_alpha (double _arg)
virtual void SetParam_beta (double _arg)
void StartOptimization () override
- Public Member Functions inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
virtual void AdvanceOneStep ()
virtual const char * GetClassName () const
virtual unsigned int GetCurrentInnerIteration () const
virtual unsigned int GetCurrentIteration () const
virtual const DerivativeType & GetGradient ()
virtual unsigned int GetLBFGSMemory () const
virtual const doubleGetLearningRate ()
virtual const unsigned long & GetNumberOfInnerIterations ()
virtual const unsigned long & GetNumberOfIterations ()
virtual const DerivativeType & GetPreviousGradient ()
virtual const ParametersType & GetPreviousPosition ()
virtual const DerivativeType & GetSearchDir ()
virtual const StopConditionTypeGetStopCondition ()
virtual const doubleGetValue ()
 ITK_DISALLOW_COPY_AND_MOVE (StochasticVarianceReducedGradientDescentOptimizer)
virtual void MetricErrorResponse (ExceptionObject &err)
virtual void ResumeOptimization ()
virtual void SetLearningRate (double _arg)
virtual void SetNumberOfIterations (unsigned long _arg)
void SetNumberOfWorkUnits (ThreadIdType numberOfThreads)
virtual void SetPreviousGradient (DerivativeType _arg)
virtual void SetPreviousPosition (ParametersType _arg)
virtual void SetUseEigen (bool _arg)
virtual void SetUseMultiThread (bool _arg)
virtual void SetUseOpenMP (bool _arg)
void StartOptimization () override
virtual void StopOptimization ()
- Public Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
virtual const char * GetClassName () const
const ParametersType & GetCurrentPosition () const override
virtual bool GetMaximize () const
virtual const ScaledCostFunctionTypeGetScaledCostFunction ()
virtual const ParametersType & GetScaledCurrentPosition ()
bool GetUseScales () const
virtual void InitializeScales ()
 ITK_DISALLOW_COPY_AND_MOVE (ScaledSingleValuedNonLinearOptimizer)
virtual void MaximizeOff ()
virtual void MaximizeOn ()
void SetCostFunction (CostFunctionType *costFunction) override
virtual void SetMaximize (bool _arg)
virtual void SetUseScales (bool arg)

Static Public Member Functions

static Pointer New ()
- Static Public Member Functions inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
static Pointer New ()
- Static Public Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
static Pointer New ()

Protected Member Functions

virtual double Compute_a (double k) const
virtual double Compute_beta (double k) const
 StandardStochasticVarianceReducedGradientOptimizer ()
virtual void UpdateCurrentTime ()
 ~StandardStochasticVarianceReducedGradientOptimizer () override=default
- Protected Member Functions inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
void PrintSelf (std::ostream &os, Indent indent) const override
 StochasticVarianceReducedGradientDescentOptimizer ()
 ~StochasticVarianceReducedGradientDescentOptimizer () override=default
- Protected Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
virtual void GetScaledDerivative (const ParametersType &parameters, DerivativeType &derivative) const
virtual MeasureType GetScaledValue (const ParametersType &parameters) const
virtual void GetScaledValueAndDerivative (const ParametersType &parameters, MeasureType &value, DerivativeType &derivative) const
void PrintSelf (std::ostream &os, Indent indent) const override
 ScaledSingleValuedNonLinearOptimizer ()
void SetCurrentPosition (const ParametersType &param) override
virtual void SetScaledCurrentPosition (const ParametersType &parameters)
 ~ScaledSingleValuedNonLinearOptimizer () override=default

Protected Attributes

double m_CurrentTime { 0.0 }
bool m_UseConstantStep
- Protected Attributes inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
unsigned long m_CurrentInnerIteration
unsigned long m_CurrentIteration { 0 }
DerivativeType m_Gradient
unsigned long m_LBFGSMemory { 0 }
double m_LearningRate { 1.0 }
ParametersType m_MeanSearchDir
unsigned long m_NumberOfInnerIterations
unsigned long m_NumberOfIterations { 100 }
DerivativeType m_PreviousGradient
ParametersType m_PreviousPosition
ParametersType m_PreviousSearchDir
ParametersType m_SearchDir
bool m_Stop { false }
StopConditionType m_StopCondition { MaximumNumberOfIterations }
ThreaderType::Pointer m_Threader { ThreaderType::New() }
double m_Value { 0.0 }
- Protected Attributes inherited from itk::ScaledSingleValuedNonLinearOptimizer
ScaledCostFunctionPointer m_ScaledCostFunction
ParametersType m_ScaledCurrentPosition

Private Attributes

double m_InitialTime { 0.0 }
double m_Param_a { 1.0 }
double m_Param_A { 1.0 }
double m_Param_alpha { 0.602 }
double m_Param_beta

Additional Inherited Members

- Protected Types inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
using ThreaderType = itk::PlatformMultiThreader
using ThreadInfoType = ThreaderType::WorkUnitInfo

Member Typedef Documentation

◆ ConstPointer

◆ Pointer

◆ Self

Standard ITK.

Definition at line 62 of file itkStandardStochasticVarianceReducedGradientDescentOptimizer.h.

◆ Superclass

Member Enumeration Documentation

◆ StopConditionType

Codes of stopping conditions The MinimumStepSize stop condition never occurs, but may be implemented in inheriting classes

Definition at line 81 of file itkStochasticVarianceReducedGradientDescentOptimizer.h.

Constructor & Destructor Documentation

◆ StandardStochasticVarianceReducedGradientOptimizer()

itk::StandardStochasticVarianceReducedGradientOptimizer::StandardStochasticVarianceReducedGradientOptimizer ( )

◆ ~StandardStochasticVarianceReducedGradientOptimizer()

itk::StandardStochasticVarianceReducedGradientOptimizer::~StandardStochasticVarianceReducedGradientOptimizer ( )

Member Function Documentation

◆ AdvanceOneStep()

void itk::StandardStochasticVarianceReducedGradientOptimizer::AdvanceOneStep ( )

Sets a new LearningRate before calling the Superclass' implementation, and updates the current time.

Reimplemented from itk::StochasticVarianceReducedGradientDescentOptimizer.

◆ Compute_a()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::Compute_a ( double  k) const

Function to compute the step size for SGD at time/iteration k.

◆ Compute_beta()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::Compute_beta ( double  k) const

Function to compute the step size for SQN at time/iteration k.

◆ GetClassName()

virtual const char * itk::StandardStochasticVarianceReducedGradientOptimizer::GetClassName ( ) const

◆ GetCurrentTime()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::GetCurrentTime ( ) const

Get the current time. This equals the CurrentIteration in this base class but may be different in inheriting classes, such as the AccelerateStochasticVarianceReducedGradient

◆ GetInitialTime()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::GetInitialTime ( ) const

◆ GetParam_a()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::GetParam_a ( ) const

◆ GetParam_A()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::GetParam_A ( ) const

◆ GetParam_alpha()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::GetParam_alpha ( ) const

◆ GetParam_beta()

virtual double itk::StandardStochasticVarianceReducedGradientOptimizer::GetParam_beta ( ) const


itk::StandardStochasticVarianceReducedGradientOptimizer::ITK_DISALLOW_COPY_AND_MOVE ( StandardStochasticVarianceReducedGradientOptimizer  )

◆ New()

static Pointer itk::StandardStochasticVarianceReducedGradientOptimizer::New ( )

Method for creation through the object factory.

◆ ResetCurrentTimeToInitialTime()

virtual void itk::StandardStochasticVarianceReducedGradientOptimizer::ResetCurrentTimeToInitialTime ( )

Set the current time to the initial time. This can be useful to 'reset' the optimisation, for example if you changed the cost function while optimisation. Be careful with this function.

Definition at line 125 of file itkStandardStochasticVarianceReducedGradientDescentOptimizer.h.

◆ SetInitialTime()

virtual void itk::StandardStochasticVarianceReducedGradientOptimizer::SetInitialTime ( double  _arg)

Set/Get the initial time. Should be >=0. This function is superfluous, since Param_A does effectively the same. However, in inheriting classes, like the AcceleratedStochasticVarianceReducedGradient the initial time may have a different function than Param_A. Default: 0.0

◆ SetParam_a()

virtual void itk::StandardStochasticVarianceReducedGradientOptimizer::SetParam_a ( double  _arg)

Set/Get a.

◆ SetParam_A()

virtual void itk::StandardStochasticVarianceReducedGradientOptimizer::SetParam_A ( double  _arg)

Set/Get A.

◆ SetParam_alpha()

virtual void itk::StandardStochasticVarianceReducedGradientOptimizer::SetParam_alpha ( double  _arg)

Set/Get alpha.

◆ SetParam_beta()

virtual void itk::StandardStochasticVarianceReducedGradientOptimizer::SetParam_beta ( double  _arg)

Set/Get beta.

◆ StartOptimization()

void itk::StandardStochasticVarianceReducedGradientOptimizer::StartOptimization ( )

Set current time to 0 and call superclass' implementation.

◆ UpdateCurrentTime()

virtual void itk::StandardStochasticVarianceReducedGradientOptimizer::UpdateCurrentTime ( )

Function to update the current time This function just increments the CurrentTime by 1. Inheriting functions may implement something smarter, for example, dependent on the progress.

Reimplemented in itk::AdaptiveStochasticVarianceReducedGradientOptimizer.

Field Documentation

◆ m_CurrentTime

double itk::StandardStochasticVarianceReducedGradientOptimizer::m_CurrentTime { 0.0 }

The current time, which serves as input for Compute_a

Definition at line 151 of file itkStandardStochasticVarianceReducedGradientDescentOptimizer.h.

◆ m_InitialTime

double itk::StandardStochasticVarianceReducedGradientOptimizer::m_InitialTime { 0.0 }

◆ m_Param_a

double itk::StandardStochasticVarianceReducedGradientOptimizer::m_Param_a { 1.0 }

Parameters, as described by Spall.

Definition at line 158 of file itkStandardStochasticVarianceReducedGradientDescentOptimizer.h.

◆ m_Param_A

double itk::StandardStochasticVarianceReducedGradientOptimizer::m_Param_A { 1.0 }

◆ m_Param_alpha

double itk::StandardStochasticVarianceReducedGradientOptimizer::m_Param_alpha { 0.602 }

◆ m_Param_beta

double itk::StandardStochasticVarianceReducedGradientOptimizer::m_Param_beta

◆ m_UseConstantStep

bool itk::StandardStochasticVarianceReducedGradientOptimizer::m_UseConstantStep

Constant step size or others, different value of k.

Definition at line 154 of file itkStandardStochasticVarianceReducedGradientDescentOptimizer.h.

Generated on Wed 12 Apr 2023 for elastix by doxygen 1.9.6 elastix logo