PLearn::ScaledGradientOptimizer Class Reference

#include <GradientOptimizer.h>

Inheritance diagram for PLearn::ScaledGradientOptimizer:

[legend]Collaboration diagram for PLearn::ScaledGradientOptimizer:


Public Member Functions
	ScaledGradientOptimizer (VarArray the_params, Var the_cost, real the_start_learning_rate=0.01, real the_decrease_constant=0.01, real the_init_learning_rate=0.003, int n_updates=1, real short_time_moving_avg_coef=0.01, real long_time_moving_avg_coef=0.001, real the_regularizer=1.0, const string &filename="", int every_iterations=1)
	ScaledGradientOptimizer (VarArray the_params, Var the_cost, real the_start_learning_rate=0.01, real the_decrease_constant=0.01, int n_updates=1, real short_time_moving_avg_coef=0.01, real long_time_moving_avg_coef=0.001, real the_regularizer=1.0, const string &filename="", int every_iterations=1)
virtual real	optimize ()
	sub-classes should define this, which is the main method
Public Attributes
real	start_learning_rate
	gradient descent specific parameters (directly modifiable by the user)
real	decrease_constant
real	init_learning_rate
real	learning_rate
Vec	eps_scale
	scaling parameter for the learning rate of each parameter
Protected Attributes
Vec	gradient
real	short_time_mac
	exponential moving average coefficient for short time scale
real	long_time_mac
	exponential moving average coefficient for long time scale
Vec	short_time_ma
	short time moving average
Vec	long_time_ma
	long time moving average
Vec	long_time_mv
	long time moving variance
Vec	long_time_md
	long time moving sdev
real	regularizer
	regularizer for update of local learning rates (eps_scale)

Constructor & Destructor Documentation

PLearn::ScaledGradientOptimizer::ScaledGradientOptimizer ( VarArray the_params,

Var the_cost,

real the_start_learning_rate = 0.01,

real the_decrease_constant = 0.01,

real the_init_learning_rate = 0.003,

int n_updates = 1,

real short_time_moving_avg_coef = 0.01,

real long_time_moving_avg_coef = 0.001,

real the_regularizer = 1.0,

const string & filename = "",

int every_iterations = 1

) [inline]

Definition at line 154 of file GradientOptimizer.h.
References decrease_constant, eps_scale, gradient, init_learning_rate, long_time_ma, long_time_mac, long_time_md, long_time_mv, regularizer, short_time_ma, short_time_mac, and start_learning_rate.

PLearn::ScaledGradientOptimizer::ScaledGradientOptimizer ( VarArray the_params,

Var the_cost,

real the_start_learning_rate = 0.01,

real the_decrease_constant = 0.01,

int n_updates = 1,

real short_time_moving_avg_coef = 0.01,

real long_time_moving_avg_coef = 0.001,

real the_regularizer = 1.0,

const string & filename = "",

int every_iterations = 1

) [inline]

Definition at line 179 of file GradientOptimizer.h.
References decrease_constant, eps_scale, PLearn::TVec< T >::fill(), gradient, long_time_ma, long_time_mac, long_time_md, long_time_mv, regularizer, short_time_ma, short_time_mac, and start_learning_rate.

Vec PLearn::ScaledGradientOptimizer::eps_scale

scaling parameter for the learning rate of each parameter

Definition at line 146 of file GradientOptimizer.h.
Referenced by optimize(), and ScaledGradientOptimizer().

real PLearn::ScaledGradientOptimizer::long_time_mac [protected]

exponential moving average coefficient for long time scale

Definition at line 129 of file GradientOptimizer.h.
Referenced by optimize(), and ScaledGradientOptimizer().

real PLearn::ScaledGradientOptimizer::regularizer [protected]

regularizer for update of local learning rates (eps_scale)

Definition at line 150 of file GradientOptimizer.h.
Referenced by optimize(), and ScaledGradientOptimizer().

real PLearn::ScaledGradientOptimizer::short_time_mac [protected]

exponential moving average coefficient for short time scale

Definition at line 127 of file GradientOptimizer.h.
Referenced by optimize(), and ScaledGradientOptimizer().

real PLearn::ScaledGradientOptimizer::start_learning_rate

gradient descent specific parameters (directly modifiable by the user)

Definition at line 142 of file GradientOptimizer.h.
Referenced by optimize(), and ScaledGradientOptimizer().

The documentation for this class was generated from the following files:

Generated on Tue Aug 17 16:24:12 2004 for PLearn by

1.3.7