PLearn::Optimizer Class Reference

#include <Optimizer.h>

Inheritance diagram for PLearn::Optimizer:

[legend]Collaboration diagram for PLearn::Optimizer:

[legend]List of all members.


Public Member Functions
	Optimizer (int n_updates=1, const string &file_name="", int every_iterations=1)
	Optimizer (VarArray the_params, Var the_cost, int n_updates=1, const string &file_name="", int every_iterations=1)
	Optimizer (VarArray the_params, Var the_cost, VarArray the_update_for_measure, int n_updates=1, const string &file_name="", int every_iterations=1)
virtual void	init ()
virtual void	build ()
	Should call simply inherited::build(), then this class's build_().
virtual void	reset ()
virtual void	setToOptimize (VarArray the_params, Var the_cost)
virtual void	setVarArrayOption (const string &optionname, VarArray value)
virtual void	setVarOption (const string &optionname, Var value)
virtual void	setVMatOption (const string &optionname, VMat value)
	PLEARN_DECLARE_ABSTRACT_OBJECT (Optimizer)
virtual void	makeDeepCopyFromShallowCopy (map< const void , void > &copies)
void	addMeasurer (Measurer &measurer)
virtual bool	measure (int t, const Vec &costs)
virtual real	optimize ()=0
	sub-classes should define this, which is the main method
virtual bool	optimizeN (VecStatsCollector &stats_coll)=0
	sub-classes should define this, which is the new main method
void	verifyGradient (real minval, real maxval, real step)
	verify gradient with uniform random initialization of parameters using step for the finite difference approximation of the gradient
void	verifyGradient (real step)
	verify gradient at the current value of the parameters using step for the finite difference approximation of the gradient
virtual void	oldwrite (ostream &out) const
virtual void	oldread (istream &in)
	DEPRECATED For backward compatibility with old saved object.
virtual	~Optimizer ()
void	computeRepartition (Vec v, int n, real mini, real maxi, Vec res, int &noutliers)
	Compute the repartition of v by splitting the interval [mini,maxi] into n intervals.
real	collectGradientStats (Vec gradient)
	Collect various statistics on the gradient.
Static Public Member Functions
void	computeGradient (Optimizer *opt, const Vec &gradient)
	Given an optimizer, compute the gradient of the cost function and store it in the "gradient" Vec.
void	computeOppositeGradient (Optimizer *opt, const Vec &gradient)
	Given an optimizer, compute the opposite of the gradient of the cost function and store it in the "gradient" Vec.
Public Attributes
VarArray	params
Var	cost
VarArray	proppath
int	nupdates
int	nstages
	number of steps to perform when calling optimizeN
int	stage
	current number of steps performed
bool	early_stop
int	early_stop_i
VarArray	update_for_measure
PStream	vlog
int	every
	if = 0 don't print or measure anything
Static Protected Member Functions
void	declareOptions (OptionList &ol)
	redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down)
Protected Attributes
Array< Measurer * >	measurers
string	filename
	call measure <every> <nupdates> iterations saving the results in the <filename>.
Private Types
typedef Object	inherited
Private Member Functions
void	build_ ()
Private Attributes
Vec	temp_grad
	used to store temp stuff for gradient stats

Member Typedef Documentation

typedef Object PLearn::Optimizer::inherited [private]

Reimplemented from PLearn::Object.
Reimplemented in PLearn::AdaptGradientOptimizer, PLearn::ConjGradientOptimizer, and PLearn::GradientOptimizer.
Definition at line 68 of file Optimizer.h.

Constructor & Destructor Documentation

PLearn::Optimizer::Optimizer ( int n_updates = 1,

const string & file_name = "",

int every_iterations = 1

)

Definition at line 54 of file Optimizer.cc.

PLearn::Optimizer::Optimizer ( VarArray the_params,

Var the_cost,

int n_updates = 1,

const string & file_name = "",

int every_iterations = 1

)

Definition at line 59 of file Optimizer.cc.

PLearn::Optimizer::Optimizer ( VarArray the_params,

Var the_cost,

VarArray the_update_for_measure,

int n_updates = 1,

const string & file_name = "",

int every_iterations = 1

)

Definition at line 65 of file Optimizer.cc.

PLearn::Optimizer::~Optimizer ( ) [virtual]

Definition at line 227 of file Optimizer.cc.

Member Function Documentation

void PLearn::Optimizer::addMeasurer ( Measurer & measurer )

Definition at line 200 of file Optimizer.cc.
References PLearn::TVec< Measurer * >::appendIfNotThereAlready(), and measurers.

void PLearn::Optimizer::build ( ) [virtual]

Should call simply inherited::build(), then this class's build_().
This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.
Reimplemented from PLearn::Object.
Reimplemented in PLearn::AdaptGradientOptimizer, PLearn::ConjGradientOptimizer, and PLearn::GradientOptimizer.
Definition at line 73 of file Optimizer.cc.
References build_().
Referenced by init(), and makeDeepCopyFromShallowCopy().

void PLearn::Optimizer::build_ ( ) [private]

This method should be redefined in subclasses and do the actual building of the object according to previously set option fields. Constructors can just set option fields, and then call build_. This method is NOT virtual, and will typically be called only from three places: a constructor, the public virtual build() method, and possibly the public virtual read method (which calls its parent's read). build_() can assume that it's parent's build_ has already been called.
Reimplemented from PLearn::Object.
Reimplemented in PLearn::AdaptGradientOptimizer, PLearn::ConjGradientOptimizer, and PLearn::GradientOptimizer.
Definition at line 79 of file Optimizer.cc.
References cost, early_stop, early_stop_i, PLearn::VarArray::fprop(), PLearn::TVec< Var >::length(), PLearn::VarArray::nelems(), params, PLearn::propagationPath(), PLearn::propagationPathToParentsOfPath(), proppath, PLearn::TVec< T >::resize(), temp_grad, and update_for_measure.
Referenced by build().

real PLearn::Optimizer::collectGradientStats ( Vec gradient )

Collect various statistics on the gradient.

void PLearn::Optimizer::computeGradient ( Optimizer * opt,

const Vec & gradient

) [static]

Given an optimizer, compute the gradient of the cost function and store it in the "gradient" Vec.

Definition at line 259 of file Optimizer.cc.
References PLearn::VarArray::clearGradient(), PLearn::VarArray::copyGradientTo(), cost, PLearn::VarArray::fbprop(), params, and proppath.

void PLearn::Optimizer::computeOppositeGradient ( Optimizer * opt,

const Vec & gradient

) [static]

Given an optimizer, compute the opposite of the gradient of the cost function and store it in the "gradient" Vec.

Definition at line 277 of file Optimizer.cc.
References PLearn::VarArray::clearGradient(), PLearn::VarArray::copyGradientTo(), cost, PLearn::VarArray::fbprop(), params, and proppath.

void PLearn::Optimizer::computeRepartition ( Vec v,

int n,

real mini,

real maxi,

Vec res,

int & noutliers

)

Compute the repartition of v by splitting the interval [mini,maxi] into n intervals.
The result is stored into res.
Definition at line 233 of file Optimizer.cc.
References PLearn::TVec< T >::clear(), k, and PLearn::TVec< T >::length().

void PLearn::Optimizer::declareOptions ( OptionList & ol ) [static, protected]

redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down)
ex: static void declareOptions(OptionList& ol) { declareOption(ol, "inputsize", &MyObject::inputsize_, OptionBase::buildoption, "the size of the input\n it must be provided"); declareOption(ol, "weights", &MyObject::weights, OptionBase::learntoption, "the learnt model weights"); inherited::declareOptions(ol); }
Reimplemented from PLearn::Object.
Reimplemented in PLearn::AdaptGradientOptimizer, PLearn::ConjGradientOptimizer, and PLearn::GradientOptimizer.
Definition at line 106 of file Optimizer.cc.
References PLearn::declareOption(), and PLearn::OptionList.

virtual void PLearn::Optimizer::init ( ) [inline, virtual]

Definition at line 114 of file Optimizer.h.
References build().

void PLearn::Optimizer::makeDeepCopyFromShallowCopy ( map< const void *, void * > & copies ) [virtual]

Definition at line 179 of file Optimizer.cc.
References build(), cost, PLearn::deepCopyField(), measurers, params, PLWARNING, PLearn::TVec< Measurer * >::size(), temp_grad, update_for_measure, and PLearn::varDeepCopyField().

bool PLearn::Optimizer::measure ( int t,

const Vec & costs

) [virtual]

Definition at line 205 of file Optimizer.cc.
References measurers, and PLearn::TVec< Measurer * >::size().

void PLearn::Optimizer::oldread ( istream & in ) [virtual]

DEPRECATED For backward compatibility with old saved object.

Reimplemented from PLearn::Object.
Definition at line 131 of file Optimizer.cc.
References every, nupdates, PLERROR, PLearn::readField(), PLearn::readFooter(), and PLearn::readHeader().

void PLearn::Optimizer::oldwrite ( ostream & out ) const [virtual]

Definition at line 123 of file Optimizer.cc.
References every, nupdates, PLearn::writeField(), PLearn::writeFooter(), and PLearn::writeHeader().

virtual real PLearn::Optimizer::optimize ( ) [pure virtual]

sub-classes should define this, which is the main method

Implemented in PLearn::AdaptGradientOptimizer, PLearn::ConjGradientOptimizer, PLearn::GradientOptimizer, and PLearn::ScaledGradientOptimizer.

virtual bool PLearn::Optimizer::optimizeN ( VecStatsCollector & stats_coll ) [pure virtual]

sub-classes should define this, which is the new main method

Implemented in PLearn::AdaptGradientOptimizer, PLearn::ConjGradientOptimizer, and PLearn::GradientOptimizer.

PLearn::Optimizer::PLEARN_DECLARE_ABSTRACT_OBJECT ( Optimizer )

void PLearn::Optimizer::reset ( ) [virtual]

Reimplemented in PLearn::ConjGradientOptimizer.
Definition at line 101 of file Optimizer.cc.
References stage.

void PLearn::Optimizer::setToOptimize ( VarArray the_params,

Var the_cost

) [virtual]

Definition at line 141 of file Optimizer.cc.
References cost, PLearn::VarArray::fprop(), params, PLearn::propagationPath(), PLearn::propagationPathToParentsOfPath(), and proppath.
Referenced by setVarArrayOption(), and setVarOption().

void PLearn::Optimizer::setVarArrayOption ( const string & optionname,

VarArray value

) [virtual]

Definition at line 152 of file Optimizer.cc.
References cost, PLERROR, setToOptimize(), and update_for_measure.

void PLearn::Optimizer::setVarOption ( const string & optionname,

Var value

) [virtual]

Definition at line 159 of file Optimizer.cc.
References params, PLERROR, and setToOptimize().

void PLearn::Optimizer::setVMatOption ( const string & optionname,

VMat value

) [virtual]

Definition at line 165 of file Optimizer.cc.
References PLERROR.

void PLearn::Optimizer::verifyGradient ( real step )

verify gradient at the current value of the parameters using step for the finite difference approximation of the gradient

Definition at line 219 of file Optimizer.cc.
References cost, PLearn::VarArray::nelems(), and params.

void PLearn::Optimizer::verifyGradient ( real minval,

real maxval,

real step

)

verify gradient with uniform random initialization of parameters using step for the finite difference approximation of the gradient

Definition at line 213 of file Optimizer.cc.
References cost, and params.

Member Data Documentation

Var PLearn::Optimizer::cost

Definition at line 72 of file Optimizer.h.
Referenced by build_(), PLearn::ConjGradientOptimizer::computeCostAndDerivative(), PLearn::ConjGradientOptimizer::computeCostValue(), computeGradient(), computeOppositeGradient(), PLearn::ConjGradientOptimizer::fletcherSearchMain(), makeDeepCopyFromShallowCopy(), setToOptimize(), setVarArrayOption(), and verifyGradient().

bool PLearn::Optimizer::early_stop

Definition at line 78 of file Optimizer.h.
Referenced by build_(), and PLearn::ConjGradientOptimizer::fletcherSearchMain().

int PLearn::Optimizer::early_stop_i

Definition at line 79 of file Optimizer.h.
Referenced by build_().

int PLearn::Optimizer::every

if = 0 don't print or measure anything

Definition at line 101 of file Optimizer.h.
Referenced by oldread(), and oldwrite().

string PLearn::Optimizer::filename [protected]

call measure <every> <nupdates> iterations saving the results in the <filename>.

Definition at line 98 of file Optimizer.h.

Array<Measurer*> PLearn::Optimizer::measurers [protected]

Definition at line 94 of file Optimizer.h.
Referenced by addMeasurer(), makeDeepCopyFromShallowCopy(), and measure().

int PLearn::Optimizer::nstages

number of steps to perform when calling optimizeN

Definition at line 75 of file Optimizer.h.

int PLearn::Optimizer::nupdates

Definition at line 74 of file Optimizer.h.
Referenced by oldread(), and oldwrite().

VarArray PLearn::Optimizer::params

Definition at line 71 of file Optimizer.h.
Referenced by build_(), PLearn::ConjGradientOptimizer::computeCostAndDerivative(), PLearn::ConjGradientOptimizer::computeCostValue(), PLearn::ConjGradientOptimizer::computeDerivative(), computeGradient(), computeOppositeGradient(), makeDeepCopyFromShallowCopy(), setToOptimize(), setVarOption(), and verifyGradient().

VarArray PLearn::Optimizer::proppath

Definition at line 73 of file Optimizer.h.
Referenced by build_(), PLearn::ConjGradientOptimizer::computeCostAndDerivative(), PLearn::ConjGradientOptimizer::computeCostValue(), computeGradient(), computeOppositeGradient(), and setToOptimize().

int PLearn::Optimizer::stage

current number of steps performed

Definition at line 76 of file Optimizer.h.
Referenced by reset().

Vec PLearn::Optimizer::temp_grad [private]

used to store temp stuff for gradient stats

Definition at line 89 of file Optimizer.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

VarArray PLearn::Optimizer::update_for_measure

Definition at line 81 of file Optimizer.h.
Referenced by build_(), makeDeepCopyFromShallowCopy(), and setVarArrayOption().

PStream PLearn::Optimizer::vlog

Definition at line 85 of file Optimizer.h.

The documentation for this class was generated from the following files:

Generated on Tue Aug 17 16:24:16 2004 for PLearn by

1.3.7

PLearn::Optimizer Class Reference

Public Member Functions

Static Public Member Functions

Public Attributes

Static Protected Member Functions

Protected Attributes

Private Types

Private Member Functions

Private Attributes

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation