PLearn::NNet Class Reference

#include <NNet.h>

Inheritance diagram for PLearn::NNet:

[legend]Collaboration diagram for PLearn::NNet:

[legend]List of all members.


Public Types
typedef PLearner	inherited
Public Member Functions
	NNet ()
virtual	~NNet ()
	PLEARN_DECLARE_OBJECT (NNet)
virtual void	build ()
	** SUBCLASS WRITING: ** This method should be redefined in subclasses, to just call inherited::build() and then build_()
virtual void	forget ()
	(Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!)
virtual int	outputsize () const
	SUBCLASS WRITING: overload this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options.
virtual TVec< string >	getTrainCostNames () const
	* SUBCLASS WRITING: * This should return the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats
virtual TVec< string >	getTestCostNames () const
	* SUBCLASS WRITING: * This should return the names of the costs computed by computeCostsFromOutpus
virtual void	train ()
	The role of the train method is to bring the learner up to stage==nstages, updating the stats with training costs measured on-line in the process.
virtual void	computeOutput (const Vec &input, Vec &output) const
	* SUBCLASS WRITING: * This should be defined in subclasses to compute the output from the input
virtual void	computeOutputAndCosts (const Vec &input, const Vec &target, Vec &output, Vec &costs) const
	Default calls computeOutput and computeCostsFromOutputs You may overload this if you have a more efficient way to compute both output and weighted costs at the same time.
virtual void	computeCostsFromOutputs (const Vec &input, const Vec &output, const Vec &target, Vec &costs) const
	* SUBCLASS WRITING: * This should be defined in subclasses to compute the weighted costs from already computed output.
virtual void	makeDeepCopyFromShallowCopy (CopiesMap &copies)
Public Attributes
Func	f
Func	test_costf
Func	output_and_target_to_cost
int	nhidden
int	nhidden2
int	noutputs
real	weight_decay
real	bias_decay
real	layer1_weight_decay
real	layer1_bias_decay
real	layer2_weight_decay
real	layer2_bias_decay
real	output_layer_weight_decay
real	output_layer_bias_decay
real	direct_in_to_out_weight_decay
real	classification_regularizer
real	margin
bool	fixed_output_weights
int	rbf_layer_size
bool	first_class_is_junk
bool	L1_penalty
real	input_reconstruction_penalty
bool	direct_in_to_out
string	output_transfer_func
string	hidden_transfer_func
real	interval_minval
real	interval_maxval
Array< string >	cost_funcs
	a list of cost functions to use in the form "[ cf1; cf2; cf3; ... ]"
PP< Optimizer >	optimizer
int	batch_size
string	initialization_method
Protected Member Functions
void	initializeParams ()
Static Protected Member Functions
void	declareOptions (OptionList &ol)
	redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down)
Protected Attributes
Var	input
Var	target
Var	sampleweight
Var	w1
Var	w2
Var	wout
Var	outbias
Var	wdirect
Var	wrec
Var	rbf_centers
Var	rbf_sigmas
Var	junk_prob
Var	output
Var	predicted_input
VarArray	costs
VarArray	penalties
Var	training_cost
Var	test_costs
VarArray	invars
VarArray	params
Vec	paramsvalues
Private Member Functions
void	build_ ()

Member Typedef Documentation

typedef PLearner PLearn::NNet::inherited

Reimplemented from PLearn::PLearner.
Definition at line 86 of file NNet.h.

Constructor & Destructor Documentation

PLearn::NNet::NNet ( )

Definition at line 83 of file NNet.cc.
References PLearn::tanh().

PLearn::NNet::~NNet ( ) [virtual]

Definition at line 112 of file NNet.cc.

Member Function Documentation

void PLearn::NNet::build ( ) [virtual]

**** SUBCLASS WRITING: **** This method should be redefined in subclasses, to just call inherited::build() and then build_()

Reimplemented from PLearn::PLearner.
Definition at line 242 of file NNet.cc.
References build_().
Referenced by train().

void PLearn::NNet::build_ ( ) [private]

**** SUBCLASS WRITING: **** This method should finalize building of the object, according to set 'options', in *any* situation.
Typical situations include:

Initial building of an object from a few user-specified options
Building of a "reloaded" object: i.e. from the complete set of all serialised options.
Updating or "re-building" of an object after a few "tuning" options (such as hyper-parameters) have been modified.

You can assume that the parent class' build_() has already been called.
A typical build method will want to know the inputsize(), targetsize() and outputsize(), and may also want to check whether train_set->hasWeights(). All these methods require a train_set to be set, so the first thing you may want to do, is check if(train_set), before doing any heavy building...
Note: build() is always called by setTrainingSet.
Reimplemented from PLearn::PLearner.
Definition at line 248 of file NNet.cc.
References PLearn::affine_transform(), PLearn::affine_transform_weight_penalty(), PLearn::TVec< Var >::append(), bias_decay, PLearn::binary_classification_loss(), c_str(), PLearn::classification_loss(), classification_regularizer, cost_funcs, costs, PLearn::cross_entropy(), direct_in_to_out, direct_in_to_out_weight_decay, PLearn::exp(), f, first_class_is_junk, fixed_output_weights, PLearn::hconcat(), hidden_transfer_func, initializeParams(), input, input_reconstruction_penalty, PLearn::PLearner::inputsize(), interval_maxval, interval_minval, invars, PLearn::TVec< Var >::isNull(), junk_prob, k, L1_penalty, layer1_bias_decay, layer1_weight_decay, layer2_bias_decay, layer2_weight_decay, PLearn::lift_output(), PLearn::log_softmax(), PLearn::VarArray::makeSharedValue(), margin, PLearn::margin_perceptron_cost(), PLearn::multiclass_loss(), PLearn::neg_log_pi(), PLearn::VarArray::nelems(), PLearn::newObject(), nhidden, nhidden2, PLearn::onehot_squared_loss(), outbias, output, output_and_target_to_cost, output_layer_bias_decay, output_layer_weight_decay, output_transfer_func, outputsize(), params, paramsvalues, penalties, PLERROR, predicted_input, PLearn::TVec< Var >::push_back(), rbf_centers, rbf_layer_size, rbf_sigmas, PLearn::TVec< T >::resize(), PLearn::TVec< Var >::resize(), sampleweight, PLearn::sigmoid(), PLearn::TVec< T >::size(), PLearn::TVec< Var >::size(), PLearn::TVec< string >::size(), PLearn::softmax(), PLearn::softplus(), PLearn::stable_cross_entropy(), PLearn::sum(), PLearn::sumabs(), PLearn::sumsquare(), PLearn::tanh(), target, PLearn::PLearner::targetsize(), test_costf, test_costs, training_cost, PLearn::transposeProduct(), PLearn::unary_hard_slope(), w1, w2, wdirect, weight_decay, wout, and wrec.
Referenced by build().

void PLearn::NNet::computeCostsFromOutputs ( const Vec & input,

const Vec & output,

const Vec & target,

Vec & costs

) const [virtual]

*** SUBCLASS WRITING: *** This should be defined in subclasses to compute the weighted costs from already computed output.
The costs should correspond to the cost names returned by getTestCostNames() NOTE: In exotic cases, the cost may also depend on some info in the input, that's why the method also gets so see it.
Implements PLearn::PLearner.
Definition at line 661 of file NNet.cc.
References output_and_target_to_cost.

void PLearn::NNet::computeOutput ( const Vec & input,

Vec & output

) const [virtual]

*** SUBCLASS WRITING: *** This should be defined in subclasses to compute the output from the input

Implements PLearn::PLearner.
Definition at line 649 of file NNet.cc.
References f.

void PLearn::NNet::computeOutputAndCosts ( const Vec & input,

const Vec & target,

Vec & output,

Vec & costs

) const [virtual]

Default calls computeOutput and computeCostsFromOutputs You may overload this if you have a more efficient way to compute both output and weighted costs at the same time.

Reimplemented from PLearn::PLearner.
Definition at line 654 of file NNet.cc.
References test_costf.

void PLearn::NNet::declareOptions ( OptionList & ol ) [static, protected]

redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down)
ex: static void declareOptions(OptionList& ol) { declareOption(ol, "inputsize", &MyObject::inputsize_, OptionBase::buildoption, "the size of the input\n it must be provided"); declareOption(ol, "weights", &MyObject::weights, OptionBase::learntoption, "the learnt model weights"); inherited::declareOptions(ol); }
Reimplemented from PLearn::PLearner.
Definition at line 116 of file NNet.cc.
References PLearn::declareOption(), and PLearn::OptionList.

void PLearn::NNet::forget ( ) [virtual]

(Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!)
*** SUBCLASS WRITING: *** A typical forget() method should do the following:

initialize a random number generator with the seed option
initialize the learner's parameters, using this random generator
stage = 0; This method is typically called by the build_() method, after it has finished setting up the parameters, and if it deemed useful to set or reset the learner in its fresh state. (remember build may be called after modifying options that do not necessarily require the learner to restart from a fresh state...) forget is also called by the setTrainingSet method, after calling build(), so it will generally be called TWICE during setTrainingSet!

Implements PLearn::PLearner.
Definition at line 744 of file NNet.cc.
References initializeParams().

TVec< string > PLearn::NNet::getTestCostNames ( ) const [virtual]

*** SUBCLASS WRITING: *** This should return the names of the costs computed by computeCostsFromOutpus

Implements PLearn::PLearner.
Definition at line 579 of file NNet.cc.
References cost_funcs.

TVec< string > PLearn::NNet::getTrainCostNames ( ) const [virtual]

*** SUBCLASS WRITING: *** This should return the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats

Implements PLearn::PLearner.
Definition at line 574 of file NNet.cc.
References cost_funcs.

void PLearn::NNet::initializeParams ( ) [protected]

Definition at line 667 of file NNet.cc.
References direct_in_to_out, PLearn::fill_random_discrete(), PLearn::fill_random_uniform(), fixed_output_weights, initialization_method, PLearn::PLearner::inputsize(), PLearn::manual_seed(), nhidden, nhidden2, optimizer, PLERROR, PLearn::TVec< T >::resize(), PLearn::seed(), PLearn::TVec< T >::size(), PLearn::sqrt(), w1, w2, wdirect, and wout.
Referenced by build_(), and forget().

void PLearn::NNet::makeDeepCopyFromShallowCopy ( CopiesMap & copies ) [virtual]

Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be. Typical implementation:
void CLASS_OF_THIS::makeDeepCopyFromShallowCopy(CopiesMap& copies) { SUPERCLASS_OF_THIS::makeDeepCopyFromShallowCopy(copies); member_ptr = member_ptr->deepCopy(copies); member_smartptr = member_smartptr->deepCopy(copies); member_mat.makeDeepCopyFromShallowCopy(copies); member_vec.makeDeepCopyFromShallowCopy(copies); ... }
Reimplemented from PLearn::PLearner.
Definition at line 753 of file NNet.cc.
References PLearn::CopiesMap, costs, PLearn::deepCopyField(), f, input, invars, junk_prob, optimizer, outbias, output, output_and_target_to_cost, params, paramsvalues, penalties, predicted_input, rbf_centers, rbf_sigmas, sampleweight, target, test_costf, test_costs, training_cost, PLearn::varDeepCopyField(), w1, w2, wdirect, wout, and wrec.

int PLearn::NNet::outputsize ( ) const [virtual]

SUBCLASS WRITING: overload this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options.

Implements PLearn::PLearner.
Definition at line 571 of file NNet.cc.
References noutputs.
Referenced by build_().

PLearn::NNet::PLEARN_DECLARE_OBJECT ( NNet )

void PLearn::NNet::train ( ) [virtual]

The role of the train method is to bring the learner up to stage==nstages, updating the stats with training costs measured on-line in the process.
*** SUBCLASS WRITING: ***
TYPICAL CODE:
static Vec input; // static so we don't reallocate/deallocate memory each time... static Vec target; // (but be careful that static means shared!) input.resize(inputsize()); // the train_set's inputsize() target.resize(targetsize()); // the train_set's targetsize() real weight;
if(!train_stats) // make a default stats collector, in case there's none train_stats = new VecStatsCollector();
if(nstages<stage) // asking to revert to a previous stage! forget(); // reset the learner to stage=0
while(stage<nstages) { clear statistics of previous epoch train_stats->forget();
... train for 1 stage, and update train_stats, using train_set->getSample(input, target, weight); and train_stats->update(train_costs)
++stage; train_stats->finalize(); // finalize statistics for this epoch }
Implements PLearn::PLearner.
Definition at line 585 of file NNet.cc.
References batch_size, build(), PLearn::endl(), f, invars, PLearn::PP< Function >::isNull(), PLearn::VMat::length(), PLearn::meanOf(), optimizer, output_and_target_to_cost, params, PLERROR, test_costf, PLearn::tostring(), training_cost, and PLearn::ProgressBar::update().

Member Data Documentation

int PLearn::NNet::batch_size

Definition at line 127 of file NNet.h.
Referenced by train().

real PLearn::NNet::bias_decay

Definition at line 97 of file NNet.h.
Referenced by build_().

real PLearn::NNet::classification_regularizer

Definition at line 105 of file NNet.h.
Referenced by build_().

Array<string> PLearn::NNet::cost_funcs

a list of cost functions to use in the form "[ cf1; cf2; cf3; ... ]"

Definition at line 122 of file NNet.h.
Referenced by build_(), getTestCostNames(), and getTrainCostNames().

VarArray PLearn::NNet::costs [protected]

Definition at line 70 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

bool PLearn::NNet::direct_in_to_out

Definition at line 114 of file NNet.h.
Referenced by build_(), and initializeParams().

real PLearn::NNet::direct_in_to_out_weight_decay

Definition at line 104 of file NNet.h.
Referenced by build_().

Func PLearn::NNet::f [mutable]

Definition at line 80 of file NNet.h.
Referenced by build_(), computeOutput(), makeDeepCopyFromShallowCopy(), and train().

bool PLearn::NNet::first_class_is_junk

Definition at line 110 of file NNet.h.
Referenced by build_().

bool PLearn::NNet::fixed_output_weights

Definition at line 107 of file NNet.h.
Referenced by build_(), and initializeParams().

string PLearn::NNet::hidden_transfer_func

Definition at line 116 of file NNet.h.
Referenced by build_().

string PLearn::NNet::initialization_method

Definition at line 130 of file NNet.h.
Referenced by initializeParams().

Var PLearn::NNet::input [protected]

Definition at line 56 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

real PLearn::NNet::input_reconstruction_penalty

Definition at line 113 of file NNet.h.
Referenced by build_().

real PLearn::NNet::interval_maxval

Definition at line 117 of file NNet.h.
Referenced by build_().

real PLearn::NNet::interval_minval

Definition at line 117 of file NNet.h.
Referenced by build_().

VarArray PLearn::NNet::invars [protected]

Definition at line 74 of file NNet.h.
Referenced by build_(), makeDeepCopyFromShallowCopy(), and train().

Var PLearn::NNet::junk_prob [protected]

Definition at line 67 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

bool PLearn::NNet::L1_penalty

Definition at line 112 of file NNet.h.
Referenced by build_().

real PLearn::NNet::layer1_bias_decay

Definition at line 99 of file NNet.h.
Referenced by build_().

real PLearn::NNet::layer1_weight_decay

Definition at line 98 of file NNet.h.
Referenced by build_().

real PLearn::NNet::layer2_bias_decay

Definition at line 101 of file NNet.h.
Referenced by build_().

real PLearn::NNet::layer2_weight_decay

Definition at line 100 of file NNet.h.
Referenced by build_().

real PLearn::NNet::margin

Definition at line 106 of file NNet.h.
Referenced by build_().

int PLearn::NNet::nhidden

Definition at line 92 of file NNet.h.
Referenced by build_(), and initializeParams().

int PLearn::NNet::nhidden2

Definition at line 93 of file NNet.h.
Referenced by build_(), and initializeParams().

int PLearn::NNet::noutputs

Definition at line 94 of file NNet.h.
Referenced by outputsize().

PP<Optimizer> PLearn::NNet::optimizer

Definition at line 125 of file NNet.h.
Referenced by initializeParams(), makeDeepCopyFromShallowCopy(), and train().

Var PLearn::NNet::outbias [protected]

Definition at line 62 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::output [protected]

Definition at line 68 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Func PLearn::NNet::output_and_target_to_cost [mutable]

Definition at line 82 of file NNet.h.
Referenced by build_(), computeCostsFromOutputs(), makeDeepCopyFromShallowCopy(), and train().

real PLearn::NNet::output_layer_bias_decay

Definition at line 103 of file NNet.h.
Referenced by build_().

real PLearn::NNet::output_layer_weight_decay

Definition at line 102 of file NNet.h.
Referenced by build_().

string PLearn::NNet::output_transfer_func

Definition at line 115 of file NNet.h.
Referenced by build_().

VarArray PLearn::NNet::params [protected]

Definition at line 75 of file NNet.h.
Referenced by build_(), makeDeepCopyFromShallowCopy(), and train().

Vec PLearn::NNet::paramsvalues [protected]

Definition at line 77 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

VarArray PLearn::NNet::penalties [protected]

Definition at line 71 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::predicted_input [protected]

Definition at line 69 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::rbf_centers [protected]

Definition at line 65 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

int PLearn::NNet::rbf_layer_size

Definition at line 109 of file NNet.h.
Referenced by build_().

Var PLearn::NNet::rbf_sigmas [protected]

Definition at line 66 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::sampleweight [protected]

Definition at line 58 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::target [protected]

Definition at line 57 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Func PLearn::NNet::test_costf [mutable]

Definition at line 81 of file NNet.h.
Referenced by build_(), computeOutputAndCosts(), makeDeepCopyFromShallowCopy(), and train().

Var PLearn::NNet::test_costs [protected]

Definition at line 73 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::training_cost [protected]

Definition at line 72 of file NNet.h.
Referenced by build_(), makeDeepCopyFromShallowCopy(), and train().

Var PLearn::NNet::w1 [protected]

Definition at line 59 of file NNet.h.
Referenced by build_(), initializeParams(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::w2 [protected]

Definition at line 60 of file NNet.h.
Referenced by build_(), initializeParams(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::wdirect [protected]

Definition at line 63 of file NNet.h.
Referenced by build_(), initializeParams(), and makeDeepCopyFromShallowCopy().

real PLearn::NNet::weight_decay

Definition at line 96 of file NNet.h.
Referenced by build_().

Var PLearn::NNet::wout [protected]

Definition at line 61 of file NNet.h.
Referenced by build_(), initializeParams(), and makeDeepCopyFromShallowCopy().

Var PLearn::NNet::wrec [protected]

Definition at line 64 of file NNet.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().

The documentation for this class was generated from the following files:

Generated on Tue Aug 17 16:27:29 2004 for PLearn by

1.3.7

PLearn::NNet Class Reference

Public Types

Public Member Functions

Public Attributes

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Private Member Functions

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation