Main Page | Namespace List | Class Hierarchy | Alphabetical List | Class List | File List | Namespace Members | Class Members | File Members

PLearn::SequentialModelSelector Class Reference

#include <SequentialModelSelector.h>

Inheritance diagram for PLearn::SequentialModelSelector:

Inheritance graph
[legend]
Collaboration diagram for PLearn::SequentialModelSelector:

Collaboration graph
[legend]
List of all members.

Public Member Functions

 SequentialModelSelector ()
 Constructor.

real sequenceCost (const Vec &sequence_errors)
 compute the cost of the given sequence of errors (based on the cost_type)

real paired_t_test (const int &m1, const int &m2, int cc=0) const
 Computes a paired t-test between common_cost[cc] series of models m1 and m2.

virtual void build ()
 simply calls inherited::build() then build_()

virtual void setExperimentDirectory (const string &_expdir)
 Redefines so that it ALSO calls the method on all the learners in the TVec models.

virtual void train ()
virtual void test (VMat testset, PP< VecStatsCollector > test_stats, VMat testoutputs=0, VMat testcosts=0) const
virtual void computeOutput (const Vec &input, Vec &output) const
 *** SUBCLASS WRITING: *** This should be defined in subclasses to compute the output from the input

virtual void computeCostsFromOutputs (const Vec &input, const Vec &output, const Vec &target, Vec &costs) const
 *** SUBCLASS WRITING: *** This should be defined in subclasses to compute the weighted costs from already computed output.

virtual void computeOutputAndCosts (const Vec &input, const Vec &target, Vec &output, Vec &costs) const
 Default calls computeOutput and computeCostsFromOutputs You may overload this if you have a more efficient way to compute both output and weighted costs at the same time.

virtual void computeCostsOnly (const Vec &input, const Vec &target, Vec &costs) const
 Default calls computeOutputAndCosts This may be overloaded if there is a more efficient way to compute the costs directly, without computing the whole output vector.

virtual void matlabSave (const string &matlab_subdir)
virtual TVec< string > getTestCostNames () const
 This should return the names of the costs computed by computeCostsFromOutputs.

virtual TVec< string > getTrainCostNames () const
 This should return the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

virtual void forget ()
 (Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!)

 PLEARN_DECLARE_OBJECT (SequentialModelSelector)
 Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be.

virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)

Public Attributes

bool stepwise_save
TVec< PP< SequentialLearner > > models
 List of all the models.

TVec< string > model_names
 If the user desires to provide a name for each model instead of model_i.

TVec< string > common_costs
int comparison_type
int comparison_window

Static Protected Member Functions

void declareOptions (OptionList &ol)
 Declare this class' options.


Protected Attributes

TMat< int > common_cost_indices
 See common_costs options for details. Row index: model. Column index: common_cost.

TVec< int > best_model
Vec sequence_costs
bool report_paired_T_tests

Private Types

typedef SequentialLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.

void checkModelNames () const

Member Typedef Documentation

typedef SequentialLearner PLearn::SequentialModelSelector::inherited [private]
 

Reimplemented from PLearn::SequentialLearner.

Definition at line 55 of file SequentialModelSelector.h.

Constructor & Destructor Documentation

PLearn::SequentialModelSelector::SequentialModelSelector (   ) 
 

Constructor.

Definition at line 48 of file SequentialModelSelector.cc.

Member Function Documentation

void PLearn::SequentialModelSelector::build (   )  [virtual]
 

simply calls inherited::build() then build_()

Reimplemented from PLearn::SequentialLearner.

Definition at line 106 of file SequentialModelSelector.cc.

References build_().

void PLearn::SequentialModelSelector::build_ (   )  [private]
 

This does the actual building.

Reimplemented from PLearn::SequentialLearner.

Definition at line 71 of file SequentialModelSelector.cc.

References best_model, common_cost_indices, common_costs, forget(), PLearn::TVec< string >::length(), PLearn::TVec< PP< SequentialLearner > >::length(), models, PLERROR, PLWARNING, PLearn::TVec< T >::resize(), PLearn::TVec< int >::resize(), PLearn::TMat< int >::resize(), and sequence_costs.

Referenced by build().

void PLearn::SequentialModelSelector::checkModelNames (   )  const [private]
 

Definition at line 460 of file SequentialModelSelector.cc.

References PLearn::TVec< string >::length(), PLearn::TVec< PP< SequentialLearner > >::length(), model_names, models, PLERROR, PLearn::TVec< string >::resize(), and PLearn::tostring().

Referenced by getTestCostNames(), and setExperimentDirectory().

void PLearn::SequentialModelSelector::computeCostsFromOutputs (  const Vec &  input,
const Vec &  output,
const Vec &  target,
Vec &  costs
)  const [virtual]
 

*** SUBCLASS WRITING: *** This should be defined in subclasses to compute the weighted costs from already computed output.

The costs should correspond to the cost names returned by getTestCostNames() NOTE: In exotic cases, the cost may also depend on some info in the input, that's why the method also gets so see it.

Reimplemented from PLearn::SequentialLearner.

Definition at line 442 of file SequentialModelSelector.cc.

References best_model, and models.

void PLearn::SequentialModelSelector::computeCostsOnly (  const Vec &  input,
const Vec &  target,
Vec &  costs
)  const [virtual]
 

Default calls computeOutputAndCosts This may be overloaded if there is a more efficient way to compute the costs directly, without computing the whole output vector.

Reimplemented from PLearn::SequentialLearner.

Definition at line 454 of file SequentialModelSelector.cc.

References best_model, and models.

void PLearn::SequentialModelSelector::computeOutput (  const Vec &  input,
Vec &  output
)  const [virtual]
 

*** SUBCLASS WRITING: *** This should be defined in subclasses to compute the output from the input

Reimplemented from PLearn::SequentialLearner.

Definition at line 437 of file SequentialModelSelector.cc.

References best_model, and models.

void PLearn::SequentialModelSelector::computeOutputAndCosts (  const Vec &  input,
const Vec &  target,
Vec &  output,
Vec &  costs
)  const [virtual]
 

Default calls computeOutput and computeCostsFromOutputs You may overload this if you have a more efficient way to compute both output and weighted costs at the same time.

Reimplemented from PLearn::SequentialLearner.

Definition at line 448 of file SequentialModelSelector.cc.

References best_model, and models.

void PLearn::SequentialModelSelector::declareOptions (  OptionList &  ol  )  [static, protected]
 

Declare this class' options.

Reimplemented from PLearn::SequentialLearner.

Definition at line 112 of file SequentialModelSelector.cc.

References PLearn::declareOption(), and PLearn::OptionList.

void PLearn::SequentialModelSelector::forget (   )  [virtual]
 

(Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!)

*** SUBCLASS WRITING: *** A typical forget() method should do the following:

  • initialize a random number generator with the seed option
  • initialize the learner's parameters, using this random generator
  • stage = 0; This method is typically called by the build_() method, after it has finished setting up the parameters, and if it deemed useful to set or reset the learner in its fresh state. (remember build may be called after modifying options that do not necessarily require the learner to restart from a fresh state...) forget is also called by the setTrainingSet method, after calling build(), so it will generally be called TWICE during setTrainingSet!

Reimplemented from PLearn::SequentialLearner.

Definition at line 200 of file SequentialModelSelector.cc.

References best_model, PLearn::TVec< int >::fill(), models, PLearn::TVec< int >::resize(), and PLearn::TVec< PP< SequentialLearner > >::size().

Referenced by build_().

TVec< string > PLearn::SequentialModelSelector::getTestCostNames (   )  const [virtual]
 

This should return the names of the costs computed by computeCostsFromOutputs.

Implements PLearn::PLearner.

Definition at line 474 of file SequentialModelSelector.cc.

References PLearn::TVec< T >::append(), checkModelNames(), common_costs, PLearn::TVec< T >::length(), PLearn::TVec< PP< SequentialLearner > >::length(), model_names, models, and report_paired_T_tests.

TVec< string > PLearn::SequentialModelSelector::getTrainCostNames (   )  const [virtual]
 

This should return the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Implements PLearn::PLearner.

Definition at line 495 of file SequentialModelSelector.cc.

References common_costs.

void PLearn::SequentialModelSelector::makeDeepCopyFromShallowCopy (  CopiesMap &  copies  )  [virtual]
 

Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be. Typical implementation:

void CLASS_OF_THIS::makeDeepCopyFromShallowCopy(CopiesMap& copies) { SUPERCLASS_OF_THIS::makeDeepCopyFromShallowCopy(copies); member_ptr = member_ptr->deepCopy(copies); member_smartptr = member_smartptr->deepCopy(copies); member_mat.makeDeepCopyFromShallowCopy(copies); member_vec.makeDeepCopyFromShallowCopy(copies); ... }

Reimplemented from PLearn::SequentialLearner.

Definition at line 498 of file SequentialModelSelector.cc.

References PLearn::CopiesMap, PLearn::deepCopyField(), and models.

void PLearn::SequentialModelSelector::matlabSave (  const string &  matlab_subdir  )  [virtual]
 

Any SequentialLearner will have the possiblity to save some data in a matlab 'readable' format. The data will be saved in expdir/matlab_subdir through the global matlabSave function (MatIO.h).

Reimplemented from PLearn::SequentialLearner.

Definition at line 505 of file SequentialModelSelector.cc.

References PLearn::add(), PLearn::TVec< T >::append(), PLearn::append_slash(), common_cost_indices, common_costs, PLearn::concat(), PLearn::PLearner::getExperimentDirectory(), PLearn::hconcat(), PLearn::TVec< T >::length(), PLearn::TVec< string >::length(), PLearn::TVec< PP< SequentialLearner > >::length(), PLearn::Mat, PLearn::matlabSave(), model_names, models, and PLearn::TVec< T >::subVec().

real PLearn::SequentialModelSelector::paired_t_test (  const int &  m1,
const int &  m2,
int  cc = 0
)  const
 

Computes a paired t-test between common_cost[cc] series of models m1 and m2.

Definition at line 182 of file SequentialModelSelector.cc.

References PLearn::TMat< T >::column(), common_cost_indices, PLearn::TVec< T >::length(), PLearn::mean(), models, PLERROR, PLearn::remove_missing(), PLearn::sqrt(), and PLearn::sumsquare().

PLearn::SequentialModelSelector::PLEARN_DECLARE_OBJECT (  SequentialModelSelector   ) 
 

Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be.

real PLearn::SequentialModelSelector::sequenceCost (  const Vec &  sequence_errors  ) 
 

compute the cost of the given sequence of errors (based on the cost_type)

Definition at line 157 of file SequentialModelSelector.cc.

References PLearn::abs(), comparison_type, comparison_window, PLearn::TVec< T >::length(), PLearn::mean(), MISSING_VALUE, PLERROR, PLearn::TVec< T >::subVec(), and PLearn::variance().

Referenced by train().

void PLearn::SequentialModelSelector::setExperimentDirectory (  const string &  _expdir  )  [virtual]
 

Redefines so that it ALSO calls the method on all the learners in the TVec models.

Reimplemented from PLearn::PLearner.

Definition at line 55 of file SequentialModelSelector.cc.

References PLearn::append_slash(), checkModelNames(), PLearn::TVec< PP< SequentialLearner > >::length(), model_names, and models.

void PLearn::SequentialModelSelector::test (  VMat  testset,
PP< VecStatsCollector >  test_stats,
VMat  testoutputs = 0,
VMat  testcosts = 0
)  const [virtual]
 

*** SUBCLASS WRITING: *** The method should:

  • call computeOutputAndCosts on the test set
  • save the outputs and the costs in the predictions & errors matrices, beginning at position last_call_train_t

Implements PLearn::SequentialLearner.

Definition at line 346 of file SequentialModelSelector.cc.

References PLearn::TVec< T >::append(), PLearn::append_slash(), best_model, PLearn::ProgressBar::close(), common_cost_indices, PLearn::endl(), PLearn::TVec< PP< SequentialLearner > >::length(), PLearn::VMat::length(), MAX, MISSING_VALUE, models, PLearn::paired_t_test(), PLWARNING, report_paired_T_tests, PLearn::saveAsciiWithoutSize(), PLearn::TVec< PP< SequentialLearner > >::size(), stepwise_save, PLearn::tostring(), PLearn::ProgressBar::update(), and PLearn::update().

void PLearn::SequentialModelSelector::train (   )  [virtual]
 

*** SUBCLASS WRITING: *** Does the actual training. Subclasses must implement this method. The method should upon entry, call setTrainingSet(training_set); The method should:

  • do nothing if we already called it with this value of train.length or a value between [last_train_t-train_step+1,last_train_t]
  • if not, train and update the value of last_train_t
  • in either case, update the value of last_call_train_t

Implements PLearn::SequentialLearner.

Definition at line 210 of file SequentialModelSelector.cc.

References PLearn::append_slash(), PLearn::argmax(), PLearn::argmin(), best_model, PLearn::ProgressBar::close(), PLearn::TMat< T >::column(), common_cost_indices, comparison_type, PLearn::endl(), PLearn::TMat< T >::hasMissing(), PLearn::TVec< PP< SequentialLearner > >::length(), PLearn::VMat::length(), MAX, models, PLERROR, PLWARNING, PLearn::remove_missing(), PLearn::saveAsciiWithoutSize(), sequence_costs, sequenceCost(), PLearn::TVec< PP< SequentialLearner > >::size(), stepwise_save, PLearn::TMat< T >::subMat(), PLearn::VMat::subMatRows(), PLearn::tostring(), PLearn::update(), and PLearn::ProgressBar::update().

Member Data Documentation

TVec<int> PLearn::SequentialModelSelector::best_model [protected]
 

Definition at line 70 of file SequentialModelSelector.h.

Referenced by build_(), computeCostsFromOutputs(), computeCostsOnly(), computeOutput(), computeOutputAndCosts(), forget(), test(), and train().

TMat<int> PLearn::SequentialModelSelector::common_cost_indices [protected]
 

See common_costs options for details. Row index: model. Column index: common_cost.

Definition at line 69 of file SequentialModelSelector.h.

Referenced by build_(), matlabSave(), paired_t_test(), test(), and train().

TVec<string> PLearn::SequentialModelSelector::common_costs
 

The names of costs that are common to all models and that the user wishes the model selector to keep track of. The first one is considered to be the main cost, the one from which models will be compared to choose the best model.

Definition at line 112 of file SequentialModelSelector.h.

Referenced by build_(), getTestCostNames(), getTrainCostNames(), and matlabSave().

int PLearn::SequentialModelSelector::comparison_type
 

From the common_costs list, the first cost is the one from which models will be compared to choose the best model. But should the best model be chosen according to the

max/min +/- 1: Mean +/- 2: Mean / Variance +/- 3: more to come.

of the cost realizations. Default: 1.

Definition at line 126 of file SequentialModelSelector.h.

Referenced by sequenceCost(), and train().

int PLearn::SequentialModelSelector::comparison_window
 

If positive, the comparison performed on the basis of common_cost[0] will be applyed only the comparison_window last elements of the cost sequence. Default: -1. (No window)

Definition at line 133 of file SequentialModelSelector.h.

Referenced by sequenceCost().

TVec< string > PLearn::SequentialModelSelector::model_names [mutable]
 

If the user desires to provide a name for each model instead of model_i.

Definition at line 105 of file SequentialModelSelector.h.

Referenced by checkModelNames(), getTestCostNames(), matlabSave(), and setExperimentDirectory().

TVec< PP<SequentialLearner> > PLearn::SequentialModelSelector::models
 

List of all the models.

Definition at line 102 of file SequentialModelSelector.h.

Referenced by build_(), checkModelNames(), computeCostsFromOutputs(), computeCostsOnly(), computeOutput(), computeOutputAndCosts(), forget(), getTestCostNames(), makeDeepCopyFromShallowCopy(), matlabSave(), paired_t_test(), setExperimentDirectory(), test(), and train().

bool PLearn::SequentialModelSelector::report_paired_T_tests [protected]
 

If true, the model selector will report as costs the paired T tests on common_cost_indices[0] for models[0] against each other model. The model selector will only report T tests once, at t=(max_seq_len-1)

Default: false.

Definition at line 80 of file SequentialModelSelector.h.

Referenced by getTestCostNames(), and test().

Vec PLearn::SequentialModelSelector::sequence_costs [protected]
 

Definition at line 71 of file SequentialModelSelector.h.

Referenced by build_(), and train().

bool PLearn::SequentialModelSelector::stepwise_save
 

Does the model selector hass to save errors at each step. Default: true.

Definition at line 99 of file SequentialModelSelector.h.

Referenced by test(), and train().

The documentation for this class was generated from the following files:
Generated on Tue Aug 17 16:27:49 2004 for PLearn by doxygen 1.3.7