PLearn::SemiSupervisedProbClassCostVariable Class Reference

#include <SemiSupervisedProbClassCostVariable.h>

Inheritance diagram for PLearn::SemiSupervisedProbClassCostVariable:

[legend]Collaboration diagram for PLearn::SemiSupervisedProbClassCostVariable:

[legend]List of all members.

Public Member Functions
	SemiSupervisedProbClassCostVariable ()
	Default constructor for persistence.
	SemiSupervisedProbClassCostVariable (Var prob_, Var target_, Var prior_, real flattenfactor=1.0)
	PLEARN_DECLARE_OBJECT (SemiSupervisedProbClassCostVariable)
virtual void	build ()
	Should call simply inherited::build(), then this class's build_().
virtual void	recomputeSize (int &l, int &w) const
	Recomputes the length l and width w that this variable should have, according to its parent variables.
virtual void	fprop ()
	compute output given input
virtual void	bprop ()
virtual void	symbolicBprop ()
	compute a piece of new Var graph that represents the symbolic derivative of this Var
virtual void	rfprop ()
Var &	prob ()
Var &	target ()
Var &	prior ()
Static Public Member Functions
void	declareOptions (OptionList &ol)
	redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down)
Protected Member Functions
void	build_ ()
Protected Attributes
real	flatten_factor
	the option
Vec	raised_prob
	temporaries for avoiding re-computations in the bprop
real	sum_raised_prob
Private Types
typedef NaryVariable	inherited

---

Detailed Description

! This is a cost function for semi-supervised training of a probabilistic classifier. ! It takes three arguments: (prob,target,prior). ! The prob should be positive and summing to 1 (e.g. the output of a softmax). ! The target is either an integer (which indexes the output vector) or a missing value. ! The prior is a vector of probabilities (they don't have to be normalized) which ! act as a prior when the target is not provided. ! There are two types of cases: when the target is given (supervised case) ! and when the target is missing (unsupervised case). ! The cost value is computed as follows: ! ! If target is not missing: ! cost = - log(prob[target]) ! Else ! cost = - (1/flatten_factor) log sum_i (prior[i] * prob[i])^flatten_factor ! ! The only option of this class is the hyper-parameter "flatten_factor" ! If prior is an empty Var, then prior[i]=1 is assumed. ! ! N.B. ! - prior can be understood as the supervised target for supervised examples ! (then the above cost becomes the classical max. likelihood, = log prob_{target}) ! - for unsupervised examples prior embodies the information available on the target; ! reasonable choices are: ! - all 1's (recommanded) ! - prior relative frequencies (conditional, if some other information can be used, heuristic) ! - output of the model trained on supervised examples only (?? not clear from theory) ! In the latter two cases it might be desirable to modulate the "strength" of the ! prior probabilities by raising them to some power. Note that prior[i] do not ! have to be normalized but they should be non-negative. ! - when flatten_factor=1 the gradient on unsupervised examples (prior=const) is 0 ! - increasing "flatten_factor" increases the confidence given to the model current prediction on unsupervised examples ! - this cost may be very sensitive to INITIALIZATION: ! - it would be wise to start from purely supervised training (i.e. a=1, or skip the unsupervised examples) ! - and gradually increase it (<=5, say) e.g. a=1, then a=1.1, then a=1.5, then a=2. ! ! N.B2: THE IMPLEMENTATION OF THIS CLASS CHEATS AND DOES NOT COMPUTE THE GRADIENTS ! WITH RESPECT TO THE target AND prior INPUTS, AS SUCH GRADIENTS SHOULD NOT BE USED. !

Definition at line 92 of file SemiSupervisedProbClassCostVariable.h.

---

Member Typedef Documentation

typedef NaryVariable PLearn::SemiSupervisedProbClassCostVariable::inherited [private]

Reimplemented from PLearn::NaryVariable. Definition at line 94 of file SemiSupervisedProbClassCostVariable.h. Referenced by SemiSupervisedProbClassCostVariable().

---

Constructor & Destructor Documentation

PLearn::SemiSupervisedProbClassCostVariable::SemiSupervisedProbClassCostVariable (   )  [inline]

Default constructor for persistence. Definition at line 106 of file SemiSupervisedProbClassCostVariable.h. References flatten_factor.

PLearn::SemiSupervisedProbClassCostVariable::SemiSupervisedProbClassCostVariable (  Var  prob_, Var  target_, Var  prior_, real  flattenfactor = 1.0 )

Definition at line 56 of file SemiSupervisedProbClassCostVariable.cc. References build_(), and inherited.

---

Member Function Documentation

void PLearn::SemiSupervisedProbClassCostVariable::bprop (   )  [virtual]

Implements PLearn::Variable. Definition at line 124 of file SemiSupervisedProbClassCostVariable.cc. References prob(), raised_prob, sum_raised_prob, and target().

void PLearn::SemiSupervisedProbClassCostVariable::build (   )  [virtual]

Should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object. Reimplemented from PLearn::Variable. Definition at line 63 of file SemiSupervisedProbClassCostVariable.cc. References build_().

void PLearn::SemiSupervisedProbClassCostVariable::build_ (   )  [protected]

This method should be redefined in subclasses and do the actual building of the object according to previously set option fields. Constructors can just set option fields, and then call build_. This method is NOT virtual, and will typically be called only from three places: a constructor, the public virtual build() method, and possibly the public virtual read method (which calls its parent's read). build_() can assume that it's parent's build_ has already been called. Reimplemented from PLearn::Variable. Definition at line 70 of file SemiSupervisedProbClassCostVariable.cc. References flatten_factor, PLearn::Variable::length(), PLERROR, raised_prob, PLearn::TVec< T >::resize(), and PLearn::TVec< Var >::size(). Referenced by build(), and SemiSupervisedProbClassCostVariable().

void PLearn::SemiSupervisedProbClassCostVariable::declareOptions (  OptionList &  ol  )  [static]

redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down) ex: static void declareOptions(OptionList& ol) { declareOption(ol, "inputsize", &MyObject::inputsize_, OptionBase::buildoption, "the size of the input\n it must be provided"); declareOption(ol, "weights", &MyObject::weights, OptionBase::learntoption, "the learnt model weights"); inherited::declareOptions(ol); } Reimplemented from PLearn::NaryVariable. Definition at line 85 of file SemiSupervisedProbClassCostVariable.cc. References PLearn::declareOption(), and PLearn::OptionList.

void PLearn::SemiSupervisedProbClassCostVariable::fprop (   )  [virtual]

compute output given input Implements PLearn::Variable. Definition at line 95 of file SemiSupervisedProbClassCostVariable.cc. References flatten_factor, PLERROR, PLearn::pow(), prior(), prob(), raised_prob, PLearn::safeflog(), PLearn::Variable::size(), sum_raised_prob, and target().

PLearn::SemiSupervisedProbClassCostVariable::PLEARN_DECLARE_OBJECT (  SemiSupervisedProbClassCostVariable   )

Var& PLearn::SemiSupervisedProbClassCostVariable::prior (   )  [inline]

Definition at line 121 of file SemiSupervisedProbClassCostVariable.h. Referenced by fprop().

Var& PLearn::SemiSupervisedProbClassCostVariable::prob (   )  [inline]

Definition at line 119 of file SemiSupervisedProbClassCostVariable.h. Referenced by bprop(), and fprop().

void PLearn::SemiSupervisedProbClassCostVariable::recomputeSize (  int &  l, int &  w )  const [virtual]

Recomputes the length l and width w that this variable should have, according to its parent variables. This is used for ex. by sizeprop() The default version stupidly returns the current dimensions, so make sure to overload it in subclasses if this is not appropriate. Reimplemented from PLearn::Variable. Definition at line 91 of file SemiSupervisedProbClassCostVariable.cc.

void PLearn::SemiSupervisedProbClassCostVariable::rfprop (   )  [virtual]

Reimplemented from PLearn::Variable. Definition at line 156 of file SemiSupervisedProbClassCostVariable.cc. References PLERROR.

void PLearn::SemiSupervisedProbClassCostVariable::symbolicBprop (   )  [virtual]

compute a piece of new Var graph that represents the symbolic derivative of this Var Reimplemented from PLearn::Variable. Definition at line 150 of file SemiSupervisedProbClassCostVariable.cc. References PLERROR.

Var& PLearn::SemiSupervisedProbClassCostVariable::target (   )  [inline]

Definition at line 120 of file SemiSupervisedProbClassCostVariable.h. Referenced by bprop(), and fprop().

---

Member Data Documentation

real PLearn::SemiSupervisedProbClassCostVariable::flatten_factor [protected]

the option Definition at line 98 of file SemiSupervisedProbClassCostVariable.h. Referenced by build_(), fprop(), and SemiSupervisedProbClassCostVariable().

Vec PLearn::SemiSupervisedProbClassCostVariable::raised_prob [protected]

temporaries for avoiding re-computations in the bprop Definition at line 101 of file SemiSupervisedProbClassCostVariable.h. Referenced by bprop(), build_(), and fprop().

real PLearn::SemiSupervisedProbClassCostVariable::sum_raised_prob [protected]

Definition at line 102 of file SemiSupervisedProbClassCostVariable.h. Referenced by bprop(), and fprop().

---

The documentation for this class was generated from the following files:

• SemiSupervisedProbClassCostVariable.h
• SemiSupervisedProbClassCostVariable.cc

---