PLearn::DiskVMatrix Class Reference

A VMatrix whose (compressed) data resides in a directory and can span several files. More...

#include <DiskVMatrix.h>

Inheritance diagram for PLearn::DiskVMatrix:

[legend]Collaboration diagram for PLearn::DiskVMatrix:

[legend]List of all members.

Public Member Functions
	DiskVMatrix ()
	DiskVMatrix *.
	DiskVMatrix (const string &the_dirname, bool readwrite=false)
	DiskVMatrix (const string &the_dirname, int the_width, bool write_double_as_float=false)
virtual void	putRow (int i, Vec v)
virtual void	appendRow (Vec v)
	This method must be implemented for matrices that are allowed to grow.
virtual void	flush ()
	For matrices stored on disk, this should flush all pending buffered write operations.
virtual void	build ()
	Should call simply inherited::build(), then this class's build_().
	PLEARN_DECLARE_OBJECT (DiskVMatrix)
virtual	~DiskVMatrix ()
Public Attributes
string	dirname
double	tolerance
Protected Member Functions
virtual void	getNewRow (int i, const Vec &v) const
	This is the only method requiring implementation in subclasses.
Static Protected Member Functions
void	declareOptions (OptionList &ol)
	redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down)
Protected Attributes
FILE *	indexf
TVec< FILE * >	dataf
bool	freshnewfile
bool	old_format
bool	swap_endians
Private Types
typedef RowBufferedVMatrix	inherited
Private Member Functions
void	build_ ()
	This does the actual building.

---

Detailed Description

A VMatrix whose (compressed) data resides in a directory and can span several files.

Each row is compressed/decompressed through the methods of VecCompressor

Definition at line 56 of file DiskVMatrix.h.

---

Member Typedef Documentation

typedef RowBufferedVMatrix PLearn::DiskVMatrix::inherited [private]

Reimplemented from PLearn::RowBufferedVMatrix. Definition at line 58 of file DiskVMatrix.h.

---

Constructor & Destructor Documentation

PLearn::DiskVMatrix::DiskVMatrix (   )

DiskVMatrix *. Definition at line 87 of file DiskVMatrix.cc.

PLearn::DiskVMatrix::DiskVMatrix (  const string &  the_dirname, bool  readwrite = false )

Opens an existing one. If directory does not exist or has missing files, an error is issued. If readwrite is true, then the files are opened in read/write mode and appendRow can be called. If readwrite is false (the default), then the files are opened in read only mode, and calling appendRow will issue an error. Definition at line 95 of file DiskVMatrix.cc. References build_(), and PLearn::remove_trailing_slash().

PLearn::DiskVMatrix::DiskVMatrix (  const string &  the_dirname, int  the_width, bool  write_double_as_float = false )

Create a new one. If directory already exist an error is issued (you may consider calling force_rmdir prior to this.) Howver if it is a file then the file is erased and replaced by a new directory (this was to allow TmpFilenames to be used with this class). Files are opened in read/write mode so appendRow can be called. Definition at line 105 of file DiskVMatrix.cc. References build_(), and PLearn::remove_trailing_slash().

PLearn::DiskVMatrix::~DiskVMatrix (   )  [virtual]

Definition at line 307 of file DiskVMatrix.cc. References dataf, indexf, PLearn::VMatrix::saveFieldInfos(), and PLearn::TVec< FILE * >::size().

---

Member Function Documentation

void PLearn::DiskVMatrix::appendRow (  Vec  v  )  [virtual]

This method must be implemented for matrices that are allowed to grow. Reimplemented from PLearn::VMatrix. Definition at line 263 of file DiskVMatrix.cc. References PLearn::binwrite_compressed(), PLearn::TVec< T >::data(), dataf, dirname, PLearn::endianswap(), indexf, PLearn::TVec< T >::length(), PLearn::new_write_compressed(), old_format, PLERROR, PLearn::TVec< FILE * >::size(), slash, swap_endians, tolerance, PLearn::tostring(), and PLearn::VMatrix::width(). Referenced by PLearn::VMatrix::saveDMAT().

void PLearn::DiskVMatrix::build (   )  [virtual]

Should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object. Reimplemented from PLearn::VMatrix. Definition at line 117 of file DiskVMatrix.cc. References build_().

void PLearn::DiskVMatrix::build_ (   )  [private]

This does the actual building. Reimplemented from PLearn::VMatrix. Definition at line 123 of file DiskVMatrix.cc. References PLearn::TVec< FILE * >::append(), PLearn::append_slash(), PLearn::byte_order(), dataf, dirname, PLearn::endianswap(), fname, PLearn::force_mkdir(), freshnewfile, PLearn::VMatrix::getFieldInfos(), header, indexf, PLearn::isdir(), PLearn::isfile(), k, PLearn::mtime(), old_format, PLERROR, PLearn::TVec< T >::resize(), slash, swap_endians, and PLearn::tostring(). Referenced by build(), and DiskVMatrix().

void PLearn::DiskVMatrix::declareOptions (  OptionList &  ol  )  [static, protected]

redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options) ( see the declareOption function further down) ex: static void declareOptions(OptionList& ol) { declareOption(ol, "inputsize", &MyObject::inputsize_, OptionBase::buildoption, "the size of the input\n it must be provided"); declareOption(ol, "weights", &MyObject::weights, OptionBase::learntoption, "the learnt model weights"); inherited::declareOptions(ol); } Reimplemented from PLearn::VMatrix. Definition at line 226 of file DiskVMatrix.cc. References PLearn::declareOption(), and PLearn::OptionList.

void PLearn::DiskVMatrix::flush (   )  [virtual]

For matrices stored on disk, this should flush all pending buffered write operations. Reimplemented from PLearn::VMatrix. Definition at line 299 of file DiskVMatrix.cc. References dataf, indexf, and PLearn::TVec< FILE * >::size().

void PLearn::DiskVMatrix::getNewRow (  int  i, const Vec &  v )  const [protected, virtual]

This is the only method requiring implementation in subclasses. Implements PLearn::RowBufferedVMatrix. Definition at line 233 of file DiskVMatrix.cc. References PLearn::binread_compressed(), PLearn::TVec< T >::data(), dataf, PLearn::endianswap(), indexf, PLearn::TVec< T >::length(), PLearn::VMatrix::length(), PLearn::new_read_compressed(), old_format, PLERROR, swap_endians, and PLearn::VMatrix::width().

PLearn::DiskVMatrix::PLEARN_DECLARE_OBJECT (  DiskVMatrix   )

void PLearn::DiskVMatrix::putRow (  int  i, Vec  v )  [virtual]

Reimplemented from PLearn::VMatrix. Definition at line 257 of file DiskVMatrix.cc. References PLERROR.

---

Member Data Documentation

TVec<FILE*> PLearn::DiskVMatrix::dataf [mutable, protected]

Definition at line 62 of file DiskVMatrix.h. Referenced by appendRow(), build_(), flush(), getNewRow(), and ~DiskVMatrix().

string PLearn::DiskVMatrix::dirname

Definition at line 69 of file DiskVMatrix.h. Referenced by appendRow(), and build_().

bool PLearn::DiskVMatrix::freshnewfile [protected]

Definition at line 63 of file DiskVMatrix.h. Referenced by build_().

FILE* PLearn::DiskVMatrix::indexf [mutable, protected]

Definition at line 61 of file DiskVMatrix.h. Referenced by appendRow(), build_(), flush(), getNewRow(), and ~DiskVMatrix().

bool PLearn::DiskVMatrix::old_format [protected]

Definition at line 64 of file DiskVMatrix.h. Referenced by appendRow(), build_(), and getNewRow().

bool PLearn::DiskVMatrix::swap_endians [protected]

Definition at line 65 of file DiskVMatrix.h. Referenced by appendRow(), build_(), and getNewRow().

double PLearn::DiskVMatrix::tolerance

Definition at line 71 of file DiskVMatrix.h. Referenced by appendRow().

---

The documentation for this class was generated from the following files:

• DiskVMatrix.h
• DiskVMatrix.cc

---