All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
FixedAffineComponent Class Reference

FixedAffineComponent is an affine transform that is supplied at network initialization time and is not trainable. More...

#include <nnet-component.h>

Inheritance diagram for FixedAffineComponent:
Collaboration diagram for FixedAffineComponent:

Public Member Functions

 FixedAffineComponent ()
 
virtual std::string Type () const
 
virtual std::string Info () const
 
void Init (const CuMatrixBase< BaseFloat > &matrix)
 matrix should be of size input-dim+1 to output-dim, last col is offset More...
 
virtual void InitFromString (std::string args)
 Initialize, typically from a line of a config file. More...
 
virtual int32 InputDim () const
 Get size of input vectors. More...
 
virtual int32 OutputDim () const
 Get size of output vectors. More...
 
virtual void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Perform forward pass propagation Input->Output. More...
 
virtual void Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
 Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
 
virtual bool BackpropNeedsInput () const
 
virtual bool BackpropNeedsOutput () const
 
virtual ComponentCopy () const
 Copy component (deep copy). More...
 
virtual void Read (std::istream &is, bool binary)
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
const CuMatrix< BaseFloat > & LinearParams () const
 
- Public Member Functions inherited from Component
 Component ()
 
virtual int32 Index () const
 Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
 
virtual void SetIndex (int32 index)
 
virtual std::vector< int32 > Context () const
 Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
 
void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
 A non-virtual propagate function that first resizes output if necessary. More...
 
virtual ~Component ()
 

Protected Member Functions

 KALDI_DISALLOW_COPY_AND_ASSIGN (FixedAffineComponent)
 

Protected Attributes

CuMatrix< BaseFloatlinear_params_
 
CuVector< BaseFloatbias_params_
 

Friends

class AffineComponent
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream. More...
 
static ComponentNewFromString (const std::string &initializer_line)
 Initialize the Component from one line that will contain first the type, e.g. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Return a new Component of the given type e.g. More...
 

Detailed Description

FixedAffineComponent is an affine transform that is supplied at network initialization time and is not trainable.

Definition at line 1454 of file nnet-component.h.

Constructor & Destructor Documentation

Definition at line 1456 of file nnet-component.h.

Referenced by FixedAffineComponent::Copy().

1456 { }

Member Function Documentation

void Backprop ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
Component to_update,
CuMatrix< BaseFloat > *  in_deriv 
) const
virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 3300 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), kaldi::kNoTrans, FixedAffineComponent::linear_params_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and CuMatrix< Real >::Resize().

3306  {
3307  in_deriv->Resize(out_deriv.NumRows(), linear_params_.NumCols());
3308  in_deriv->AddMatMat(1.0, out_deriv, kNoTrans, linear_params_, kNoTrans, 0.0);
3309 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
void AddMatMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType transA, const CuMatrixBase< Real > &B, MatrixTransposeType transB, Real beta)
C = alpha * A(^T)*B(^T) + beta * C.
Definition: cu-matrix.cc:1141
CuMatrix< BaseFloat > linear_params_
virtual bool BackpropNeedsInput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 1481 of file nnet-component.h.

1481 { return false; }
virtual bool BackpropNeedsOutput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 1482 of file nnet-component.h.

1482 { return false; }
Component * Copy ( ) const
virtual

Copy component (deep copy).

Implements Component.

Definition at line 3311 of file nnet-component.cc.

References FixedAffineComponent::bias_params_, FixedAffineComponent::FixedAffineComponent(), and FixedAffineComponent::linear_params_.

Referenced by AffineComponent::CollapseWithNext(), and AffineComponent::CollapseWithPrevious().

3311  {
3313  ans->linear_params_ = linear_params_;
3314  ans->bias_params_ = bias_params_;
3315  return ans;
3316 }
CuMatrix< BaseFloat > linear_params_
CuVector< BaseFloat > bias_params_
std::string Info ( ) const
virtual

Reimplemented from Component.

Definition at line 3272 of file nnet-component.cc.

References FixedAffineComponent::bias_params_, CuVectorBase< Real >::Dim(), Component::Info(), kaldi::kTrans, FixedAffineComponent::linear_params_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), kaldi::TraceMatMat(), and kaldi::VecVec().

3272  {
3273  std::stringstream stream;
3274  BaseFloat linear_params_size = static_cast<BaseFloat>(linear_params_.NumRows())
3275  * static_cast<BaseFloat>(linear_params_.NumCols()),
3276  linear_params_stddev =
3277  std::sqrt(TraceMatMat(linear_params_,
3279  linear_params_size),
3280  bias_params_stddev = std::sqrt(VecVec(bias_params_, bias_params_) /
3281  bias_params_.Dim());
3282 
3283  stream << Component::Info() << ", linear-params-stddev=" << linear_params_stddev
3284  << ", bias-params-stddev=" << bias_params_stddev;
3285  return stream.str();
3286 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
float BaseFloat
Definition: kaldi-types.h:29
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
virtual std::string Info() const
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
We need to declare this here as it will be a friend function.
CuMatrix< BaseFloat > linear_params_
CuVector< BaseFloat > bias_params_
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:36
void Init ( const CuMatrixBase< BaseFloat > &  matrix)

matrix should be of size input-dim+1 to output-dim, last col is offset

Definition at line 3245 of file nnet-component.cc.

References FixedAffineComponent::bias_params_, CuVectorBase< Real >::CopyColFromMat(), KALDI_ASSERT, FixedAffineComponent::linear_params_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrixBase< Real >::Range(), and CuVector< Real >::Resize().

Referenced by FixedAffineComponent::InitFromString(), and kaldi::nnet2::UnitTestFixedAffineComponent().

3245  {
3246  KALDI_ASSERT(mat.NumCols() > 1);
3247  linear_params_ = mat.Range(0, mat.NumRows(),
3248  0, mat.NumCols() - 1);
3249  bias_params_.Resize(mat.NumRows());
3250  bias_params_.CopyColFromMat(mat, mat.NumCols() - 1);
3251 }
void CopyColFromMat(const CuMatrixBase< Real > &mat, MatrixIndexT col)
Definition: cu-vector.cc:79
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:892
CuSubMatrix< Real > Range(const MatrixIndexT row_offset, const MatrixIndexT num_rows, const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:532
CuMatrix< BaseFloat > linear_params_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuVector< BaseFloat > bias_params_
void InitFromString ( std::string  args)
virtual

Initialize, typically from a line of a config file.

The "args" will contain any parameters that need to be passed to the Component, e.g. dimensions.

Implements Component.

Definition at line 3254 of file nnet-component.cc.

References FixedAffineComponent::Init(), KALDI_ASSERT, KALDI_ERR, CuMatrixBase< Real >::NumRows(), kaldi::nnet2::ParseFromString(), CuMatrix< Real >::Read(), Input::Stream(), and FixedAffineComponent::Type().

3254  {
3255  std::string orig_args = args;
3256  std::string filename;
3257  bool ok = ParseFromString("matrix", &args, &filename);
3258 
3259  if (!ok || !args.empty())
3260  KALDI_ERR << "Invalid initializer for layer of type "
3261  << Type() << ": \"" << orig_args << "\"";
3262 
3263  bool binary;
3264  Input ki(filename, &binary);
3265  CuMatrix<BaseFloat> mat;
3266  mat.Read(ki.Stream(), binary);
3267  KALDI_ASSERT(mat.NumRows() != 0);
3268  Init(mat);
3269 }
bool ParseFromString(const std::string &name, std::string *string, int32 *param)
Functions used in Init routines.
void Init(const CuMatrixBase< BaseFloat > &matrix)
matrix should be of size input-dim+1 to output-dim, last col is offset
#define KALDI_ERR
Definition: kaldi-error.h:127
virtual std::string Type() const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual int32 InputDim ( ) const
inlinevirtual

Get size of input vectors.

Implements Component.

Definition at line 1467 of file nnet-component.h.

References FixedAffineComponent::linear_params_, and CuMatrixBase< Real >::NumCols().

Referenced by AffineComponent::CollapseWithPrevious().

1467 { return linear_params_.NumCols(); }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
CuMatrix< BaseFloat > linear_params_
KALDI_DISALLOW_COPY_AND_ASSIGN ( FixedAffineComponent  )
protected
const CuMatrix<BaseFloat>& LinearParams ( ) const
inline

Definition at line 1488 of file nnet-component.h.

References FixedAffineComponent::linear_params_.

1488 { return linear_params_; }
CuMatrix< BaseFloat > linear_params_
virtual int32 OutputDim ( ) const
inlinevirtual

Get size of output vectors.

Implements Component.

Definition at line 1468 of file nnet-component.h.

References FixedAffineComponent::linear_params_, and CuMatrixBase< Real >::NumRows().

Referenced by AffineComponent::CollapseWithNext().

1468 { return linear_params_.NumRows(); }
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
CuMatrix< BaseFloat > linear_params_
void Propagate ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 3288 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), CuMatrixBase< Real >::AddVecToRows(), FixedAffineComponent::bias_params_, ChunkInfo::CheckSize(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, FixedAffineComponent::linear_params_, and ChunkInfo::NumChunks().

3291  {
3292  in_info.CheckSize(in);
3293  out_info.CheckSize(*out);
3294  KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());
3295 
3296  out->AddMatMat(1.0, in, kNoTrans, linear_params_, kTrans, 0.0);
3297  out->AddVecToRows(1.0, bias_params_);
3298 }
void AddVecToRows(Real alpha, const CuVectorBase< Real > &row, Real beta=1.0)
(for each row r of *this), r = alpha * row + beta * r
Definition: cu-matrix.cc:1111
void AddMatMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType transA, const CuMatrixBase< Real > &B, MatrixTransposeType transB, Real beta)
C = alpha * A(^T)*B(^T) + beta * C.
Definition: cu-matrix.cc:1141
CuMatrix< BaseFloat > linear_params_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuVector< BaseFloat > bias_params_
void Read ( std::istream &  is,
bool  binary 
)
virtual

Implements Component.

Definition at line 3328 of file nnet-component.cc.

References FixedAffineComponent::bias_params_, kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), FixedAffineComponent::linear_params_, CuVector< Real >::Read(), and CuMatrix< Real >::Read().

3328  {
3329  ExpectOneOrTwoTokens(is, binary, "<FixedAffineComponent>", "<LinearParams>");
3330  linear_params_.Read(is, binary);
3331  ExpectToken(is, binary, "<BiasParams>");
3332  bias_params_.Read(is, binary);
3333  ExpectToken(is, binary, "</FixedAffineComponent>");
3334 }
void Read(std::istream &is, bool binary)
I/O.
Definition: cu-vector.cc:862
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:188
void Read(std::istream &is, bool binary)
I/O functions.
Definition: cu-matrix.cc:459
CuMatrix< BaseFloat > linear_params_
static void ExpectOneOrTwoTokens(std::istream &is, bool binary, const std::string &token1, const std::string &token2)
CuVector< BaseFloat > bias_params_
virtual std::string Type ( ) const
inlinevirtual

Implements Component.

Definition at line 1457 of file nnet-component.h.

Referenced by FixedAffineComponent::InitFromString().

1457 { return "FixedAffineComponent"; }
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Definition at line 3319 of file nnet-component.cc.

References FixedAffineComponent::bias_params_, FixedAffineComponent::linear_params_, CuVector< Real >::Write(), CuMatrixBase< Real >::Write(), and kaldi::WriteToken().

3319  {
3320  WriteToken(os, binary, "<FixedAffineComponent>");
3321  WriteToken(os, binary, "<LinearParams>");
3322  linear_params_.Write(os, binary);
3323  WriteToken(os, binary, "<BiasParams>");
3324  bias_params_.Write(os, binary);
3325  WriteToken(os, binary, "</FixedAffineComponent>");
3326 }
void Write(std::ostream &is, bool binary) const
Definition: cu-vector.cc:872
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
CuMatrix< BaseFloat > linear_params_
CuVector< BaseFloat > bias_params_
void Write(std::ostream &os, bool binary) const
Definition: cu-matrix.cc:467

Friends And Related Function Documentation

friend class AffineComponent
friend

Definition at line 1490 of file nnet-component.h.

Member Data Documentation


The documentation for this class was generated from the following files: