All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
FixedBiasComponent Class Reference

FixedBiasComponent applies a fixed per-element bias; it's similar to the AddShift component in the nnet1 setup (and only needed for nnet1 model conversion. More...

#include <nnet-component.h>

Inheritance diagram for FixedBiasComponent:
Collaboration diagram for FixedBiasComponent:

Public Member Functions

 FixedBiasComponent ()
 
virtual std::string Type () const
 
virtual std::string Info () const
 
void Init (const CuVectorBase< BaseFloat > &scales)
 
virtual void InitFromString (std::string args)
 Initialize, typically from a line of a config file. More...
 
virtual int32 InputDim () const
 Get size of input vectors. More...
 
virtual int32 OutputDim () const
 Get size of output vectors. More...
 
virtual void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Perform forward pass propagation Input->Output. More...
 
virtual void Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
 Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
 
virtual bool BackpropNeedsInput () const
 
virtual bool BackpropNeedsOutput () const
 
virtual ComponentCopy () const
 Copy component (deep copy). More...
 
virtual void Read (std::istream &is, bool binary)
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
- Public Member Functions inherited from Component
 Component ()
 
virtual int32 Index () const
 Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
 
virtual void SetIndex (int32 index)
 
virtual std::vector< int32 > Context () const
 Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
 
void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
 A non-virtual propagate function that first resizes output if necessary. More...
 
virtual ~Component ()
 

Protected Member Functions

 KALDI_DISALLOW_COPY_AND_ASSIGN (FixedBiasComponent)
 

Protected Attributes

CuVector< BaseFloatbias_
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream. More...
 
static ComponentNewFromString (const std::string &initializer_line)
 Initialize the Component from one line that will contain first the type, e.g. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Return a new Component of the given type e.g. More...
 

Detailed Description

FixedBiasComponent applies a fixed per-element bias; it's similar to the AddShift component in the nnet1 setup (and only needed for nnet1 model conversion.

Definition at line 1542 of file nnet-component.h.

Constructor & Destructor Documentation

FixedBiasComponent ( )
inline

Definition at line 1544 of file nnet-component.h.

Referenced by FixedBiasComponent::Copy().

1544 { }

Member Function Documentation

void Backprop ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
Component to_update,
CuMatrix< BaseFloat > *  in_deriv 
) const
virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 3446 of file nnet-component.cc.

3452  {
3453  *in_deriv = out_deriv;
3454 }
virtual bool BackpropNeedsInput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 1568 of file nnet-component.h.

1568 { return false; }
virtual bool BackpropNeedsOutput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 1569 of file nnet-component.h.

1569 { return false; }
Component * Copy ( ) const
virtual

Copy component (deep copy).

Implements Component.

Definition at line 3456 of file nnet-component.cc.

References FixedBiasComponent::bias_, and FixedBiasComponent::FixedBiasComponent().

3456  {
3458  ans->bias_ = bias_;
3459  return ans;
3460 }
std::string Info ( ) const
virtual

Reimplemented from Component.

Definition at line 3427 of file nnet-component.cc.

References FixedBiasComponent::bias_, CuVectorBase< Real >::Dim(), Component::Info(), CuVectorBase< Real >::Sum(), and kaldi::VecVec().

3427  {
3428  std::stringstream stream;
3429  BaseFloat bias_size = static_cast<BaseFloat>(bias_.Dim()),
3430  bias_mean = bias_.Sum() / bias_size,
3431  bias_stddev = std::sqrt(VecVec(bias_, bias_) / bias_size)
3432  - (bias_mean * bias_mean);
3433  stream << Component::Info() << ", bias-mean=" << bias_mean
3434  << ", bias-stddev=" << bias_stddev;
3435  return stream.str();
3436 }
float BaseFloat
Definition: kaldi-types.h:29
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
virtual std::string Info() const
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:36
Real Sum() const
Definition: cu-vector.cc:268
void Init ( const CuVectorBase< BaseFloat > &  scales)

Definition at line 3407 of file nnet-component.cc.

References FixedBiasComponent::bias_, CuVectorBase< Real >::Dim(), and KALDI_ASSERT.

Referenced by kaldi::ConvertAddShiftComponent(), FixedBiasComponent::InitFromString(), and kaldi::nnet2::UnitTestFixedBiasComponent().

3407  {
3408  KALDI_ASSERT(bias.Dim() != 0);
3409  bias_ = bias;
3410 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InitFromString ( std::string  args)
virtual

Initialize, typically from a line of a config file.

The "args" will contain any parameters that need to be passed to the Component, e.g. dimensions.

Implements Component.

Definition at line 3412 of file nnet-component.cc.

References FixedBiasComponent::Init(), KALDI_ERR, kaldi::nnet2::ParseFromString(), kaldi::ReadKaldiObject(), and FixedBiasComponent::Type().

3412  {
3413  std::string orig_args = args;
3414  std::string filename;
3415  bool ok = ParseFromString("bias", &args, &filename);
3416 
3417  if (!ok || !args.empty())
3418  KALDI_ERR << "Invalid initializer for layer of type "
3419  << Type() << ": \"" << orig_args << "\"";
3420 
3421  CuVector<BaseFloat> vec;
3422  ReadKaldiObject(filename, &vec);
3423  Init(vec);
3424 }
void Init(const CuVectorBase< BaseFloat > &scales)
void ReadKaldiObject(const std::string &filename, Matrix< float > *m)
Definition: kaldi-io.cc:818
bool ParseFromString(const std::string &name, std::string *string, int32 *param)
Functions used in Init routines.
virtual std::string Type() const
#define KALDI_ERR
Definition: kaldi-error.h:127
virtual int32 InputDim ( ) const
inlinevirtual

Get size of input vectors.

Implements Component.

Definition at line 1554 of file nnet-component.h.

References FixedBiasComponent::bias_, and CuVectorBase< Real >::Dim().

1554 { return bias_.Dim(); }
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
KALDI_DISALLOW_COPY_AND_ASSIGN ( FixedBiasComponent  )
protected
virtual int32 OutputDim ( ) const
inlinevirtual

Get size of output vectors.

Implements Component.

Definition at line 1555 of file nnet-component.h.

References FixedBiasComponent::bias_, and CuVectorBase< Real >::Dim().

1555 { return bias_.Dim(); }
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
void Propagate ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 3438 of file nnet-component.cc.

References CuMatrixBase< Real >::AddVecToRows(), FixedBiasComponent::bias_, and CuMatrixBase< Real >::CopyFromMat().

3441  {
3442  out->CopyFromMat(in);
3443  out->AddVecToRows(1.0, bias_, 1.0);
3444 }
void AddVecToRows(Real alpha, const CuVectorBase< Real > &row, Real beta=1.0)
(for each row r of *this), r = alpha * row + beta * r
Definition: cu-matrix.cc:1111
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:337
void Read ( std::istream &  is,
bool  binary 
)
virtual

Implements Component.

Definition at line 3470 of file nnet-component.cc.

References FixedBiasComponent::bias_, kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), and CuVector< Real >::Read().

3470  {
3471  ExpectOneOrTwoTokens(is, binary, "<FixedBiasComponent>", "<Bias>");
3472  bias_.Read(is, binary);
3473  ExpectToken(is, binary, "</FixedBiasComponent>");
3474 }
void Read(std::istream &is, bool binary)
I/O.
Definition: cu-vector.cc:862
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:188
static void ExpectOneOrTwoTokens(std::istream &is, bool binary, const std::string &token1, const std::string &token2)
virtual std::string Type ( ) const
inlinevirtual

Implements Component.

Definition at line 1545 of file nnet-component.h.

Referenced by FixedBiasComponent::InitFromString().

1545 { return "FixedBiasComponent"; }
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Definition at line 3463 of file nnet-component.cc.

References FixedBiasComponent::bias_, CuVector< Real >::Write(), and kaldi::WriteToken().

3463  {
3464  WriteToken(os, binary, "<FixedBiasComponent>");
3465  WriteToken(os, binary, "<Bias>");
3466  bias_.Write(os, binary);
3467  WriteToken(os, binary, "</FixedBiasComponent>");
3468 }
void Write(std::ostream &is, bool binary) const
Definition: cu-vector.cc:872
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134

Member Data Documentation


The documentation for this class was generated from the following files: