All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
AffineComponent Class Reference

#include <nnet-component.h>

Inheritance diagram for AffineComponent:
Collaboration diagram for AffineComponent:

Public Member Functions

 AffineComponent (const AffineComponent &other)
 
 AffineComponent (const CuMatrixBase< BaseFloat > &linear_params, const CuVectorBase< BaseFloat > &bias_params, BaseFloat learning_rate)
 
virtual int32 InputDim () const
 Get size of input vectors. More...
 
virtual int32 OutputDim () const
 Get size of output vectors. More...
 
void Init (BaseFloat learning_rate, int32 input_dim, int32 output_dim, BaseFloat param_stddev, BaseFloat bias_stddev)
 
void Init (BaseFloat learning_rate, std::string matrix_filename)
 
virtual void Resize (int32 input_dim, int32 output_dim)
 
ComponentCollapseWithNext (const AffineComponent &next) const
 
ComponentCollapseWithNext (const FixedAffineComponent &next) const
 
ComponentCollapseWithNext (const FixedScaleComponent &next) const
 
ComponentCollapseWithPrevious (const FixedAffineComponent &prev) const
 
virtual std::string Info () const
 
virtual void InitFromString (std::string args)
 Initialize, typically from a line of a config file. More...
 
 AffineComponent ()
 
virtual std::string Type () const
 
virtual bool BackpropNeedsInput () const
 
virtual bool BackpropNeedsOutput () const
 
virtual void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Perform forward pass propagation Input->Output. More...
 
virtual void Scale (BaseFloat scale)
 This new virtual function scales the parameters by this amount. More...
 
virtual void Add (BaseFloat alpha, const UpdatableComponent &other)
 This new virtual function adds the parameters of another updatable component, times some constant, to the current parameters. More...
 
virtual void Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
 Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
 
virtual void SetZero (bool treat_as_gradient)
 Set parameters to zero, and if treat_as_gradient is true, we'll be treating this as a gradient so set the learning rate to 1 and make any other changes necessary (there's a variable we have to set for the MixtureProbComponent). More...
 
virtual void Read (std::istream &is, bool binary)
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
virtual BaseFloat DotProduct (const UpdatableComponent &other) const
 Here, "other" is a component of the same specific type. More...
 
virtual ComponentCopy () const
 Copy component (deep copy). More...
 
virtual void PerturbParams (BaseFloat stddev)
 We introduce a new virtual function that only applies to class UpdatableComponent. More...
 
virtual void SetParams (const VectorBase< BaseFloat > &bias, const MatrixBase< BaseFloat > &linear)
 
const CuVector< BaseFloat > & BiasParams ()
 
const CuMatrix< BaseFloat > & LinearParams ()
 
virtual int32 GetParameterDim () const
 The following new virtual function returns the total dimension of the parameters in this class. More...
 
virtual void Vectorize (VectorBase< BaseFloat > *params) const
 Turns the parameters into vector form. More...
 
virtual void UnVectorize (const VectorBase< BaseFloat > &params)
 Converts the parameters from vector form. More...
 
virtual void LimitRank (int32 dimension, AffineComponent **a, AffineComponent **b) const
 This function is for getting a low-rank approximations of this AffineComponent by two AffineComponents. More...
 
void Widen (int32 new_dimension, BaseFloat param_stddev, BaseFloat bias_stddev, std::vector< NonlinearComponent * > c2, AffineComponent *c3)
 This function is implemented in widen-nnet.cc. More...
 
- Public Member Functions inherited from UpdatableComponent
 UpdatableComponent (const UpdatableComponent &other)
 
void Init (BaseFloat learning_rate)
 
 UpdatableComponent (BaseFloat learning_rate)
 
 UpdatableComponent ()
 
virtual ~UpdatableComponent ()
 
void SetLearningRate (BaseFloat lrate)
 Sets the learning rate of gradient descent. More...
 
BaseFloat LearningRate () const
 Gets the learning rate of gradient descent. More...
 
- Public Member Functions inherited from Component
 Component ()
 
virtual int32 Index () const
 Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
 
virtual void SetIndex (int32 index)
 
virtual std::vector< int32 > Context () const
 Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
 
void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
 A non-virtual propagate function that first resizes output if necessary. More...
 
virtual ~Component ()
 

Protected Member Functions

virtual void Update (const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv)
 
virtual void UpdateSimple (const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv)
 
const AffineComponentoperator= (const AffineComponent &other)
 

Protected Attributes

CuMatrix< BaseFloatlinear_params_
 
CuVector< BaseFloatbias_params_
 
bool is_gradient_
 
- Protected Attributes inherited from UpdatableComponent
BaseFloat learning_rate_
 learning rate (0.0..0.01) More...
 

Friends

class SoftmaxComponent
 
class AffineComponentPreconditionedOnline
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream. More...
 
static ComponentNewFromString (const std::string &initializer_line)
 Initialize the Component from one line that will contain first the type, e.g. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Return a new Component of the given type e.g. More...
 

Detailed Description

Definition at line 843 of file nnet-component.h.

Constructor & Destructor Documentation

AffineComponent ( const AffineComponent other)

Definition at line 1017 of file nnet-component.cc.

1017  :
1018  UpdatableComponent(component),
1019  linear_params_(component.linear_params_),
1020  bias_params_(component.bias_params_),
1021  is_gradient_(component.is_gradient_) { }
CuVector< BaseFloat > bias_params_
CuMatrix< BaseFloat > linear_params_
AffineComponent ( const CuMatrixBase< BaseFloat > &  linear_params,
const CuVectorBase< BaseFloat > &  bias_params,
BaseFloat  learning_rate 
)

Definition at line 1023 of file nnet-component.cc.

References CuVectorBase< Real >::Dim(), AffineComponent::is_gradient_, KALDI_ASSERT, and CuMatrixBase< Real >::NumRows().

1025  :
1026  UpdatableComponent(learning_rate),
1027  linear_params_(linear_params),
1028  bias_params_(bias_params) {
1029  KALDI_ASSERT(linear_params.NumRows() == bias_params.Dim()&&
1030  bias_params.Dim() != 0);
1031  is_gradient_ = false;
1032 }
CuVector< BaseFloat > bias_params_
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
AffineComponent ( )
inline

Definition at line 876 of file nnet-component.h.

Referenced by AffineComponent::Copy().

876 : is_gradient_(false) { } // use Init to really initialize.

Member Function Documentation

void Add ( BaseFloat  alpha,
const UpdatableComponent other 
)
virtual

This new virtual function adds the parameters of another updatable component, times some constant, to the current parameters.

Implements UpdatableComponent.

Definition at line 1009 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), AffineComponent::bias_params_, KALDI_ASSERT, and AffineComponent::linear_params_.

1009  {
1010  const AffineComponent *other =
1011  dynamic_cast<const AffineComponent*>(&other_in);
1012  KALDI_ASSERT(other != NULL);
1013  linear_params_.AddMat(alpha, other->linear_params_);
1014  bias_params_.AddVec(alpha, other->bias_params_);
1015 }
CuVector< BaseFloat > bias_params_
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:939
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1126
void Backprop ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
Component to_update,
CuMatrix< BaseFloat > *  in_deriv 
) const
virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 1176 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), AffineComponent::InputDim(), AffineComponent::is_gradient_, kaldi::kNoTrans, AffineComponent::linear_params_, CuMatrixBase< Real >::NumRows(), CuMatrix< Real >::Resize(), AffineComponent::Update(), and AffineComponent::UpdateSimple().

1182  {
1183  AffineComponent *to_update = dynamic_cast<AffineComponent*>(to_update_in);
1184  in_deriv->Resize(out_deriv.NumRows(), InputDim());
1185  // Propagate the derivative back to the input.
1186  in_deriv->AddMatMat(1.0, out_deriv, kNoTrans, linear_params_, kNoTrans,
1187  0.0);
1188 
1189  if (to_update != NULL) {
1190  // Next update the model (must do this 2nd so the derivatives we propagate
1191  // are accurate, in case this == to_update_in.)
1192  if (to_update->is_gradient_)
1193  to_update->UpdateSimple(in_value, out_deriv);
1194  else // the call below is to a virtual function that may be re-implemented
1195  to_update->Update(in_value, out_deriv); // by child classes.
1196  }
1197 }
virtual int32 InputDim() const
Get size of input vectors.
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
void AddMatMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType transA, const CuMatrixBase< Real > &B, MatrixTransposeType transB, Real beta)
C = alpha * A(^T)*B(^T) + beta * C.
Definition: cu-matrix.cc:1142
CuMatrix< BaseFloat > linear_params_
virtual bool BackpropNeedsInput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 878 of file nnet-component.h.

878 { return true; }
virtual bool BackpropNeedsOutput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 879 of file nnet-component.h.

879 { return false; }
const CuVector<BaseFloat>& BiasParams ( )
inline

Definition at line 903 of file nnet-component.h.

References AffineComponent::bias_params_.

Referenced by kaldi::nnet2::FixNnet(), and LimitRankClass::operator()().

903 { return bias_params_; }
CuVector< BaseFloat > bias_params_
Component * CollapseWithNext ( const AffineComponent next) const

Definition at line 1295 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), CuVectorBase< Real >::AddMatVec(), AffineComponent::bias_params_, AffineComponent::Copy(), AffineComponent::InputDim(), KALDI_ASSERT, kaldi::kNoTrans, AffineComponent::linear_params_, AffineComponent::OutputDim(), and CuMatrix< Real >::Resize().

Referenced by Nnet::Collapse(), and Nnet::ResizeOutputLayer().

1296  {
1297  AffineComponent *ans = dynamic_cast<AffineComponent*>(this->Copy());
1298  KALDI_ASSERT(ans != NULL);
1299  // Note: it's possible that "ans" is really of a derived type such
1300  // as AffineComponentPreconditioned, but this will still work.
1301  // the "copy" call will copy things like learning rates, "alpha" value
1302  // for preconditioned component, etc.
1303  ans->linear_params_.Resize(next_component.OutputDim(), InputDim());
1304  ans->bias_params_ = next_component.bias_params_;
1305 
1306  ans->linear_params_.AddMatMat(1.0, next_component.linear_params_, kNoTrans,
1307  this->linear_params_, kNoTrans, 0.0);
1308  ans->bias_params_.AddMatVec(1.0, next_component.linear_params_, kNoTrans,
1309  this->bias_params_, 1.0);
1310  return ans;
1311 }
virtual int32 InputDim() const
Get size of input vectors.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual Component * Copy() const
Copy component (deep copy).
Component * CollapseWithNext ( const FixedAffineComponent next) const

Definition at line 1313 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), CuVectorBase< Real >::AddMatVec(), FixedAffineComponent::bias_params_, FixedAffineComponent::Copy(), AffineComponent::InputDim(), KALDI_ASSERT, kaldi::kNoTrans, FixedAffineComponent::linear_params_, FixedAffineComponent::OutputDim(), and CuMatrix< Real >::Resize().

1314  {
1315  // If at least one was non-updatable, make the whole non-updatable.
1316  FixedAffineComponent *ans =
1317  dynamic_cast<FixedAffineComponent*>(next_component.Copy());
1318  KALDI_ASSERT(ans != NULL);
1319  ans->linear_params_.Resize(next_component.OutputDim(), InputDim());
1320  ans->bias_params_ = next_component.bias_params_;
1321 
1322  ans->linear_params_.AddMatMat(1.0, next_component.linear_params_, kNoTrans,
1323  this->linear_params_, kNoTrans, 0.0);
1324  ans->bias_params_.AddMatVec(1.0, next_component.linear_params_, kNoTrans,
1325  this->bias_params_, 1.0);
1326  return ans;
1327 }
virtual int32 InputDim() const
Get size of input vectors.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Component * CollapseWithNext ( const FixedScaleComponent next) const

Definition at line 1329 of file nnet-component.cc.

References AffineComponent::Copy(), FixedScaleComponent::InputDim(), KALDI_ASSERT, AffineComponent::OutputDim(), and FixedScaleComponent::scales_.

1330  {
1331  KALDI_ASSERT(this->OutputDim() == next_component.InputDim());
1332  AffineComponent *ans =
1333  dynamic_cast<AffineComponent*>(this->Copy());
1334  KALDI_ASSERT(ans != NULL);
1335  ans->linear_params_.MulRowsVec(next_component.scales_);
1336  ans->bias_params_.MulElements(next_component.scales_);
1337 
1338  return ans;
1339 }
virtual int32 OutputDim() const
Get size of output vectors.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual Component * Copy() const
Copy component (deep copy).
Component * CollapseWithPrevious ( const FixedAffineComponent prev) const

Definition at line 1343 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), CuVectorBase< Real >::AddMatVec(), AffineComponent::bias_params_, FixedAffineComponent::bias_params_, FixedAffineComponent::Copy(), FixedAffineComponent::InputDim(), KALDI_ASSERT, kaldi::kNoTrans, AffineComponent::linear_params_, FixedAffineComponent::linear_params_, AffineComponent::OutputDim(), and CuMatrix< Real >::Resize().

1344  {
1345  // If at least one was non-updatable, make the whole non-updatable.
1346  FixedAffineComponent *ans =
1347  dynamic_cast<FixedAffineComponent*>(prev_component.Copy());
1348  KALDI_ASSERT(ans != NULL);
1349 
1350  ans->linear_params_.Resize(this->OutputDim(), prev_component.InputDim());
1351  ans->bias_params_ = this->bias_params_;
1352 
1353  ans->linear_params_.AddMatMat(1.0, this->linear_params_, kNoTrans,
1354  prev_component.linear_params_, kNoTrans, 0.0);
1355  ans->bias_params_.AddMatVec(1.0, this->linear_params_, kNoTrans,
1356  prev_component.bias_params_, 1.0);
1357  return ans;
1358 }
CuVector< BaseFloat > bias_params_
virtual int32 OutputDim() const
Get size of output vectors.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void AddMatVec(const Real alpha, const CuMatrixBase< Real > &M, MatrixTransposeType trans, const CuVectorBase< Real > &v, const Real beta)
Definition: cu-vector.cc:439
CuMatrix< BaseFloat > linear_params_
Component * Copy ( ) const
virtual

Copy component (deep copy).

Implements Component.

Reimplemented in AffineComponentPreconditionedOnline, and AffineComponentPreconditioned.

Definition at line 1080 of file nnet-component.cc.

References AffineComponent::AffineComponent(), AffineComponent::bias_params_, AffineComponent::is_gradient_, UpdatableComponent::learning_rate_, and AffineComponent::linear_params_.

Referenced by AffineComponent::CollapseWithNext(), and AffineComponent::LimitRank().

1080  {
1081  AffineComponent *ans = new AffineComponent();
1082  ans->learning_rate_ = learning_rate_;
1083  ans->linear_params_ = linear_params_;
1084  ans->bias_params_ = bias_params_;
1085  ans->is_gradient_ = is_gradient_;
1086  return ans;
1087 }
CuVector< BaseFloat > bias_params_
BaseFloat learning_rate_
learning rate (0.0..0.01)
CuMatrix< BaseFloat > linear_params_
BaseFloat DotProduct ( const UpdatableComponent other) const
virtual

Here, "other" is a component of the same specific type.

This function computes the dot product in parameters, and is computed while automatically adjusting learning rates; typically, one of the two will actually contain the gradient.

Implements UpdatableComponent.

Definition at line 1089 of file nnet-component.cc.

References AffineComponent::bias_params_, kaldi::kTrans, AffineComponent::linear_params_, kaldi::TraceMatMat(), and kaldi::VecVec().

1089  {
1090  const AffineComponent *other =
1091  dynamic_cast<const AffineComponent*>(&other_in);
1092  return TraceMatMat(linear_params_, other->linear_params_, kTrans)
1093  + VecVec(bias_params_, other->bias_params_);
1094 }
CuVector< BaseFloat > bias_params_
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
We need to declare this here as it will be a friend function.
CuMatrix< BaseFloat > linear_params_
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:36
int32 GetParameterDim ( ) const
virtual

The following new virtual function returns the total dimension of the parameters in this class.

E.g. used for L-BFGS update

Reimplemented from UpdatableComponent.

Definition at line 1247 of file nnet-component.cc.

References AffineComponent::InputDim(), and AffineComponent::OutputDim().

1247  {
1248  return (InputDim() + 1) * OutputDim();
1249 }
virtual int32 OutputDim() const
Get size of output vectors.
virtual int32 InputDim() const
Get size of input vectors.
std::string Info ( ) const
virtual

Reimplemented from UpdatableComponent.

Reimplemented in AffineComponentPreconditionedOnline, and AffineComponentPreconditioned.

Definition at line 1063 of file nnet-component.cc.

References AffineComponent::bias_params_, CuVectorBase< Real >::Dim(), AffineComponent::InputDim(), kaldi::kTrans, UpdatableComponent::LearningRate(), AffineComponent::linear_params_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), AffineComponent::OutputDim(), kaldi::TraceMatMat(), AffineComponent::Type(), and kaldi::VecVec().

1063  {
1064  std::stringstream stream;
1065  BaseFloat linear_params_size = static_cast<BaseFloat>(linear_params_.NumRows())
1066  * static_cast<BaseFloat>(linear_params_.NumCols());
1067  BaseFloat linear_stddev =
1069  linear_params_size),
1070  bias_stddev = std::sqrt(VecVec(bias_params_, bias_params_) /
1071  bias_params_.Dim());
1072  stream << Type() << ", input-dim=" << InputDim()
1073  << ", output-dim=" << OutputDim()
1074  << ", linear-params-stddev=" << linear_stddev
1075  << ", bias-params-stddev=" << bias_stddev
1076  << ", learning-rate=" << LearningRate();
1077  return stream.str();
1078 }
CuVector< BaseFloat > bias_params_
virtual int32 OutputDim() const
Get size of output vectors.
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
virtual int32 InputDim() const
Get size of input vectors.
BaseFloat LearningRate() const
Gets the learning rate of gradient descent.
float BaseFloat
Definition: kaldi-types.h:29
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
We need to declare this here as it will be a friend function.
virtual std::string Type() const
CuMatrix< BaseFloat > linear_params_
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:36
void Init ( BaseFloat  learning_rate,
int32  input_dim,
int32  output_dim,
BaseFloat  param_stddev,
BaseFloat  bias_stddev 
)

Definition at line 1096 of file nnet-component.cc.

References AffineComponent::bias_params_, UpdatableComponent::Init(), KALDI_ASSERT, AffineComponent::linear_params_, CuVector< Real >::Resize(), CuMatrix< Real >::Resize(), CuVectorBase< Real >::Scale(), CuMatrixBase< Real >::Scale(), CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

Referenced by kaldi::nnet2::GenRandomNnet(), AffineComponent::InitFromString(), and kaldi::nnet2::UnitTestAffineComponent().

1098  {
1099  UpdatableComponent::Init(learning_rate);
1100  linear_params_.Resize(output_dim, input_dim);
1101  bias_params_.Resize(output_dim);
1102  KALDI_ASSERT(output_dim > 0 && input_dim > 0 && param_stddev >= 0.0);
1103  linear_params_.SetRandn(); // sets to random normally distributed noise.
1104  linear_params_.Scale(param_stddev);
1106  bias_params_.Scale(bias_stddev);
1107 }
void Scale(Real value)
Definition: cu-vector.cc:1105
CuVector< BaseFloat > bias_params_
void Scale(Real value)
Definition: cu-matrix.cc:608
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:892
void Init(BaseFloat learning_rate)
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
void Init ( BaseFloat  learning_rate,
std::string  matrix_filename 
)

Definition at line 1109 of file nnet-component.cc.

References AffineComponent::bias_params_, CuVectorBase< Real >::CopyColFromMat(), CuMatrixBase< Real >::CopyFromMat(), UpdatableComponent::Init(), KALDI_ASSERT, AffineComponent::linear_params_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrixBase< Real >::Range(), kaldi::ReadKaldiObject(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

1110  {
1111  UpdatableComponent::Init(learning_rate);
1112  CuMatrix<BaseFloat> mat;
1113  ReadKaldiObject(matrix_filename, &mat); // will abort on failure.
1114  KALDI_ASSERT(mat.NumCols() >= 2);
1115  int32 input_dim = mat.NumCols() - 1, output_dim = mat.NumRows();
1116  linear_params_.Resize(output_dim, input_dim);
1117  bias_params_.Resize(output_dim);
1118  linear_params_.CopyFromMat(mat.Range(0, output_dim, 0, input_dim));
1119  bias_params_.CopyColFromMat(mat, input_dim);
1120 }
CuVector< BaseFloat > bias_params_
void CopyColFromMat(const CuMatrixBase< Real > &mat, MatrixIndexT col)
Definition: cu-vector.cc:79
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:337
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:892
void ReadKaldiObject(const std::string &filename, Matrix< float > *m)
Definition: kaldi-io.cc:818
void Init(BaseFloat learning_rate)
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
void InitFromString ( std::string  args)
virtual

Initialize, typically from a line of a config file.

The "args" will contain any parameters that need to be passed to the Component, e.g. dimensions.

Implements Component.

Reimplemented in AffineComponentPreconditionedOnline, and AffineComponentPreconditioned.

Definition at line 1122 of file nnet-component.cc.

References AffineComponent::Init(), AffineComponent::InputDim(), KALDI_ASSERT, KALDI_ERR, UpdatableComponent::learning_rate_, AffineComponent::OutputDim(), and kaldi::nnet2::ParseFromString().

Referenced by kaldi::nnet2::UnitTestAffineComponent().

1122  {
1123  std::string orig_args(args);
1124  bool ok = true;
1125  BaseFloat learning_rate = learning_rate_;
1126  std::string matrix_filename;
1127  int32 input_dim = -1, output_dim = -1;
1128  ParseFromString("learning-rate", &args, &learning_rate); // optional.
1129  if (ParseFromString("matrix", &args, &matrix_filename)) {
1130  Init(learning_rate, matrix_filename);
1131  if (ParseFromString("input-dim", &args, &input_dim))
1132  KALDI_ASSERT(input_dim == InputDim() &&
1133  "input-dim mismatch vs. matrix.");
1134  if (ParseFromString("output-dim", &args, &output_dim))
1135  KALDI_ASSERT(output_dim == OutputDim() &&
1136  "output-dim mismatch vs. matrix.");
1137  } else {
1138  ok = ok && ParseFromString("input-dim", &args, &input_dim);
1139  ok = ok && ParseFromString("output-dim", &args, &output_dim);
1140  BaseFloat param_stddev = 1.0 / std::sqrt(input_dim),
1141  bias_stddev = 1.0;
1142  ParseFromString("param-stddev", &args, &param_stddev);
1143  ParseFromString("bias-stddev", &args, &bias_stddev);
1144  Init(learning_rate, input_dim, output_dim,
1145  param_stddev, bias_stddev);
1146  }
1147  if (!args.empty())
1148  KALDI_ERR << "Could not process these elements in initializer: "
1149  << args;
1150  if (!ok)
1151  KALDI_ERR << "Bad initializer " << orig_args;
1152 }
void Init(BaseFloat learning_rate, int32 input_dim, int32 output_dim, BaseFloat param_stddev, BaseFloat bias_stddev)
virtual int32 OutputDim() const
Get size of output vectors.
virtual int32 InputDim() const
Get size of input vectors.
bool ParseFromString(const std::string &name, std::string *string, int32 *param)
Functions used in Init routines.
float BaseFloat
Definition: kaldi-types.h:29
BaseFloat learning_rate_
learning rate (0.0..0.01)
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void LimitRank ( int32  dimension,
AffineComponent **  a,
AffineComponent **  b 
) const
virtual

This function is for getting a low-rank approximations of this AffineComponent by two AffineComponents.

Definition at line 1261 of file nnet-component.cc.

References AffineComponent::bias_params_, AffineComponent::Copy(), MatrixBase< Real >::DestructiveSvd(), AffineComponent::InputDim(), KALDI_ASSERT, KALDI_LOG, kaldi::kCopyData, kaldi::kSetZero, AffineComponent::linear_params_, MatrixBase< Real >::MulRowsVec(), MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), CuVector< Real >::Resize(), Vector< Real >::Resize(), Matrix< Real >::Resize(), kaldi::SortSvd(), and VectorBase< Real >::Sum().

Referenced by Nnet::LimitRankOfLastLayer().

1262  {
1263  KALDI_ASSERT(d <= InputDim());
1264 
1265  // We'll limit the rank of just the linear part, keeping the bias vector full.
1266  Matrix<BaseFloat> M (linear_params_);
1267  int32 rows = M.NumRows(), cols = M.NumCols(), rc_min = std::min(rows, cols);
1268  Vector<BaseFloat> s(rc_min);
1269  Matrix<BaseFloat> U(rows, rc_min), Vt(rc_min, cols);
1270  // Do the destructive svd M = U diag(s) V^T. It actually outputs the transpose of V.
1271  M.DestructiveSvd(&s, &U, &Vt);
1272  SortSvd(&s, &U, &Vt); // Sort the singular values from largest to smallest.
1273  BaseFloat old_svd_sum = s.Sum();
1274  U.Resize(rows, d, kCopyData);
1275  s.Resize(d, kCopyData);
1276  Vt.Resize(d, cols, kCopyData);
1277  BaseFloat new_svd_sum = s.Sum();
1278  KALDI_LOG << "Reduced rank from "
1279  << rc_min << " to " << d << ", SVD sum reduced from "
1280  << old_svd_sum << " to " << new_svd_sum;
1281 
1282  // U.MulColsVec(s); // U <-- U diag(s)
1283  Vt.MulRowsVec(s); // Vt <-- diag(s) Vt.
1284 
1285  *a = dynamic_cast<AffineComponent*>(this->Copy());
1286  *b = dynamic_cast<AffineComponent*>(this->Copy());
1287 
1288  (*a)->bias_params_.Resize(d, kSetZero);
1289  (*a)->linear_params_ = Vt;
1290 
1291  (*b)->bias_params_ = this->bias_params_;
1292  (*b)->linear_params_ = U;
1293 }
CuVector< BaseFloat > bias_params_
virtual int32 InputDim() const
Get size of input vectors.
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
#define KALDI_LOG
Definition: kaldi-error.h:133
virtual Component * Copy() const
Copy component (deep copy).
void SortSvd(VectorBase< Real > *s, MatrixBase< Real > *U, MatrixBase< Real > *Vt, bool sort_on_absolute_value)
Function to ensure that SVD is sorted.
const CuMatrix<BaseFloat>& LinearParams ( )
inline

Definition at line 904 of file nnet-component.h.

References AffineComponent::linear_params_.

Referenced by kaldi::nnet2::FixNnet(), and LimitRankClass::operator()().

904 { return linear_params_; }
CuMatrix< BaseFloat > linear_params_
const AffineComponent& operator= ( const AffineComponent other)
protected
void PerturbParams ( BaseFloat  stddev)
virtual

We introduce a new virtual function that only applies to class UpdatableComponent.

This is used in testing.

Implements UpdatableComponent.

Definition at line 1053 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), AffineComponent::bias_params_, AffineComponent::linear_params_, CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

1053  {
1054  CuMatrix<BaseFloat> temp_linear_params(linear_params_);
1055  temp_linear_params.SetRandn();
1056  linear_params_.AddMat(stddev, temp_linear_params);
1057 
1058  CuVector<BaseFloat> temp_bias_params(bias_params_);
1059  temp_bias_params.SetRandn();
1060  bias_params_.AddVec(stddev, temp_bias_params);
1061 }
CuVector< BaseFloat > bias_params_
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:939
CuMatrix< BaseFloat > linear_params_
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1126
void Propagate ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 1155 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), AffineComponent::bias_params_, ChunkInfo::CheckSize(), CuMatrixBase< Real >::CopyRowsFromVec(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, AffineComponent::linear_params_, and ChunkInfo::NumChunks().

1158  {
1159  in_info.CheckSize(in);
1160  out_info.CheckSize(*out);
1161  KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());
1162 
1163  // No need for asserts as they'll happen within the matrix operations.
1164  out->CopyRowsFromVec(bias_params_); // copies bias_params_ to each row
1165  // of *out.
1166  out->AddMatMat(1.0, in, kNoTrans, linear_params_, kTrans, 1.0);
1167 }
CuVector< BaseFloat > bias_params_
void CopyRowsFromVec(const CuVectorBase< Real > &v)
This function has two modes of operation.
Definition: cu-matrix.cc:2146
void AddMatMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType transA, const CuMatrixBase< Real > &B, MatrixTransposeType transB, Real beta)
C = alpha * A(^T)*B(^T) + beta * C.
Definition: cu-matrix.cc:1142
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
void Read ( std::istream &  is,
bool  binary 
)
virtual

Implements Component.

Reimplemented in AffineComponentPreconditionedOnline, and AffineComponentPreconditioned.

Definition at line 1199 of file nnet-component.cc.

References AffineComponent::bias_params_, kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), AffineComponent::is_gradient_, KALDI_ASSERT, UpdatableComponent::learning_rate_, AffineComponent::linear_params_, CuVector< Real >::Read(), CuMatrix< Real >::Read(), kaldi::ReadBasicType(), kaldi::ReadToken(), and AffineComponent::Type().

1199  {
1200  std::ostringstream ostr_beg, ostr_end;
1201  ostr_beg << "<" << Type() << ">"; // e.g. "<AffineComponent>"
1202  ostr_end << "</" << Type() << ">"; // e.g. "</AffineComponent>"
1203  // might not see the "<AffineComponent>" part because
1204  // of how ReadNew() works.
1205  ExpectOneOrTwoTokens(is, binary, ostr_beg.str(), "<LearningRate>");
1206  ReadBasicType(is, binary, &learning_rate_);
1207  ExpectToken(is, binary, "<LinearParams>");
1208  linear_params_.Read(is, binary);
1209  ExpectToken(is, binary, "<BiasParams>");
1210  bias_params_.Read(is, binary);
1211  std::string tok;
1212  // back-compatibility code. TODO: re-do this later.
1213  ReadToken(is, binary, &tok);
1214  if (tok == "<AvgInput>") { // discard the following.
1215  CuVector<BaseFloat> avg_input;
1216  avg_input.Read(is, binary);
1217  BaseFloat avg_input_count;
1218  ExpectToken(is, binary, "<AvgInputCount>");
1219  ReadBasicType(is, binary, &avg_input_count);
1220  ReadToken(is, binary, &tok);
1221  }
1222  if (tok == "<IsGradient>") {
1223  ReadBasicType(is, binary, &is_gradient_);
1224  ExpectToken(is, binary, ostr_end.str());
1225  } else {
1226  is_gradient_ = false;
1227  KALDI_ASSERT(tok == ostr_end.str());
1228  }
1229 }
CuVector< BaseFloat > bias_params_
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
void Read(std::istream &is, bool binary)
I/O.
Definition: cu-vector.cc:862
void ReadToken(std::istream &is, bool binary, std::string *str)
ReadToken gets the next token and puts it in str (exception on failure).
Definition: io-funcs.cc:154
float BaseFloat
Definition: kaldi-types.h:29
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:188
BaseFloat learning_rate_
learning rate (0.0..0.01)
void Read(std::istream &is, bool binary)
I/O functions.
Definition: cu-matrix.cc:459
virtual std::string Type() const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
static void ExpectOneOrTwoTokens(std::istream &is, bool binary, const std::string &token1, const std::string &token2)
CuMatrix< BaseFloat > linear_params_
void Resize ( int32  input_dim,
int32  output_dim 
)
virtual

Reimplemented in AffineComponentPreconditionedOnline.

Definition at line 1003 of file nnet-component.cc.

References AffineComponent::bias_params_, KALDI_ASSERT, AffineComponent::linear_params_, CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

Referenced by Nnet::ResizeOutputLayer().

1003  {
1004  KALDI_ASSERT(input_dim > 0 && output_dim > 0);
1005  bias_params_.Resize(output_dim);
1006  linear_params_.Resize(output_dim, input_dim);
1007 }
CuVector< BaseFloat > bias_params_
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:892
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
void Scale ( BaseFloat  scale)
virtual

This new virtual function scales the parameters by this amount.

Implements UpdatableComponent.

Definition at line 997 of file nnet-component.cc.

References AffineComponent::bias_params_, AffineComponent::linear_params_, CuVectorBase< Real >::Scale(), and CuMatrixBase< Real >::Scale().

997  {
998  linear_params_.Scale(scale);
999  bias_params_.Scale(scale);
1000 }
void Scale(Real value)
Definition: cu-vector.cc:1105
CuVector< BaseFloat > bias_params_
void Scale(Real value)
Definition: cu-matrix.cc:608
CuMatrix< BaseFloat > linear_params_
void SetParams ( const VectorBase< BaseFloat > &  bias,
const MatrixBase< BaseFloat > &  linear 
)
virtual

Definition at line 1046 of file nnet-component.cc.

References AffineComponent::bias_params_, CuVectorBase< Real >::Dim(), KALDI_ASSERT, AffineComponent::linear_params_, and CuMatrixBase< Real >::NumRows().

Referenced by kaldi::nnet2::FixNnet(), SoftmaxComponent::MixUp(), and LimitRankClass::operator()().

1047  {
1048  bias_params_ = bias;
1049  linear_params_ = linear;
1051 }
CuVector< BaseFloat > bias_params_
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
void SetZero ( bool  treat_as_gradient)
virtual

Set parameters to zero, and if treat_as_gradient is true, we'll be treating this as a gradient so set the learning rate to 1 and make any other changes necessary (there's a variable we have to set for the MixtureProbComponent).

Implements UpdatableComponent.

Definition at line 1036 of file nnet-component.cc.

References AffineComponent::bias_params_, AffineComponent::is_gradient_, AffineComponent::linear_params_, UpdatableComponent::SetLearningRate(), CuVectorBase< Real >::SetZero(), and CuMatrixBase< Real >::SetZero().

1036  {
1037  if (treat_as_gradient) {
1038  SetLearningRate(1.0);
1039  }
1042  if (treat_as_gradient)
1043  is_gradient_ = true;
1044 }
CuVector< BaseFloat > bias_params_
void SetZero()
Math operations, some calling kernels.
Definition: cu-matrix.cc:474
void SetZero()
Math operations.
Definition: cu-vector.cc:988
CuMatrix< BaseFloat > linear_params_
void SetLearningRate(BaseFloat lrate)
Sets the learning rate of gradient descent.
virtual std::string Type ( ) const
inlinevirtual

Implements Component.

Reimplemented in AffineComponentPreconditionedOnline, and AffineComponentPreconditioned.

Definition at line 877 of file nnet-component.h.

Referenced by AffineComponent::Info(), AffineComponent::Read(), and AffineComponent::Write().

877 { return "AffineComponent"; }
void UnVectorize ( const VectorBase< BaseFloat > &  params)
virtual

Converts the parameters from vector form.

Reimplemented from UpdatableComponent.

Definition at line 1255 of file nnet-component.cc.

References AffineComponent::bias_params_, CuVectorBase< Real >::CopyFromVec(), CuMatrixBase< Real >::CopyRowsFromVec(), AffineComponent::InputDim(), AffineComponent::linear_params_, AffineComponent::OutputDim(), and VectorBase< Real >::Range().

1255  {
1258  OutputDim()));
1259 }
CuVector< BaseFloat > bias_params_
virtual int32 OutputDim() const
Get size of output vectors.
void CopyRowsFromVec(const CuVectorBase< Real > &v)
This function has two modes of operation.
Definition: cu-matrix.cc:2146
virtual int32 InputDim() const
Get size of input vectors.
void CopyFromVec(const CuVectorBase< Real > &src)
Copy functions; these will crash if the dimension do not match.
Definition: cu-vector.cc:970
CuMatrix< BaseFloat > linear_params_
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:92
virtual void Update ( const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv 
)
inlineprotectedvirtual

Reimplemented in AffineComponentPreconditionedOnline, and AffineComponentPreconditioned.

Definition at line 925 of file nnet-component.h.

References AffineComponent::UpdateSimple().

Referenced by AffineComponent::Backprop().

927  {
928  UpdateSimple(in_value, out_deriv);
929  }
virtual void UpdateSimple(const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv)
void UpdateSimple ( const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv 
)
protectedvirtual

Definition at line 1169 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), CuVectorBase< Real >::AddRowSumMat(), AffineComponent::bias_params_, kaldi::kNoTrans, kaldi::kTrans, UpdatableComponent::learning_rate_, and AffineComponent::linear_params_.

Referenced by AffineComponent::Backprop(), and AffineComponent::Update().

1170  {
1171  bias_params_.AddRowSumMat(learning_rate_, out_deriv, 1.0);
1173  in_value, kNoTrans, 1.0);
1174 }
CuVector< BaseFloat > bias_params_
void AddRowSumMat(Real alpha, const CuMatrixBase< Real > &mat, Real beta=1.0)
Sum the rows of the matrix, add to vector.
Definition: cu-vector.cc:1166
BaseFloat learning_rate_
learning rate (0.0..0.01)
void AddMatMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType transA, const CuMatrixBase< Real > &B, MatrixTransposeType transB, Real beta)
C = alpha * A(^T)*B(^T) + beta * C.
Definition: cu-matrix.cc:1142
CuMatrix< BaseFloat > linear_params_
void Vectorize ( VectorBase< BaseFloat > *  params) const
virtual

Turns the parameters into vector form.

We put the vector form on the CPU, because in the kinds of situations where we do this, we'll tend to use too much memory for the GPU.

Reimplemented from UpdatableComponent.

Definition at line 1250 of file nnet-component.cc.

References AffineComponent::bias_params_, AffineComponent::InputDim(), AffineComponent::linear_params_, AffineComponent::OutputDim(), and VectorBase< Real >::Range().

1250  {
1251  params->Range(0, InputDim() * OutputDim()).CopyRowsFromMat(linear_params_);
1252  params->Range(InputDim() * OutputDim(),
1253  OutputDim()).CopyFromVec(bias_params_);
1254 }
CuVector< BaseFloat > bias_params_
virtual int32 OutputDim() const
Get size of output vectors.
virtual int32 InputDim() const
Get size of input vectors.
CuMatrix< BaseFloat > linear_params_
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:92
void Widen ( int32  new_dimension,
BaseFloat  param_stddev,
BaseFloat  bias_stddev,
std::vector< NonlinearComponent * >  c2,
AffineComponent c3 
)

This function is implemented in widen-nnet.cc.

Definition at line 28 of file widen-nnet.cc.

References AffineComponent::bias_params_, rnnlm::i, AffineComponent::InputDim(), KALDI_ASSERT, KALDI_WARN, kaldi::kCopyData, AffineComponent::linear_params_, AffineComponent::OutputDim(), CuVectorBase< Real >::Range(), CuMatrixBase< Real >::Range(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

Referenced by kaldi::nnet2::WidenNnet().

34  {
35  int32 old_dim = this->OutputDim(), extra_dim = new_dim - old_dim;
36  KALDI_ASSERT(!c2.empty());
37  if (new_dim <= old_dim) {
38  KALDI_WARN << "Not widening component because new dim "
39  << new_dim << " <= old dim " << old_dim;
40  return;
41  }
42 
43  this->bias_params_.Resize(new_dim,
44  kCopyData);
45  this->bias_params_.Range(old_dim, extra_dim).SetRandn();
46  this->bias_params_.Range(old_dim, extra_dim).Scale(bias_stddev);
47 
48  this->linear_params_.Resize(new_dim, InputDim(), kCopyData);
49  this->linear_params_.Range(old_dim, extra_dim,
50  0, InputDim()).SetRandn();
51  this->linear_params_.Range(old_dim, extra_dim,
52  0, InputDim()).Scale(param_stddev);
53 
54  for (size_t i = 0; i < c2.size(); i++) // Change dimension of nonlinear
55  c2[i]->SetDim(new_dim); // components
56 
57  // Change dimension of next affine component [extend with zeros,
58  // so the existing outputs do not change in value]
59  c3->linear_params_.Resize(c3->OutputDim(), new_dim, kCopyData);
60 }
CuVector< BaseFloat > bias_params_
virtual int32 OutputDim() const
Get size of output vectors.
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:892
virtual int32 InputDim() const
Get size of input vectors.
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
#define KALDI_WARN
Definition: kaldi-error.h:130
CuSubMatrix< Real > Range(const MatrixIndexT row_offset, const MatrixIndexT num_rows, const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:532
CuSubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Definition: cu-vector.h:132
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
CuMatrix< BaseFloat > linear_params_
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Reimplemented in AffineComponentPreconditionedOnline, and AffineComponentPreconditioned.

Definition at line 1231 of file nnet-component.cc.

References AffineComponent::bias_params_, AffineComponent::is_gradient_, UpdatableComponent::learning_rate_, AffineComponent::linear_params_, AffineComponent::Type(), CuVector< Real >::Write(), CuMatrixBase< Real >::Write(), kaldi::WriteBasicType(), and kaldi::WriteToken().

1231  {
1232  std::ostringstream ostr_beg, ostr_end;
1233  ostr_beg << "<" << Type() << ">"; // e.g. "<AffineComponent>"
1234  ostr_end << "</" << Type() << ">"; // e.g. "</AffineComponent>"
1235  WriteToken(os, binary, ostr_beg.str());
1236  WriteToken(os, binary, "<LearningRate>");
1237  WriteBasicType(os, binary, learning_rate_);
1238  WriteToken(os, binary, "<LinearParams>");
1239  linear_params_.Write(os, binary);
1240  WriteToken(os, binary, "<BiasParams>");
1241  bias_params_.Write(os, binary);
1242  WriteToken(os, binary, "<IsGradient>");
1243  WriteBasicType(os, binary, is_gradient_);
1244  WriteToken(os, binary, ostr_end.str());
1245 }
CuVector< BaseFloat > bias_params_
void Write(std::ostream &is, bool binary) const
Definition: cu-vector.cc:872
BaseFloat learning_rate_
learning rate (0.0..0.01)
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
virtual std::string Type() const
void WriteBasicType(std::ostream &os, bool binary, T t)
WriteBasicType is the name of the write function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:34
CuMatrix< BaseFloat > linear_params_
void Write(std::ostream &os, bool binary) const
Definition: cu-matrix.cc:467

Friends And Related Function Documentation

Definition at line 923 of file nnet-component.h.

friend class SoftmaxComponent
friend

Definition at line 844 of file nnet-component.h.

Member Data Documentation


The documentation for this class was generated from the following files: