All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
ConvolutionComponent Class Reference

WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted. More...

#include <nnet-simple-component.h>

Inheritance diagram for ConvolutionComponent:
Collaboration diagram for ConvolutionComponent:

Public Types

enum  TensorVectorizationType { kYzx = 0, kZyx = 1 }
 

Public Member Functions

 ConvolutionComponent ()
 
 ConvolutionComponent (const ConvolutionComponent &component)
 
 ConvolutionComponent (const CuMatrixBase< BaseFloat > &filter_params, const CuVectorBase< BaseFloat > &bias_params, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, TensorVectorizationType input_vectorization, BaseFloat learning_rate)
 
virtual int32 InputDim () const
 Returns input-dimension of this component. More...
 
virtual int32 OutputDim () const
 Returns output-dimension of this component. More...
 
virtual std::string Info () const
 Returns some text-form information about this component, for diagnostics. More...
 
virtual void InitFromConfig (ConfigLine *cfl)
 Initialize, from a ConfigLine object. More...
 
virtual std::string Type () const
 Returns a string such as "SigmoidComponent", describing the type of the object. More...
 
virtual int32 Properties () const
 Return bitmask of the component's properties. More...
 
virtual void * Propagate (const ComponentPrecomputedIndexes *indexes, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Propagate function. More...
 
virtual void Backprop (const std::string &debug_info, const ComponentPrecomputedIndexes *indexes, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &, const CuMatrixBase< BaseFloat > &out_deriv, void *memo, Component *to_update_in, CuMatrixBase< BaseFloat > *in_deriv) const
 Backprop function; depending on which of the arguments 'to_update' and 'in_deriv' are non-NULL, this can compute input-data derivatives and/or perform model update. More...
 
void Update (const std::string &debug_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv, const std::vector< CuSubMatrix< BaseFloat > * > &out_deriv_batch)
 
virtual void Read (std::istream &is, bool binary)
 Read function (used after we know the type of the Component); accepts input that is missing the token that describes the component type, in case it has already been consumed. More...
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
virtual ComponentCopy () const
 Copies component (deep copy). More...
 
virtual void Scale (BaseFloat scale)
 This virtual function when called by. More...
 
virtual void Add (BaseFloat alpha, const Component &other)
 This virtual function when called by – an UpdatableComponent adds the parameters of another updatable component, times some constant, to the current parameters. More...
 
virtual void PerturbParams (BaseFloat stddev)
 This function is to be used in testing. More...
 
virtual BaseFloat DotProduct (const UpdatableComponent &other) const
 Computes dot-product between parameters of two instances of a Component. More...
 
virtual int32 NumParameters () const
 The following new virtual function returns the total dimension of the parameters in this class. More...
 
virtual void Vectorize (VectorBase< BaseFloat > *params) const
 Turns the parameters into vector form. More...
 
virtual void UnVectorize (const VectorBase< BaseFloat > &params)
 Converts the parameters from vector form. More...
 
void SetParams (const VectorBase< BaseFloat > &bias, const MatrixBase< BaseFloat > &filter)
 
const CuVector< BaseFloat > & BiasParams () const
 
const CuMatrix< BaseFloat > & LinearParams () const
 
void Init (int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, int32 num_filters, TensorVectorizationType input_vectorization, BaseFloat param_stddev, BaseFloat bias_stddev)
 
void Init (int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, TensorVectorizationType input_vectorization, std::string matrix_filename)
 
void Resize (int32 input_dim, int32 output_dim)
 
void Update (const std::string &debug_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv)
 
- Public Member Functions inherited from UpdatableComponent
 UpdatableComponent (const UpdatableComponent &other)
 
 UpdatableComponent ()
 
virtual ~UpdatableComponent ()
 
virtual void SetUnderlyingLearningRate (BaseFloat lrate)
 Sets the learning rate of gradient descent- gets multiplied by learning_rate_factor_. More...
 
virtual void SetActualLearningRate (BaseFloat lrate)
 Sets the learning rate directly, bypassing learning_rate_factor_. More...
 
virtual void SetAsGradient ()
 Sets is_gradient_ to true and sets learning_rate_ to 1, ignoring learning_rate_factor_. More...
 
virtual void SetLearningRateFactor (BaseFloat lrate_factor)
 
virtual void FreezeNaturalGradient (bool freeze)
 freezes/unfreezes NaturalGradient updates, if applicable (to be overriden by components that use Natural Gradient). More...
 
BaseFloat LearningRate () const
 Gets the learning rate of gradient descent. More...
 
BaseFloat MaxChange () const
 Gets per-component max-change value. More...
 
- Public Member Functions inherited from Component
virtual void StoreStats (const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, void *memo)
 This function may store stats on average activation values, and for some component types, the average value of the derivative of the nonlinearity. More...
 
virtual void ZeroStats ()
 Components that provide an implementation of StoreStats should also provide an implementation of ZeroStats(), to set those stats to zero. More...
 
virtual void GetInputIndexes (const MiscComputationInfo &misc_info, const Index &output_index, std::vector< Index > *desired_indexes) const
 This function only does something interesting for non-simple Components. More...
 
virtual bool IsComputable (const MiscComputationInfo &misc_info, const Index &output_index, const IndexSet &input_index_set, std::vector< Index > *used_inputs) const
 This function only does something interesting for non-simple Components, and it exists to make it possible to manage optionally-required inputs. More...
 
virtual void ReorderIndexes (std::vector< Index > *input_indexes, std::vector< Index > *output_indexes) const
 This function only does something interesting for non-simple Components. More...
 
virtual
ComponentPrecomputedIndexes
PrecomputeIndexes (const MiscComputationInfo &misc_info, const std::vector< Index > &input_indexes, const std::vector< Index > &output_indexes, bool need_backprop) const
 This function must return NULL for simple Components. More...
 
virtual void DeleteMemo (void *memo) const
 This virtual function only needs to be overwritten by Components that return a non-NULL memo from their Propagate() function. More...
 
 Component ()
 
virtual ~Component ()
 

Private Member Functions

void InputToInputPatches (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
 
void InderivPatchesToInderiv (const CuMatrix< BaseFloat > &in_deriv_patches, CuMatrixBase< BaseFloat > *in_deriv) const
 
const ConvolutionComponentoperator= (const ConvolutionComponent &other)
 

Private Attributes

int32 input_x_dim_
 
int32 input_y_dim_
 
int32 input_z_dim_
 
int32 filt_x_dim_
 
int32 filt_y_dim_
 
int32 filt_x_step_
 
int32 filt_y_step_
 
TensorVectorizationType input_vectorization_
 
CuMatrix< BaseFloatfilter_params_
 
CuVector< BaseFloatbias_params_
 
bool is_gradient_
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream (works out its type). Dies on error. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Returns a new Component of the given type e.g. More...
 
- Protected Member Functions inherited from UpdatableComponent
void InitLearningRatesFromConfig (ConfigLine *cfl)
 
std::string ReadUpdatableCommon (std::istream &is, bool binary)
 
void WriteUpdatableCommon (std::ostream &is, bool binary) const
 
- Protected Attributes inherited from UpdatableComponent
BaseFloat learning_rate_
 learning rate (typically 0.0..0.01) More...
 
BaseFloat learning_rate_factor_
 learning rate factor (normally 1.0, but can be set to another < value so that when < you call SetLearningRate(), that value will be scaled by this factor. More...
 
bool is_gradient_
 True if this component is to be treated as a gradient rather than as parameters. More...
 
BaseFloat max_change_
 configuration value for imposing max-change More...
 

Detailed Description

WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted.

ConvolutionalComponent implements 2d-convolution. It uses 3D filters on 3D inputs, but the 3D filters hop only over 2 dimensions as it has same size as the input along the 3rd dimension. Input : A matrix where each row is a vectorized 3D-tensor. The 3D tensor has dimensions x: (e.g. time) y: (e.g. frequency) z: (e.g. channels like features/delta/delta-delta)

The component supports input vectorizations of type zyx and yzx. The default vectorization type is zyx. e.g. for input vectorization of type zyx the input is vectorized by spanning axes z, y and x of the tensor in that order. Given 3d tensor A with sizes (2, 2, 2) along the three dimensions the zyx vectorized input looks like A(0,0,0) A(0,0,1) A(0,1,0) A(0,1,1) A(1,0,0) A(1,0,1) A(1,1,0) A(1,1,1)

Output : The output is also a 3D tensor vectorized in the zyx format. The channel axis (z) in the output corresponds to the output of different filters. The first channel corresponds to the first filter i.e., first row of the filter_params_ matrix.

Note: The component has to support yzx input vectorization as the binaries like add-deltas generate yz vectorized output. These input vectors are concatenated using the Append descriptor across time steps to form a yzx vectorized 3D tensor input. e.g. Append(Offset(input, -1), input, Offset(input, 1))

For information on the hyperparameters and parameters of this component see the variable declarations.

Propagation: ------------ Convolution operation consists of a dot-products between the filter tensor and input tensor patch, for various shifts of filter tensor along the x and y axes input tensor. (Note: there is no shift along z-axis as the filter and input tensor have same size along this axis).

For a particular shift (i,j) of the filter tensor along input tensor dimensions x and y, the elements of the input tensor which overlap with the filter form the input tensor patch. This patch is vectorized in zyx format. All the patches corresponding to various samples in the mini-batch are stacked into a matrix, where each row corresponds to one patch. Let this matrix be represented by X_{i,j}. The dot products with various filters are computed simultaneously by computing the matrix product with the filter_params_ matrix (W) Y_{i,j} = X_{i,j}*W^T. Each row of W corresponds to one filter 3D tensor vectorized in zyx format.

All the matrix products corresponding to various shifts (i,j) of the filter tensor are computed simultaneously using the AddMatMatBatched call of CuMatrixBase class.

BackPropagation: ---------------- Backpropagation to compute the input derivative ( X_{i,j}) consists of the a series of matrix products. {i,j} = {i,j}*W where {i,j} corresponds to the output derivative for a particular shift of the filter.

Once again these matrix products are computed simultaneously.

Update: ------- The weight gradient is computed as = {i,j} (X_{i,j}^T *{i,j})

Definition at line 1682 of file nnet-simple-component.h.

Member Enumeration Documentation

Enumerator
kYzx 
kZyx 

Definition at line 1684 of file nnet-simple-component.h.

Constructor & Destructor Documentation

Definition at line 3326 of file nnet-simple-component.cc.

Referenced by ConvolutionComponent::Copy().

Definition at line 3334 of file nnet-simple-component.cc.

3335  :
3336  UpdatableComponent(component),
3337  input_x_dim_(component.input_x_dim_),
3338  input_y_dim_(component.input_y_dim_),
3339  input_z_dim_(component.input_z_dim_),
3340  filt_x_dim_(component.filt_x_dim_),
3341  filt_y_dim_(component.filt_y_dim_),
3342  filt_x_step_(component.filt_x_step_),
3343  filt_y_step_(component.filt_y_step_),
3344  input_vectorization_(component.input_vectorization_),
3345  filter_params_(component.filter_params_),
3346  bias_params_(component.bias_params_),
3347  is_gradient_(component.is_gradient_) {}
TensorVectorizationType input_vectorization_
ConvolutionComponent ( const CuMatrixBase< BaseFloat > &  filter_params,
const CuVectorBase< BaseFloat > &  bias_params,
int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
TensorVectorizationType  input_vectorization,
BaseFloat  learning_rate 
)

Definition at line 3349 of file nnet-simple-component.cc.

References CuVectorBase< Real >::Dim(), ConvolutionComponent::is_gradient_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and UpdatableComponent::SetUnderlyingLearningRate().

3356  :
3357  input_x_dim_(input_x_dim),
3358  input_y_dim_(input_y_dim),
3359  input_z_dim_(input_z_dim),
3360  filt_x_dim_(filt_x_dim),
3361  filt_y_dim_(filt_y_dim),
3362  filt_x_step_(filt_x_step),
3363  filt_y_step_(filt_y_step),
3364  input_vectorization_(input_vectorization),
3365  filter_params_(filter_params),
3366  bias_params_(bias_params){
3367  KALDI_ASSERT(filter_params.NumRows() == bias_params.Dim() &&
3368  bias_params.Dim() != 0);
3369  KALDI_ASSERT(filter_params.NumCols() == filt_x_dim * filt_y_dim * input_z_dim);
3370  SetUnderlyingLearningRate(learning_rate);
3371  is_gradient_ = false;
3372 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
TensorVectorizationType input_vectorization_
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual void SetUnderlyingLearningRate(BaseFloat lrate)
Sets the learning rate of gradient descent- gets multiplied by learning_rate_factor_.

Member Function Documentation

void Add ( BaseFloat  alpha,
const Component other 
)
virtual

This virtual function when called by – an UpdatableComponent adds the parameters of another updatable component, times some constant, to the current parameters.

– a NonlinearComponent it relates to adding stats Otherwise it should do nothing.

Reimplemented from Component.

Definition at line 3642 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, and KALDI_ASSERT.

3642  {
3643  const ConvolutionComponent *other =
3644  dynamic_cast<const ConvolutionComponent*>(&other_in);
3645  KALDI_ASSERT(other != NULL);
3646  filter_params_.AddMat(alpha, other->filter_params_);
3647  bias_params_.AddVec(alpha, other->bias_params_);
3648 }
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:939
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1126
void Backprop ( const std::string &  debug_info,
const ComponentPrecomputedIndexes indexes,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
void *  memo,
Component to_update,
CuMatrixBase< BaseFloat > *  in_deriv 
) const
virtual

Backprop function; depending on which of the arguments 'to_update' and 'in_deriv' are non-NULL, this can compute input-data derivatives and/or perform model update.

Parameters
[in]debug_infoThe component name, to be printed out in any warning messages.
[in]indexesA pointer to some information output by this class's PrecomputeIndexes function (will be NULL for simple components, i.e. those that don't do things like splicing).
[in]in_valueThe matrix that was given as input to the Propagate function. Will be ignored (and may be empty) if Properties()&kBackpropNeedsInput == 0.
[in]out_valueThe matrix that was output from the Propagate function. Will be ignored (and may be empty) if Properties()&kBackpropNeedsOutput == 0
[in]out_derivThe derivative at the output of this component.
[in]memoThis will normally be NULL, but for component types that set the flag kUsesMemo, this will be the return value of the Propagate() function that corresponds to this Backprop() function. Ownership of any pointers is not transferred to the Backprop function; DeleteMemo() will be called to delete it.
[out]to_updateIf model update is desired, the Component to be updated, else NULL. Does not have to be identical to this. If supplied, you can assume that to_update->Properties() & kUpdatableComponent is nonzero.
[out]in_derivThe derivative at the input of this component, if needed (else NULL). If Properties()&kBackpropInPlace, may be the same matrix as out_deriv. If Properties()&kBackpropAdds, this is added to by the Backprop routine, else it is set. The component code chooses which mode to work in, based on convenience.

Implements Component.

Definition at line 3737 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::InderivPatchesToInderiv(), ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, KALDI_ASSERT, kaldi::kNoTrans, kaldi::kSetZero, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and ConvolutionComponent::Update().

3744  {
3745  ConvolutionComponent *to_update =
3746  dynamic_cast<ConvolutionComponent*>(to_update_in);
3747  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
3748  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
3749  num_filters = filter_params_.NumRows(),
3750  num_frames = out_deriv.NumRows(),
3751  filter_dim = filter_params_.NumCols();
3752 
3753  KALDI_ASSERT(out_deriv.NumRows() == num_frames &&
3754  out_deriv.NumCols() ==
3755  (num_filters * num_x_steps * num_y_steps));
3756 
3757  // Compute inderiv patches
3758  CuMatrix<BaseFloat> in_deriv_patches(num_frames,
3759  num_x_steps * num_y_steps * filter_dim,
3760  kSetZero);
3761 
3762  std::vector<CuSubMatrix<BaseFloat>* > patch_deriv_batch, out_deriv_batch,
3763  filter_params_batch;
3764  CuSubMatrix<BaseFloat>* filter_params_elem = new CuSubMatrix<BaseFloat>(
3765  filter_params_, 0, filter_params_.NumRows(), 0, filter_params_.NumCols());
3766 
3767  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
3768  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
3769  int32 patch_number = x_step * num_y_steps + y_step;
3770 
3771  patch_deriv_batch.push_back(new CuSubMatrix<BaseFloat>(
3772  in_deriv_patches.ColRange(
3773  patch_number * filter_dim, filter_dim)));
3774  out_deriv_batch.push_back(new CuSubMatrix<BaseFloat>(out_deriv.ColRange(
3775  patch_number * num_filters, num_filters)));
3776  filter_params_batch.push_back(filter_params_elem);
3777  }
3778  }
3779  AddMatMatBatched<BaseFloat>(1.0, patch_deriv_batch,
3780  out_deriv_batch, kNoTrans,
3781  filter_params_batch, kNoTrans, 0.0);
3782 
3783  if (in_deriv) {
3784  // combine the derivatives from the individual input deriv patches
3785  // to compute input deriv matrix
3786  InderivPatchesToInderiv(in_deriv_patches, in_deriv);
3787  }
3788 
3789  if (to_update != NULL) {
3790  to_update->Update(debug_info, in_value, out_deriv, out_deriv_batch);
3791  }
3792 
3793  // release memory
3794  delete filter_params_elem;
3795  for (int32 p = 0; p < patch_deriv_batch.size(); p++) {
3796  delete patch_deriv_batch[p];
3797  delete out_deriv_batch[p];
3798  }
3799 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:544
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InderivPatchesToInderiv(const CuMatrix< BaseFloat > &in_deriv_patches, CuMatrixBase< BaseFloat > *in_deriv) const
const CuVector<BaseFloat>& BiasParams ( ) const
inline

Definition at line 1747 of file nnet-simple-component.h.

References ConvolutionComponent::bias_params_.

1747 { return bias_params_; }
Component * Copy ( ) const
virtual

Copies component (deep copy).

Implements Component.

Definition at line 3947 of file nnet-simple-component.cc.

References ConvolutionComponent::ConvolutionComponent().

3947  {
3948  ConvolutionComponent *ans = new ConvolutionComponent(*this);
3949  return ans;
3950 }
BaseFloat DotProduct ( const UpdatableComponent other) const
virtual

Computes dot-product between parameters of two instances of a Component.

Can be used for computing parameter-norm of an UpdatableComponent.

Implements UpdatableComponent.

Definition at line 3940 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, kaldi::kTrans, kaldi::TraceMatMat(), and kaldi::VecVec().

3940  {
3941  const ConvolutionComponent *other =
3942  dynamic_cast<const ConvolutionComponent*>(&other_in);
3943  return TraceMatMat(filter_params_, other->filter_params_, kTrans)
3944  + VecVec(bias_params_, other->bias_params_);
3945 }
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
We need to declare this here as it will be a friend function.
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:36
void InderivPatchesToInderiv ( const CuMatrix< BaseFloat > &  in_deriv_patches,
CuMatrixBase< BaseFloat > *  in_deriv 
) const
private

Definition at line 3681 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddCols(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, CuMatrixBase< Real >::NumCols(), kaldi::nnet3::RearrangeIndexes(), kaldi::nnet3::YzxVectorIndex(), and kaldi::nnet3::ZyxVectorIndex().

Referenced by ConvolutionComponent::Backprop().

3683  {
3684 
3685  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
3686  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
3687  filt_x_step = filt_x_step_,
3688  filt_y_step = filt_y_step_,
3689  filt_x_dim = filt_x_dim_,
3690  filt_y_dim = filt_y_dim_,
3691  input_x_dim = input_x_dim_,
3692  input_y_dim = input_y_dim_,
3693  input_z_dim = input_z_dim_,
3694  filter_dim = filter_params_.NumCols();
3695 
3696  // Compute the reverse column_map from the matrix with input
3697  // derivative patches to input derivative matrix
3698  std::vector<std::vector<int32> > reverse_column_map(in_deriv->NumCols());
3699  int32 rev_col_map_size = reverse_column_map.size();
3700  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
3701  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
3702  int32 patch_number = x_step * num_y_steps + y_step;
3703  int32 patch_start_index = patch_number * filter_dim;
3704  for (int32 x = 0, index = patch_start_index; x < filt_x_dim; x++) {
3705  for (int32 y = 0; y < filt_y_dim; y++) {
3706  for (int32 z = 0; z < input_z_dim; z++, index++) {
3707  int32 vector_index;
3708  if (input_vectorization_ == kZyx) {
3709  vector_index = ZyxVectorIndex(x_step * filt_x_step + x,
3710  y_step * filt_y_step + y, z,
3711  input_x_dim, input_y_dim,
3712  input_z_dim);
3713  } else {
3715  vector_index = YzxVectorIndex(x_step * filt_x_step + x,
3716  y_step * filt_y_step + y, z,
3717  input_x_dim, input_y_dim,
3718  input_z_dim);
3719  }
3720  KALDI_ASSERT(vector_index < rev_col_map_size);
3721  reverse_column_map[vector_index].push_back(index);
3722  }
3723  }
3724  }
3725  }
3726  }
3727  std::vector<std::vector<int32> > rearranged_column_map;
3728  RearrangeIndexes(reverse_column_map, &rearranged_column_map);
3729  for (int32 p = 0; p < rearranged_column_map.size(); p++) {
3730  CuArray<int32> cu_cols(rearranged_column_map[p]);
3731  in_deriv->AddCols(in_deriv_patches, cu_cols);
3732  }
3733 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
TensorVectorizationType input_vectorization_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
int32 YzxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
void AddCols(const CuMatrixBase< Real > &src, const CuArray< MatrixIndexT > &indices)
Add column indices[r] of src to column r.
Definition: cu-matrix.cc:2481
int32 ZyxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
void RearrangeIndexes(const std::vector< std::vector< int32 > > &in, std::vector< std::vector< int32 > > *out)
std::string Info ( ) const
virtual

Returns some text-form information about this component, for diagnostics.

Starts with the type of the component. E.g. "SigmoidComponent dim=900", although most components will have much more info.

Reimplemented from UpdatableComponent.

Definition at line 3441 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, UpdatableComponent::Info(), ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, CuMatrixBase< Real >::NumRows(), and kaldi::nnet3::PrintParameterStats().

3441  {
3442  std::ostringstream stream;
3443  stream << UpdatableComponent::Info()
3444  << ", input-x-dim=" << input_x_dim_
3445  << ", input-y-dim=" << input_y_dim_
3446  << ", input-z-dim=" << input_z_dim_
3447  << ", filt-x-dim=" << filt_x_dim_
3448  << ", filt-y-dim=" << filt_y_dim_
3449  << ", filt-x-step=" << filt_x_step_
3450  << ", filt-y-step=" << filt_y_step_
3451  << ", input-vectorization=" << input_vectorization_
3452  << ", num-filters=" << filter_params_.NumRows();
3453  PrintParameterStats(stream, "filter-params", filter_params_);
3454  PrintParameterStats(stream, "bias-params", bias_params_, true);
3455  return stream.str();
3456 }
TensorVectorizationType input_vectorization_
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
virtual std::string Info() const
Returns some text-form information about this component, for diagnostics.
void PrintParameterStats(std::ostringstream &os, const std::string &name, const CuVectorBase< BaseFloat > &params, bool include_mean)
Print to 'os' some information about the mean and standard deviation of some parameters, used in Info() functions in nnet-simple-component.cc.
Definition: nnet-parse.cc:520
void Init ( int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
int32  num_filters,
TensorVectorizationType  input_vectorization,
BaseFloat  param_stddev,
BaseFloat  bias_stddev 
)

Definition at line 3388 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, CuVector< Real >::Resize(), CuMatrix< Real >::Resize(), CuVectorBase< Real >::Scale(), CuMatrixBase< Real >::Scale(), CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

Referenced by ConvolutionComponent::InitFromConfig().

3393  {
3394  input_x_dim_ = input_x_dim;
3395  input_y_dim_ = input_y_dim;
3396  input_z_dim_ = input_z_dim;
3397  filt_x_dim_ = filt_x_dim;
3398  filt_y_dim_ = filt_y_dim;
3399  filt_x_step_ = filt_x_step;
3400  filt_y_step_ = filt_y_step;
3401  input_vectorization_ = input_vectorization;
3404  int32 filter_dim = filt_x_dim_ * filt_y_dim_ * input_z_dim_;
3405  filter_params_.Resize(num_filters, filter_dim);
3406  bias_params_.Resize(num_filters);
3407  KALDI_ASSERT(param_stddev >= 0.0 && bias_stddev >= 0.0);
3409  filter_params_.Scale(param_stddev);
3411  bias_params_.Scale(bias_stddev);
3412 }
void Scale(Real value)
Definition: cu-vector.cc:1105
void Scale(Real value)
Definition: cu-matrix.cc:608
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:892
TensorVectorizationType input_vectorization_
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Init ( int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
TensorVectorizationType  input_vectorization,
std::string  matrix_filename 
)

Definition at line 3415 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::CopyColFromMat(), CuMatrixBase< Real >::CopyFromMat(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrixBase< Real >::Range(), kaldi::ReadKaldiObject(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

3420  {
3421  input_x_dim_ = input_x_dim;
3422  input_y_dim_ = input_y_dim;
3423  input_z_dim_ = input_z_dim;
3424  filt_x_dim_ = filt_x_dim;
3425  filt_y_dim_ = filt_y_dim;
3426  filt_x_step_ = filt_x_step;
3427  filt_y_step_ = filt_y_step;
3428  input_vectorization_ = input_vectorization;
3429  CuMatrix<BaseFloat> mat;
3430  ReadKaldiObject(matrix_filename, &mat);
3431  int32 filter_dim = (filt_x_dim_ * filt_y_dim_ * input_z_dim_);
3432  int32 num_filters = mat.NumRows();
3433  KALDI_ASSERT(mat.NumCols() == (filter_dim + 1));
3434  filter_params_.Resize(num_filters, filter_dim);
3435  bias_params_.Resize(num_filters);
3436  filter_params_.CopyFromMat(mat.Range(0, num_filters, 0, filter_dim));
3437  bias_params_.CopyColFromMat(mat, filter_dim);
3438 }
void CopyColFromMat(const CuMatrixBase< Real > &mat, MatrixIndexT col)
Definition: cu-vector.cc:79
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:337
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:892
TensorVectorizationType input_vectorization_
void ReadKaldiObject(const std::string &filename, Matrix< float > *m)
Definition: kaldi-io.cc:818
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InitFromConfig ( ConfigLine cfl)
virtual

Initialize, from a ConfigLine object.

Parameters
[in]cflA ConfigLine containing any parameters that are needed for initialization. For example: "dim=100 param-stddev=0.1"

Implements Component.

Definition at line 3459 of file nnet-simple-component.cc.

References ConfigLine::GetValue(), ConfigLine::HasUnusedValues(), ConvolutionComponent::Init(), UpdatableComponent::InitLearningRatesFromConfig(), KALDI_ERR, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, ConfigLine::UnusedValues(), and ConfigLine::WholeLine().

3459  {
3460  bool ok = true;
3461  std::string matrix_filename;
3462  int32 input_x_dim = -1, input_y_dim = -1, input_z_dim = -1,
3463  filt_x_dim = -1, filt_y_dim = -1,
3464  filt_x_step = -1, filt_y_step = -1,
3465  num_filters = -1;
3466  std::string input_vectorization_order = "zyx";
3468  ok = ok && cfl->GetValue("input-x-dim", &input_x_dim);
3469  ok = ok && cfl->GetValue("input-y-dim", &input_y_dim);
3470  ok = ok && cfl->GetValue("input-z-dim", &input_z_dim);
3471  ok = ok && cfl->GetValue("filt-x-dim", &filt_x_dim);
3472  ok = ok && cfl->GetValue("filt-y-dim", &filt_y_dim);
3473  ok = ok && cfl->GetValue("filt-x-step", &filt_x_step);
3474  ok = ok && cfl->GetValue("filt-y-step", &filt_y_step);
3475 
3476  if (!ok)
3477  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
3478  // optional argument
3479  TensorVectorizationType input_vectorization;
3480  cfl->GetValue("input-vectorization-order", &input_vectorization_order);
3481  if (input_vectorization_order.compare("zyx") == 0) {
3482  input_vectorization = kZyx;
3483  } else if (input_vectorization_order.compare("yzx") == 0) {
3484  input_vectorization = kYzx;
3485  } else {
3486  KALDI_ERR << "Unknown or unsupported input vectorization order "
3487  << input_vectorization_order
3488  << " accepted candidates are 'yzx' and 'zyx'";
3489  }
3490 
3491  if (cfl->GetValue("matrix", &matrix_filename)) {
3492  // initialize from prefined parameter matrix
3493  Init(input_x_dim, input_y_dim, input_z_dim,
3494  filt_x_dim, filt_y_dim,
3495  filt_x_step, filt_y_step,
3496  input_vectorization,
3497  matrix_filename);
3498  } else {
3499  ok = ok && cfl->GetValue("num-filters", &num_filters);
3500  if (!ok)
3501  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
3502  // initialize from configuration
3503  int32 filter_input_dim = filt_x_dim * filt_y_dim * input_z_dim;
3504  BaseFloat param_stddev = 1.0 / std::sqrt(filter_input_dim), bias_stddev = 1.0;
3505  cfl->GetValue("param-stddev", &param_stddev);
3506  cfl->GetValue("bias-stddev", &bias_stddev);
3507  Init(input_x_dim, input_y_dim, input_z_dim,
3508  filt_x_dim, filt_y_dim, filt_x_step, filt_y_step, num_filters,
3509  input_vectorization, param_stddev, bias_stddev);
3510  }
3511  if (cfl->HasUnusedValues())
3512  KALDI_ERR << "Could not process these elements in initializer: "
3513  << cfl->UnusedValues();
3514  if (!ok)
3515  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
3516 }
void Init(int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, int32 num_filters, TensorVectorizationType input_vectorization, BaseFloat param_stddev, BaseFloat bias_stddev)
void InitLearningRatesFromConfig(ConfigLine *cfl)
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 InputDim ( ) const
virtual
void InputToInputPatches ( const CuMatrixBase< BaseFloat > &  in,
CuMatrix< BaseFloat > *  patches 
) const
private

Definition at line 3539 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::CopyCols(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, CuMatrixBase< Real >::NumCols(), kaldi::nnet3::YzxVectorIndex(), and kaldi::nnet3::ZyxVectorIndex().

Referenced by ConvolutionComponent::Propagate(), and ConvolutionComponent::Update().

3541  {
3542  int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_);
3543  int32 num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_);
3544  const int32 filt_x_step = filt_x_step_,
3545  filt_y_step = filt_y_step_,
3546  filt_x_dim = filt_x_dim_,
3547  filt_y_dim = filt_y_dim_,
3548  input_x_dim = input_x_dim_,
3549  input_y_dim = input_y_dim_,
3550  input_z_dim = input_z_dim_,
3551  filter_dim = filter_params_.NumCols();
3552 
3553  std::vector<int32> column_map(patches->NumCols());
3554  int32 column_map_size = column_map.size();
3555  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
3556  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
3557  int32 patch_number = x_step * num_y_steps + y_step;
3558  int32 patch_start_index = patch_number * filter_dim;
3559  for (int32 x = 0, index = patch_start_index; x < filt_x_dim; x++) {
3560  for (int32 y = 0; y < filt_y_dim; y++) {
3561  for (int32 z = 0; z < input_z_dim; z++, index++) {
3562  KALDI_ASSERT(index < column_map_size);
3563  if (input_vectorization_ == kZyx) {
3564  column_map[index] = ZyxVectorIndex(x_step * filt_x_step + x,
3565  y_step * filt_y_step + y, z,
3566  input_x_dim, input_y_dim,
3567  input_z_dim);
3568  } else if (input_vectorization_ == kYzx) {
3569  column_map[index] = YzxVectorIndex(x_step * filt_x_step + x,
3570  y_step * filt_y_step + y, z,
3571  input_x_dim, input_y_dim,
3572  input_z_dim);
3573  }
3574  }
3575  }
3576  }
3577  }
3578  }
3579  CuArray<int32> cu_cols(column_map);
3580  patches->CopyCols(in, cu_cols);
3581 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
TensorVectorizationType input_vectorization_
void CopyCols(const CuMatrixBase< Real > &src, const CuArray< MatrixIndexT > &indexes)
Copies column r from column indexes[r] of src.
Definition: cu-matrix.cc:2436
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
int32 YzxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
int32 ZyxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
const CuMatrix<BaseFloat>& LinearParams ( ) const
inline

Definition at line 1748 of file nnet-simple-component.h.

References ConvolutionComponent::filter_params_.

1748 { return filter_params_; }
int32 NumParameters ( ) const
virtual

The following new virtual function returns the total dimension of the parameters in this class.

Reimplemented from UpdatableComponent.

Definition at line 3969 of file nnet-simple-component.cc.

References ConvolutionComponent::filter_params_, CuMatrixBase< Real >::NumCols(), and CuMatrixBase< Real >::NumRows().

Referenced by ConvolutionComponent::UnVectorize(), and ConvolutionComponent::Vectorize().

3969  {
3970  return (filter_params_.NumCols() + 1) * filter_params_.NumRows();
3971 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
const ConvolutionComponent& operator= ( const ConvolutionComponent other)
private
int32 OutputDim ( ) const
virtual

Returns output-dimension of this component.

Implements Component.

Definition at line 3380 of file nnet-simple-component.cc.

References ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, and CuMatrixBase< Real >::NumRows().

3380  {
3381  int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_);
3382  int32 num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_);
3383  int32 num_filters = filter_params_.NumRows();
3384  return num_x_steps * num_y_steps * num_filters;
3385 }
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
void PerturbParams ( BaseFloat  stddev)
virtual

This function is to be used in testing.

It adds unit noise times "stddev" to the parameters of the component.

Implements UpdatableComponent.

Definition at line 3952 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

3952  {
3953  CuMatrix<BaseFloat> temp_filter_params(filter_params_);
3954  temp_filter_params.SetRandn();
3955  filter_params_.AddMat(stddev, temp_filter_params);
3956 
3957  CuVector<BaseFloat> temp_bias_params(bias_params_);
3958  temp_bias_params.SetRandn();
3959  bias_params_.AddVec(stddev, temp_bias_params);
3960 }
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:939
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1126
void * Propagate ( const ComponentPrecomputedIndexes indexes,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Propagate function.

Parameters
[in]indexesA pointer to some information output by this class's PrecomputeIndexes function (will be NULL for simple components, i.e. those that don't do things like splicing).
[in]inThe input to this component. Num-columns == InputDim().
[out]outThe output of this component. Num-columns == OutputDim(). Note: output of this component will be added to the initial value of "out" if Properties()&kPropagateAdds != 0; otherwise the output will be set and the initial value ignored. Each Component chooses whether it is more convenient implementation-wise to add or set, and the calling code has to deal with it.
Returns
Normally returns NULL, but may return a non-NULL value for components which have the flag kUsesMemo set. This value will be passed into the corresponding Backprop routine.

Implements Component.

Definition at line 3586 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddVecToRows(), ConvolutionComponent::bias_params_, CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::InputToInputPatches(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, kaldi::kUndefined, CuMatrixBase< Real >::NumCols(), and CuMatrixBase< Real >::NumRows().

3588  {
3589  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
3590  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
3591  num_filters = filter_params_.NumRows(),
3592  num_frames = in.NumRows(),
3593  filter_dim = filter_params_.NumCols();
3594  KALDI_ASSERT((*out).NumRows() == num_frames &&
3595  (*out).NumCols() == (num_filters * num_x_steps * num_y_steps));
3596 
3597  CuMatrix<BaseFloat> patches(num_frames,
3598  num_x_steps * num_y_steps * filter_dim,
3599  kUndefined);
3600  InputToInputPatches(in, &patches);
3601  CuSubMatrix<BaseFloat>* filter_params_elem = new CuSubMatrix<BaseFloat>(
3602  filter_params_, 0, filter_params_.NumRows(), 0, filter_params_.NumCols());
3603  std::vector<CuSubMatrix<BaseFloat>* > tgt_batch, patch_batch,
3604  filter_params_batch;
3605 
3606  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
3607  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
3608  int32 patch_number = x_step * num_y_steps + y_step;
3609  tgt_batch.push_back(new CuSubMatrix<BaseFloat>(
3610  out->ColRange(patch_number * num_filters, num_filters)));
3611  patch_batch.push_back(new CuSubMatrix<BaseFloat>(
3612  patches.ColRange(patch_number * filter_dim, filter_dim)));
3613  filter_params_batch.push_back(filter_params_elem);
3614  tgt_batch[patch_number]->AddVecToRows(1.0, bias_params_, 1.0); // add bias
3615  }
3616  }
3617  // apply all filters
3618  AddMatMatBatched<BaseFloat>(1.0, tgt_batch, patch_batch,
3619  kNoTrans, filter_params_batch,
3620  kTrans, 1.0);
3621  // release memory
3622  delete filter_params_elem;
3623  for (int32 p = 0; p < tgt_batch.size(); p++) {
3624  delete tgt_batch[p];
3625  delete patch_batch[p];
3626  }
3627  return NULL;
3628 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:544
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
void InputToInputPatches(const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual int32 Properties ( ) const
inlinevirtual
void Read ( std::istream &  is,
bool  binary 
)
virtual

Read function (used after we know the type of the Component); accepts input that is missing the token that describes the component type, in case it has already been consumed.

Implements Component.

Definition at line 3878 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, kaldi::nnet3::ExpectToken(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, ConvolutionComponent::is_gradient_, KALDI_ASSERT, CuVector< Real >::Read(), CuMatrix< Real >::Read(), kaldi::ReadBasicType(), kaldi::ReadToken(), and UpdatableComponent::ReadUpdatableCommon().

3878  {
3879  ReadUpdatableCommon(is, binary); // Read opening tag and learning rate.
3880  ExpectToken(is, binary, "<InputXDim>");
3881  ReadBasicType(is, binary, &input_x_dim_);
3882  ExpectToken(is, binary, "<InputYDim>");
3883  ReadBasicType(is, binary, &input_y_dim_);
3884  ExpectToken(is, binary, "<InputZDim>");
3885  ReadBasicType(is, binary, &input_z_dim_);
3886  ExpectToken(is, binary, "<FiltXDim>");
3887  ReadBasicType(is, binary, &filt_x_dim_);
3888  ExpectToken(is, binary, "<FiltYDim>");
3889  ReadBasicType(is, binary, &filt_y_dim_);
3890  ExpectToken(is, binary, "<FiltXStep>");
3891  ReadBasicType(is, binary, &filt_x_step_);
3892  ExpectToken(is, binary, "<FiltYStep>");
3893  ReadBasicType(is, binary, &filt_y_step_);
3894  ExpectToken(is, binary, "<InputVectorization>");
3895  int32 input_vectorization;
3896  ReadBasicType(is, binary, &input_vectorization);
3897  input_vectorization_ = static_cast<TensorVectorizationType>(input_vectorization);
3898  ExpectToken(is, binary, "<FilterParams>");
3899  filter_params_.Read(is, binary);
3900  ExpectToken(is, binary, "<BiasParams>");
3901  bias_params_.Read(is, binary);
3902  std::string tok;
3903  ReadToken(is, binary, &tok);
3904  if (tok == "<IsGradient>") {
3905  ReadBasicType(is, binary, &is_gradient_);
3906  ExpectToken(is, binary, "</ConvolutionComponent>");
3907  } else {
3908  is_gradient_ = false;
3909  KALDI_ASSERT(tok == "</ConvolutionComponent>");
3910  }
3911 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
void Read(std::istream &is, bool binary)
I/O.
Definition: cu-vector.cc:862
void ReadToken(std::istream &is, bool binary, std::string *str)
ReadToken gets the next token and puts it in str (exception on failure).
Definition: io-funcs.cc:154
TensorVectorizationType input_vectorization_
static void ExpectToken(const std::string &token, const std::string &what_we_are_parsing, const std::string **next_token)
std::string ReadUpdatableCommon(std::istream &is, bool binary)
void Read(std::istream &is, bool binary)
I/O functions.
Definition: cu-matrix.cc:459
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Resize ( int32  input_dim,
int32  output_dim 
)
void Scale ( BaseFloat  scale)
virtual

This virtual function when called by.

by "scale" when called by an UpdatableComponent. stores stats, like BatchNormComponent– it relates to scaling activation stats, not parameters.

Reimplemented from Component.

Definition at line 3631 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, CuVectorBase< Real >::Scale(), CuMatrixBase< Real >::Scale(), CuVectorBase< Real >::SetZero(), and CuMatrixBase< Real >::SetZero().

3631  {
3632  if (scale == 0.0) {
3635  } else {
3636  filter_params_.Scale(scale);
3637  bias_params_.Scale(scale);
3638  }
3639 }
void Scale(Real value)
Definition: cu-vector.cc:1105
void Scale(Real value)
Definition: cu-matrix.cc:608
void SetZero()
Math operations, some calling kernels.
Definition: cu-matrix.cc:474
void SetZero()
Math operations.
Definition: cu-vector.cc:988
void SetParams ( const VectorBase< BaseFloat > &  bias,
const MatrixBase< BaseFloat > &  filter 
)

Definition at line 3962 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, and CuMatrixBase< Real >::NumRows().

3963  {
3964  bias_params_ = bias;
3965  filter_params_ = filter;
3967 }
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual std::string Type ( ) const
inlinevirtual

Returns a string such as "SigmoidComponent", describing the type of the object.

Implements Component.

Definition at line 1707 of file nnet-simple-component.h.

1707 { return "ConvolutionComponent"; }
void UnVectorize ( const VectorBase< BaseFloat > &  params)
virtual

Converts the parameters from vector form.

Reimplemented from UpdatableComponent.

Definition at line 3979 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::CopyFromVec(), CuMatrixBase< Real >::CopyRowsFromVec(), VectorBase< Real >::Dim(), CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), ConvolutionComponent::NumParameters(), CuMatrixBase< Real >::NumRows(), and VectorBase< Real >::Range().

3979  {
3980  KALDI_ASSERT(params.Dim() == this->NumParameters());
3981  int32 num_filter_params = filter_params_.NumCols() * filter_params_.NumRows();
3982  filter_params_.CopyRowsFromVec(params.Range(0, num_filter_params));
3983  bias_params_.CopyFromVec(params.Range(num_filter_params, bias_params_.Dim()));
3984 }
void CopyRowsFromVec(const CuVectorBase< Real > &v)
This function has two modes of operation.
Definition: cu-matrix.cc:2146
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
void CopyFromVec(const CuVectorBase< Real > &src)
Copy functions; these will crash if the dimension do not match.
Definition: cu-vector.cc:970
virtual int32 NumParameters() const
The following new virtual function returns the total dimension of the parameters in this class...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:92
void Update ( const std::string &  debug_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
const std::vector< CuSubMatrix< BaseFloat > * > &  out_deriv_batch 
)

Definition at line 3804 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuMatrixBase< Real >::AddMatBlocks(), CuVectorBase< Real >::AddRowSumMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::InputToInputPatches(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kSetZero, kaldi::kTrans, kaldi::kUndefined, UpdatableComponent::learning_rate_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

Referenced by ConvolutionComponent::Backprop().

3807  {
3808  // useful dims
3809  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
3810  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
3811  num_filters = filter_params_.NumRows(),
3812  num_frames = out_deriv.NumRows(),
3813  filter_dim = filter_params_.NumCols();
3814  KALDI_ASSERT(out_deriv.NumRows() == num_frames &&
3815  out_deriv.NumCols() ==
3816  (num_filters * num_x_steps * num_y_steps));
3817 
3818 
3819  CuMatrix<BaseFloat> filters_grad;
3820  CuVector<BaseFloat> bias_grad;
3821 
3822  CuMatrix<BaseFloat> input_patches(num_frames,
3823  filter_dim * num_x_steps * num_y_steps,
3824  kUndefined);
3825  InputToInputPatches(in_value, &input_patches);
3826 
3827  filters_grad.Resize(num_filters, filter_dim, kSetZero); // reset
3828  bias_grad.Resize(num_filters, kSetZero); // reset
3829 
3830  // create a single large matrix holding the smaller matrices
3831  // from the vector container filters_grad_batch along the rows
3832  CuMatrix<BaseFloat> filters_grad_blocks_batch(
3833  num_x_steps * num_y_steps * filters_grad.NumRows(),
3834  filters_grad.NumCols());
3835 
3836  std::vector<CuSubMatrix<BaseFloat>* > filters_grad_batch, input_patch_batch;
3837 
3838  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
3839  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
3840  int32 patch_number = x_step * num_y_steps + y_step;
3841  filters_grad_batch.push_back(new CuSubMatrix<BaseFloat>(
3842  filters_grad_blocks_batch.RowRange(
3843  patch_number * filters_grad.NumRows(), filters_grad.NumRows())));
3844 
3845  input_patch_batch.push_back(new CuSubMatrix<BaseFloat>(
3846  input_patches.ColRange(patch_number * filter_dim, filter_dim)));
3847  }
3848  }
3849 
3850  AddMatMatBatched<BaseFloat>(1.0, filters_grad_batch, out_deriv_batch, kTrans,
3851  input_patch_batch, kNoTrans, 1.0);
3852 
3853  // add the row blocks together to filters_grad
3854  filters_grad.AddMatBlocks(1.0, filters_grad_blocks_batch);
3855 
3856  // create a matrix holding the col blocks sum of out_deriv
3857  CuMatrix<BaseFloat> out_deriv_col_blocks_sum(out_deriv.NumRows(),
3858  num_filters);
3859 
3860  // add the col blocks together to out_deriv_col_blocks_sum
3861  out_deriv_col_blocks_sum.AddMatBlocks(1.0, out_deriv);
3862 
3863  bias_grad.AddRowSumMat(1.0, out_deriv_col_blocks_sum, 1.0);
3864 
3865  // release memory
3866  for (int32 p = 0; p < input_patch_batch.size(); p++) {
3867  delete filters_grad_batch[p];
3868  delete input_patch_batch[p];
3869  }
3870 
3871  //
3872  // update
3873  //
3874  filter_params_.AddMat(learning_rate_, filters_grad);
3875  bias_params_.AddVec(learning_rate_, bias_grad);
3876 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
BaseFloat learning_rate_
learning rate (typically 0.0..0.01)
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:939
void InputToInputPatches(const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1126
void Update ( const std::string &  debug_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv 
)
void Vectorize ( VectorBase< BaseFloat > *  params) const
virtual

Turns the parameters into vector form.

We put the vector form on the CPU, because in the kinds of situations where we do this, we'll tend to use too much memory for the GPU.

Reimplemented from UpdatableComponent.

Definition at line 3973 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, VectorBase< Real >::Dim(), CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), ConvolutionComponent::NumParameters(), CuMatrixBase< Real >::NumRows(), and VectorBase< Real >::Range().

3973  {
3974  KALDI_ASSERT(params->Dim() == this->NumParameters());
3975  int32 num_filter_params = filter_params_.NumCols() * filter_params_.NumRows();
3976  params->Range(0, num_filter_params).CopyRowsFromMat(filter_params_);
3977  params->Range(num_filter_params, bias_params_.Dim()).CopyFromVec(bias_params_);
3978 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:67
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
virtual int32 NumParameters() const
The following new virtual function returns the total dimension of the parameters in this class...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:92
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Definition at line 3913 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, ConvolutionComponent::is_gradient_, CuVector< Real >::Write(), CuMatrixBase< Real >::Write(), kaldi::WriteBasicType(), kaldi::WriteToken(), and UpdatableComponent::WriteUpdatableCommon().

3913  {
3914  WriteUpdatableCommon(os, binary); // write opening tag and learning rate.
3915  WriteToken(os, binary, "<InputXDim>");
3916  WriteBasicType(os, binary, input_x_dim_);
3917  WriteToken(os, binary, "<InputYDim>");
3918  WriteBasicType(os, binary, input_y_dim_);
3919  WriteToken(os, binary, "<InputZDim>");
3920  WriteBasicType(os, binary, input_z_dim_);
3921  WriteToken(os, binary, "<FiltXDim>");
3922  WriteBasicType(os, binary, filt_x_dim_);
3923  WriteToken(os, binary, "<FiltYDim>");
3924  WriteBasicType(os, binary, filt_y_dim_);
3925  WriteToken(os, binary, "<FiltXStep>");
3926  WriteBasicType(os, binary, filt_x_step_);
3927  WriteToken(os, binary, "<FiltYStep>");
3928  WriteBasicType(os, binary, filt_y_step_);
3929  WriteToken(os, binary, "<InputVectorization>");
3930  WriteBasicType(os, binary, static_cast<int32>(input_vectorization_));
3931  WriteToken(os, binary, "<FilterParams>");
3932  filter_params_.Write(os, binary);
3933  WriteToken(os, binary, "<BiasParams>");
3934  bias_params_.Write(os, binary);
3935  WriteToken(os, binary, "<IsGradient>");
3936  WriteBasicType(os, binary, is_gradient_);
3937  WriteToken(os, binary, "</ConvolutionComponent>");
3938 }
TensorVectorizationType input_vectorization_
void Write(std::ostream &is, bool binary) const
Definition: cu-vector.cc:872
void WriteUpdatableCommon(std::ostream &is, bool binary) const
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
void WriteBasicType(std::ostream &os, bool binary, T t)
WriteBasicType is the name of the write function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:34
void Write(std::ostream &os, bool binary) const
Definition: cu-matrix.cc:467

Member Data Documentation


The documentation for this class was generated from the following files: