All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
ConvolutionComponent Class Reference

WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted. More...

#include <nnet-simple-component.h>

Inheritance diagram for ConvolutionComponent:
Collaboration diagram for ConvolutionComponent:

Public Types

enum  TensorVectorizationType { kYzx = 0, kZyx = 1 }
 

Public Member Functions

 ConvolutionComponent ()
 
 ConvolutionComponent (const ConvolutionComponent &component)
 
 ConvolutionComponent (const CuMatrixBase< BaseFloat > &filter_params, const CuVectorBase< BaseFloat > &bias_params, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, TensorVectorizationType input_vectorization, BaseFloat learning_rate)
 
virtual int32 InputDim () const
 Returns input-dimension of this component. More...
 
virtual int32 OutputDim () const
 Returns output-dimension of this component. More...
 
virtual std::string Info () const
 Returns some text-form information about this component, for diagnostics. More...
 
virtual void InitFromConfig (ConfigLine *cfl)
 Initialize, from a ConfigLine object. More...
 
virtual std::string Type () const
 Returns a string such as "SigmoidComponent", describing the type of the object. More...
 
virtual int32 Properties () const
 Return bitmask of the component's properties. More...
 
virtual void * Propagate (const ComponentPrecomputedIndexes *indexes, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Propagate function. More...
 
virtual void Backprop (const std::string &debug_info, const ComponentPrecomputedIndexes *indexes, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &, const CuMatrixBase< BaseFloat > &out_deriv, void *memo, Component *to_update_in, CuMatrixBase< BaseFloat > *in_deriv) const
 Backprop function; depending on which of the arguments 'to_update' and 'in_deriv' are non-NULL, this can compute input-data derivatives and/or perform model update. More...
 
void Update (const std::string &debug_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv, const std::vector< CuSubMatrix< BaseFloat > * > &out_deriv_batch)
 
virtual void Read (std::istream &is, bool binary)
 Read function (used after we know the type of the Component); accepts input that is missing the token that describes the component type, in case it has already been consumed. More...
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
virtual ComponentCopy () const
 Copies component (deep copy). More...
 
virtual void Scale (BaseFloat scale)
 This virtual function when called on – an UpdatableComponent scales the parameters by "scale" when called by an UpdatableComponent. More...
 
virtual void Add (BaseFloat alpha, const Component &other)
 This virtual function when called by – an UpdatableComponent adds the parameters of another updatable component, times some constant, to the current parameters. More...
 
virtual void PerturbParams (BaseFloat stddev)
 This function is to be used in testing. More...
 
virtual BaseFloat DotProduct (const UpdatableComponent &other) const
 Computes dot-product between parameters of two instances of a Component. More...
 
virtual int32 NumParameters () const
 The following new virtual function returns the total dimension of the parameters in this class. More...
 
virtual void Vectorize (VectorBase< BaseFloat > *params) const
 Turns the parameters into vector form. More...
 
virtual void UnVectorize (const VectorBase< BaseFloat > &params)
 Converts the parameters from vector form. More...
 
void SetParams (const VectorBase< BaseFloat > &bias, const MatrixBase< BaseFloat > &filter)
 
const CuVector< BaseFloat > & BiasParams () const
 
const CuMatrix< BaseFloat > & LinearParams () const
 
void Init (int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, int32 num_filters, TensorVectorizationType input_vectorization, BaseFloat param_stddev, BaseFloat bias_stddev)
 
void Init (int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, TensorVectorizationType input_vectorization, std::string matrix_filename)
 
void Resize (int32 input_dim, int32 output_dim)
 
void Update (const std::string &debug_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv)
 
- Public Member Functions inherited from UpdatableComponent
 UpdatableComponent (const UpdatableComponent &other)
 
 UpdatableComponent ()
 
virtual ~UpdatableComponent ()
 
virtual void SetUnderlyingLearningRate (BaseFloat lrate)
 Sets the learning rate of gradient descent- gets multiplied by learning_rate_factor_. More...
 
virtual void SetActualLearningRate (BaseFloat lrate)
 Sets the learning rate directly, bypassing learning_rate_factor_. More...
 
virtual void SetAsGradient ()
 Sets is_gradient_ to true and sets learning_rate_ to 1, ignoring learning_rate_factor_. More...
 
virtual BaseFloat LearningRateFactor ()
 
virtual void SetLearningRateFactor (BaseFloat lrate_factor)
 
void SetUpdatableConfigs (const UpdatableComponent &other)
 
virtual void FreezeNaturalGradient (bool freeze)
 freezes/unfreezes NaturalGradient updates, if applicable (to be overriden by components that use Natural Gradient). More...
 
BaseFloat LearningRate () const
 Gets the learning rate to be used in gradient descent. More...
 
BaseFloat MaxChange () const
 Returns the per-component max-change value, which is interpreted as the maximum change (in l2 norm) in parameters that is allowed per minibatch for this component. More...
 
void SetMaxChange (BaseFloat max_change)
 
BaseFloat L2Regularization () const
 Returns the l2 regularization constant, which may be set in any updatable component (usually from the config file). More...
 
void SetL2Regularization (BaseFloat a)
 
- Public Member Functions inherited from Component
virtual void StoreStats (const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, void *memo)
 This function may store stats on average activation values, and for some component types, the average value of the derivative of the nonlinearity. More...
 
virtual void ZeroStats ()
 Components that provide an implementation of StoreStats should also provide an implementation of ZeroStats(), to set those stats to zero. More...
 
virtual void GetInputIndexes (const MiscComputationInfo &misc_info, const Index &output_index, std::vector< Index > *desired_indexes) const
 This function only does something interesting for non-simple Components. More...
 
virtual bool IsComputable (const MiscComputationInfo &misc_info, const Index &output_index, const IndexSet &input_index_set, std::vector< Index > *used_inputs) const
 This function only does something interesting for non-simple Components, and it exists to make it possible to manage optionally-required inputs. More...
 
virtual void ReorderIndexes (std::vector< Index > *input_indexes, std::vector< Index > *output_indexes) const
 This function only does something interesting for non-simple Components. More...
 
virtual
ComponentPrecomputedIndexes
PrecomputeIndexes (const MiscComputationInfo &misc_info, const std::vector< Index > &input_indexes, const std::vector< Index > &output_indexes, bool need_backprop) const
 This function must return NULL for simple Components. More...
 
virtual void DeleteMemo (void *memo) const
 This virtual function only needs to be overwritten by Components that return a non-NULL memo from their Propagate() function. More...
 
 Component ()
 
virtual ~Component ()
 

Private Member Functions

void InputToInputPatches (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
 
void InderivPatchesToInderiv (const CuMatrix< BaseFloat > &in_deriv_patches, CuMatrixBase< BaseFloat > *in_deriv) const
 
const ConvolutionComponentoperator= (const ConvolutionComponent &other)
 

Private Attributes

int32 input_x_dim_
 
int32 input_y_dim_
 
int32 input_z_dim_
 
int32 filt_x_dim_
 
int32 filt_y_dim_
 
int32 filt_x_step_
 
int32 filt_y_step_
 
TensorVectorizationType input_vectorization_
 
CuMatrix< BaseFloatfilter_params_
 
CuVector< BaseFloatbias_params_
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream (works out its type). Dies on error. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Returns a new Component of the given type e.g. More...
 
- Protected Member Functions inherited from UpdatableComponent
void InitLearningRatesFromConfig (ConfigLine *cfl)
 
std::string ReadUpdatableCommon (std::istream &is, bool binary)
 
void WriteUpdatableCommon (std::ostream &is, bool binary) const
 
- Protected Attributes inherited from UpdatableComponent
BaseFloat learning_rate_
 learning rate (typically 0.0..0.01) More...
 
BaseFloat learning_rate_factor_
 learning rate factor (normally 1.0, but can be set to another < value so that when < you call SetLearningRate(), that value will be scaled by this factor. More...
 
BaseFloat l2_regularize_
 L2 regularization constant. More...
 
bool is_gradient_
 True if this component is to be treated as a gradient rather than as parameters. More...
 
BaseFloat max_change_
 configuration value for imposing max-change More...
 

Detailed Description

WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted.

ConvolutionalComponent implements 2d-convolution. It uses 3D filters on 3D inputs, but the 3D filters hop only over 2 dimensions as it has same size as the input along the 3rd dimension. Input : A matrix where each row is a vectorized 3D-tensor. The 3D tensor has dimensions x: (e.g. time) y: (e.g. frequency) z: (e.g. channels like features/delta/delta-delta)

The component supports input vectorizations of type zyx and yzx. The default vectorization type is zyx. e.g. for input vectorization of type zyx the input is vectorized by spanning axes z, y and x of the tensor in that order. Given 3d tensor A with sizes (2, 2, 2) along the three dimensions the zyx vectorized input looks like A(0,0,0) A(0,0,1) A(0,1,0) A(0,1,1) A(1,0,0) A(1,0,1) A(1,1,0) A(1,1,1)

Output : The output is also a 3D tensor vectorized in the zyx format. The channel axis (z) in the output corresponds to the output of different filters. The first channel corresponds to the first filter i.e., first row of the filter_params_ matrix.

Note: The component has to support yzx input vectorization as the binaries like add-deltas generate yz vectorized output. These input vectors are concatenated using the Append descriptor across time steps to form a yzx vectorized 3D tensor input. e.g. Append(Offset(input, -1), input, Offset(input, 1))

For information on the hyperparameters and parameters of this component see the variable declarations.

Propagation: ------------ Convolution operation consists of a dot-products between the filter tensor and input tensor patch, for various shifts of filter tensor along the x and y axes input tensor. (Note: there is no shift along z-axis as the filter and input tensor have same size along this axis).

For a particular shift (i,j) of the filter tensor along input tensor dimensions x and y, the elements of the input tensor which overlap with the filter form the input tensor patch. This patch is vectorized in zyx format. All the patches corresponding to various samples in the mini-batch are stacked into a matrix, where each row corresponds to one patch. Let this matrix be represented by X_{i,j}. The dot products with various filters are computed simultaneously by computing the matrix product with the filter_params_ matrix (W) Y_{i,j} = X_{i,j}*W^T. Each row of W corresponds to one filter 3D tensor vectorized in zyx format.

All the matrix products corresponding to various shifts (i,j) of the filter tensor are computed simultaneously using the AddMatMatBatched call of CuMatrixBase class.

BackPropagation: ---------------- Backpropagation to compute the input derivative ( X_{i,j}) consists of the a series of matrix products. {i,j} = {i,j}*W where {i,j} corresponds to the output derivative for a particular shift of the filter.

Once again these matrix products are computed simultaneously.

Update: ------- The weight gradient is computed as = {i,j} (X_{i,j}^T *{i,j})

Definition at line 1964 of file nnet-simple-component.h.

Member Enumeration Documentation

Enumerator
kYzx 
kZyx 

Definition at line 1966 of file nnet-simple-component.h.

Constructor & Destructor Documentation

Definition at line 3830 of file nnet-simple-component.cc.

3831  :
3832  UpdatableComponent(component),
3833  input_x_dim_(component.input_x_dim_),
3834  input_y_dim_(component.input_y_dim_),
3835  input_z_dim_(component.input_z_dim_),
3836  filt_x_dim_(component.filt_x_dim_),
3837  filt_y_dim_(component.filt_y_dim_),
3838  filt_x_step_(component.filt_x_step_),
3839  filt_y_step_(component.filt_y_step_),
3840  input_vectorization_(component.input_vectorization_),
3841  filter_params_(component.filter_params_),
3842  bias_params_(component.bias_params_) { }
TensorVectorizationType input_vectorization_
ConvolutionComponent ( const CuMatrixBase< BaseFloat > &  filter_params,
const CuVectorBase< BaseFloat > &  bias_params,
int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
TensorVectorizationType  input_vectorization,
BaseFloat  learning_rate 
)

Definition at line 3844 of file nnet-simple-component.cc.

References CuVectorBase< Real >::Dim(), UpdatableComponent::is_gradient_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and UpdatableComponent::SetUnderlyingLearningRate().

3851  :
3852  input_x_dim_(input_x_dim),
3853  input_y_dim_(input_y_dim),
3854  input_z_dim_(input_z_dim),
3855  filt_x_dim_(filt_x_dim),
3856  filt_y_dim_(filt_y_dim),
3857  filt_x_step_(filt_x_step),
3858  filt_y_step_(filt_y_step),
3859  input_vectorization_(input_vectorization),
3860  filter_params_(filter_params),
3861  bias_params_(bias_params){
3862  KALDI_ASSERT(filter_params.NumRows() == bias_params.Dim() &&
3863  bias_params.Dim() != 0);
3864  KALDI_ASSERT(filter_params.NumCols() == filt_x_dim * filt_y_dim * input_z_dim);
3865  SetUnderlyingLearningRate(learning_rate);
3866  is_gradient_ = false;
3867 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
TensorVectorizationType input_vectorization_
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
bool is_gradient_
True if this component is to be treated as a gradient rather than as parameters.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual void SetUnderlyingLearningRate(BaseFloat lrate)
Sets the learning rate of gradient descent- gets multiplied by learning_rate_factor_.

Member Function Documentation

void Add ( BaseFloat  alpha,
const Component other 
)
virtual

This virtual function when called by – an UpdatableComponent adds the parameters of another updatable component, times some constant, to the current parameters.

– a NonlinearComponent (or another component that stores stats, like BatchNormComponent)– it relates to adding stats. Otherwise it will normally do nothing.

Reimplemented from Component.

Definition at line 4137 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, and KALDI_ASSERT.

4137  {
4138  const ConvolutionComponent *other =
4139  dynamic_cast<const ConvolutionComponent*>(&other_in);
4140  KALDI_ASSERT(other != NULL);
4141  filter_params_.AddMat(alpha, other->filter_params_);
4142  bias_params_.AddVec(alpha, other->bias_params_);
4143 }
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:941
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1182
void Backprop ( const std::string &  debug_info,
const ComponentPrecomputedIndexes indexes,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
void *  memo,
Component to_update,
CuMatrixBase< BaseFloat > *  in_deriv 
) const
virtual

Backprop function; depending on which of the arguments 'to_update' and 'in_deriv' are non-NULL, this can compute input-data derivatives and/or perform model update.

Parameters
[in]debug_infoThe component name, to be printed out in any warning messages.
[in]indexesA pointer to some information output by this class's PrecomputeIndexes function (will be NULL for simple components, i.e. those that don't do things like splicing).
[in]in_valueThe matrix that was given as input to the Propagate function. Will be ignored (and may be empty) if Properties()&kBackpropNeedsInput == 0.
[in]out_valueThe matrix that was output from the Propagate function. Will be ignored (and may be empty) if Properties()&kBackpropNeedsOutput == 0
[in]out_derivThe derivative at the output of this component.
[in]memoThis will normally be NULL, but for component types that set the flag kUsesMemo, this will be the return value of the Propagate() function that corresponds to this Backprop() function. Ownership of any pointers is not transferred to the Backprop function; DeleteMemo() will be called to delete it.
[out]to_updateIf model update is desired, the Component to be updated, else NULL. Does not have to be identical to this. If supplied, you can assume that to_update->Properties() & kUpdatableComponent is nonzero.
[out]in_derivThe derivative at the input of this component, if needed (else NULL). If Properties()&kBackpropInPlace, may be the same matrix as out_deriv. If Properties()&kBackpropAdds, this is added to by the Backprop routine, else it is set. The component code chooses which mode to work in, based on convenience.

Implements Component.

Definition at line 4232 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::InderivPatchesToInderiv(), ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, KALDI_ASSERT, kaldi::kNoTrans, kaldi::kSetZero, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and ConvolutionComponent::Update().

4239  {
4240  ConvolutionComponent *to_update =
4241  dynamic_cast<ConvolutionComponent*>(to_update_in);
4242  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4243  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4244  num_filters = filter_params_.NumRows(),
4245  num_frames = out_deriv.NumRows(),
4246  filter_dim = filter_params_.NumCols();
4247 
4248  KALDI_ASSERT(out_deriv.NumRows() == num_frames &&
4249  out_deriv.NumCols() ==
4250  (num_filters * num_x_steps * num_y_steps));
4251 
4252  // Compute inderiv patches
4253  CuMatrix<BaseFloat> in_deriv_patches(num_frames,
4254  num_x_steps * num_y_steps * filter_dim,
4255  kSetZero);
4256 
4257  std::vector<CuSubMatrix<BaseFloat>* > patch_deriv_batch, out_deriv_batch,
4258  filter_params_batch;
4259  CuSubMatrix<BaseFloat>* filter_params_elem = new CuSubMatrix<BaseFloat>(
4260  filter_params_, 0, filter_params_.NumRows(), 0, filter_params_.NumCols());
4261 
4262  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4263  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4264  int32 patch_number = x_step * num_y_steps + y_step;
4265 
4266  patch_deriv_batch.push_back(new CuSubMatrix<BaseFloat>(
4267  in_deriv_patches.ColRange(
4268  patch_number * filter_dim, filter_dim)));
4269  out_deriv_batch.push_back(new CuSubMatrix<BaseFloat>(out_deriv.ColRange(
4270  patch_number * num_filters, num_filters)));
4271  filter_params_batch.push_back(filter_params_elem);
4272  }
4273  }
4274  AddMatMatBatched<BaseFloat>(1.0, patch_deriv_batch,
4275  out_deriv_batch, kNoTrans,
4276  filter_params_batch, kNoTrans, 0.0);
4277 
4278  if (in_deriv) {
4279  // combine the derivatives from the individual input deriv patches
4280  // to compute input deriv matrix
4281  InderivPatchesToInderiv(in_deriv_patches, in_deriv);
4282  }
4283 
4284  if (to_update != NULL) {
4285  to_update->Update(debug_info, in_value, out_deriv, out_deriv_batch);
4286  }
4287 
4288  // release memory
4289  delete filter_params_elem;
4290  for (int32 p = 0; p < patch_deriv_batch.size(); p++) {
4291  delete patch_deriv_batch[p];
4292  delete out_deriv_batch[p];
4293  }
4294 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:608
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InderivPatchesToInderiv(const CuMatrix< BaseFloat > &in_deriv_patches, CuMatrixBase< BaseFloat > *in_deriv) const
const CuVector<BaseFloat>& BiasParams ( ) const
inline

Definition at line 2029 of file nnet-simple-component.h.

References ConvolutionComponent::bias_params_.

2029 { return bias_params_; }
Component * Copy ( ) const
virtual

Copies component (deep copy).

Implements Component.

Definition at line 4442 of file nnet-simple-component.cc.

References ConvolutionComponent::ConvolutionComponent().

4442  {
4443  ConvolutionComponent *ans = new ConvolutionComponent(*this);
4444  return ans;
4445 }
BaseFloat DotProduct ( const UpdatableComponent other) const
virtual

Computes dot-product between parameters of two instances of a Component.

Can be used for computing parameter-norm of an UpdatableComponent.

Implements UpdatableComponent.

Definition at line 4435 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, kaldi::kTrans, kaldi::TraceMatMat(), and kaldi::VecVec().

4435  {
4436  const ConvolutionComponent *other =
4437  dynamic_cast<const ConvolutionComponent*>(&other_in);
4438  return TraceMatMat(filter_params_, other->filter_params_, kTrans)
4439  + VecVec(bias_params_, other->bias_params_);
4440 }
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans=kNoTrans)
We need to declare this here as it will be a friend function.
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:37
void InderivPatchesToInderiv ( const CuMatrix< BaseFloat > &  in_deriv_patches,
CuMatrixBase< BaseFloat > *  in_deriv 
) const
private

Definition at line 4176 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddCols(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, CuMatrixBase< Real >::NumCols(), kaldi::nnet3::RearrangeIndexes(), kaldi::nnet3::YzxVectorIndex(), and kaldi::nnet3::ZyxVectorIndex().

Referenced by ConvolutionComponent::Backprop().

4178  {
4179 
4180  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4181  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4182  filt_x_step = filt_x_step_,
4183  filt_y_step = filt_y_step_,
4184  filt_x_dim = filt_x_dim_,
4185  filt_y_dim = filt_y_dim_,
4186  input_x_dim = input_x_dim_,
4187  input_y_dim = input_y_dim_,
4188  input_z_dim = input_z_dim_,
4189  filter_dim = filter_params_.NumCols();
4190 
4191  // Compute the reverse column_map from the matrix with input
4192  // derivative patches to input derivative matrix
4193  std::vector<std::vector<int32> > reverse_column_map(in_deriv->NumCols());
4194  int32 rev_col_map_size = reverse_column_map.size();
4195  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4196  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4197  int32 patch_number = x_step * num_y_steps + y_step;
4198  int32 patch_start_index = patch_number * filter_dim;
4199  for (int32 x = 0, index = patch_start_index; x < filt_x_dim; x++) {
4200  for (int32 y = 0; y < filt_y_dim; y++) {
4201  for (int32 z = 0; z < input_z_dim; z++, index++) {
4202  int32 vector_index;
4203  if (input_vectorization_ == kZyx) {
4204  vector_index = ZyxVectorIndex(x_step * filt_x_step + x,
4205  y_step * filt_y_step + y, z,
4206  input_x_dim, input_y_dim,
4207  input_z_dim);
4208  } else {
4210  vector_index = YzxVectorIndex(x_step * filt_x_step + x,
4211  y_step * filt_y_step + y, z,
4212  input_x_dim, input_y_dim,
4213  input_z_dim);
4214  }
4215  KALDI_ASSERT(vector_index < rev_col_map_size);
4216  reverse_column_map[vector_index].push_back(index);
4217  }
4218  }
4219  }
4220  }
4221  }
4222  std::vector<std::vector<int32> > rearranged_column_map;
4223  RearrangeIndexes(reverse_column_map, &rearranged_column_map);
4224  for (int32 p = 0; p < rearranged_column_map.size(); p++) {
4225  CuArray<int32> cu_cols(rearranged_column_map[p]);
4226  in_deriv->AddCols(in_deriv_patches, cu_cols);
4227  }
4228 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
void AddCols(const CuMatrixBase< Real > &src, const CuArrayBase< MatrixIndexT > &indices)
Add column indices[r] of src to column r.
Definition: cu-matrix.cc:2668
TensorVectorizationType input_vectorization_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
int32 YzxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
int32 ZyxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
void RearrangeIndexes(const std::vector< std::vector< int32 > > &in, std::vector< std::vector< int32 > > *out)
std::string Info ( ) const
virtual

Returns some text-form information about this component, for diagnostics.

Starts with the type of the component. E.g. "SigmoidComponent dim=900", although most components will have much more info.

Reimplemented from UpdatableComponent.

Definition at line 3936 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, UpdatableComponent::Info(), ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, CuMatrixBase< Real >::NumRows(), and kaldi::nnet3::PrintParameterStats().

3936  {
3937  std::ostringstream stream;
3938  stream << UpdatableComponent::Info()
3939  << ", input-x-dim=" << input_x_dim_
3940  << ", input-y-dim=" << input_y_dim_
3941  << ", input-z-dim=" << input_z_dim_
3942  << ", filt-x-dim=" << filt_x_dim_
3943  << ", filt-y-dim=" << filt_y_dim_
3944  << ", filt-x-step=" << filt_x_step_
3945  << ", filt-y-step=" << filt_y_step_
3946  << ", input-vectorization=" << input_vectorization_
3947  << ", num-filters=" << filter_params_.NumRows();
3948  PrintParameterStats(stream, "filter-params", filter_params_);
3949  PrintParameterStats(stream, "bias-params", bias_params_, true);
3950  return stream.str();
3951 }
TensorVectorizationType input_vectorization_
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
virtual std::string Info() const
Returns some text-form information about this component, for diagnostics.
void PrintParameterStats(std::ostringstream &os, const std::string &name, const CuVectorBase< BaseFloat > &params, bool include_mean)
Print to 'os' some information about the mean and standard deviation of some parameters, used in Info() functions in nnet-simple-component.cc.
Definition: nnet-parse.cc:530
void Init ( int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
int32  num_filters,
TensorVectorizationType  input_vectorization,
BaseFloat  param_stddev,
BaseFloat  bias_stddev 
)

Definition at line 3883 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, CuVector< Real >::Resize(), CuMatrix< Real >::Resize(), CuVectorBase< Real >::Scale(), CuMatrixBase< Real >::Scale(), CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

Referenced by ConvolutionComponent::InitFromConfig().

3888  {
3889  input_x_dim_ = input_x_dim;
3890  input_y_dim_ = input_y_dim;
3891  input_z_dim_ = input_z_dim;
3892  filt_x_dim_ = filt_x_dim;
3893  filt_y_dim_ = filt_y_dim;
3894  filt_x_step_ = filt_x_step;
3895  filt_y_step_ = filt_y_step;
3896  input_vectorization_ = input_vectorization;
3899  int32 filter_dim = filt_x_dim_ * filt_y_dim_ * input_z_dim_;
3900  filter_params_.Resize(num_filters, filter_dim);
3901  bias_params_.Resize(num_filters);
3902  KALDI_ASSERT(param_stddev >= 0.0 && bias_stddev >= 0.0);
3904  filter_params_.Scale(param_stddev);
3906  bias_params_.Scale(bias_stddev);
3907 }
void Scale(Real value)
Definition: cu-vector.cc:1161
void Scale(Real value)
Definition: cu-matrix.cc:610
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:941
TensorVectorizationType input_vectorization_
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:49
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Init ( int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
TensorVectorizationType  input_vectorization,
std::string  matrix_filename 
)

Definition at line 3910 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::CopyColFromMat(), CuMatrixBase< Real >::CopyFromMat(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrixBase< Real >::Range(), kaldi::ReadKaldiObject(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

3915  {
3916  input_x_dim_ = input_x_dim;
3917  input_y_dim_ = input_y_dim;
3918  input_z_dim_ = input_z_dim;
3919  filt_x_dim_ = filt_x_dim;
3920  filt_y_dim_ = filt_y_dim;
3921  filt_x_step_ = filt_x_step;
3922  filt_y_step_ = filt_y_step;
3923  input_vectorization_ = input_vectorization;
3924  CuMatrix<BaseFloat> mat;
3925  ReadKaldiObject(matrix_filename, &mat);
3926  int32 filter_dim = (filt_x_dim_ * filt_y_dim_ * input_z_dim_);
3927  int32 num_filters = mat.NumRows();
3928  KALDI_ASSERT(mat.NumCols() == (filter_dim + 1));
3929  filter_params_.Resize(num_filters, filter_dim);
3930  bias_params_.Resize(num_filters);
3931  filter_params_.CopyFromMat(mat.Range(0, num_filters, 0, filter_dim));
3932  bias_params_.CopyColFromMat(mat, filter_dim);
3933 }
void CopyColFromMat(const CuMatrixBase< Real > &mat, MatrixIndexT col)
Definition: cu-vector.cc:101
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:339
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:941
TensorVectorizationType input_vectorization_
void ReadKaldiObject(const std::string &filename, Matrix< float > *m)
Definition: kaldi-io.cc:832
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:49
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InitFromConfig ( ConfigLine cfl)
virtual

Initialize, from a ConfigLine object.

Parameters
[in]cflA ConfigLine containing any parameters that are needed for initialization. For example: "dim=100 param-stddev=0.1"

Implements Component.

Definition at line 3954 of file nnet-simple-component.cc.

References ConfigLine::GetValue(), ConfigLine::HasUnusedValues(), ConvolutionComponent::Init(), UpdatableComponent::InitLearningRatesFromConfig(), KALDI_ERR, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, ConfigLine::UnusedValues(), and ConfigLine::WholeLine().

3954  {
3955  bool ok = true;
3956  std::string matrix_filename;
3957  int32 input_x_dim = -1, input_y_dim = -1, input_z_dim = -1,
3958  filt_x_dim = -1, filt_y_dim = -1,
3959  filt_x_step = -1, filt_y_step = -1,
3960  num_filters = -1;
3961  std::string input_vectorization_order = "zyx";
3963  ok = ok && cfl->GetValue("input-x-dim", &input_x_dim);
3964  ok = ok && cfl->GetValue("input-y-dim", &input_y_dim);
3965  ok = ok && cfl->GetValue("input-z-dim", &input_z_dim);
3966  ok = ok && cfl->GetValue("filt-x-dim", &filt_x_dim);
3967  ok = ok && cfl->GetValue("filt-y-dim", &filt_y_dim);
3968  ok = ok && cfl->GetValue("filt-x-step", &filt_x_step);
3969  ok = ok && cfl->GetValue("filt-y-step", &filt_y_step);
3970 
3971  if (!ok)
3972  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
3973  // optional argument
3974  TensorVectorizationType input_vectorization;
3975  cfl->GetValue("input-vectorization-order", &input_vectorization_order);
3976  if (input_vectorization_order.compare("zyx") == 0) {
3977  input_vectorization = kZyx;
3978  } else if (input_vectorization_order.compare("yzx") == 0) {
3979  input_vectorization = kYzx;
3980  } else {
3981  KALDI_ERR << "Unknown or unsupported input vectorization order "
3982  << input_vectorization_order
3983  << " accepted candidates are 'yzx' and 'zyx'";
3984  }
3985 
3986  if (cfl->GetValue("matrix", &matrix_filename)) {
3987  // initialize from prefined parameter matrix
3988  Init(input_x_dim, input_y_dim, input_z_dim,
3989  filt_x_dim, filt_y_dim,
3990  filt_x_step, filt_y_step,
3991  input_vectorization,
3992  matrix_filename);
3993  } else {
3994  ok = ok && cfl->GetValue("num-filters", &num_filters);
3995  if (!ok)
3996  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
3997  // initialize from configuration
3998  int32 filter_input_dim = filt_x_dim * filt_y_dim * input_z_dim;
3999  BaseFloat param_stddev = 1.0 / std::sqrt(filter_input_dim), bias_stddev = 1.0;
4000  cfl->GetValue("param-stddev", &param_stddev);
4001  cfl->GetValue("bias-stddev", &bias_stddev);
4002  Init(input_x_dim, input_y_dim, input_z_dim,
4003  filt_x_dim, filt_y_dim, filt_x_step, filt_y_step, num_filters,
4004  input_vectorization, param_stddev, bias_stddev);
4005  }
4006  if (cfl->HasUnusedValues())
4007  KALDI_ERR << "Could not process these elements in initializer: "
4008  << cfl->UnusedValues();
4009  if (!ok)
4010  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
4011 }
void Init(int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, int32 num_filters, TensorVectorizationType input_vectorization, BaseFloat param_stddev, BaseFloat bias_stddev)
void InitLearningRatesFromConfig(ConfigLine *cfl)
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 InputDim ( ) const
virtual
void InputToInputPatches ( const CuMatrixBase< BaseFloat > &  in,
CuMatrix< BaseFloat > *  patches 
) const
private

Definition at line 4034 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::CopyCols(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, CuMatrixBase< Real >::NumCols(), kaldi::nnet3::YzxVectorIndex(), and kaldi::nnet3::ZyxVectorIndex().

Referenced by ConvolutionComponent::Propagate(), and ConvolutionComponent::Update().

4036  {
4037  int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_);
4038  int32 num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_);
4039  const int32 filt_x_step = filt_x_step_,
4040  filt_y_step = filt_y_step_,
4041  filt_x_dim = filt_x_dim_,
4042  filt_y_dim = filt_y_dim_,
4043  input_x_dim = input_x_dim_,
4044  input_y_dim = input_y_dim_,
4045  input_z_dim = input_z_dim_,
4046  filter_dim = filter_params_.NumCols();
4047 
4048  std::vector<int32> column_map(patches->NumCols());
4049  int32 column_map_size = column_map.size();
4050  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4051  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4052  int32 patch_number = x_step * num_y_steps + y_step;
4053  int32 patch_start_index = patch_number * filter_dim;
4054  for (int32 x = 0, index = patch_start_index; x < filt_x_dim; x++) {
4055  for (int32 y = 0; y < filt_y_dim; y++) {
4056  for (int32 z = 0; z < input_z_dim; z++, index++) {
4057  KALDI_ASSERT(index < column_map_size);
4058  if (input_vectorization_ == kZyx) {
4059  column_map[index] = ZyxVectorIndex(x_step * filt_x_step + x,
4060  y_step * filt_y_step + y, z,
4061  input_x_dim, input_y_dim,
4062  input_z_dim);
4063  } else if (input_vectorization_ == kYzx) {
4064  column_map[index] = YzxVectorIndex(x_step * filt_x_step + x,
4065  y_step * filt_y_step + y, z,
4066  input_x_dim, input_y_dim,
4067  input_z_dim);
4068  }
4069  }
4070  }
4071  }
4072  }
4073  }
4074  CuArray<int32> cu_cols(column_map);
4075  patches->CopyCols(in, cu_cols);
4076 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
TensorVectorizationType input_vectorization_
void CopyCols(const CuMatrixBase< Real > &src, const CuArrayBase< MatrixIndexT > &indexes)
Copies column r from column indexes[r] of src.
Definition: cu-matrix.cc:2623
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
int32 YzxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
int32 ZyxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
const CuMatrix<BaseFloat>& LinearParams ( ) const
inline

Definition at line 2030 of file nnet-simple-component.h.

References ConvolutionComponent::filter_params_.

2030 { return filter_params_; }
int32 NumParameters ( ) const
virtual

The following new virtual function returns the total dimension of the parameters in this class.

Reimplemented from UpdatableComponent.

Definition at line 4464 of file nnet-simple-component.cc.

References ConvolutionComponent::filter_params_, CuMatrixBase< Real >::NumCols(), and CuMatrixBase< Real >::NumRows().

Referenced by ConvolutionComponent::UnVectorize(), and ConvolutionComponent::Vectorize().

4464  {
4465  return (filter_params_.NumCols() + 1) * filter_params_.NumRows();
4466 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
const ConvolutionComponent& operator= ( const ConvolutionComponent other)
private
int32 OutputDim ( ) const
virtual

Returns output-dimension of this component.

Implements Component.

Definition at line 3875 of file nnet-simple-component.cc.

References ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, and CuMatrixBase< Real >::NumRows().

3875  {
3876  int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_);
3877  int32 num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_);
3878  int32 num_filters = filter_params_.NumRows();
3879  return num_x_steps * num_y_steps * num_filters;
3880 }
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
void PerturbParams ( BaseFloat  stddev)
virtual

This function is to be used in testing.

It adds unit noise times "stddev" to the parameters of the component.

Implements UpdatableComponent.

Definition at line 4447 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

4447  {
4448  CuMatrix<BaseFloat> temp_filter_params(filter_params_);
4449  temp_filter_params.SetRandn();
4450  filter_params_.AddMat(stddev, temp_filter_params);
4451 
4452  CuVector<BaseFloat> temp_bias_params(bias_params_);
4453  temp_bias_params.SetRandn();
4454  bias_params_.AddVec(stddev, temp_bias_params);
4455 }
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:941
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1182
void * Propagate ( const ComponentPrecomputedIndexes indexes,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Propagate function.

Parameters
[in]indexesA pointer to some information output by this class's PrecomputeIndexes function (will be NULL for simple components, i.e. those that don't do things like splicing).
[in]inThe input to this component. Num-columns == InputDim().
[out]outThe output of this component. Num-columns == OutputDim(). Note: output of this component will be added to the initial value of "out" if Properties()&kPropagateAdds != 0; otherwise the output will be set and the initial value ignored. Each Component chooses whether it is more convenient implementation-wise to add or set, and the calling code has to deal with it.
Returns
Normally returns NULL, but may return a non-NULL value for components which have the flag kUsesMemo set. This value will be passed into the corresponding Backprop routine.

Implements Component.

Definition at line 4081 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddVecToRows(), ConvolutionComponent::bias_params_, CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::InputToInputPatches(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, kaldi::kUndefined, CuMatrixBase< Real >::NumCols(), and CuMatrixBase< Real >::NumRows().

4083  {
4084  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4085  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4086  num_filters = filter_params_.NumRows(),
4087  num_frames = in.NumRows(),
4088  filter_dim = filter_params_.NumCols();
4089  KALDI_ASSERT((*out).NumRows() == num_frames &&
4090  (*out).NumCols() == (num_filters * num_x_steps * num_y_steps));
4091 
4092  CuMatrix<BaseFloat> patches(num_frames,
4093  num_x_steps * num_y_steps * filter_dim,
4094  kUndefined);
4095  InputToInputPatches(in, &patches);
4096  CuSubMatrix<BaseFloat>* filter_params_elem = new CuSubMatrix<BaseFloat>(
4097  filter_params_, 0, filter_params_.NumRows(), 0, filter_params_.NumCols());
4098  std::vector<CuSubMatrix<BaseFloat>* > tgt_batch, patch_batch,
4099  filter_params_batch;
4100 
4101  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4102  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4103  int32 patch_number = x_step * num_y_steps + y_step;
4104  tgt_batch.push_back(new CuSubMatrix<BaseFloat>(
4105  out->ColRange(patch_number * num_filters, num_filters)));
4106  patch_batch.push_back(new CuSubMatrix<BaseFloat>(
4107  patches.ColRange(patch_number * filter_dim, filter_dim)));
4108  filter_params_batch.push_back(filter_params_elem);
4109  tgt_batch[patch_number]->AddVecToRows(1.0, bias_params_, 1.0); // add bias
4110  }
4111  }
4112  // apply all filters
4113  AddMatMatBatched<BaseFloat>(1.0, tgt_batch, patch_batch,
4114  kNoTrans, filter_params_batch,
4115  kTrans, 1.0);
4116  // release memory
4117  delete filter_params_elem;
4118  for (int32 p = 0; p < tgt_batch.size(); p++) {
4119  delete tgt_batch[p];
4120  delete patch_batch[p];
4121  }
4122  return NULL;
4123 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:608
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
void InputToInputPatches(const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual int32 Properties ( ) const
inlinevirtual
void Read ( std::istream &  is,
bool  binary 
)
virtual

Read function (used after we know the type of the Component); accepts input that is missing the token that describes the component type, in case it has already been consumed.

Implements Component.

Definition at line 4373 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, kaldi::nnet3::ExpectToken(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, UpdatableComponent::is_gradient_, KALDI_ASSERT, CuVector< Real >::Read(), CuMatrix< Real >::Read(), kaldi::ReadBasicType(), kaldi::ReadToken(), and UpdatableComponent::ReadUpdatableCommon().

4373  {
4374  ReadUpdatableCommon(is, binary); // Read opening tag and learning rate.
4375  ExpectToken(is, binary, "<InputXDim>");
4376  ReadBasicType(is, binary, &input_x_dim_);
4377  ExpectToken(is, binary, "<InputYDim>");
4378  ReadBasicType(is, binary, &input_y_dim_);
4379  ExpectToken(is, binary, "<InputZDim>");
4380  ReadBasicType(is, binary, &input_z_dim_);
4381  ExpectToken(is, binary, "<FiltXDim>");
4382  ReadBasicType(is, binary, &filt_x_dim_);
4383  ExpectToken(is, binary, "<FiltYDim>");
4384  ReadBasicType(is, binary, &filt_y_dim_);
4385  ExpectToken(is, binary, "<FiltXStep>");
4386  ReadBasicType(is, binary, &filt_x_step_);
4387  ExpectToken(is, binary, "<FiltYStep>");
4388  ReadBasicType(is, binary, &filt_y_step_);
4389  ExpectToken(is, binary, "<InputVectorization>");
4390  int32 input_vectorization;
4391  ReadBasicType(is, binary, &input_vectorization);
4392  input_vectorization_ = static_cast<TensorVectorizationType>(input_vectorization);
4393  ExpectToken(is, binary, "<FilterParams>");
4394  filter_params_.Read(is, binary);
4395  ExpectToken(is, binary, "<BiasParams>");
4396  bias_params_.Read(is, binary);
4397  std::string tok;
4398  ReadToken(is, binary, &tok);
4399  if (tok == "<IsGradient>") {
4400  ReadBasicType(is, binary, &is_gradient_);
4401  ExpectToken(is, binary, "</ConvolutionComponent>");
4402  } else {
4403  is_gradient_ = false;
4404  KALDI_ASSERT(tok == "</ConvolutionComponent>");
4405  }
4406 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
void Read(std::istream &is, bool binary)
I/O.
Definition: cu-vector.cc:911
void ReadToken(std::istream &is, bool binary, std::string *str)
ReadToken gets the next token and puts it in str (exception on failure).
Definition: io-funcs.cc:154
TensorVectorizationType input_vectorization_
static void ExpectToken(const std::string &token, const std::string &what_we_are_parsing, const std::string **next_token)
std::string ReadUpdatableCommon(std::istream &is, bool binary)
void Read(std::istream &is, bool binary)
I/O functions.
Definition: cu-matrix.cc:461
bool is_gradient_
True if this component is to be treated as a gradient rather than as parameters.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Resize ( int32  input_dim,
int32  output_dim 
)
void Scale ( BaseFloat  scale)
virtual

This virtual function when called on – an UpdatableComponent scales the parameters by "scale" when called by an UpdatableComponent.

– a Nonlinear component (or another component that stores stats, like BatchNormComponent)– it relates to scaling activation stats, not parameters. Otherwise it will normally do nothing.

Reimplemented from Component.

Definition at line 4126 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, CuVectorBase< Real >::Scale(), CuMatrixBase< Real >::Scale(), CuVectorBase< Real >::SetZero(), and CuMatrixBase< Real >::SetZero().

4126  {
4127  if (scale == 0.0) {
4130  } else {
4131  filter_params_.Scale(scale);
4132  bias_params_.Scale(scale);
4133  }
4134 }
void Scale(Real value)
Definition: cu-vector.cc:1161
void Scale(Real value)
Definition: cu-matrix.cc:610
void SetZero()
Math operations, some calling kernels.
Definition: cu-matrix.cc:476
void SetZero()
Math operations.
Definition: cu-vector.cc:1044
void SetParams ( const VectorBase< BaseFloat > &  bias,
const MatrixBase< BaseFloat > &  filter 
)

Definition at line 4457 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, and CuMatrixBase< Real >::NumRows().

4458  {
4459  bias_params_ = bias;
4460  filter_params_ = filter;
4462 }
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual std::string Type ( ) const
inlinevirtual

Returns a string such as "SigmoidComponent", describing the type of the object.

Implements Component.

Definition at line 1989 of file nnet-simple-component.h.

1989 { return "ConvolutionComponent"; }
void UnVectorize ( const VectorBase< BaseFloat > &  params)
virtual

Converts the parameters from vector form.

Reimplemented from UpdatableComponent.

Definition at line 4474 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::CopyFromVec(), CuMatrixBase< Real >::CopyRowsFromVec(), VectorBase< Real >::Dim(), CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), ConvolutionComponent::NumParameters(), CuMatrixBase< Real >::NumRows(), and VectorBase< Real >::Range().

4474  {
4475  KALDI_ASSERT(params.Dim() == this->NumParameters());
4476  int32 num_filter_params = filter_params_.NumCols() * filter_params_.NumRows();
4477  filter_params_.CopyRowsFromVec(params.Range(0, num_filter_params));
4478  bias_params_.CopyFromVec(params.Range(num_filter_params, bias_params_.Dim()));
4479 }
void CopyRowsFromVec(const CuVectorBase< Real > &v)
This function has two modes of operation.
Definition: cu-matrix.cc:2282
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
void CopyFromVec(const CuVectorBase< Real > &src)
Copy functions; these will crash if the dimension do not match.
Definition: cu-vector.cc:1026
virtual int32 NumParameters() const
The following new virtual function returns the total dimension of the parameters in this class...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:63
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:93
void Update ( const std::string &  debug_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
const std::vector< CuSubMatrix< BaseFloat > * > &  out_deriv_batch 
)

Definition at line 4299 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuMatrixBase< Real >::AddMatBlocks(), CuVectorBase< Real >::AddRowSumMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::InputToInputPatches(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kSetZero, kaldi::kTrans, kaldi::kUndefined, UpdatableComponent::learning_rate_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

Referenced by ConvolutionComponent::Backprop().

4302  {
4303  // useful dims
4304  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4305  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4306  num_filters = filter_params_.NumRows(),
4307  num_frames = out_deriv.NumRows(),
4308  filter_dim = filter_params_.NumCols();
4309  KALDI_ASSERT(out_deriv.NumRows() == num_frames &&
4310  out_deriv.NumCols() ==
4311  (num_filters * num_x_steps * num_y_steps));
4312 
4313 
4314  CuMatrix<BaseFloat> filters_grad;
4315  CuVector<BaseFloat> bias_grad;
4316 
4317  CuMatrix<BaseFloat> input_patches(num_frames,
4318  filter_dim * num_x_steps * num_y_steps,
4319  kUndefined);
4320  InputToInputPatches(in_value, &input_patches);
4321 
4322  filters_grad.Resize(num_filters, filter_dim, kSetZero); // reset
4323  bias_grad.Resize(num_filters, kSetZero); // reset
4324 
4325  // create a single large matrix holding the smaller matrices
4326  // from the vector container filters_grad_batch along the rows
4327  CuMatrix<BaseFloat> filters_grad_blocks_batch(
4328  num_x_steps * num_y_steps * filters_grad.NumRows(),
4329  filters_grad.NumCols());
4330 
4331  std::vector<CuSubMatrix<BaseFloat>* > filters_grad_batch, input_patch_batch;
4332 
4333  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4334  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4335  int32 patch_number = x_step * num_y_steps + y_step;
4336  filters_grad_batch.push_back(new CuSubMatrix<BaseFloat>(
4337  filters_grad_blocks_batch.RowRange(
4338  patch_number * filters_grad.NumRows(), filters_grad.NumRows())));
4339 
4340  input_patch_batch.push_back(new CuSubMatrix<BaseFloat>(
4341  input_patches.ColRange(patch_number * filter_dim, filter_dim)));
4342  }
4343  }
4344 
4345  AddMatMatBatched<BaseFloat>(1.0, filters_grad_batch, out_deriv_batch, kTrans,
4346  input_patch_batch, kNoTrans, 1.0);
4347 
4348  // add the row blocks together to filters_grad
4349  filters_grad.AddMatBlocks(1.0, filters_grad_blocks_batch);
4350 
4351  // create a matrix holding the col blocks sum of out_deriv
4352  CuMatrix<BaseFloat> out_deriv_col_blocks_sum(out_deriv.NumRows(),
4353  num_filters);
4354 
4355  // add the col blocks together to out_deriv_col_blocks_sum
4356  out_deriv_col_blocks_sum.AddMatBlocks(1.0, out_deriv);
4357 
4358  bias_grad.AddRowSumMat(1.0, out_deriv_col_blocks_sum, 1.0);
4359 
4360  // release memory
4361  for (int32 p = 0; p < input_patch_batch.size(); p++) {
4362  delete filters_grad_batch[p];
4363  delete input_patch_batch[p];
4364  }
4365 
4366  //
4367  // update
4368  //
4369  filter_params_.AddMat(learning_rate_, filters_grad);
4370  bias_params_.AddVec(learning_rate_, bias_grad);
4371 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
BaseFloat learning_rate_
learning rate (typically 0.0..0.01)
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:941
void InputToInputPatches(const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1182
void Update ( const std::string &  debug_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv 
)
void Vectorize ( VectorBase< BaseFloat > *  params) const
virtual

Turns the parameters into vector form.

We put the vector form on the CPU, because in the kinds of situations where we do this, we'll tend to use too much memory for the GPU.

Reimplemented from UpdatableComponent.

Definition at line 4468 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, VectorBase< Real >::Dim(), CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), ConvolutionComponent::NumParameters(), CuMatrixBase< Real >::NumRows(), and VectorBase< Real >::Range().

4468  {
4469  KALDI_ASSERT(params->Dim() == this->NumParameters());
4470  int32 num_filter_params = filter_params_.NumCols() * filter_params_.NumRows();
4471  params->Range(0, num_filter_params).CopyRowsFromMat(filter_params_);
4472  params->Range(num_filter_params, bias_params_.Dim()).CopyFromVec(bias_params_);
4473 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
virtual int32 NumParameters() const
The following new virtual function returns the total dimension of the parameters in this class...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:63
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:93
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Definition at line 4408 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, UpdatableComponent::is_gradient_, CuVector< Real >::Write(), CuMatrixBase< Real >::Write(), kaldi::WriteBasicType(), kaldi::WriteToken(), and UpdatableComponent::WriteUpdatableCommon().

4408  {
4409  WriteUpdatableCommon(os, binary); // write opening tag and learning rate.
4410  WriteToken(os, binary, "<InputXDim>");
4411  WriteBasicType(os, binary, input_x_dim_);
4412  WriteToken(os, binary, "<InputYDim>");
4413  WriteBasicType(os, binary, input_y_dim_);
4414  WriteToken(os, binary, "<InputZDim>");
4415  WriteBasicType(os, binary, input_z_dim_);
4416  WriteToken(os, binary, "<FiltXDim>");
4417  WriteBasicType(os, binary, filt_x_dim_);
4418  WriteToken(os, binary, "<FiltYDim>");
4419  WriteBasicType(os, binary, filt_y_dim_);
4420  WriteToken(os, binary, "<FiltXStep>");
4421  WriteBasicType(os, binary, filt_x_step_);
4422  WriteToken(os, binary, "<FiltYStep>");
4423  WriteBasicType(os, binary, filt_y_step_);
4424  WriteToken(os, binary, "<InputVectorization>");
4425  WriteBasicType(os, binary, static_cast<int32>(input_vectorization_));
4426  WriteToken(os, binary, "<FilterParams>");
4427  filter_params_.Write(os, binary);
4428  WriteToken(os, binary, "<BiasParams>");
4429  bias_params_.Write(os, binary);
4430  WriteToken(os, binary, "<IsGradient>");
4431  WriteBasicType(os, binary, is_gradient_);
4432  WriteToken(os, binary, "</ConvolutionComponent>");
4433 }
TensorVectorizationType input_vectorization_
void Write(std::ostream &is, bool binary) const
Definition: cu-vector.cc:921
void WriteUpdatableCommon(std::ostream &is, bool binary) const
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
bool is_gradient_
True if this component is to be treated as a gradient rather than as parameters.
void WriteBasicType(std::ostream &os, bool binary, T t)
WriteBasicType is the name of the write function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:34
void Write(std::ostream &os, bool binary) const
Definition: cu-matrix.cc:469

Member Data Documentation


The documentation for this class was generated from the following files: