All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
ConvolutionComponent Class Reference

WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted. More...

#include <nnet-simple-component.h>

Inheritance diagram for ConvolutionComponent:
Collaboration diagram for ConvolutionComponent:

Public Types

enum  TensorVectorizationType { kYzx = 0, kZyx = 1 }
 

Public Member Functions

 ConvolutionComponent ()
 
 ConvolutionComponent (const ConvolutionComponent &component)
 
 ConvolutionComponent (const CuMatrixBase< BaseFloat > &filter_params, const CuVectorBase< BaseFloat > &bias_params, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, TensorVectorizationType input_vectorization, BaseFloat learning_rate)
 
virtual int32 InputDim () const
 Returns input-dimension of this component. More...
 
virtual int32 OutputDim () const
 Returns output-dimension of this component. More...
 
virtual std::string Info () const
 Returns some text-form information about this component, for diagnostics. More...
 
virtual void InitFromConfig (ConfigLine *cfl)
 Initialize, from a ConfigLine object. More...
 
virtual std::string Type () const
 Returns a string such as "SigmoidComponent", describing the type of the object. More...
 
virtual int32 Properties () const
 Return bitmask of the component's properties. More...
 
virtual void * Propagate (const ComponentPrecomputedIndexes *indexes, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Propagate function. More...
 
virtual void Backprop (const std::string &debug_info, const ComponentPrecomputedIndexes *indexes, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &, const CuMatrixBase< BaseFloat > &out_deriv, void *memo, Component *to_update_in, CuMatrixBase< BaseFloat > *in_deriv) const
 Backprop function; depending on which of the arguments 'to_update' and 'in_deriv' are non-NULL, this can compute input-data derivatives and/or perform model update. More...
 
void Update (const std::string &debug_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv, const std::vector< CuSubMatrix< BaseFloat > * > &out_deriv_batch)
 
virtual void Read (std::istream &is, bool binary)
 Read function (used after we know the type of the Component); accepts input that is missing the token that describes the component type, in case it has already been consumed. More...
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
virtual ComponentCopy () const
 Copies component (deep copy). More...
 
virtual void Scale (BaseFloat scale)
 This virtual function when called on – an UpdatableComponent scales the parameters by "scale" when called by an UpdatableComponent. More...
 
virtual void Add (BaseFloat alpha, const Component &other)
 This virtual function when called by – an UpdatableComponent adds the parameters of another updatable component, times some constant, to the current parameters. More...
 
virtual void PerturbParams (BaseFloat stddev)
 This function is to be used in testing. More...
 
virtual BaseFloat DotProduct (const UpdatableComponent &other) const
 Computes dot-product between parameters of two instances of a Component. More...
 
virtual int32 NumParameters () const
 The following new virtual function returns the total dimension of the parameters in this class. More...
 
virtual void Vectorize (VectorBase< BaseFloat > *params) const
 Turns the parameters into vector form. More...
 
virtual void UnVectorize (const VectorBase< BaseFloat > &params)
 Converts the parameters from vector form. More...
 
void SetParams (const VectorBase< BaseFloat > &bias, const MatrixBase< BaseFloat > &filter)
 
const CuVector< BaseFloat > & BiasParams () const
 
const CuMatrix< BaseFloat > & LinearParams () const
 
void Init (int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, int32 num_filters, TensorVectorizationType input_vectorization, BaseFloat param_stddev, BaseFloat bias_stddev)
 
void Init (int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, TensorVectorizationType input_vectorization, std::string matrix_filename)
 
void Resize (int32 input_dim, int32 output_dim)
 
void Update (const std::string &debug_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_deriv)
 
- Public Member Functions inherited from UpdatableComponent
 UpdatableComponent (const UpdatableComponent &other)
 
 UpdatableComponent ()
 
virtual ~UpdatableComponent ()
 
virtual void SetUnderlyingLearningRate (BaseFloat lrate)
 Sets the learning rate of gradient descent- gets multiplied by learning_rate_factor_. More...
 
virtual void SetActualLearningRate (BaseFloat lrate)
 Sets the learning rate directly, bypassing learning_rate_factor_. More...
 
virtual void SetAsGradient ()
 Sets is_gradient_ to true and sets learning_rate_ to 1, ignoring learning_rate_factor_. More...
 
virtual BaseFloat LearningRateFactor ()
 
virtual void SetLearningRateFactor (BaseFloat lrate_factor)
 
void SetUpdatableConfigs (const UpdatableComponent &other)
 
virtual void FreezeNaturalGradient (bool freeze)
 freezes/unfreezes NaturalGradient updates, if applicable (to be overriden by components that use Natural Gradient). More...
 
BaseFloat LearningRate () const
 Gets the learning rate to be used in gradient descent. More...
 
BaseFloat MaxChange () const
 Returns the per-component max-change value, which is interpreted as the maximum change (in l2 norm) in parameters that is allowed per minibatch for this component. More...
 
void SetMaxChange (BaseFloat max_change)
 
BaseFloat L2Regularization () const
 Returns the l2 regularization constant, which may be set in any updatable component (usually from the config file). More...
 
void SetL2Regularization (BaseFloat a)
 
- Public Member Functions inherited from Component
virtual void StoreStats (const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, void *memo)
 This function may store stats on average activation values, and for some component types, the average value of the derivative of the nonlinearity. More...
 
virtual void ZeroStats ()
 Components that provide an implementation of StoreStats should also provide an implementation of ZeroStats(), to set those stats to zero. More...
 
virtual void GetInputIndexes (const MiscComputationInfo &misc_info, const Index &output_index, std::vector< Index > *desired_indexes) const
 This function only does something interesting for non-simple Components. More...
 
virtual bool IsComputable (const MiscComputationInfo &misc_info, const Index &output_index, const IndexSet &input_index_set, std::vector< Index > *used_inputs) const
 This function only does something interesting for non-simple Components, and it exists to make it possible to manage optionally-required inputs. More...
 
virtual void ReorderIndexes (std::vector< Index > *input_indexes, std::vector< Index > *output_indexes) const
 This function only does something interesting for non-simple Components. More...
 
virtual
ComponentPrecomputedIndexes
PrecomputeIndexes (const MiscComputationInfo &misc_info, const std::vector< Index > &input_indexes, const std::vector< Index > &output_indexes, bool need_backprop) const
 This function must return NULL for simple Components. More...
 
virtual void DeleteMemo (void *memo) const
 This virtual function only needs to be overwritten by Components that return a non-NULL memo from their Propagate() function. More...
 
 Component ()
 
virtual ~Component ()
 

Private Member Functions

void InputToInputPatches (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
 
void InderivPatchesToInderiv (const CuMatrix< BaseFloat > &in_deriv_patches, CuMatrixBase< BaseFloat > *in_deriv) const
 
const ConvolutionComponentoperator= (const ConvolutionComponent &other)
 

Private Attributes

int32 input_x_dim_
 
int32 input_y_dim_
 
int32 input_z_dim_
 
int32 filt_x_dim_
 
int32 filt_y_dim_
 
int32 filt_x_step_
 
int32 filt_y_step_
 
TensorVectorizationType input_vectorization_
 
CuMatrix< BaseFloatfilter_params_
 
CuVector< BaseFloatbias_params_
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream (works out its type). Dies on error. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Returns a new Component of the given type e.g. More...
 
- Protected Member Functions inherited from UpdatableComponent
void InitLearningRatesFromConfig (ConfigLine *cfl)
 
std::string ReadUpdatableCommon (std::istream &is, bool binary)
 
void WriteUpdatableCommon (std::ostream &is, bool binary) const
 
- Protected Attributes inherited from UpdatableComponent
BaseFloat learning_rate_
 learning rate (typically 0.0..0.01) More...
 
BaseFloat learning_rate_factor_
 learning rate factor (normally 1.0, but can be set to another < value so that when < you call SetLearningRate(), that value will be scaled by this factor. More...
 
BaseFloat l2_regularize_
 L2 regularization constant. More...
 
bool is_gradient_
 True if this component is to be treated as a gradient rather than as parameters. More...
 
BaseFloat max_change_
 configuration value for imposing max-change More...
 

Detailed Description

WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted.

ConvolutionalComponent implements 2d-convolution. It uses 3D filters on 3D inputs, but the 3D filters hop only over 2 dimensions as it has same size as the input along the 3rd dimension. Input : A matrix where each row is a vectorized 3D-tensor. The 3D tensor has dimensions x: (e.g. time) y: (e.g. frequency) z: (e.g. channels like features/delta/delta-delta)

The component supports input vectorizations of type zyx and yzx. The default vectorization type is zyx. e.g. for input vectorization of type zyx the input is vectorized by spanning axes z, y and x of the tensor in that order. Given 3d tensor A with sizes (2, 2, 2) along the three dimensions the zyx vectorized input looks like A(0,0,0) A(0,0,1) A(0,1,0) A(0,1,1) A(1,0,0) A(1,0,1) A(1,1,0) A(1,1,1)

Output : The output is also a 3D tensor vectorized in the zyx format. The channel axis (z) in the output corresponds to the output of different filters. The first channel corresponds to the first filter i.e., first row of the filter_params_ matrix.

Note: The component has to support yzx input vectorization as the binaries like add-deltas generate yz vectorized output. These input vectors are concatenated using the Append descriptor across time steps to form a yzx vectorized 3D tensor input. e.g. Append(Offset(input, -1), input, Offset(input, 1))

For information on the hyperparameters and parameters of this component see the variable declarations.

Propagation: ------------ Convolution operation consists of a dot-products between the filter tensor and input tensor patch, for various shifts of filter tensor along the x and y axes input tensor. (Note: there is no shift along z-axis as the filter and input tensor have same size along this axis).

For a particular shift (i,j) of the filter tensor along input tensor dimensions x and y, the elements of the input tensor which overlap with the filter form the input tensor patch. This patch is vectorized in zyx format. All the patches corresponding to various samples in the mini-batch are stacked into a matrix, where each row corresponds to one patch. Let this matrix be represented by X_{i,j}. The dot products with various filters are computed simultaneously by computing the matrix product with the filter_params_ matrix (W) Y_{i,j} = X_{i,j}*W^T. Each row of W corresponds to one filter 3D tensor vectorized in zyx format.

All the matrix products corresponding to various shifts (i,j) of the filter tensor are computed simultaneously using the AddMatMatBatched call of CuMatrixBase class.

BackPropagation: ---------------- Backpropagation to compute the input derivative ( X_{i,j}) consists of the a series of matrix products. {i,j} = {i,j}*W where {i,j} corresponds to the output derivative for a particular shift of the filter.

Once again these matrix products are computed simultaneously.

Update: ------- The weight gradient is computed as = {i,j} (X_{i,j}^T *{i,j})

Definition at line 1895 of file nnet-simple-component.h.

Member Enumeration Documentation

Enumerator
kYzx 
kZyx 

Definition at line 1897 of file nnet-simple-component.h.

Constructor & Destructor Documentation

Definition at line 3828 of file nnet-simple-component.cc.

3829  :
3830  UpdatableComponent(component),
3831  input_x_dim_(component.input_x_dim_),
3832  input_y_dim_(component.input_y_dim_),
3833  input_z_dim_(component.input_z_dim_),
3834  filt_x_dim_(component.filt_x_dim_),
3835  filt_y_dim_(component.filt_y_dim_),
3836  filt_x_step_(component.filt_x_step_),
3837  filt_y_step_(component.filt_y_step_),
3838  input_vectorization_(component.input_vectorization_),
3839  filter_params_(component.filter_params_),
3840  bias_params_(component.bias_params_) { }
TensorVectorizationType input_vectorization_
ConvolutionComponent ( const CuMatrixBase< BaseFloat > &  filter_params,
const CuVectorBase< BaseFloat > &  bias_params,
int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
TensorVectorizationType  input_vectorization,
BaseFloat  learning_rate 
)

Definition at line 3842 of file nnet-simple-component.cc.

References CuVectorBase< Real >::Dim(), UpdatableComponent::is_gradient_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and UpdatableComponent::SetUnderlyingLearningRate().

3849  :
3850  input_x_dim_(input_x_dim),
3851  input_y_dim_(input_y_dim),
3852  input_z_dim_(input_z_dim),
3853  filt_x_dim_(filt_x_dim),
3854  filt_y_dim_(filt_y_dim),
3855  filt_x_step_(filt_x_step),
3856  filt_y_step_(filt_y_step),
3857  input_vectorization_(input_vectorization),
3858  filter_params_(filter_params),
3859  bias_params_(bias_params){
3860  KALDI_ASSERT(filter_params.NumRows() == bias_params.Dim() &&
3861  bias_params.Dim() != 0);
3862  KALDI_ASSERT(filter_params.NumCols() == filt_x_dim * filt_y_dim * input_z_dim);
3863  SetUnderlyingLearningRate(learning_rate);
3864  is_gradient_ = false;
3865 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
TensorVectorizationType input_vectorization_
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
bool is_gradient_
True if this component is to be treated as a gradient rather than as parameters.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual void SetUnderlyingLearningRate(BaseFloat lrate)
Sets the learning rate of gradient descent- gets multiplied by learning_rate_factor_.

Member Function Documentation

void Add ( BaseFloat  alpha,
const Component other 
)
virtual

This virtual function when called by – an UpdatableComponent adds the parameters of another updatable component, times some constant, to the current parameters.

– a NonlinearComponent (or another component that stores stats, like BatchNormComponent)– it relates to adding stats. Otherwise it will normally do nothing.

Reimplemented from Component.

Definition at line 4135 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, and KALDI_ASSERT.

4135  {
4136  const ConvolutionComponent *other =
4137  dynamic_cast<const ConvolutionComponent*>(&other_in);
4138  KALDI_ASSERT(other != NULL);
4139  filter_params_.AddMat(alpha, other->filter_params_);
4140  bias_params_.AddVec(alpha, other->bias_params_);
4141 }
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:941
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1182
void Backprop ( const std::string &  debug_info,
const ComponentPrecomputedIndexes indexes,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
void *  memo,
Component to_update,
CuMatrixBase< BaseFloat > *  in_deriv 
) const
virtual

Backprop function; depending on which of the arguments 'to_update' and 'in_deriv' are non-NULL, this can compute input-data derivatives and/or perform model update.

Parameters
[in]debug_infoThe component name, to be printed out in any warning messages.
[in]indexesA pointer to some information output by this class's PrecomputeIndexes function (will be NULL for simple components, i.e. those that don't do things like splicing).
[in]in_valueThe matrix that was given as input to the Propagate function. Will be ignored (and may be empty) if Properties()&kBackpropNeedsInput == 0.
[in]out_valueThe matrix that was output from the Propagate function. Will be ignored (and may be empty) if Properties()&kBackpropNeedsOutput == 0
[in]out_derivThe derivative at the output of this component.
[in]memoThis will normally be NULL, but for component types that set the flag kUsesMemo, this will be the return value of the Propagate() function that corresponds to this Backprop() function. Ownership of any pointers is not transferred to the Backprop function; DeleteMemo() will be called to delete it.
[out]to_updateIf model update is desired, the Component to be updated, else NULL. Does not have to be identical to this. If supplied, you can assume that to_update->Properties() & kUpdatableComponent is nonzero.
[out]in_derivThe derivative at the input of this component, if needed (else NULL). If Properties()&kBackpropInPlace, may be the same matrix as out_deriv. If Properties()&kBackpropAdds, this is added to by the Backprop routine, else it is set. The component code chooses which mode to work in, based on convenience.

Implements Component.

Definition at line 4230 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::InderivPatchesToInderiv(), ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, KALDI_ASSERT, kaldi::kNoTrans, kaldi::kSetZero, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and ConvolutionComponent::Update().

4237  {
4238  ConvolutionComponent *to_update =
4239  dynamic_cast<ConvolutionComponent*>(to_update_in);
4240  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4241  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4242  num_filters = filter_params_.NumRows(),
4243  num_frames = out_deriv.NumRows(),
4244  filter_dim = filter_params_.NumCols();
4245 
4246  KALDI_ASSERT(out_deriv.NumRows() == num_frames &&
4247  out_deriv.NumCols() ==
4248  (num_filters * num_x_steps * num_y_steps));
4249 
4250  // Compute inderiv patches
4251  CuMatrix<BaseFloat> in_deriv_patches(num_frames,
4252  num_x_steps * num_y_steps * filter_dim,
4253  kSetZero);
4254 
4255  std::vector<CuSubMatrix<BaseFloat>* > patch_deriv_batch, out_deriv_batch,
4256  filter_params_batch;
4257  CuSubMatrix<BaseFloat>* filter_params_elem = new CuSubMatrix<BaseFloat>(
4258  filter_params_, 0, filter_params_.NumRows(), 0, filter_params_.NumCols());
4259 
4260  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4261  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4262  int32 patch_number = x_step * num_y_steps + y_step;
4263 
4264  patch_deriv_batch.push_back(new CuSubMatrix<BaseFloat>(
4265  in_deriv_patches.ColRange(
4266  patch_number * filter_dim, filter_dim)));
4267  out_deriv_batch.push_back(new CuSubMatrix<BaseFloat>(out_deriv.ColRange(
4268  patch_number * num_filters, num_filters)));
4269  filter_params_batch.push_back(filter_params_elem);
4270  }
4271  }
4272  AddMatMatBatched<BaseFloat>(1.0, patch_deriv_batch,
4273  out_deriv_batch, kNoTrans,
4274  filter_params_batch, kNoTrans, 0.0);
4275 
4276  if (in_deriv) {
4277  // combine the derivatives from the individual input deriv patches
4278  // to compute input deriv matrix
4279  InderivPatchesToInderiv(in_deriv_patches, in_deriv);
4280  }
4281 
4282  if (to_update != NULL) {
4283  to_update->Update(debug_info, in_value, out_deriv, out_deriv_batch);
4284  }
4285 
4286  // release memory
4287  delete filter_params_elem;
4288  for (int32 p = 0; p < patch_deriv_batch.size(); p++) {
4289  delete patch_deriv_batch[p];
4290  delete out_deriv_batch[p];
4291  }
4292 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:592
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InderivPatchesToInderiv(const CuMatrix< BaseFloat > &in_deriv_patches, CuMatrixBase< BaseFloat > *in_deriv) const
const CuVector<BaseFloat>& BiasParams ( ) const
inline

Definition at line 1960 of file nnet-simple-component.h.

References ConvolutionComponent::bias_params_.

1960 { return bias_params_; }
Component * Copy ( ) const
virtual

Copies component (deep copy).

Implements Component.

Definition at line 4440 of file nnet-simple-component.cc.

References ConvolutionComponent::ConvolutionComponent().

4440  {
4441  ConvolutionComponent *ans = new ConvolutionComponent(*this);
4442  return ans;
4443 }
BaseFloat DotProduct ( const UpdatableComponent other) const
virtual

Computes dot-product between parameters of two instances of a Component.

Can be used for computing parameter-norm of an UpdatableComponent.

Implements UpdatableComponent.

Definition at line 4433 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, kaldi::kTrans, kaldi::TraceMatMat(), and kaldi::VecVec().

4433  {
4434  const ConvolutionComponent *other =
4435  dynamic_cast<const ConvolutionComponent*>(&other_in);
4436  return TraceMatMat(filter_params_, other->filter_params_, kTrans)
4437  + VecVec(bias_params_, other->bias_params_);
4438 }
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans=kNoTrans)
We need to declare this here as it will be a friend function.
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:37
void InderivPatchesToInderiv ( const CuMatrix< BaseFloat > &  in_deriv_patches,
CuMatrixBase< BaseFloat > *  in_deriv 
) const
private

Definition at line 4174 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddCols(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, CuMatrixBase< Real >::NumCols(), kaldi::nnet3::RearrangeIndexes(), kaldi::nnet3::YzxVectorIndex(), and kaldi::nnet3::ZyxVectorIndex().

Referenced by ConvolutionComponent::Backprop().

4176  {
4177 
4178  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4179  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4180  filt_x_step = filt_x_step_,
4181  filt_y_step = filt_y_step_,
4182  filt_x_dim = filt_x_dim_,
4183  filt_y_dim = filt_y_dim_,
4184  input_x_dim = input_x_dim_,
4185  input_y_dim = input_y_dim_,
4186  input_z_dim = input_z_dim_,
4187  filter_dim = filter_params_.NumCols();
4188 
4189  // Compute the reverse column_map from the matrix with input
4190  // derivative patches to input derivative matrix
4191  std::vector<std::vector<int32> > reverse_column_map(in_deriv->NumCols());
4192  int32 rev_col_map_size = reverse_column_map.size();
4193  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4194  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4195  int32 patch_number = x_step * num_y_steps + y_step;
4196  int32 patch_start_index = patch_number * filter_dim;
4197  for (int32 x = 0, index = patch_start_index; x < filt_x_dim; x++) {
4198  for (int32 y = 0; y < filt_y_dim; y++) {
4199  for (int32 z = 0; z < input_z_dim; z++, index++) {
4200  int32 vector_index;
4201  if (input_vectorization_ == kZyx) {
4202  vector_index = ZyxVectorIndex(x_step * filt_x_step + x,
4203  y_step * filt_y_step + y, z,
4204  input_x_dim, input_y_dim,
4205  input_z_dim);
4206  } else {
4208  vector_index = YzxVectorIndex(x_step * filt_x_step + x,
4209  y_step * filt_y_step + y, z,
4210  input_x_dim, input_y_dim,
4211  input_z_dim);
4212  }
4213  KALDI_ASSERT(vector_index < rev_col_map_size);
4214  reverse_column_map[vector_index].push_back(index);
4215  }
4216  }
4217  }
4218  }
4219  }
4220  std::vector<std::vector<int32> > rearranged_column_map;
4221  RearrangeIndexes(reverse_column_map, &rearranged_column_map);
4222  for (int32 p = 0; p < rearranged_column_map.size(); p++) {
4223  CuArray<int32> cu_cols(rearranged_column_map[p]);
4224  in_deriv->AddCols(in_deriv_patches, cu_cols);
4225  }
4226 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
void AddCols(const CuMatrixBase< Real > &src, const CuArrayBase< MatrixIndexT > &indices)
Add column indices[r] of src to column r.
Definition: cu-matrix.cc:2637
TensorVectorizationType input_vectorization_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
int32 YzxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
int32 ZyxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
void RearrangeIndexes(const std::vector< std::vector< int32 > > &in, std::vector< std::vector< int32 > > *out)
std::string Info ( ) const
virtual

Returns some text-form information about this component, for diagnostics.

Starts with the type of the component. E.g. "SigmoidComponent dim=900", although most components will have much more info.

Reimplemented from UpdatableComponent.

Definition at line 3934 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, UpdatableComponent::Info(), ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, CuMatrixBase< Real >::NumRows(), and kaldi::nnet3::PrintParameterStats().

3934  {
3935  std::ostringstream stream;
3936  stream << UpdatableComponent::Info()
3937  << ", input-x-dim=" << input_x_dim_
3938  << ", input-y-dim=" << input_y_dim_
3939  << ", input-z-dim=" << input_z_dim_
3940  << ", filt-x-dim=" << filt_x_dim_
3941  << ", filt-y-dim=" << filt_y_dim_
3942  << ", filt-x-step=" << filt_x_step_
3943  << ", filt-y-step=" << filt_y_step_
3944  << ", input-vectorization=" << input_vectorization_
3945  << ", num-filters=" << filter_params_.NumRows();
3946  PrintParameterStats(stream, "filter-params", filter_params_);
3947  PrintParameterStats(stream, "bias-params", bias_params_, true);
3948  return stream.str();
3949 }
TensorVectorizationType input_vectorization_
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
virtual std::string Info() const
Returns some text-form information about this component, for diagnostics.
void PrintParameterStats(std::ostringstream &os, const std::string &name, const CuVectorBase< BaseFloat > &params, bool include_mean)
Print to 'os' some information about the mean and standard deviation of some parameters, used in Info() functions in nnet-simple-component.cc.
Definition: nnet-parse.cc:530
void Init ( int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
int32  num_filters,
TensorVectorizationType  input_vectorization,
BaseFloat  param_stddev,
BaseFloat  bias_stddev 
)

Definition at line 3881 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, CuVector< Real >::Resize(), CuMatrix< Real >::Resize(), CuVectorBase< Real >::Scale(), CuMatrixBase< Real >::Scale(), CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

Referenced by ConvolutionComponent::InitFromConfig().

3886  {
3887  input_x_dim_ = input_x_dim;
3888  input_y_dim_ = input_y_dim;
3889  input_z_dim_ = input_z_dim;
3890  filt_x_dim_ = filt_x_dim;
3891  filt_y_dim_ = filt_y_dim;
3892  filt_x_step_ = filt_x_step;
3893  filt_y_step_ = filt_y_step;
3894  input_vectorization_ = input_vectorization;
3897  int32 filter_dim = filt_x_dim_ * filt_y_dim_ * input_z_dim_;
3898  filter_params_.Resize(num_filters, filter_dim);
3899  bias_params_.Resize(num_filters);
3900  KALDI_ASSERT(param_stddev >= 0.0 && bias_stddev >= 0.0);
3902  filter_params_.Scale(param_stddev);
3904  bias_params_.Scale(bias_stddev);
3905 }
void Scale(Real value)
Definition: cu-vector.cc:1161
void Scale(Real value)
Definition: cu-matrix.cc:610
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:941
TensorVectorizationType input_vectorization_
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:49
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Init ( int32  input_x_dim,
int32  input_y_dim,
int32  input_z_dim,
int32  filt_x_dim,
int32  filt_y_dim,
int32  filt_x_step,
int32  filt_y_step,
TensorVectorizationType  input_vectorization,
std::string  matrix_filename 
)

Definition at line 3908 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::CopyColFromMat(), CuMatrixBase< Real >::CopyFromMat(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrixBase< Real >::Range(), kaldi::ReadKaldiObject(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

3913  {
3914  input_x_dim_ = input_x_dim;
3915  input_y_dim_ = input_y_dim;
3916  input_z_dim_ = input_z_dim;
3917  filt_x_dim_ = filt_x_dim;
3918  filt_y_dim_ = filt_y_dim;
3919  filt_x_step_ = filt_x_step;
3920  filt_y_step_ = filt_y_step;
3921  input_vectorization_ = input_vectorization;
3922  CuMatrix<BaseFloat> mat;
3923  ReadKaldiObject(matrix_filename, &mat);
3924  int32 filter_dim = (filt_x_dim_ * filt_y_dim_ * input_z_dim_);
3925  int32 num_filters = mat.NumRows();
3926  KALDI_ASSERT(mat.NumCols() == (filter_dim + 1));
3927  filter_params_.Resize(num_filters, filter_dim);
3928  bias_params_.Resize(num_filters);
3929  filter_params_.CopyFromMat(mat.Range(0, num_filters, 0, filter_dim));
3930  bias_params_.CopyColFromMat(mat, filter_dim);
3931 }
void CopyColFromMat(const CuMatrixBase< Real > &mat, MatrixIndexT col)
Definition: cu-vector.cc:101
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:339
void Resize(MatrixIndexT dim, MatrixResizeType t=kSetZero)
Allocate the memory.
Definition: cu-vector.cc:941
TensorVectorizationType input_vectorization_
void ReadKaldiObject(const std::string &filename, Matrix< float > *m)
Definition: kaldi-io.cc:829
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:49
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InitFromConfig ( ConfigLine cfl)
virtual

Initialize, from a ConfigLine object.

Parameters
[in]cflA ConfigLine containing any parameters that are needed for initialization. For example: "dim=100 param-stddev=0.1"

Implements Component.

Definition at line 3952 of file nnet-simple-component.cc.

References ConfigLine::GetValue(), ConfigLine::HasUnusedValues(), ConvolutionComponent::Init(), UpdatableComponent::InitLearningRatesFromConfig(), KALDI_ERR, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, ConfigLine::UnusedValues(), and ConfigLine::WholeLine().

3952  {
3953  bool ok = true;
3954  std::string matrix_filename;
3955  int32 input_x_dim = -1, input_y_dim = -1, input_z_dim = -1,
3956  filt_x_dim = -1, filt_y_dim = -1,
3957  filt_x_step = -1, filt_y_step = -1,
3958  num_filters = -1;
3959  std::string input_vectorization_order = "zyx";
3961  ok = ok && cfl->GetValue("input-x-dim", &input_x_dim);
3962  ok = ok && cfl->GetValue("input-y-dim", &input_y_dim);
3963  ok = ok && cfl->GetValue("input-z-dim", &input_z_dim);
3964  ok = ok && cfl->GetValue("filt-x-dim", &filt_x_dim);
3965  ok = ok && cfl->GetValue("filt-y-dim", &filt_y_dim);
3966  ok = ok && cfl->GetValue("filt-x-step", &filt_x_step);
3967  ok = ok && cfl->GetValue("filt-y-step", &filt_y_step);
3968 
3969  if (!ok)
3970  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
3971  // optional argument
3972  TensorVectorizationType input_vectorization;
3973  cfl->GetValue("input-vectorization-order", &input_vectorization_order);
3974  if (input_vectorization_order.compare("zyx") == 0) {
3975  input_vectorization = kZyx;
3976  } else if (input_vectorization_order.compare("yzx") == 0) {
3977  input_vectorization = kYzx;
3978  } else {
3979  KALDI_ERR << "Unknown or unsupported input vectorization order "
3980  << input_vectorization_order
3981  << " accepted candidates are 'yzx' and 'zyx'";
3982  }
3983 
3984  if (cfl->GetValue("matrix", &matrix_filename)) {
3985  // initialize from prefined parameter matrix
3986  Init(input_x_dim, input_y_dim, input_z_dim,
3987  filt_x_dim, filt_y_dim,
3988  filt_x_step, filt_y_step,
3989  input_vectorization,
3990  matrix_filename);
3991  } else {
3992  ok = ok && cfl->GetValue("num-filters", &num_filters);
3993  if (!ok)
3994  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
3995  // initialize from configuration
3996  int32 filter_input_dim = filt_x_dim * filt_y_dim * input_z_dim;
3997  BaseFloat param_stddev = 1.0 / std::sqrt(filter_input_dim), bias_stddev = 1.0;
3998  cfl->GetValue("param-stddev", &param_stddev);
3999  cfl->GetValue("bias-stddev", &bias_stddev);
4000  Init(input_x_dim, input_y_dim, input_z_dim,
4001  filt_x_dim, filt_y_dim, filt_x_step, filt_y_step, num_filters,
4002  input_vectorization, param_stddev, bias_stddev);
4003  }
4004  if (cfl->HasUnusedValues())
4005  KALDI_ERR << "Could not process these elements in initializer: "
4006  << cfl->UnusedValues();
4007  if (!ok)
4008  KALDI_ERR << "Bad initializer " << cfl->WholeLine();
4009 }
void Init(int32 input_x_dim, int32 input_y_dim, int32 input_z_dim, int32 filt_x_dim, int32 filt_y_dim, int32 filt_x_step, int32 filt_y_step, int32 num_filters, TensorVectorizationType input_vectorization, BaseFloat param_stddev, BaseFloat bias_stddev)
void InitLearningRatesFromConfig(ConfigLine *cfl)
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 InputDim ( ) const
virtual
void InputToInputPatches ( const CuMatrixBase< BaseFloat > &  in,
CuMatrix< BaseFloat > *  patches 
) const
private

Definition at line 4032 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::CopyCols(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, KALDI_ASSERT, ConvolutionComponent::kYzx, ConvolutionComponent::kZyx, CuMatrixBase< Real >::NumCols(), kaldi::nnet3::YzxVectorIndex(), and kaldi::nnet3::ZyxVectorIndex().

Referenced by ConvolutionComponent::Propagate(), and ConvolutionComponent::Update().

4034  {
4035  int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_);
4036  int32 num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_);
4037  const int32 filt_x_step = filt_x_step_,
4038  filt_y_step = filt_y_step_,
4039  filt_x_dim = filt_x_dim_,
4040  filt_y_dim = filt_y_dim_,
4041  input_x_dim = input_x_dim_,
4042  input_y_dim = input_y_dim_,
4043  input_z_dim = input_z_dim_,
4044  filter_dim = filter_params_.NumCols();
4045 
4046  std::vector<int32> column_map(patches->NumCols());
4047  int32 column_map_size = column_map.size();
4048  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4049  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4050  int32 patch_number = x_step * num_y_steps + y_step;
4051  int32 patch_start_index = patch_number * filter_dim;
4052  for (int32 x = 0, index = patch_start_index; x < filt_x_dim; x++) {
4053  for (int32 y = 0; y < filt_y_dim; y++) {
4054  for (int32 z = 0; z < input_z_dim; z++, index++) {
4055  KALDI_ASSERT(index < column_map_size);
4056  if (input_vectorization_ == kZyx) {
4057  column_map[index] = ZyxVectorIndex(x_step * filt_x_step + x,
4058  y_step * filt_y_step + y, z,
4059  input_x_dim, input_y_dim,
4060  input_z_dim);
4061  } else if (input_vectorization_ == kYzx) {
4062  column_map[index] = YzxVectorIndex(x_step * filt_x_step + x,
4063  y_step * filt_y_step + y, z,
4064  input_x_dim, input_y_dim,
4065  input_z_dim);
4066  }
4067  }
4068  }
4069  }
4070  }
4071  }
4072  CuArray<int32> cu_cols(column_map);
4073  patches->CopyCols(in, cu_cols);
4074 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
TensorVectorizationType input_vectorization_
void CopyCols(const CuMatrixBase< Real > &src, const CuArrayBase< MatrixIndexT > &indexes)
Copies column r from column indexes[r] of src.
Definition: cu-matrix.cc:2592
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
int32 YzxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
int32 ZyxVectorIndex(int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
const CuMatrix<BaseFloat>& LinearParams ( ) const
inline

Definition at line 1961 of file nnet-simple-component.h.

References ConvolutionComponent::filter_params_.

1961 { return filter_params_; }
int32 NumParameters ( ) const
virtual

The following new virtual function returns the total dimension of the parameters in this class.

Reimplemented from UpdatableComponent.

Definition at line 4462 of file nnet-simple-component.cc.

References ConvolutionComponent::filter_params_, CuMatrixBase< Real >::NumCols(), and CuMatrixBase< Real >::NumRows().

Referenced by ConvolutionComponent::UnVectorize(), and ConvolutionComponent::Vectorize().

4462  {
4463  return (filter_params_.NumCols() + 1) * filter_params_.NumRows();
4464 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
const ConvolutionComponent& operator= ( const ConvolutionComponent other)
private
int32 OutputDim ( ) const
virtual

Returns output-dimension of this component.

Implements Component.

Definition at line 3873 of file nnet-simple-component.cc.

References ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, and CuMatrixBase< Real >::NumRows().

3873  {
3874  int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_);
3875  int32 num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_);
3876  int32 num_filters = filter_params_.NumRows();
3877  return num_x_steps * num_y_steps * num_filters;
3878 }
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
void PerturbParams ( BaseFloat  stddev)
virtual

This function is to be used in testing.

It adds unit noise times "stddev" to the parameters of the component.

Implements UpdatableComponent.

Definition at line 4445 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, CuVectorBase< Real >::SetRandn(), and CuMatrixBase< Real >::SetRandn().

4445  {
4446  CuMatrix<BaseFloat> temp_filter_params(filter_params_);
4447  temp_filter_params.SetRandn();
4448  filter_params_.AddMat(stddev, temp_filter_params);
4449 
4450  CuVector<BaseFloat> temp_bias_params(bias_params_);
4451  temp_bias_params.SetRandn();
4452  bias_params_.AddVec(stddev, temp_bias_params);
4453 }
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:941
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1182
void * Propagate ( const ComponentPrecomputedIndexes indexes,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Propagate function.

Parameters
[in]indexesA pointer to some information output by this class's PrecomputeIndexes function (will be NULL for simple components, i.e. those that don't do things like splicing).
[in]inThe input to this component. Num-columns == InputDim().
[out]outThe output of this component. Num-columns == OutputDim(). Note: output of this component will be added to the initial value of "out" if Properties()&kPropagateAdds != 0; otherwise the output will be set and the initial value ignored. Each Component chooses whether it is more convenient implementation-wise to add or set, and the calling code has to deal with it.
Returns
Normally returns NULL, but may return a non-NULL value for components which have the flag kUsesMemo set. This value will be passed into the corresponding Backprop routine.

Implements Component.

Definition at line 4079 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddVecToRows(), ConvolutionComponent::bias_params_, CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::InputToInputPatches(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, kaldi::kUndefined, CuMatrixBase< Real >::NumCols(), and CuMatrixBase< Real >::NumRows().

4081  {
4082  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4083  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4084  num_filters = filter_params_.NumRows(),
4085  num_frames = in.NumRows(),
4086  filter_dim = filter_params_.NumCols();
4087  KALDI_ASSERT((*out).NumRows() == num_frames &&
4088  (*out).NumCols() == (num_filters * num_x_steps * num_y_steps));
4089 
4090  CuMatrix<BaseFloat> patches(num_frames,
4091  num_x_steps * num_y_steps * filter_dim,
4092  kUndefined);
4093  InputToInputPatches(in, &patches);
4094  CuSubMatrix<BaseFloat>* filter_params_elem = new CuSubMatrix<BaseFloat>(
4095  filter_params_, 0, filter_params_.NumRows(), 0, filter_params_.NumCols());
4096  std::vector<CuSubMatrix<BaseFloat>* > tgt_batch, patch_batch,
4097  filter_params_batch;
4098 
4099  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4100  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4101  int32 patch_number = x_step * num_y_steps + y_step;
4102  tgt_batch.push_back(new CuSubMatrix<BaseFloat>(
4103  out->ColRange(patch_number * num_filters, num_filters)));
4104  patch_batch.push_back(new CuSubMatrix<BaseFloat>(
4105  patches.ColRange(patch_number * filter_dim, filter_dim)));
4106  filter_params_batch.push_back(filter_params_elem);
4107  tgt_batch[patch_number]->AddVecToRows(1.0, bias_params_, 1.0); // add bias
4108  }
4109  }
4110  // apply all filters
4111  AddMatMatBatched<BaseFloat>(1.0, tgt_batch, patch_batch,
4112  kNoTrans, filter_params_batch,
4113  kTrans, 1.0);
4114  // release memory
4115  delete filter_params_elem;
4116  for (int32 p = 0; p < tgt_batch.size(); p++) {
4117  delete tgt_batch[p];
4118  delete patch_batch[p];
4119  }
4120  return NULL;
4121 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:592
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
void InputToInputPatches(const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual int32 Properties ( ) const
inlinevirtual
void Read ( std::istream &  is,
bool  binary 
)
virtual

Read function (used after we know the type of the Component); accepts input that is missing the token that describes the component type, in case it has already been consumed.

Implements Component.

Definition at line 4371 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, kaldi::nnet3::ExpectToken(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, UpdatableComponent::is_gradient_, KALDI_ASSERT, CuVector< Real >::Read(), CuMatrix< Real >::Read(), kaldi::ReadBasicType(), kaldi::ReadToken(), and UpdatableComponent::ReadUpdatableCommon().

4371  {
4372  ReadUpdatableCommon(is, binary); // Read opening tag and learning rate.
4373  ExpectToken(is, binary, "<InputXDim>");
4374  ReadBasicType(is, binary, &input_x_dim_);
4375  ExpectToken(is, binary, "<InputYDim>");
4376  ReadBasicType(is, binary, &input_y_dim_);
4377  ExpectToken(is, binary, "<InputZDim>");
4378  ReadBasicType(is, binary, &input_z_dim_);
4379  ExpectToken(is, binary, "<FiltXDim>");
4380  ReadBasicType(is, binary, &filt_x_dim_);
4381  ExpectToken(is, binary, "<FiltYDim>");
4382  ReadBasicType(is, binary, &filt_y_dim_);
4383  ExpectToken(is, binary, "<FiltXStep>");
4384  ReadBasicType(is, binary, &filt_x_step_);
4385  ExpectToken(is, binary, "<FiltYStep>");
4386  ReadBasicType(is, binary, &filt_y_step_);
4387  ExpectToken(is, binary, "<InputVectorization>");
4388  int32 input_vectorization;
4389  ReadBasicType(is, binary, &input_vectorization);
4390  input_vectorization_ = static_cast<TensorVectorizationType>(input_vectorization);
4391  ExpectToken(is, binary, "<FilterParams>");
4392  filter_params_.Read(is, binary);
4393  ExpectToken(is, binary, "<BiasParams>");
4394  bias_params_.Read(is, binary);
4395  std::string tok;
4396  ReadToken(is, binary, &tok);
4397  if (tok == "<IsGradient>") {
4398  ReadBasicType(is, binary, &is_gradient_);
4399  ExpectToken(is, binary, "</ConvolutionComponent>");
4400  } else {
4401  is_gradient_ = false;
4402  KALDI_ASSERT(tok == "</ConvolutionComponent>");
4403  }
4404 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
void Read(std::istream &is, bool binary)
I/O.
Definition: cu-vector.cc:911
void ReadToken(std::istream &is, bool binary, std::string *str)
ReadToken gets the next token and puts it in str (exception on failure).
Definition: io-funcs.cc:154
TensorVectorizationType input_vectorization_
static void ExpectToken(const std::string &token, const std::string &what_we_are_parsing, const std::string **next_token)
std::string ReadUpdatableCommon(std::istream &is, bool binary)
void Read(std::istream &is, bool binary)
I/O functions.
Definition: cu-matrix.cc:461
bool is_gradient_
True if this component is to be treated as a gradient rather than as parameters.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Resize ( int32  input_dim,
int32  output_dim 
)
void Scale ( BaseFloat  scale)
virtual

This virtual function when called on – an UpdatableComponent scales the parameters by "scale" when called by an UpdatableComponent.

– a Nonlinear component (or another component that stores stats, like BatchNormComponent)– it relates to scaling activation stats, not parameters. Otherwise it will normally do nothing.

Reimplemented from Component.

Definition at line 4124 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filter_params_, CuVectorBase< Real >::Scale(), CuMatrixBase< Real >::Scale(), CuVectorBase< Real >::SetZero(), and CuMatrixBase< Real >::SetZero().

4124  {
4125  if (scale == 0.0) {
4128  } else {
4129  filter_params_.Scale(scale);
4130  bias_params_.Scale(scale);
4131  }
4132 }
void Scale(Real value)
Definition: cu-vector.cc:1161
void Scale(Real value)
Definition: cu-matrix.cc:610
void SetZero()
Math operations, some calling kernels.
Definition: cu-matrix.cc:476
void SetZero()
Math operations.
Definition: cu-vector.cc:1044
void SetParams ( const VectorBase< BaseFloat > &  bias,
const MatrixBase< BaseFloat > &  filter 
)

Definition at line 4455 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, and CuMatrixBase< Real >::NumRows().

4456  {
4457  bias_params_ = bias;
4458  filter_params_ = filter;
4460 }
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual std::string Type ( ) const
inlinevirtual

Returns a string such as "SigmoidComponent", describing the type of the object.

Implements Component.

Definition at line 1920 of file nnet-simple-component.h.

1920 { return "ConvolutionComponent"; }
void UnVectorize ( const VectorBase< BaseFloat > &  params)
virtual

Converts the parameters from vector form.

Reimplemented from UpdatableComponent.

Definition at line 4472 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, CuVectorBase< Real >::CopyFromVec(), CuMatrixBase< Real >::CopyRowsFromVec(), VectorBase< Real >::Dim(), CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), ConvolutionComponent::NumParameters(), CuMatrixBase< Real >::NumRows(), and VectorBase< Real >::Range().

4472  {
4473  KALDI_ASSERT(params.Dim() == this->NumParameters());
4474  int32 num_filter_params = filter_params_.NumCols() * filter_params_.NumRows();
4475  filter_params_.CopyRowsFromVec(params.Range(0, num_filter_params));
4476  bias_params_.CopyFromVec(params.Range(num_filter_params, bias_params_.Dim()));
4477 }
void CopyRowsFromVec(const CuVectorBase< Real > &v)
This function has two modes of operation.
Definition: cu-matrix.cc:2282
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
void CopyFromVec(const CuVectorBase< Real > &src)
Copy functions; these will crash if the dimension do not match.
Definition: cu-vector.cc:1026
virtual int32 NumParameters() const
The following new virtual function returns the total dimension of the parameters in this class...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:63
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:93
void Update ( const std::string &  debug_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
const std::vector< CuSubMatrix< BaseFloat > * > &  out_deriv_batch 
)

Definition at line 4297 of file nnet-simple-component.cc.

References CuMatrixBase< Real >::AddMat(), CuMatrixBase< Real >::AddMatBlocks(), CuVectorBase< Real >::AddRowSumMat(), CuVectorBase< Real >::AddVec(), ConvolutionComponent::bias_params_, CuMatrixBase< Real >::ColRange(), ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::InputToInputPatches(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kSetZero, kaldi::kTrans, kaldi::kUndefined, UpdatableComponent::learning_rate_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuVector< Real >::Resize(), and CuMatrix< Real >::Resize().

Referenced by ConvolutionComponent::Backprop().

4300  {
4301  // useful dims
4302  const int32 num_x_steps = (1 + (input_x_dim_ - filt_x_dim_) / filt_x_step_),
4303  num_y_steps = (1 + (input_y_dim_ - filt_y_dim_) / filt_y_step_),
4304  num_filters = filter_params_.NumRows(),
4305  num_frames = out_deriv.NumRows(),
4306  filter_dim = filter_params_.NumCols();
4307  KALDI_ASSERT(out_deriv.NumRows() == num_frames &&
4308  out_deriv.NumCols() ==
4309  (num_filters * num_x_steps * num_y_steps));
4310 
4311 
4312  CuMatrix<BaseFloat> filters_grad;
4313  CuVector<BaseFloat> bias_grad;
4314 
4315  CuMatrix<BaseFloat> input_patches(num_frames,
4316  filter_dim * num_x_steps * num_y_steps,
4317  kUndefined);
4318  InputToInputPatches(in_value, &input_patches);
4319 
4320  filters_grad.Resize(num_filters, filter_dim, kSetZero); // reset
4321  bias_grad.Resize(num_filters, kSetZero); // reset
4322 
4323  // create a single large matrix holding the smaller matrices
4324  // from the vector container filters_grad_batch along the rows
4325  CuMatrix<BaseFloat> filters_grad_blocks_batch(
4326  num_x_steps * num_y_steps * filters_grad.NumRows(),
4327  filters_grad.NumCols());
4328 
4329  std::vector<CuSubMatrix<BaseFloat>* > filters_grad_batch, input_patch_batch;
4330 
4331  for (int32 x_step = 0; x_step < num_x_steps; x_step++) {
4332  for (int32 y_step = 0; y_step < num_y_steps; y_step++) {
4333  int32 patch_number = x_step * num_y_steps + y_step;
4334  filters_grad_batch.push_back(new CuSubMatrix<BaseFloat>(
4335  filters_grad_blocks_batch.RowRange(
4336  patch_number * filters_grad.NumRows(), filters_grad.NumRows())));
4337 
4338  input_patch_batch.push_back(new CuSubMatrix<BaseFloat>(
4339  input_patches.ColRange(patch_number * filter_dim, filter_dim)));
4340  }
4341  }
4342 
4343  AddMatMatBatched<BaseFloat>(1.0, filters_grad_batch, out_deriv_batch, kTrans,
4344  input_patch_batch, kNoTrans, 1.0);
4345 
4346  // add the row blocks together to filters_grad
4347  filters_grad.AddMatBlocks(1.0, filters_grad_blocks_batch);
4348 
4349  // create a matrix holding the col blocks sum of out_deriv
4350  CuMatrix<BaseFloat> out_deriv_col_blocks_sum(out_deriv.NumRows(),
4351  num_filters);
4352 
4353  // add the col blocks together to out_deriv_col_blocks_sum
4354  out_deriv_col_blocks_sum.AddMatBlocks(1.0, out_deriv);
4355 
4356  bias_grad.AddRowSumMat(1.0, out_deriv_col_blocks_sum, 1.0);
4357 
4358  // release memory
4359  for (int32 p = 0; p < input_patch_batch.size(); p++) {
4360  delete filters_grad_batch[p];
4361  delete input_patch_batch[p];
4362  }
4363 
4364  //
4365  // update
4366  //
4367  filter_params_.AddMat(learning_rate_, filters_grad);
4368  bias_params_.AddVec(learning_rate_, bias_grad);
4369 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
BaseFloat learning_rate_
learning rate (typically 0.0..0.01)
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:941
void InputToInputPatches(const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *patches) const
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void AddVec(Real alpha, const CuVectorBase< Real > &vec, Real beta=1.0)
Definition: cu-vector.cc:1182
void Update ( const std::string &  debug_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_deriv 
)
void Vectorize ( VectorBase< BaseFloat > *  params) const
virtual

Turns the parameters into vector form.

We put the vector form on the CPU, because in the kinds of situations where we do this, we'll tend to use too much memory for the GPU.

Reimplemented from UpdatableComponent.

Definition at line 4466 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, VectorBase< Real >::Dim(), CuVectorBase< Real >::Dim(), ConvolutionComponent::filter_params_, KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), ConvolutionComponent::NumParameters(), CuMatrixBase< Real >::NumRows(), and VectorBase< Real >::Range().

4466  {
4467  KALDI_ASSERT(params->Dim() == this->NumParameters());
4468  int32 num_filter_params = filter_params_.NumCols() * filter_params_.NumRows();
4469  params->Range(0, num_filter_params).CopyRowsFromMat(filter_params_);
4470  params->Range(num_filter_params, bias_params_.Dim()).CopyFromVec(bias_params_);
4471 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:206
MatrixIndexT Dim() const
Dimensions.
Definition: cu-vector.h:68
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:205
virtual int32 NumParameters() const
The following new virtual function returns the total dimension of the parameters in this class...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:63
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:93
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Definition at line 4406 of file nnet-simple-component.cc.

References ConvolutionComponent::bias_params_, ConvolutionComponent::filt_x_dim_, ConvolutionComponent::filt_x_step_, ConvolutionComponent::filt_y_dim_, ConvolutionComponent::filt_y_step_, ConvolutionComponent::filter_params_, ConvolutionComponent::input_vectorization_, ConvolutionComponent::input_x_dim_, ConvolutionComponent::input_y_dim_, ConvolutionComponent::input_z_dim_, UpdatableComponent::is_gradient_, CuVector< Real >::Write(), CuMatrixBase< Real >::Write(), kaldi::WriteBasicType(), kaldi::WriteToken(), and UpdatableComponent::WriteUpdatableCommon().

4406  {
4407  WriteUpdatableCommon(os, binary); // write opening tag and learning rate.
4408  WriteToken(os, binary, "<InputXDim>");
4409  WriteBasicType(os, binary, input_x_dim_);
4410  WriteToken(os, binary, "<InputYDim>");
4411  WriteBasicType(os, binary, input_y_dim_);
4412  WriteToken(os, binary, "<InputZDim>");
4413  WriteBasicType(os, binary, input_z_dim_);
4414  WriteToken(os, binary, "<FiltXDim>");
4415  WriteBasicType(os, binary, filt_x_dim_);
4416  WriteToken(os, binary, "<FiltYDim>");
4417  WriteBasicType(os, binary, filt_y_dim_);
4418  WriteToken(os, binary, "<FiltXStep>");
4419  WriteBasicType(os, binary, filt_x_step_);
4420  WriteToken(os, binary, "<FiltYStep>");
4421  WriteBasicType(os, binary, filt_y_step_);
4422  WriteToken(os, binary, "<InputVectorization>");
4423  WriteBasicType(os, binary, static_cast<int32>(input_vectorization_));
4424  WriteToken(os, binary, "<FilterParams>");
4425  filter_params_.Write(os, binary);
4426  WriteToken(os, binary, "<BiasParams>");
4427  bias_params_.Write(os, binary);
4428  WriteToken(os, binary, "<IsGradient>");
4429  WriteBasicType(os, binary, is_gradient_);
4430  WriteToken(os, binary, "</ConvolutionComponent>");
4431 }
TensorVectorizationType input_vectorization_
void Write(std::ostream &is, bool binary) const
Definition: cu-vector.cc:921
void WriteUpdatableCommon(std::ostream &is, bool binary) const
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
bool is_gradient_
True if this component is to be treated as a gradient rather than as parameters.
void WriteBasicType(std::ostream &os, bool binary, T t)
WriteBasicType is the name of the write function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:34
void Write(std::ostream &os, bool binary) const
Definition: cu-matrix.cc:469

Member Data Documentation


The documentation for this class was generated from the following files: