All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
BlockSoftmax Class Reference

#include <nnet-activation.h>

Inheritance diagram for BlockSoftmax:
Collaboration diagram for BlockSoftmax:

Public Member Functions

 BlockSoftmax (int32 dim_in, int32 dim_out)
 
 ~BlockSoftmax ()
 
ComponentCopy () const
 Copy component (deep copy),. More...
 
ComponentType GetType () const
 Get Type Identification of the component,. More...
 
void InitData (std::istream &is)
 Virtual interface for initialization and I/O,. More...
 
void ReadData (std::istream &is, bool binary)
 Reads the component content. More...
 
void WriteData (std::ostream &os, bool binary) const
 Writes the component content. More...
 
void PropagateFnc (const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out)
 Abstract interface for propagation/backpropagation. More...
 
void BackpropagateFnc (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrixBase< BaseFloat > *in_diff)
 Backward pass transformation (to be implemented by descending class...) More...
 
std::string Info () const
 Print some additional info (after <ComponentName> and the dims),. More...
 
- Public Member Functions inherited from Component
 Component (int32 input_dim, int32 output_dim)
 Generic interface of a component,. More...
 
virtual ~Component ()
 
virtual bool IsUpdatable () const
 Check if componeny has 'Updatable' interface (trainable components),. More...
 
virtual bool IsMultistream () const
 Check if component has 'Recurrent' interface (trainable and recurrent),. More...
 
int32 InputDim () const
 Get the dimension of the input,. More...
 
int32 OutputDim () const
 Get the dimension of the output,. More...
 
void Propagate (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out)
 Perform forward-pass propagation 'in' -> 'out',. More...
 
void Backpropagate (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrix< BaseFloat > *in_diff)
 Perform backward-pass propagation 'out_diff' -> 'in_diff'. More...
 
void Write (std::ostream &os, bool binary) const
 Write the component to a stream,. More...
 
virtual std::string InfoGradient () const
 Print some additional info about gradient (after <...> and dims),. More...
 

Public Attributes

std::vector< int32 > block_dims
 
std::vector< int32 > block_offset
 

Additional Inherited Members

- Public Types inherited from Component
enum  ComponentType {
  kUnknown = 0x0, kUpdatableComponent = 0x0100, kAffineTransform, kLinearTransform,
  kConvolutionalComponent, kConvolutional2DComponent, kLstmProjected, kBlstmProjected,
  kRecurrentComponent, kActivationFunction = 0x0200, kSoftmax, kHiddenSoftmax,
  kBlockSoftmax, kSigmoid, kTanh, kParametricRelu,
  kDropout, kLengthNormComponent, kTranform = 0x0400, kRbm,
  kSplice, kCopy, kTranspose, kBlockLinearity,
  kAddShift, kRescale, kKlHmm = 0x0800, kSentenceAveragingComponent,
  kSimpleSentenceAveragingComponent, kAveragePoolingComponent, kAveragePooling2DComponent, kMaxPoolingComponent,
  kMaxPooling2DComponent, kFramePoolingComponent, kParallelComponent, kMultiBasisComponent
}
 Component type identification mechanism,. More...
 
- Static Public Member Functions inherited from Component
static const char * TypeToMarker (ComponentType t)
 Converts component type to marker,. More...
 
static ComponentType MarkerToType (const std::string &s)
 Converts marker to component type (case insensitive),. More...
 
static ComponentInit (const std::string &conf_line)
 Initialize component from a line in config file,. More...
 
static ComponentRead (std::istream &is, bool binary)
 Read the component from a stream (static method),. More...
 
- Static Public Attributes inherited from Component
static const struct key_value kMarkerMap []
 The table with pairs of Component types and markers (defined in nnet-component.cc),. More...
 
- Protected Attributes inherited from Component
int32 input_dim_
 Data members,. More...
 
int32 output_dim_
 Dimension of the output of the Component,. More...
 

Detailed Description

Definition at line 113 of file nnet-activation.h.

Constructor & Destructor Documentation

BlockSoftmax ( int32  dim_in,
int32  dim_out 
)
inline

Definition at line 115 of file nnet-activation.h.

Referenced by BlockSoftmax::Copy().

115  :
116  Component(dim_in, dim_out)
117  { }
Component(int32 input_dim, int32 output_dim)
Generic interface of a component,.
~BlockSoftmax ( )
inline

Definition at line 119 of file nnet-activation.h.

120  { }

Member Function Documentation

void BackpropagateFnc ( const CuMatrixBase< BaseFloat > &  in,
const CuMatrixBase< BaseFloat > &  out,
const CuMatrixBase< BaseFloat > &  out_diff,
CuMatrixBase< BaseFloat > *  in_diff 
)
inlinevirtual

Backward pass transformation (to be implemented by descending class...)

Implements Component.

Definition at line 174 of file nnet-activation.h.

References CuVectorBase< Real >::Add(), CuVectorBase< Real >::AddColSumMat(), BlockSoftmax::block_dims, BlockSoftmax::block_offset, CuMatrixBase< Real >::ColRange(), CuMatrixBase< Real >::CopyFromMat(), CuMatrixBase< Real >::MulRowsVec(), CuMatrixBase< Real >::NumRows(), and CuVectorBase< Real >::Scale().

177  {
178  // copy the error derivative:
179  // (assuming we already got softmax-cross-entropy derivative in out_diff)
180  in_diff->CopyFromMat(out_diff);
181 
182  // Set the derivatives to zero for the matrix-lines in which
183  // the sum of 'derivatives' was 1.0 (i.e. there was no target):
184  for (int32 bl = 0; bl < block_dims.size(); bl++) {
185  // get the block,
186  CuSubMatrix<BaseFloat> diff_bl =
187  in_diff->ColRange(block_offset[bl], block_dims[bl]);
188  // get the sum of each row,
189  CuVector<BaseFloat> row_sum(diff_bl.NumRows());
190  row_sum.AddColSumMat(1.0, diff_bl, 0.0); // 0: keep as-is, 1: zero-out
191  // we'll scale rows by 0/1 masks,
192  CuVector<BaseFloat> row_diff_mask(row_sum);
193  row_diff_mask.Scale(-1.0); // 0: keep as-is, -1: zero-out
194  row_diff_mask.Add(1.0); // 1: keep as-is, 0: zero-out
195  // here we should have only 0's and 1's,
196  diff_bl.MulRowsVec(row_diff_mask);
197  }
198  }
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:337
std::vector< int32 > block_offset
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:544
std::vector< int32 > block_dims
Component* Copy ( ) const
inlinevirtual

Copy component (deep copy),.

Implements Component.

Definition at line 122 of file nnet-activation.h.

References BlockSoftmax::BlockSoftmax().

122 { return new BlockSoftmax(*this); }
BlockSoftmax(int32 dim_in, int32 dim_out)
ComponentType GetType ( ) const
inlinevirtual

Get Type Identification of the component,.

Implements Component.

Definition at line 123 of file nnet-activation.h.

References Component::kBlockSoftmax.

std::string Info ( ) const
inlinevirtual

Print some additional info (after <ComponentName> and the dims),.

Reimplemented from Component.

Definition at line 200 of file nnet-activation.h.

References BlockSoftmax::block_dims, and kaldi::nnet1::ToString().

200  {
201  return "\n softmax-dims " + ToString(block_dims);
202  }
std::string ToString(const T &t)
Convert basic type to a string (please don't overuse),.
Definition: nnet-utils.h:52
std::vector< int32 > block_dims
void InitData ( std::istream &  is)
inlinevirtual

Virtual interface for initialization and I/O,.

Initialize internal data of a component

Reimplemented from Component.

Definition at line 125 of file nnet-activation.h.

References BlockSoftmax::block_dims, rnnlm::i, KALDI_ASSERT, KALDI_ERR, Component::OutputDim(), kaldi::ReadToken(), and kaldi::SplitStringToIntegers().

125  {
126  // parse config
127  std::string token,
128  dims_str;
129  while (is >> std::ws, !is.eof()) {
130  ReadToken(is, false, &token);
131  if (token == "<BlockDims>") is >> dims_str;
132  else KALDI_ERR << "Unknown token " << token << ", a typo in config?"
133  << " (BlockDims)";
134  }
135  // parse dims,
136  if (!kaldi::SplitStringToIntegers(dims_str, ",:", false, &block_dims))
137  KALDI_ERR << "Invalid block-dims " << dims_str;
138  // sanity check
139  int32 sum = 0;
140  for (int32 i = 0; i < block_dims.size(); i++) {
141  sum += block_dims[i];
142  }
143  KALDI_ASSERT(sum == OutputDim());
144  }
bool SplitStringToIntegers(const std::string &full, const char *delim, bool omit_empty_strings, std::vector< I > *out)
Split a string (e.g.
Definition: text-utils.h:68
int32 OutputDim() const
Get the dimension of the output,.
void ReadToken(std::istream &is, bool binary, std::string *str)
ReadToken gets the next token and puts it in str (exception on failure).
Definition: io-funcs.cc:154
#define KALDI_ERR
Definition: kaldi-error.h:127
std::vector< int32 > block_dims
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void PropagateFnc ( const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
)
inlinevirtual

Abstract interface for propagation/backpropagation.

Forward pass transformation (to be implemented by descending class...)

Implements Component.

Definition at line 160 of file nnet-activation.h.

References CuMatrixBase< Real >::ApplySoftMaxPerRow(), BlockSoftmax::block_dims, BlockSoftmax::block_offset, and CuMatrixBase< Real >::ColRange().

161  {
162  // perform softmax per block:
163  for (int32 bl = 0; bl < block_dims.size(); bl++) {
164  // get the blocks,
165  CuSubMatrix<BaseFloat> in_bl =
166  in.ColRange(block_offset[bl], block_dims[bl]);
167  CuSubMatrix<BaseFloat> out_bl =
168  out->ColRange(block_offset[bl], block_dims[bl]);
169  // y = e^x_j/sum_j(e^x_j),
170  out_bl.ApplySoftMaxPerRow(in_bl);
171  }
172  }
std::vector< int32 > block_offset
CuSubMatrix< Real > ColRange(const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Definition: cu-matrix.h:544
std::vector< int32 > block_dims
void ReadData ( std::istream &  is,
bool  binary 
)
inlinevirtual

Reads the component content.

Reimplemented from Component.

Definition at line 146 of file nnet-activation.h.

References BlockSoftmax::block_dims, BlockSoftmax::block_offset, rnnlm::i, KALDI_ASSERT, Component::OutputDim(), and kaldi::ReadIntegerVector().

146  {
147  ReadIntegerVector(is, binary, &block_dims);
148  block_offset.resize(block_dims.size()+1, 0);
149  for (int32 i = 0; i < block_dims.size(); i++) {
151  }
152  // check
154  }
int32 OutputDim() const
Get the dimension of the output,.
std::vector< int32 > block_offset
void ReadIntegerVector(std::istream &is, bool binary, std::vector< T > *v)
Function for reading STL vector of integer types.
Definition: io-funcs-inl.h:232
std::vector< int32 > block_dims
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void WriteData ( std::ostream &  os,
bool  binary 
) const
inlinevirtual

Writes the component content.

Reimplemented from Component.

Definition at line 156 of file nnet-activation.h.

References BlockSoftmax::block_dims, and kaldi::WriteIntegerVector().

156  {
157  WriteIntegerVector(os, binary, block_dims);
158  }
std::vector< int32 > block_dims
void WriteIntegerVector(std::ostream &os, bool binary, const std::vector< T > &v)
Function for writing STL vectors of integer types.
Definition: io-funcs-inl.h:198

Member Data Documentation

std::vector<int32> block_offset

The documentation for this class was generated from the following file: