LengthNormComponent Class Reference

Rescale the matrix-rows to have unit length (L2-norm). More...

#include <nnet-various.h>

Inheritance diagram for LengthNormComponent:
Collaboration diagram for LengthNormComponent:

Public Member Functions

 LengthNormComponent (int32 dim_in, int32 dim_out)
 
 ~LengthNormComponent ()
 
ComponentCopy () const
 Copy component (deep copy),. More...
 
ComponentType GetType () const
 Get Type Identification of the component,. More...
 
void PropagateFnc (const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out)
 Abstract interface for propagation/backpropagation. More...
 
void BackpropagateFnc (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrixBase< BaseFloat > *in_diff)
 Backward pass transformation (to be implemented by descending class...) More...
 
- Public Member Functions inherited from Component
 Component (int32 input_dim, int32 output_dim)
 Generic interface of a component,. More...
 
virtual ~Component ()
 
virtual bool IsUpdatable () const
 Check if componeny has 'Updatable' interface (trainable components),. More...
 
virtual bool IsMultistream () const
 Check if component has 'Recurrent' interface (trainable and recurrent),. More...
 
int32 InputDim () const
 Get the dimension of the input,. More...
 
int32 OutputDim () const
 Get the dimension of the output,. More...
 
void Propagate (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out)
 Perform forward-pass propagation 'in' -> 'out',. More...
 
void Backpropagate (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrix< BaseFloat > *in_diff)
 Perform backward-pass propagation 'out_diff' -> 'in_diff'. More...
 
void Write (std::ostream &os, bool binary) const
 Write the component to a stream,. More...
 
virtual std::string Info () const
 Print some additional info (after <ComponentName> and the dims),. More...
 
virtual std::string InfoGradient () const
 Print some additional info about gradient (after <...> and dims),. More...
 

Private Attributes

CuMatrix< BaseFloatl2_aux_
 auxiliary matrix for L2 norm computation, More...
 
CuVector< BaseFloatrow_scales_
 normalization scale of each row, More...
 

Additional Inherited Members

- Public Types inherited from Component
enum  ComponentType {
  kUnknown = 0x0, kUpdatableComponent = 0x0100, kAffineTransform, kLinearTransform,
  kConvolutionalComponent, kLstmProjected, kBlstmProjected, kRecurrentComponent,
  kActivationFunction = 0x0200, kSoftmax, kHiddenSoftmax, kBlockSoftmax,
  kSigmoid, kTanh, kParametricRelu, kDropout,
  kLengthNormComponent, kTranform = 0x0400, kRbm, kSplice,
  kCopy, kTranspose, kBlockLinearity, kAddShift,
  kRescale, kKlHmm = 0x0800, kSentenceAveragingComponent, kSimpleSentenceAveragingComponent,
  kAveragePoolingComponent, kMaxPoolingComponent, kFramePoolingComponent, kParallelComponent,
  kMultiBasisComponent
}
 Component type identification mechanism,. More...
 
- Static Public Member Functions inherited from Component
static const char * TypeToMarker (ComponentType t)
 Converts component type to marker,. More...
 
static ComponentType MarkerToType (const std::string &s)
 Converts marker to component type (case insensitive),. More...
 
static ComponentInit (const std::string &conf_line)
 Initialize component from a line in config file,. More...
 
static ComponentRead (std::istream &is, bool binary)
 Read the component from a stream (static method),. More...
 
- Static Public Attributes inherited from Component
static const struct key_value kMarkerMap []
 The table with pairs of Component types and markers (defined in nnet-component.cc),. More...
 
- Protected Member Functions inherited from Component
virtual void InitData (std::istream &is)
 Virtual interface for initialization and I/O,. More...
 
virtual void ReadData (std::istream &is, bool binary)
 Reads the component content. More...
 
virtual void WriteData (std::ostream &os, bool binary) const
 Writes the component content. More...
 
- Protected Attributes inherited from Component
int32 input_dim_
 Data members,. More...
 
int32 output_dim_
 Dimension of the output of the Component,. More...
 

Detailed Description

Rescale the matrix-rows to have unit length (L2-norm).

Definition at line 244 of file nnet-various.h.

Constructor & Destructor Documentation

◆ LengthNormComponent()

LengthNormComponent ( int32  dim_in,
int32  dim_out 
)
inline

Definition at line 246 of file nnet-various.h.

246  :
247  Component(dim_in, dim_out)
248  { }
Component(int32 input_dim, int32 output_dim)
Generic interface of a component,.

◆ ~LengthNormComponent()

~LengthNormComponent ( )
inline

Definition at line 250 of file nnet-various.h.

251  { }

Member Function Documentation

◆ BackpropagateFnc()

void BackpropagateFnc ( const CuMatrixBase< BaseFloat > &  in,
const CuMatrixBase< BaseFloat > &  out,
const CuMatrixBase< BaseFloat > &  out_diff,
CuMatrixBase< BaseFloat > *  in_diff 
)
inlinevirtual

Backward pass transformation (to be implemented by descending class...)

Implements Component.

Definition at line 273 of file nnet-various.h.

References CuMatrixBase< Real >::CopyFromMat(), and CuMatrixBase< Real >::MulRowsVec().

276  {
277  in_diff->CopyFromMat(out_diff);
278  in_diff->MulRowsVec(row_scales_); // diff_by_x(s * x) = s,
279  }
CuVector< BaseFloat > row_scales_
normalization scale of each row,
Definition: nnet-various.h:283

◆ Copy()

Component* Copy ( ) const
inlinevirtual

Copy component (deep copy),.

Implements Component.

Definition at line 253 of file nnet-various.h.

253 { return new LengthNormComponent(*this); }
LengthNormComponent(int32 dim_in, int32 dim_out)
Definition: nnet-various.h:246

◆ GetType()

ComponentType GetType ( ) const
inlinevirtual

Get Type Identification of the component,.

Implements Component.

Definition at line 254 of file nnet-various.h.

References Component::kLengthNormComponent.

◆ PropagateFnc()

void PropagateFnc ( const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
)
inlinevirtual

Abstract interface for propagation/backpropagation.

Forward pass transformation (to be implemented by descending class...)

Implements Component.

Definition at line 256 of file nnet-various.h.

References CuMatrixBase< Real >::CopyFromMat(), CuMatrixBase< Real >::MulElements(), CuMatrixBase< Real >::MulRowsVec(), and CuMatrixBase< Real >::NumRows().

257  {
258  // resize vector when needed,
259  if (row_scales_.Dim() != in.NumRows()) {
260  row_scales_.Resize(in.NumRows());
261  }
262  // get the normalization scalars,
263  l2_aux_ = in;
264  l2_aux_.MulElements(l2_aux_); // x^2,
265  row_scales_.AddColSumMat(1.0, l2_aux_, 0.0); // sum_of_cols(x^2),
266  row_scales_.ApplyPow(0.5); // L2norm = sqrt(sum_of_cols(x^2)),
267  row_scales_.InvertElements(); // 1/L2norm,
268  // compute the output,
269  out->CopyFromMat(in);
270  out->MulRowsVec(row_scales_); // re-normalize,
271  }
CuMatrix< BaseFloat > l2_aux_
auxiliary matrix for L2 norm computation,
Definition: nnet-various.h:282
CuVector< BaseFloat > row_scales_
normalization scale of each row,
Definition: nnet-various.h:283

Member Data Documentation

◆ l2_aux_

CuMatrix<BaseFloat> l2_aux_
private

auxiliary matrix for L2 norm computation,

Definition at line 282 of file nnet-various.h.

◆ row_scales_

CuVector<BaseFloat> row_scales_
private

normalization scale of each row,

Definition at line 283 of file nnet-various.h.


The documentation for this class was generated from the following file: