All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
LogSoftmaxComponent Class Reference

#include <nnet-component.h>

Inheritance diagram for LogSoftmaxComponent:
Collaboration diagram for LogSoftmaxComponent:

Public Member Functions

 LogSoftmaxComponent (int32 dim)
 
 LogSoftmaxComponent (const LogSoftmaxComponent &other)
 
 LogSoftmaxComponent ()
 
virtual std::string Type () const
 
virtual bool BackpropNeedsInput () const
 
virtual bool BackpropNeedsOutput () const
 
virtual void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Perform forward pass propagation Input->Output. More...
 
virtual void Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
 Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
 
virtual ComponentCopy () const
 Copy component (deep copy). More...
 
- Public Member Functions inherited from NonlinearComponent
void Init (int32 dim)
 
 NonlinearComponent (int32 dim)
 
 NonlinearComponent ()
 
 NonlinearComponent (const NonlinearComponent &other)
 
virtual int32 InputDim () const
 Get size of input vectors. More...
 
virtual int32 OutputDim () const
 Get size of output vectors. More...
 
virtual void InitFromString (std::string args)
 We implement InitFromString at this level. More...
 
virtual void Read (std::istream &is, bool binary)
 We implement Read at this level as it just needs the Type(). More...
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
void Scale (BaseFloat scale)
 
void Add (BaseFloat alpha, const NonlinearComponent &other)
 
const CuVector< double > & ValueSum () const
 
const CuVector< double > & DerivSum () const
 
double Count () const
 
void SetDim (int32 dim)
 
- Public Member Functions inherited from Component
 Component ()
 
virtual int32 Index () const
 Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
 
virtual void SetIndex (int32 index)
 
virtual std::vector< int32 > Context () const
 Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
 
void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
 A non-virtual propagate function that first resizes output if necessary. More...
 
virtual std::string Info () const
 
virtual ~Component ()
 

Private Member Functions

LogSoftmaxComponentoperator= (const LogSoftmaxComponent &other)
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream. More...
 
static ComponentNewFromString (const std::string &initializer_line)
 Initialize the Component from one line that will contain first the type, e.g. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Return a new Component of the given type e.g. More...
 
- Protected Member Functions inherited from NonlinearComponent
void UpdateStats (const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > *deriv=NULL)
 
const NonlinearComponentoperator= (const NonlinearComponent &other)
 
- Protected Attributes inherited from NonlinearComponent
int32 dim_
 
CuVector< double > value_sum_
 
CuVector< double > deriv_sum_
 
double count_
 
std::mutex mutex_
 

Detailed Description

Definition at line 810 of file nnet-component.h.

Constructor & Destructor Documentation

LogSoftmaxComponent ( int32  dim)
inlineexplicit

Definition at line 812 of file nnet-component.h.

LogSoftmaxComponent ( const LogSoftmaxComponent other)
inlineexplicit

Definition at line 813 of file nnet-component.h.

LogSoftmaxComponent ( )
inline

Definition at line 814 of file nnet-component.h.

Referenced by LogSoftmaxComponent::Copy().

814 { }

Member Function Documentation

void Backprop ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
Component to_update,
CuMatrix< BaseFloat > *  in_deriv 
) const
virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 965 of file nnet-component.cc.

References CuMatrixBase< Real >::DiffLogSoftmaxPerRow(), KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrix< Real >::Resize(), kaldi::SameDim(), and NonlinearComponent::UpdateStats().

971  {
972  /*
973  Let the output be y, then
974  y_i = x_i - log(sum_i exp(x_i))
975  where x_i is the input to the component. The Jacobian matrix of this
976  function is
977  J = I - 1 exp(y^T)
978  where 1 is a vector of ones. Let the derivative vector at the output be e,
979  and at the input be d, then we have
980  d = e - exp(y) Sum(e)
981  d_i = e_i - exp(y_i) Sum(e)
982  */
983  in_deriv->Resize(out_deriv.NumRows(), out_deriv.NumCols());
984  KALDI_ASSERT(SameDim(out_value, out_deriv) && SameDim(out_value, *in_deriv));
985 
986  in_deriv->DiffLogSoftmaxPerRow(out_value, out_deriv);
987 
988  // Updates stats.
989  if (to_update != NULL) {
990  NonlinearComponent *to_update_nonlinear =
991  dynamic_cast<NonlinearComponent*>(to_update);
992  to_update_nonlinear->UpdateStats(out_value);
993  }
994 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
void DiffLogSoftmaxPerRow(const CuMatrixBase< Real > &out_value, const CuMatrixBase< Real > &out_deriv)
Differentiate backward through the log softmax function.
Definition: cu-matrix.cc:1749
bool SameDim(const MatrixBase< Real > &M, const MatrixBase< Real > &N)
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual bool BackpropNeedsInput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 816 of file nnet-component.h.

816 { return false; }
virtual bool BackpropNeedsOutput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 817 of file nnet-component.h.

817 { return true; }
virtual Component* Copy ( ) const
inlinevirtual

Copy component (deep copy).

Implements Component.

Definition at line 831 of file nnet-component.h.

References LogSoftmaxComponent::LogSoftmaxComponent().

831 { return new LogSoftmaxComponent(*this); }
LogSoftmaxComponent& operator= ( const LogSoftmaxComponent other)
private
void Propagate ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 949 of file nnet-component.cc.

References CuMatrixBase< Real >::ApplyFloor(), CuMatrixBase< Real >::ApplyLogSoftMaxPerRow(), ChunkInfo::CheckSize(), KALDI_ASSERT, kaldi::Log(), and ChunkInfo::NumChunks().

952  {
953  in_info.CheckSize(in);
954  out_info.CheckSize(*out);
955  KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());
956 
957  // Applies log softmax function to each row of the output. For each row, we do
958  // x_i = x_i - log(sum_j exp(x_j))
959  out->ApplyLogSoftMaxPerRow(in);
960 
961  // Just to be consistent with SoftmaxComponent::Propagate()
962  out->ApplyFloor(Log(1.0e-20));
963 }
void ApplyFloor(Real floor_val)
Definition: cu-matrix.cc:2365
double Log(double x)
Definition: kaldi-math.h:100
void ApplyLogSoftMaxPerRow(const CuMatrixBase< Real > &src)
LogSoftmax nonlinearity Y = LogSoftmax(X) : Yij = Xij - log(sum_k(e^Xik)), done to each row...
Definition: cu-matrix.cc:1586
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
virtual std::string Type ( ) const
inlinevirtual

Implements Component.

Definition at line 815 of file nnet-component.h.

815 { return "LogSoftmaxComponent"; }

The documentation for this class was generated from the following files: