All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
PnormComponent Class Reference

#include <nnet-component.h>

Inheritance diagram for PnormComponent:
Collaboration diagram for PnormComponent:

Public Member Functions

void Init (int32 input_dim, int32 output_dim, BaseFloat p)
 
 PnormComponent (int32 input_dim, int32 output_dim, BaseFloat p)
 
 PnormComponent ()
 
virtual std::string Type () const
 
virtual void InitFromString (std::string args)
 Initialize, typically from a line of a config file. More...
 
virtual int32 InputDim () const
 Get size of input vectors. More...
 
virtual int32 OutputDim () const
 Get size of output vectors. More...
 
virtual void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Perform forward pass propagation Input->Output. More...
 
virtual void Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
 Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
 
virtual bool BackpropNeedsInput () const
 
virtual bool BackpropNeedsOutput () const
 
virtual ComponentCopy () const
 Copy component (deep copy). More...
 
virtual void Read (std::istream &is, bool binary)
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
virtual std::string Info () const
 
- Public Member Functions inherited from Component
 Component ()
 
virtual int32 Index () const
 Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
 
virtual void SetIndex (int32 index)
 
virtual std::vector< int32 > Context () const
 Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
 
void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
 A non-virtual propagate function that first resizes output if necessary. More...
 
virtual ~Component ()
 

Protected Attributes

int32 input_dim_
 
int32 output_dim_
 
BaseFloat p_
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream. More...
 
static ComponentNewFromString (const std::string &initializer_line)
 Initialize the Component from one line that will contain first the type, e.g. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Return a new Component of the given type e.g. More...
 

Detailed Description

Definition at line 514 of file nnet-component.h.

Constructor & Destructor Documentation

PnormComponent ( int32  input_dim,
int32  output_dim,
BaseFloat  p 
)
inlineexplicit

Definition at line 517 of file nnet-component.h.

References PnormComponent::Init().

517  {
518  Init(input_dim, output_dim, p);
519  }
void Init(int32 input_dim, int32 output_dim, BaseFloat p)
PnormComponent ( )
inline

Member Function Documentation

void Backprop ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
Component to_update,
CuMatrix< BaseFloat > *  in_deriv 
) const
virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 524 of file nnet-component.cc.

References CuMatrixBase< Real >::DiffGroupPnorm(), kaldi::kSetZero, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), PnormComponent::p_, and CuMatrix< Real >::Resize().

531  {
532  in_deriv->Resize(in_value.NumRows(), in_value.NumCols(), kSetZero);
533  in_deriv->DiffGroupPnorm(in_value, out_value, out_deriv, p_);
534 }
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
void DiffGroupPnorm(const CuMatrixBase< Real > &in_value, const CuMatrixBase< Real > &out_value, const CuMatrixBase< Real > &out_deriv, Real power)
Differentiate backward through the GroupPnorm function.
Definition: cu-matrix.cc:826
virtual bool BackpropNeedsInput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 537 of file nnet-component.h.

537 { return true; }
virtual bool BackpropNeedsOutput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 538 of file nnet-component.h.

538 { return true; }
virtual Component* Copy ( ) const
inlinevirtual
std::string Info ( ) const
virtual

Reimplemented from Component.

Definition at line 557 of file nnet-component.cc.

References PnormComponent::input_dim_, PnormComponent::output_dim_, PnormComponent::p_, and PnormComponent::Type().

557  {
558  std::stringstream stream;
559  stream << Type() << ", input-dim = " << input_dim_
560  << ", output-dim = " << output_dim_
561  << ", p = " << p_;
562  return stream.str();
563 }
virtual std::string Type() const
void Init ( int32  input_dim,
int32  output_dim,
BaseFloat  p 
)

Definition at line 488 of file nnet-component.cc.

References PnormComponent::input_dim_, KALDI_ASSERT, PnormComponent::output_dim_, and PnormComponent::p_.

Referenced by PnormComponent::InitFromString(), and PnormComponent::PnormComponent().

488  {
489  input_dim_ = input_dim;
490  output_dim_ = output_dim;
491  if (input_dim_ == 0)
492  input_dim_ = 10 * output_dim_; // default group size : 10
493  p_ = p;
494  KALDI_ASSERT(input_dim_ > 0 && output_dim_ >= 0 && p_ >= 0);
496 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InitFromString ( std::string  args)
virtual

Initialize, typically from a line of a config file.

The "args" will contain any parameters that need to be passed to the Component, e.g. dimensions.

Implements Component.

Definition at line 498 of file nnet-component.cc.

References PnormComponent::Init(), KALDI_ERR, kaldi::nnet2::ParseFromString(), and PnormComponent::Type().

498  {
499  std::string orig_args(args);
500  int32 input_dim = 0;
501  int32 output_dim = 0;
502  BaseFloat p = 2;
503  bool ok = ParseFromString("output-dim", &args, &output_dim) &&
504  ParseFromString("input-dim", &args, &input_dim);
505  ParseFromString("p", &args, &p);
506  if (!ok || !args.empty() || output_dim <= 0)
507  KALDI_ERR << "Invalid initializer for layer of type "
508  << Type() << ": \"" << orig_args << "\"";
509  Init(input_dim, output_dim, p);
510 }
bool ParseFromString(const std::string &name, std::string *string, int32 *param)
Functions used in Init routines.
float BaseFloat
Definition: kaldi-types.h:29
void Init(int32 input_dim, int32 output_dim, BaseFloat p)
virtual std::string Type() const
#define KALDI_ERR
Definition: kaldi-error.h:127
virtual int32 InputDim ( ) const
inlinevirtual

Get size of input vectors.

Implements Component.

Definition at line 523 of file nnet-component.h.

References PnormComponent::input_dim_.

523 { return input_dim_; }
virtual int32 OutputDim ( ) const
inlinevirtual

Get size of output vectors.

Implements Component.

Definition at line 524 of file nnet-component.h.

References PnormComponent::output_dim_.

524 { return output_dim_; }
void Propagate ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 513 of file nnet-component.cc.

References ChunkInfo::CheckSize(), CuMatrixBase< Real >::GroupPnorm(), KALDI_ASSERT, ChunkInfo::NumChunks(), and PnormComponent::p_.

516  {
517  in_info.CheckSize(in);
518  out_info.CheckSize(*out);
519  KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());
520 
521  out->GroupPnorm(in, p_);
522 }
void GroupPnorm(const CuMatrixBase< Real > &src, Real pow)
Apply the function y(i) = (sum_{j = i*G}^{(i+1)*G-1} x_j ^ (power)) ^ (1 / p) where G = x...
Definition: cu-matrix.cc:1427
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Read ( std::istream &  is,
bool  binary 
)
virtual

Implements Component.

Definition at line 536 of file nnet-component.cc.

References kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), PnormComponent::input_dim_, PnormComponent::output_dim_, PnormComponent::p_, and kaldi::ReadBasicType().

536  {
537  ExpectOneOrTwoTokens(is, binary, "<PnormComponent>", "<InputDim>");
538  ReadBasicType(is, binary, &input_dim_);
539  ExpectToken(is, binary, "<OutputDim>");
540  ReadBasicType(is, binary, &output_dim_);
541  ExpectToken(is, binary, "<P>");
542  ReadBasicType(is, binary, &p_);
543  ExpectToken(is, binary, "</PnormComponent>");
544 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:188
static void ExpectOneOrTwoTokens(std::istream &is, bool binary, const std::string &token1, const std::string &token2)
virtual std::string Type ( ) const
inlinevirtual

Implements Component.

Definition at line 521 of file nnet-component.h.

Referenced by PnormComponent::Info(), and PnormComponent::InitFromString().

521 { return "PnormComponent"; }
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Definition at line 546 of file nnet-component.cc.

References PnormComponent::input_dim_, PnormComponent::output_dim_, PnormComponent::p_, kaldi::WriteBasicType(), and kaldi::WriteToken().

546  {
547  WriteToken(os, binary, "<PnormComponent>");
548  WriteToken(os, binary, "<InputDim>");
549  WriteBasicType(os, binary, input_dim_);
550  WriteToken(os, binary, "<OutputDim>");
551  WriteBasicType(os, binary, output_dim_);
552  WriteToken(os, binary, "<P>");
553  WriteBasicType(os, binary, p_);
554  WriteToken(os, binary, "</PnormComponent>");
555 }
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
void WriteBasicType(std::ostream &os, bool binary, T t)
WriteBasicType is the name of the write function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:34

Member Data Documentation


The documentation for this class was generated from the following files: