All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
PowerComponent Class Reference

Take the absoute values of an input vector to a power. More...

#include <nnet-component.h>

Inheritance diagram for PowerComponent:
Collaboration diagram for PowerComponent:

Public Member Functions

void Init (int32 dim, BaseFloat power=2)
 
 PowerComponent (int32 dim, BaseFloat power=2)
 
 PowerComponent ()
 
virtual std::string Type () const
 
virtual void InitFromString (std::string args)
 We implement InitFromString at this level. More...
 
virtual int32 InputDim () const
 Get size of input vectors. More...
 
virtual int32 OutputDim () const
 Get size of output vectors. More...
 
virtual void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Perform forward pass propagation Input->Output. More...
 
virtual void Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
 Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
 
virtual bool BackpropNeedsInput () const
 
virtual bool BackpropNeedsOutput () const
 
virtual ComponentCopy () const
 Copy component (deep copy). More...
 
virtual void Read (std::istream &is, bool binary)
 We implement Read at this level as it just needs the Type(). More...
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
virtual std::string Info () const
 
- Public Member Functions inherited from NonlinearComponent
void Init (int32 dim)
 
 NonlinearComponent (int32 dim)
 
 NonlinearComponent ()
 
 NonlinearComponent (const NonlinearComponent &other)
 
void Scale (BaseFloat scale)
 
void Add (BaseFloat alpha, const NonlinearComponent &other)
 
const CuVector< double > & ValueSum () const
 
const CuVector< double > & DerivSum () const
 
double Count () const
 
void SetDim (int32 dim)
 
- Public Member Functions inherited from Component
 Component ()
 
virtual int32 Index () const
 Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
 
virtual void SetIndex (int32 index)
 
virtual std::vector< int32 > Context () const
 Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
 
void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
 A non-virtual propagate function that first resizes output if necessary. More...
 
virtual ~Component ()
 

Private Attributes

int32 dim_
 
BaseFloat power_
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream. More...
 
static ComponentNewFromString (const std::string &initializer_line)
 Initialize the Component from one line that will contain first the type, e.g. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Return a new Component of the given type e.g. More...
 
- Protected Member Functions inherited from NonlinearComponent
void UpdateStats (const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > *deriv=NULL)
 
const NonlinearComponentoperator= (const NonlinearComponent &other)
 
- Protected Attributes inherited from NonlinearComponent
int32 dim_
 
CuVector< double > value_sum_
 
CuVector< double > deriv_sum_
 
double count_
 
std::mutex mutex_
 

Detailed Description

Take the absoute values of an input vector to a power.

The derivative for zero input will be treated as zero.

Definition at line 637 of file nnet-component.h.

Constructor & Destructor Documentation

PowerComponent ( int32  dim,
BaseFloat  power = 2 
)
inlineexplicit

Definition at line 640 of file nnet-component.h.

References PowerComponent::Init().

640  {
641  Init(dim, power);
642  }
void Init(int32 dim, BaseFloat power=2)
PowerComponent ( )
inline

Definition at line 643 of file nnet-component.h.

Referenced by PowerComponent::Copy().

Member Function Documentation

void Backprop ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
Component to_update,
CuMatrix< BaseFloat > *  in_deriv 
) const
virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 726 of file nnet-component.cc.

References CuMatrixBase< Real >::ApplyPowAbs(), CuMatrixBase< Real >::CopyFromMat(), CuMatrixBase< Real >::MulElements(), CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), PowerComponent::power_, CuMatrix< Real >::Resize(), and CuMatrixBase< Real >::Scale().

732  {
733  in_deriv->Resize(in_value.NumRows(), in_value.NumCols());
734  // in scalar terms: in_deriv += p * in_value^(p-1) * out_deriv
735  in_deriv->CopyFromMat(in_value);
736  in_deriv->ApplyPowAbs(power_ - 1.0, true);
737  in_deriv->Scale(power_);
738  in_deriv->MulElements(out_deriv);
739 }
void Scale(Real value)
Definition: cu-matrix.cc:608
MatrixIndexT NumCols() const
Definition: cu-matrix.h:196
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:337
void Resize(MatrixIndexT rows, MatrixIndexT cols, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Allocate the memory.
Definition: cu-matrix.cc:47
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:195
void ApplyPowAbs(Real power, bool include_sign=false)
Definition: cu-matrix.cc:2291
void MulElements(const CuMatrixBase< Real > &A)
Multiply two matrices elementwise: C = C .* A.
Definition: cu-matrix.cc:652
virtual bool BackpropNeedsInput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 660 of file nnet-component.h.

660 { return true; }
virtual bool BackpropNeedsOutput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 661 of file nnet-component.h.

661 { return true; }
virtual Component* Copy ( ) const
inlinevirtual

Copy component (deep copy).

Implements Component.

Definition at line 662 of file nnet-component.h.

References PowerComponent::dim_, PowerComponent::power_, and PowerComponent::PowerComponent().

std::string Info ( ) const
virtual

Reimplemented from Component.

Definition at line 762 of file nnet-component.cc.

References PowerComponent::dim_, PowerComponent::power_, and PowerComponent::Type().

762  {
763  std::stringstream stream;
764  stream << Type() << ", dim = " << dim_
765  << ", power = " << power_;
766  return stream.str();
767 }
virtual std::string Type() const
void Init ( int32  dim,
BaseFloat  power = 2 
)

Definition at line 692 of file nnet-component.cc.

References PowerComponent::dim_, KALDI_ASSERT, and PowerComponent::power_.

Referenced by PowerComponent::InitFromString(), and PowerComponent::PowerComponent().

692  {
693  dim_ = dim;
694  power_ = power;
695  KALDI_ASSERT(dim > 0 && power >= 0);
696 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void InitFromString ( std::string  args)
virtual

We implement InitFromString at this level.

Reimplemented from NonlinearComponent.

Definition at line 698 of file nnet-component.cc.

References PowerComponent::Init(), KALDI_ERR, kaldi::nnet2::ParseFromString(), and PowerComponent::Type().

698  {
699  std::string orig_args(args);
700  int32 dim;
701  BaseFloat power = 2.0;
702  ParseFromString("power", &args, &power); // Optional.
703  // Accept either "dim" or "input-dim" to specify the input dim.
704  // "input-dim" is the canonical one; "dim" simplifies the testing code.
705  bool ok = (ParseFromString("dim", &args, &dim) ||
706  ParseFromString("input-dim", &args, &dim));
707  if (!ok || !args.empty() || dim <= 0)
708  KALDI_ERR << "Invalid initializer for layer of type "
709  << Type() << ": \"" << orig_args << "\"";
710  Init(dim, power);
711 }
virtual std::string Type() const
bool ParseFromString(const std::string &name, std::string *string, int32 *param)
Functions used in Init routines.
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ERR
Definition: kaldi-error.h:127
void Init(int32 dim, BaseFloat power=2)
virtual int32 InputDim ( ) const
inlinevirtual

Get size of input vectors.

Reimplemented from NonlinearComponent.

Definition at line 646 of file nnet-component.h.

References PowerComponent::dim_.

646 { return dim_; }
virtual int32 OutputDim ( ) const
inlinevirtual

Get size of output vectors.

Reimplemented from NonlinearComponent.

Definition at line 647 of file nnet-component.h.

References PowerComponent::dim_.

647 { return dim_; }
void Propagate ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 713 of file nnet-component.cc.

References CuMatrixBase< Real >::ApplyPowAbs(), ChunkInfo::CheckSize(), CuMatrixBase< Real >::CopyFromMat(), KALDI_ASSERT, ChunkInfo::NumChunks(), and PowerComponent::power_.

716  {
717  in_info.CheckSize(in);
718  out_info.CheckSize(*out);
719  KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());
720 
721  // Apply power operation to each element of the input...
722  out->CopyFromMat(in);
723  out->ApplyPowAbs(power_);
724 }
void CopyFromMat(const MatrixBase< OtherReal > &src, MatrixTransposeType trans=kNoTrans)
Definition: cu-matrix.cc:337
void ApplyPowAbs(Real power, bool include_sign=false)
Definition: cu-matrix.cc:2291
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Read ( std::istream &  is,
bool  binary 
)
virtual

We implement Read at this level as it just needs the Type().

Reimplemented from NonlinearComponent.

Definition at line 741 of file nnet-component.cc.

References PowerComponent::dim_, kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), PowerComponent::power_, and kaldi::ReadBasicType().

741  {
742  ExpectOneOrTwoTokens(is, binary, "<PowerComponent>", "<InputDim>");
743  ReadBasicType(is, binary, &dim_);
744  ExpectToken(is, binary, "<OutputDim>");
745  ReadBasicType(is, binary, &dim_);
746  ExpectToken(is, binary, "<Power>");
747  ReadBasicType(is, binary, &power_);
748  ExpectToken(is, binary, "</PowerComponent>");
749 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:188
static void ExpectOneOrTwoTokens(std::istream &is, bool binary, const std::string &token1, const std::string &token2)
virtual std::string Type ( ) const
inlinevirtual

Implements Component.

Definition at line 644 of file nnet-component.h.

Referenced by PowerComponent::Info(), and PowerComponent::InitFromString().

644 { return "PowerComponent"; }
void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Reimplemented from NonlinearComponent.

Definition at line 751 of file nnet-component.cc.

References PowerComponent::dim_, PowerComponent::power_, kaldi::WriteBasicType(), and kaldi::WriteToken().

751  {
752  WriteToken(os, binary, "<PowerComponent>");
753  WriteToken(os, binary, "<InputDim>");
754  WriteBasicType(os, binary, dim_);
755  WriteToken(os, binary, "<OutputDim>");
756  WriteBasicType(os, binary, dim_);
757  WriteToken(os, binary, "<Power>");
758  WriteBasicType(os, binary, power_);
759  WriteToken(os, binary, "</PowerComponent>");
760 }
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
void WriteBasicType(std::ostream &os, bool binary, T t)
WriteBasicType is the name of the write function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:34

Member Data Documentation


The documentation for this class was generated from the following files: