PowerComponent Class Reference

Take the absoute values of an input vector to a power. More...

#include <nnet-component.h>

Inheritance diagram for PowerComponent:

Collaboration diagram for PowerComponent:

Public Member Functions
void	Init (int32 dim, BaseFloat power=2)
	PowerComponent (int32 dim, BaseFloat power=2)
	PowerComponent ()
virtual std::string	Type () const
virtual void	InitFromString (std::string args)
	We implement InitFromString at this level. More...
virtual int32	InputDim () const
	Get size of input vectors. More...
virtual int32	OutputDim () const
	Get size of output vectors. More...
virtual void	Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
	Perform forward pass propagation Input->Output. More...
virtual void	Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
	Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
virtual bool	BackpropNeedsInput () const
virtual bool	BackpropNeedsOutput () const
virtual Component *	Copy () const
	Copy component (deep copy). More...
virtual void	Read (std::istream &is, bool binary)
	We implement Read at this level as it just needs the Type(). More...
virtual void	Write (std::ostream &os, bool binary) const
	Write component to stream. More...
virtual std::string	Info () const
Public Member Functions inherited from NonlinearComponent
void	Init (int32 dim)
	NonlinearComponent (int32 dim)
	NonlinearComponent ()
	NonlinearComponent (const NonlinearComponent &other)
void	Scale (BaseFloat scale)
void	Add (BaseFloat alpha, const NonlinearComponent &other)
const CuVector< double > &	ValueSum () const
const CuVector< double > &	DerivSum () const
double	Count () const
void	SetDim (int32 dim)
Public Member Functions inherited from Component
	Component ()
virtual int32	Index () const
	Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
virtual void	SetIndex (int32 index)
virtual std::vector< int32 >	Context () const
	Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
void	Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
	A non-virtual propagate function that first resizes output if necessary. More...
virtual	~Component ()

Private Attributes
int32	dim_
BaseFloat	power_

Additional Inherited Members
Static Public Member Functions inherited from Component
static Component *	ReadNew (std::istream &is, bool binary)
	Read component from stream. More...
static Component *	NewFromString (const std::string &initializer_line)
	Initialize the Component from one line that will contain first the type, e.g. More...
static Component *	NewComponentOfType (const std::string &type)
	Return a new Component of the given type e.g. More...
Protected Member Functions inherited from NonlinearComponent
void	UpdateStats (const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > *deriv=NULL)
const NonlinearComponent &	operator= (const NonlinearComponent &other)
Protected Attributes inherited from NonlinearComponent
int32	dim_
CuVector< double >	value_sum_
CuVector< double >	deriv_sum_
double	count_
std::mutex	mutex_

Detailed Description

Take the absoute values of an input vector to a power.

The derivative for zero input will be treated as zero.

Definition at line 637 of file nnet-component.h.

Constructor & Destructor Documentation

PowerComponent() [1/2]

PowerComponent ( int32  dim, BaseFloat  power = 2  )

inlineexplicit

Definition at line 640 of file nnet-component.h.

                                                          {
    Init(dim, power);
  }

PowerComponent() [2/2]

PowerComponent ( )

inline

Definition at line 643 of file nnet-component.h.

: dim_(0), power_(2) { }

Member Function Documentation

Backprop()

void Backprop ( const ChunkInfo &  in_info, const ChunkInfo &  out_info, const CuMatrixBase< BaseFloat > &  in_value, const CuMatrixBase< BaseFloat > &  out_value, const CuMatrixBase< BaseFloat > &  out_deriv, Component *  to_update, CuMatrix< BaseFloat > *  in_deriv  ) const

virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 726 of file nnet-component.cc.

References CuMatrixBase< Real >::ApplyPowAbs(), CuMatrixBase< Real >::CopyFromMat(), CuMatrixBase< Real >::MulElements(), CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrix< Real >::Resize(), and CuMatrixBase< Real >::Scale().

                                                                    {
  in_deriv->Resize(in_value.NumRows(), in_value.NumCols());
  // in scalar terms: in_deriv += p * in_value^(p-1) * out_deriv
  in_deriv->CopyFromMat(in_value);
  in_deriv->ApplyPowAbs(power_ - 1.0, true);
  in_deriv->Scale(power_);
  in_deriv->MulElements(out_deriv);
}

BackpropNeedsInput()

virtual bool BackpropNeedsInput ( ) const

inlinevirtual

Reimplemented from Component.

Definition at line 660 of file nnet-component.h.

{ return true; }

BackpropNeedsOutput()

virtual bool BackpropNeedsOutput ( ) const

inlinevirtual

Reimplemented from Component.

Definition at line 661 of file nnet-component.h.

{ return true; }

Copy()

virtual Component* Copy ( ) const

inlinevirtual

Copy component (deep copy).

Implements Component.

Definition at line 662 of file nnet-component.h.

{ return new PowerComponent(dim_, power_); }

Info()

std::string Info ( ) const

virtual

Reimplemented from Component.

Definition at line 762 of file nnet-component.cc.

References NonlinearComponent::dim_, and Component::Type().

                                     {
  std::stringstream stream;
  stream << Type() << ", dim = " << dim_
     << ", power = " << power_;
  return stream.str();
}

Init()

void Init	(	int32	dim,
		BaseFloat	power = 2
	)

Definition at line 692 of file nnet-component.cc.

References NonlinearComponent::dim_, and KALDI_ASSERT.

                                                    {
  dim_ = dim;
  power_ = power;
  KALDI_ASSERT(dim > 0 && power >= 0);
}

InitFromString()

void InitFromString ( std::string  args )

virtual

We implement InitFromString at this level.

Reimplemented from NonlinearComponent.

Definition at line 698 of file nnet-component.cc.

References NonlinearComponent::Init(), KALDI_ERR, kaldi::nnet2::ParseFromString(), and Component::Type().

                                                  {
  std::string orig_args(args);
  int32 dim;
  BaseFloat power = 2.0;
  ParseFromString("power", &args, &power); // Optional.
  // Accept either "dim" or "input-dim" to specify the input dim.
  // "input-dim" is the canonical one; "dim" simplifies the testing code.
  bool ok = (ParseFromString("dim", &args, &dim) ||
             ParseFromString("input-dim", &args, &dim));
  if (!ok || !args.empty() || dim <= 0)
    KALDI_ERR << "Invalid initializer for layer of type "
              << Type() << ": \"" << orig_args << "\"";
  Init(dim, power);
}

InputDim()

virtual int32 InputDim ( ) const

inlinevirtual

Get size of input vectors.

Reimplemented from NonlinearComponent.

Definition at line 646 of file nnet-component.h.

{ return dim_; }

OutputDim()

virtual int32 OutputDim ( ) const

inlinevirtual

Get size of output vectors.

Reimplemented from NonlinearComponent.

Definition at line 647 of file nnet-component.h.

References Component::Propagate().

{ return dim_; }

Propagate()

void Propagate ( const ChunkInfo &  in_info, const ChunkInfo &  out_info, const CuMatrixBase< BaseFloat > &  in, CuMatrixBase< BaseFloat > *  out  ) const

virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 713 of file nnet-component.cc.

References CuMatrixBase< Real >::ApplyPowAbs(), ChunkInfo::CheckSize(), CuMatrixBase< Real >::CopyFromMat(), KALDI_ASSERT, and ChunkInfo::NumChunks().

                                                                    {
  in_info.CheckSize(in);
  out_info.CheckSize(*out);
  KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());

  // Apply power operation to each element of the input...
  out->CopyFromMat(in);
  out->ApplyPowAbs(power_);
}

Read()

void Read ( std::istream &  is, bool  binary  )

virtual

We implement Read at this level as it just needs the Type().

Reimplemented from NonlinearComponent.

Definition at line 741 of file nnet-component.cc.

References NonlinearComponent::dim_, kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), and kaldi::ReadBasicType().

                                                     {
  ExpectOneOrTwoTokens(is, binary, "<PowerComponent>", "<InputDim>");
  ReadBasicType(is, binary, &dim_);
  ExpectToken(is, binary, "<OutputDim>");
  ReadBasicType(is, binary, &dim_);
  ExpectToken(is, binary, "<Power>");
  ReadBasicType(is, binary, &power_);
  ExpectToken(is, binary, "</PowerComponent>");
}

Type()

virtual std::string Type ( ) const

inlinevirtual

Implements Component.

Definition at line 644 of file nnet-component.h.

{ return "PowerComponent"; }

Write()

void Write ( std::ostream &  os, bool  binary  ) const

virtual

Write component to stream.

Reimplemented from NonlinearComponent.

Definition at line 751 of file nnet-component.cc.

References NonlinearComponent::dim_, kaldi::WriteBasicType(), and kaldi::WriteToken().

                                                            {
  WriteToken(os, binary, "<PowerComponent>");
  WriteToken(os, binary, "<InputDim>");
  WriteBasicType(os, binary, dim_);
  WriteToken(os, binary, "<OutputDim>");
  WriteBasicType(os, binary, dim_);
  WriteToken(os, binary, "<Power>");
  WriteBasicType(os, binary, power_);
  WriteToken(os, binary, "</PowerComponent>");
}

Member Data Documentation

dim_

int32 dim_

private

Definition at line 672 of file nnet-component.h.

power_

BaseFloat power_

private

Definition at line 673 of file nnet-component.h.

---

The documentation for this class was generated from the following files:

• nnet2/nnet-component.h
• nnet2/nnet-component.cc