FixedAffineComponent Class Reference

FixedAffineComponent is an affine transform that is supplied at network initialization time and is not trainable. More...

#include <nnet-component.h>

Inheritance diagram for FixedAffineComponent:

Collaboration diagram for FixedAffineComponent:

Public Member Functions
	FixedAffineComponent ()
virtual std::string	Type () const
virtual std::string	Info () const
void	Init (const CuMatrixBase< BaseFloat > &matrix)
	matrix should be of size input-dim+1 to output-dim, last col is offset More...
virtual void	InitFromString (std::string args)
	Initialize, typically from a line of a config file. More...
virtual int32	InputDim () const
	Get size of input vectors. More...
virtual int32	OutputDim () const
	Get size of output vectors. More...
virtual void	Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
	Perform forward pass propagation Input->Output. More...
virtual void	Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
	Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
virtual bool	BackpropNeedsInput () const
virtual bool	BackpropNeedsOutput () const
virtual Component *	Copy () const
	Copy component (deep copy). More...
virtual void	Read (std::istream &is, bool binary)
virtual void	Write (std::ostream &os, bool binary) const
	Write component to stream. More...
const CuMatrix< BaseFloat > &	LinearParams () const
Public Member Functions inherited from Component
	Component ()
virtual int32	Index () const
	Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
virtual void	SetIndex (int32 index)
virtual std::vector< int32 >	Context () const
	Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
void	Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
	A non-virtual propagate function that first resizes output if necessary. More...
virtual	~Component ()

Protected Member Functions
	KALDI_DISALLOW_COPY_AND_ASSIGN (FixedAffineComponent)

Protected Attributes
CuMatrix< BaseFloat >	linear_params_
CuVector< BaseFloat >	bias_params_

Friends
class	AffineComponent

Additional Inherited Members
Static Public Member Functions inherited from Component
static Component *	ReadNew (std::istream &is, bool binary)
	Read component from stream. More...
static Component *	NewFromString (const std::string &initializer_line)
	Initialize the Component from one line that will contain first the type, e.g. More...
static Component *	NewComponentOfType (const std::string &type)
	Return a new Component of the given type e.g. More...

Detailed Description

FixedAffineComponent is an affine transform that is supplied at network initialization time and is not trainable.

Definition at line 1454 of file nnet-component.h.

Constructor & Destructor Documentation

FixedAffineComponent()

FixedAffineComponent ( )

inline

Definition at line 1456 of file nnet-component.h.

{ }

Member Function Documentation

Backprop()

void Backprop ( const ChunkInfo &  in_info, const ChunkInfo &  out_info, const CuMatrixBase< BaseFloat > &  in_value, const CuMatrixBase< BaseFloat > &  out_value, const CuMatrixBase< BaseFloat > &  out_deriv, Component *  to_update, CuMatrix< BaseFloat > *  in_deriv  ) const

virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 3300 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), kaldi::kNoTrans, AffineComponent::linear_params_, CuMatrixBase< Real >::NumRows(), and CuMatrix< Real >::Resize().

                                                                          {
  in_deriv->Resize(out_deriv.NumRows(), linear_params_.NumCols());
  in_deriv->AddMatMat(1.0, out_deriv, kNoTrans, linear_params_, kNoTrans, 0.0);
}

BackpropNeedsInput()

virtual bool BackpropNeedsInput ( ) const

inlinevirtual

Reimplemented from Component.

Definition at line 1481 of file nnet-component.h.

{ return false; }

BackpropNeedsOutput()

virtual bool BackpropNeedsOutput ( ) const

inlinevirtual

Reimplemented from Component.

Definition at line 1482 of file nnet-component.h.

References kaldi::cu::Copy().

{ return false; }

Copy()

Component * Copy ( ) const

virtual

Copy component (deep copy).

Implements Component.

Definition at line 3311 of file nnet-component.cc.

References AffineComponent::bias_params_, FixedAffineComponent::bias_params_, AffineComponent::linear_params_, and FixedAffineComponent::linear_params_.

Referenced by AffineComponent::CollapseWithNext(), and AffineComponent::CollapseWithPrevious().

                                            {
  FixedAffineComponent *ans = new FixedAffineComponent();
  ans->linear_params_ = linear_params_;
  ans->bias_params_ = bias_params_;
  return ans;
}

Info()

std::string Info ( ) const

virtual

Reimplemented from Component.

Definition at line 3272 of file nnet-component.cc.

References AffineComponent::bias_params_, Component::Info(), kaldi::kTrans, AffineComponent::linear_params_, kaldi::TraceMatMat(), and kaldi::VecVec().

                                           {
  std::stringstream stream;
  BaseFloat linear_params_size = static_cast<BaseFloat>(linear_params_.NumRows())
      * static_cast<BaseFloat>(linear_params_.NumCols()),
      linear_params_stddev =
      std::sqrt(TraceMatMat(linear_params_,
                            linear_params_, kTrans) /
                linear_params_size),
      bias_params_stddev = std::sqrt(VecVec(bias_params_, bias_params_) /
                                     bias_params_.Dim());

  stream << Component::Info() << ", linear-params-stddev=" << linear_params_stddev
         << ", bias-params-stddev=" << bias_params_stddev;
  return stream.str();
}

Init()

void Init

(

const CuMatrixBase< BaseFloat > &

matrix

)

matrix should be of size input-dim+1 to output-dim, last col is offset

Definition at line 3245 of file nnet-component.cc.

References AffineComponent::bias_params_, KALDI_ASSERT, AffineComponent::linear_params_, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and CuMatrixBase< Real >::Range().

Referenced by kaldi::nnet2::UnitTestFixedAffineComponent().

                                                                  {
  KALDI_ASSERT(mat.NumCols() > 1);
  linear_params_ = mat.Range(0, mat.NumRows(),
                             0, mat.NumCols() - 1);
  bias_params_.Resize(mat.NumRows());
  bias_params_.CopyColFromMat(mat, mat.NumCols() - 1);
}

InitFromString()

void InitFromString ( std::string  args )

virtual

Initialize, typically from a line of a config file.

The "args" will contain any parameters that need to be passed to the Component, e.g. dimensions.

Implements Component.

Definition at line 3254 of file nnet-component.cc.

References AffineComponentPreconditionedOnline::Init(), KALDI_ASSERT, KALDI_ERR, CuMatrixBase< Real >::NumRows(), kaldi::nnet2::ParseFromString(), CuMatrix< Real >::Read(), Input::Stream(), and AffineComponentPreconditionedOnline::Type().

                                                        {
  std::string orig_args = args;
  std::string filename;
  bool ok = ParseFromString("matrix", &args, &filename);

  if (!ok || !args.empty())
    KALDI_ERR << "Invalid initializer for layer of type "
              << Type() << ": \"" << orig_args << "\"";

  bool binary;
  Input ki(filename, &binary);
  CuMatrix<BaseFloat> mat;
  mat.Read(ki.Stream(), binary);
  KALDI_ASSERT(mat.NumRows() != 0);
  Init(mat);
}

InputDim()

virtual int32 InputDim ( ) const

inlinevirtual

Get size of input vectors.

Implements Component.

Definition at line 1467 of file nnet-component.h.

Referenced by AffineComponent::CollapseWithPrevious().

{ return linear_params_.NumCols(); }

KALDI_DISALLOW_COPY_AND_ASSIGN()

KALDI_DISALLOW_COPY_AND_ASSIGN ( FixedAffineComponent  )

protected

LinearParams()

const CuMatrix<BaseFloat>& LinearParams ( ) const

inline

Definition at line 1488 of file nnet-component.h.

{ return linear_params_; }

OutputDim()

virtual int32 OutputDim ( ) const

inlinevirtual

Get size of output vectors.

Implements Component.

Definition at line 1468 of file nnet-component.h.

References Component::Propagate().

Referenced by AffineComponent::CollapseWithNext().

{ return linear_params_.NumRows(); }

Propagate()

void Propagate ( const ChunkInfo &  in_info, const ChunkInfo &  out_info, const CuMatrixBase< BaseFloat > &  in, CuMatrixBase< BaseFloat > *  out  ) const

virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 3288 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMatMat(), CuMatrixBase< Real >::AddVecToRows(), AffineComponent::bias_params_, ChunkInfo::CheckSize(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, AffineComponent::linear_params_, and ChunkInfo::NumChunks().

                                                                          {
  in_info.CheckSize(in);
  out_info.CheckSize(*out);
  KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());

  out->AddMatMat(1.0, in, kNoTrans, linear_params_, kTrans, 0.0);
  out->AddVecToRows(1.0, bias_params_);
}

Read()

void Read ( std::istream &  is, bool  binary  )

virtual

Implements Component.

Definition at line 3328 of file nnet-component.cc.

References AffineComponent::bias_params_, kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), and AffineComponent::linear_params_.

                                                           {
  ExpectOneOrTwoTokens(is, binary, "<FixedAffineComponent>", "<LinearParams>");
  linear_params_.Read(is, binary);
  ExpectToken(is, binary, "<BiasParams>");
  bias_params_.Read(is, binary);
  ExpectToken(is, binary, "</FixedAffineComponent>");
}

Type()

virtual std::string Type ( ) const

inlinevirtual

Implements Component.

Definition at line 1457 of file nnet-component.h.

{ return "FixedAffineComponent"; }

Write()

void Write ( std::ostream &  os, bool  binary  ) const

virtual

Write component to stream.

Implements Component.

Definition at line 3319 of file nnet-component.cc.

References AffineComponent::bias_params_, AffineComponent::linear_params_, and kaldi::WriteToken().

                                                                  {
  WriteToken(os, binary, "<FixedAffineComponent>");
  WriteToken(os, binary, "<LinearParams>");
  linear_params_.Write(os, binary);
  WriteToken(os, binary, "<BiasParams>");
  bias_params_.Write(os, binary);
  WriteToken(os, binary, "</FixedAffineComponent>");
}

Friends And Related Function Documentation

AffineComponent

friend class AffineComponent

friend

Definition at line 1490 of file nnet-component.h.

Member Data Documentation

bias_params_

CuVector<BaseFloat> bias_params_

protected

Definition at line 1492 of file nnet-component.h.

Referenced by AffineComponent::CollapseWithNext(), AffineComponent::CollapseWithPrevious(), and FixedAffineComponent::Copy().

linear_params_

CuMatrix<BaseFloat> linear_params_

protected

Definition at line 1491 of file nnet-component.h.

Referenced by AffineComponent::CollapseWithNext(), AffineComponent::CollapseWithPrevious(), and FixedAffineComponent::Copy().

---

The documentation for this class was generated from the following files:

• nnet2/nnet-component.h
• nnet2/nnet-component.cc