#include <nnet-component.h>

Inheritance diagram for TanhComponent:

Collaboration diagram for TanhComponent:

[legend]

Public Member Functions
	TanhComponent (int32 dim)

	TanhComponent (const TanhComponent &other)

	TanhComponent ()

virtual std::string	Type () const

virtual Component *	Copy () const
	Copy component (deep copy). More...

virtual bool	BackpropNeedsInput () const

virtual bool	BackpropNeedsOutput () const

virtual void	Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
	Perform forward pass propagation Input->Output. More...

virtual void	Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component to_update, CuMatrix< BaseFloat > in_deriv) const
	Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...

Public Member Functions inherited from NonlinearComponent
void	Init (int32 dim)

	NonlinearComponent (int32 dim)

	NonlinearComponent ()

	NonlinearComponent (const NonlinearComponent &other)

virtual int32	InputDim () const
	Get size of input vectors. More...

virtual int32	OutputDim () const
	Get size of output vectors. More...

virtual void	InitFromString (std::string args)
	We implement InitFromString at this level. More...

virtual void	Read (std::istream &is, bool binary)
	We implement Read at this level as it just needs the Type(). More...

virtual void	Write (std::ostream &os, bool binary) const
	Write component to stream. More...

void	Scale (BaseFloat scale)

void	Add (BaseFloat alpha, const NonlinearComponent &other)

const CuVector< double > &	ValueSum () const

const CuVector< double > &	DerivSum () const

double	Count () const

void	SetDim (int32 dim)

Public Member Functions inherited from Component
	Component ()

virtual int32	Index () const
	Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...

virtual void	SetIndex (int32 index)

virtual std::vector< int32 >	Context () const
	Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...

void	Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
	A non-virtual propagate function that first resizes output if necessary. More...

virtual std::string	Info () const

virtual	~Component ()

Private Member Functions
TanhComponent &	operator= (const TanhComponent &other)

Additional Inherited Members
Static Public Member Functions inherited from Component
static Component *	ReadNew (std::istream &is, bool binary)
	Read component from stream. More...

static Component *	NewFromString (const std::string &initializer_line)
	Initialize the Component from one line that will contain first the type, e.g. More...

static Component *	NewComponentOfType (const std::string &type)
	Return a new Component of the given type e.g. More...

Protected Member Functions inherited from NonlinearComponent
void	UpdateStats (const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > *deriv=NULL)

const NonlinearComponent &	operator= (const NonlinearComponent &other)

Protected Attributes inherited from NonlinearComponent
int32	dim_

CuVector< double >	value_sum_

CuVector< double >	deriv_sum_

double	count_

std::mutex	mutex_

Detailed Description

Definition at line 610 of file nnet-component.h.

Constructor & Destructor Documentation

◆ TanhComponent() [1/3]

TanhComponent ( int32 dim )

inlineexplicit

Definition at line 612 of file nnet-component.h.

612 : NonlinearComponent(dim) { }

kaldi::nnet2::NonlinearComponent::NonlinearComponent

NonlinearComponent()

Definition: nnet-component.h:356

◆ TanhComponent() [2/3]

TanhComponent ( const TanhComponent & other )

inlineexplicit

Definition at line 613 of file nnet-component.h.

613 : NonlinearComponent(other) { }

kaldi::nnet2::NonlinearComponent::NonlinearComponent

NonlinearComponent()

Definition: nnet-component.h:356

◆ TanhComponent() [3/3]

TanhComponent ( )

inline

Definition at line 614 of file nnet-component.h.

614 { }

Member Function Documentation

◆ Backprop()

void Backprop	(	const ChunkInfo &	in_info,
		const ChunkInfo &	out_info,
		const CuMatrixBase< BaseFloat > &	in_value,
		const CuMatrixBase< BaseFloat > &	out_value,
		const CuMatrixBase< BaseFloat > &	out_deriv,
		Component *	to_update,
		CuMatrix< BaseFloat > *	in_deriv
	)		const

virtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 664 of file nnet-component.cc.

References CuMatrixBase< Real >::Add(), CuMatrixBase< Real >::ApplyPow(), CuMatrixBase< Real >::CopyFromMat(), CuMatrixBase< Real >::MulElements(), CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrix< Real >::Resize(), CuMatrixBase< Real >::Scale(), and NonlinearComponent::UpdateStats().

                                                                   {
   /*
     Note on the derivative of the tanh function:
     tanh'(x) = sech^2(x) = -(tanh(x)+1) (tanh(x)-1) = 1 - tanh^2(x)
 
     The element by element equation of what we're doing would be:
     in_deriv = out_deriv * (1.0 - out_value^2).
     We can accomplish this via calls to the matrix library. */
 
   in_deriv->Resize(out_deriv.NumRows(), out_deriv.NumCols());
   in_deriv->CopyFromMat(out_value);
   in_deriv->ApplyPow(2.0);
   in_deriv->Scale(-1.0);
   in_deriv->Add(1.0);
   // now in_deriv = (1.0 - out_value^2), the element-by-element derivative of
   // the nonlinearity.
   if (to_update != NULL)
     dynamic_cast<NonlinearComponent*>(to_update)->UpdateStats(out_value,
                                                               in_deriv);
   in_deriv->MulElements(out_deriv);
 }

◆ BackpropNeedsInput()

virtual bool BackpropNeedsInput ( ) const

inlinevirtual

Reimplemented from Component.

Definition at line 617 of file nnet-component.h.

617 { return false; }

◆ BackpropNeedsOutput()

virtual bool BackpropNeedsOutput ( ) const

inlinevirtual

Reimplemented from Component.

Definition at line 618 of file nnet-component.h.

References Component::Propagate().

618 { return true; }

◆ Copy()

virtual Component* Copy ( ) const

inlinevirtual

Copy component (deep copy).

Implements Component.

Definition at line 616 of file nnet-component.h.

616 { return new TanhComponent(*this); }

kaldi::nnet2::TanhComponent::TanhComponent

TanhComponent()

Definition: nnet-component.h:614

◆ operator=()

TanhComponent& operator= ( const TanhComponent & other )

private

◆ Propagate()

void Propagate	(	const ChunkInfo &	in_info,
		const ChunkInfo &	out_info,
		const CuMatrixBase< BaseFloat > &	in,
		CuMatrixBase< BaseFloat > *	out
	)		const

virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 650 of file nnet-component.cc.

References ChunkInfo::CheckSize(), KALDI_ASSERT, ChunkInfo::NumChunks(), and CuMatrixBase< Real >::Tanh().

                                                                    {
   // Apply tanh function to each element of the output...
   // the tanh function may be written as -1 + ( 2 / (1 + e^{-2 x})),
   // which is a scaled and shifted sigmoid.
 
   in_info.CheckSize(in);
   out_info.CheckSize(*out);
   KALDI_ASSERT(in_info.NumChunks() == out_info.NumChunks());
   out->Tanh(in);
 }

◆ Type()

virtual std::string Type ( ) const

inlinevirtual

Implements Component.

Definition at line 615 of file nnet-component.h.

615 { return "TanhComponent"; }

The documentation for this class was generated from the following files:

nnet2/nnet-component.h
nnet2/nnet-component.cc

Public Member Functions

Private Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ TanhComponent() [1/3]

◆ TanhComponent() [2/3]

◆ TanhComponent() [3/3]

Member Function Documentation

◆ Backprop()

◆ BackpropNeedsInput()

◆ BackpropNeedsOutput()

◆ Copy()

◆ operator=()

◆ Propagate()

◆ Type()