Class UpdatableComponent is a Component which has trainable parameters, it contains SGD training hyper-parameters in NnetTrainOptions. More...

#include <nnet-component.h>

Inheritance diagram for UpdatableComponent:

[legend]

Collaboration diagram for UpdatableComponent:

[legend]

Public Member Functions
	UpdatableComponent (int32 input_dim, int32 output_dim)

virtual	~UpdatableComponent ()

bool	IsUpdatable () const
	Check if contains trainable parameters,. More...

virtual int32	NumParams () const =0
	Number of trainable parameters,. More...

virtual void	GetGradient (VectorBase< BaseFloat > *gradient) const =0
	Get gradient reshaped as a vector,. More...

virtual void	GetParams (VectorBase< BaseFloat > *params) const =0
	Get the trainable parameters reshaped as a vector,. More...

virtual void	SetParams (const VectorBase< BaseFloat > &params)=0
	Set the trainable parameters from, reshaped as a vector,. More...

virtual void	Update (const CuMatrixBase< BaseFloat > &input, const CuMatrixBase< BaseFloat > &diff)=0
	Compute gradient and update parameters,. More...

virtual void	SetTrainOptions (const NnetTrainOptions &opts)
	Set the training options to the component,. More...

const NnetTrainOptions &	GetTrainOptions () const
	Get the training options from the component,. More...

virtual void	SetLearnRateCoef (BaseFloat val)
	Set the learn-rate coefficient,. More...

virtual void	SetBiasLearnRateCoef (BaseFloat val)
	Set the learn-rate coefficient for bias,. More...

virtual void	InitData (std::istream &is)=0
	Initialize the content of the component by the 'line' from the prototype,. More...

Public Member Functions inherited from Component
	Component (int32 input_dim, int32 output_dim)
	Generic interface of a component,. More...

virtual	~Component ()

virtual Component *	Copy () const =0
	Copy component (deep copy),. More...

virtual ComponentType	GetType () const =0
	Get Type Identification of the component,. More...

virtual bool	IsMultistream () const
	Check if component has 'Recurrent' interface (trainable and recurrent),. More...

int32	InputDim () const
	Get the dimension of the input,. More...

int32	OutputDim () const
	Get the dimension of the output,. More...

void	Propagate (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out)
	Perform forward-pass propagation 'in' -> 'out',. More...

void	Backpropagate (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrix< BaseFloat > *in_diff)
	Perform backward-pass propagation 'out_diff' -> 'in_diff'. More...

void	Write (std::ostream &os, bool binary) const
	Write the component to a stream,. More...

virtual std::string	Info () const
	Print some additional info (after <ComponentName> and the dims),. More...

virtual std::string	InfoGradient () const
	Print some additional info about gradient (after <...> and dims),. More...

Protected Attributes
NnetTrainOptions	opts_
	Option-class with training hyper-parameters,. More...

BaseFloat	learn_rate_coef_
	Scalar applied to learning rate for weight matrices (to be used in ::Update method),. More...

BaseFloat	bias_learn_rate_coef_
	Scalar applied to learning rate for bias (to be used in ::Update method),. More...

Protected Attributes inherited from Component
int32	input_dim_
	Data members,. More...

int32	output_dim_
	Dimension of the output of the Component,. More...

Additional Inherited Members
Public Types inherited from Component
enum	ComponentType { kUnknown = 0x0, kUpdatableComponent = 0x0100, kAffineTransform, kLinearTransform, kConvolutionalComponent, kLstmProjected, kBlstmProjected, kRecurrentComponent, kActivationFunction = 0x0200, kSoftmax, kHiddenSoftmax, kBlockSoftmax, kSigmoid, kTanh, kParametricRelu, kDropout, kLengthNormComponent, kTranform = 0x0400, kRbm, kSplice, kCopy, kTranspose, kBlockLinearity, kAddShift, kRescale, kKlHmm = 0x0800, kSentenceAveragingComponent, kSimpleSentenceAveragingComponent, kAveragePoolingComponent, kMaxPoolingComponent, kFramePoolingComponent, kParallelComponent, kMultiBasisComponent }
	Component type identification mechanism,. More...

Static Public Member Functions inherited from Component
static const char *	TypeToMarker (ComponentType t)
	Converts component type to marker,. More...

static ComponentType	MarkerToType (const std::string &s)
	Converts marker to component type (case insensitive),. More...

static Component *	Init (const std::string &conf_line)
	Initialize component from a line in config file,. More...

static Component *	Read (std::istream &is, bool binary)
	Read the component from a stream (static method),. More...

Static Public Attributes inherited from Component
static const struct key_value	kMarkerMap []
	The table with pairs of Component types and markers (defined in nnet-component.cc),. More...

Protected Member Functions inherited from Component
virtual void	PropagateFnc (const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out)=0
	Abstract interface for propagation/backpropagation. More...

virtual void	BackpropagateFnc (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrixBase< BaseFloat > *in_diff)=0
	Backward pass transformation (to be implemented by descending class...) More...

virtual void	ReadData (std::istream &is, bool binary)
	Reads the component content. More...

virtual void	WriteData (std::ostream &os, bool binary) const
	Writes the component content. More...

Detailed Description

Class UpdatableComponent is a Component which has trainable parameters, it contains SGD training hyper-parameters in NnetTrainOptions.

The constants 'learning_rate_coef_' and 'bias_learn_rate_coef_' are separate, and should be stored by ::WriteData(...),

Definition at line 208 of file nnet-component.h.

Constructor & Destructor Documentation

◆ UpdatableComponent()

UpdatableComponent	(	int32	input_dim,
		int32	output_dim
	)

inline

Definition at line 210 of file nnet-component.h.

                                                        :
     Component(input_dim, output_dim),
     learn_rate_coef_(1.0),
     bias_learn_rate_coef_(1.0)
   { }

◆ ~UpdatableComponent()

virtual ~UpdatableComponent ( )

inlinevirtual

Definition at line 216 of file nnet-component.h.

217 { }

Member Function Documentation

◆ GetGradient()

virtual void GetGradient ( VectorBase< BaseFloat > * gradient ) const

pure virtual

Get gradient reshaped as a vector,.

Implemented in Rescale, AddShift, BlstmProjected, SentenceAveragingComponent, ConvolutionalComponent, MultiBasisComponent, LstmProjected, FramePoolingComponent, ParallelComponent, RecurrentComponent, AffineTransform, LinearTransform, and ParametricRelu.

Referenced by Nnet::GetGradient().

◆ GetParams()

virtual void GetParams ( VectorBase< BaseFloat > * params ) const

pure virtual

Get the trainable parameters reshaped as a vector,.

Implemented in Rescale, AddShift, BlstmProjected, SentenceAveragingComponent, ConvolutionalComponent, LstmProjected, MultiBasisComponent, FramePoolingComponent, ParallelComponent, RecurrentComponent, AffineTransform, LinearTransform, and ParametricRelu.

Referenced by Nnet::GetParams().

◆ GetTrainOptions()

const NnetTrainOptions& GetTrainOptions ( ) const

inline

Get the training options from the component,.

Definition at line 246 of file nnet-component.h.

                                                   {
     return opts_;
   }

◆ InitData()

virtual void InitData ( std::istream & is )

pure virtual

Initialize the content of the component by the 'line' from the prototype,.

Reimplemented from Component.

Implemented in Rescale, AddShift, SentenceAveragingComponent, ConvolutionalComponent, BlstmProjected, LstmProjected, FramePoolingComponent, RecurrentComponent, ParametricRelu, ParallelComponent, MultiBasisComponent, AffineTransform, and LinearTransform.

◆ IsUpdatable()

bool IsUpdatable ( ) const

inlinevirtual

Check if contains trainable parameters,.

Reimplemented from Component.

Definition at line 220 of file nnet-component.h.

                            {
     return true;
   }

◆ NumParams()

virtual int32 NumParams ( ) const

pure virtual

Number of trainable parameters,.

Implemented in Rescale, AddShift, BlstmProjected, SentenceAveragingComponent, ConvolutionalComponent, MultiBasisComponent, LstmProjected, FramePoolingComponent, ParallelComponent, RecurrentComponent, AffineTransform, LinearTransform, and ParametricRelu.

Referenced by Nnet::GetGradient(), Nnet::GetParams(), and Nnet::SetParams().

◆ SetBiasLearnRateCoef()

virtual void SetBiasLearnRateCoef ( BaseFloat val )

inlinevirtual

Set the learn-rate coefficient for bias,.

Reimplemented in MultiBasisComponent, and ParallelComponent.

Definition at line 256 of file nnet-component.h.

References Component::InitData().

Referenced by main(), ParallelComponent::SetBiasLearnRateCoef(), and MultiBasisComponent::SetBiasLearnRateCoef().

                                                    {
     bias_learn_rate_coef_ = val;
   }

◆ SetLearnRateCoef()

virtual void SetLearnRateCoef ( BaseFloat val )

inlinevirtual

Set the learn-rate coefficient,.

Reimplemented in Rescale, MultiBasisComponent, AddShift, and ParallelComponent.

Definition at line 251 of file nnet-component.h.

Referenced by main(), ParallelComponent::SetLearnRateCoef(), and MultiBasisComponent::SetLearnRateCoef().

                                                {
     learn_rate_coef_ = val;
   }

◆ SetParams()

virtual void SetParams ( const VectorBase< BaseFloat > & params )

pure virtual

Set the trainable parameters from, reshaped as a vector,.

Implemented in Rescale, BlstmProjected, AddShift, LstmProjected, SentenceAveragingComponent, ConvolutionalComponent, MultiBasisComponent, FramePoolingComponent, ParallelComponent, RecurrentComponent, AffineTransform, LinearTransform, and ParametricRelu.

Referenced by Nnet::SetParams().

◆ SetTrainOptions()

virtual void SetTrainOptions ( const NnetTrainOptions & opts )

inlinevirtual

Set the training options to the component,.

Reimplemented in MultiBasisComponent, ParallelComponent, and SentenceAveragingComponent.

Definition at line 241 of file nnet-component.h.

Referenced by SentenceAveragingComponent::SetTrainOptions().

                                                              {
     opts_ = opts;
   }

◆ Update()

virtual void Update	(	const CuMatrixBase< BaseFloat > &	input,
		const CuMatrixBase< BaseFloat > &	diff
	)

pure virtual

Compute gradient and update parameters,.

Implemented in BlstmProjected, LstmProjected, Rescale, ConvolutionalComponent, AddShift, MultiBasisComponent, RecurrentComponent, ParallelComponent, SentenceAveragingComponent, FramePoolingComponent, AffineTransform, LinearTransform, and ParametricRelu.

Referenced by Nnet::Backpropagate().

Member Data Documentation

◆ bias_learn_rate_coef_

BaseFloat bias_learn_rate_coef_

protected

Scalar applied to learning rate for bias (to be used in ::Update method),.

Definition at line 273 of file nnet-component.h.

Referenced by AffineTransform::Info(), ConvolutionalComponent::Info(), LstmProjected::Info(), BlstmProjected::Info(), AffineTransform::InfoGradient(), RecurrentComponent::InfoGradient(), ConvolutionalComponent::InfoGradient(), LstmProjected::InfoGradient(), BlstmProjected::InfoGradient(), AffineTransform::InitData(), RecurrentComponent::InitData(), LstmProjected::InitData(), BlstmProjected::InitData(), ConvolutionalComponent::InitData(), AffineTransform::ReadData(), RecurrentComponent::ReadData(), LstmProjected::ReadData(), ConvolutionalComponent::ReadData(), BlstmProjected::ReadData(), AffineTransform::Update(), RecurrentComponent::Update(), ConvolutionalComponent::Update(), LstmProjected::Update(), BlstmProjected::Update(), AffineTransform::WriteData(), RecurrentComponent::WriteData(), LstmProjected::WriteData(), ConvolutionalComponent::WriteData(), and BlstmProjected::WriteData().

◆ learn_rate_coef_

BaseFloat learn_rate_coef_

protected

Scalar applied to learning rate for weight matrices (to be used in ::Update method),.

Definition at line 269 of file nnet-component.h.

Referenced by LinearTransform::Info(), AffineTransform::Info(), ConvolutionalComponent::Info(), LstmProjected::Info(), BlstmProjected::Info(), LinearTransform::InfoGradient(), AffineTransform::InfoGradient(), RecurrentComponent::InfoGradient(), FramePoolingComponent::InfoGradient(), ConvolutionalComponent::InfoGradient(), LstmProjected::InfoGradient(), BlstmProjected::InfoGradient(), AffineTransform::InitData(), LinearTransform::InitData(), RecurrentComponent::InitData(), FramePoolingComponent::InitData(), LstmProjected::InitData(), BlstmProjected::InitData(), ConvolutionalComponent::InitData(), AffineTransform::ReadData(), RecurrentComponent::ReadData(), LinearTransform::ReadData(), LstmProjected::ReadData(), FramePoolingComponent::ReadData(), ConvolutionalComponent::ReadData(), BlstmProjected::ReadData(), LinearTransform::Update(), AffineTransform::Update(), FramePoolingComponent::Update(), RecurrentComponent::Update(), ConvolutionalComponent::Update(), LstmProjected::Update(), BlstmProjected::Update(), AffineTransform::WriteData(), RecurrentComponent::WriteData(), LinearTransform::WriteData(), LstmProjected::WriteData(), FramePoolingComponent::WriteData(), ConvolutionalComponent::WriteData(), and BlstmProjected::WriteData().

◆ opts_

NnetTrainOptions opts_

protected

Option-class with training hyper-parameters,.

Definition at line 265 of file nnet-component.h.

Referenced by LstmProjected::BackpropagateFnc(), BlstmProjected::BackpropagateFnc(), ParametricRelu::Update(), LinearTransform::Update(), AffineTransform::Update(), FramePoolingComponent::Update(), RecurrentComponent::Update(), ConvolutionalComponent::Update(), LstmProjected::Update(), and BlstmProjected::Update().

The documentation for this class was generated from the following file:

nnet/nnet-component.h

Public Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ UpdatableComponent()

◆ ~UpdatableComponent()

Member Function Documentation

◆ GetGradient()

◆ GetParams()

◆ GetTrainOptions()

◆ InitData()

◆ IsUpdatable()

◆ NumParams()

◆ SetBiasLearnRateCoef()

◆ SetLearnRateCoef()

◆ SetParams()

◆ SetTrainOptions()

◆ Update()

Member Data Documentation

◆ bias_learn_rate_coef_

◆ learn_rate_coef_

◆ opts_