AdditiveNoiseComponent Class Reference

This is a bit similar to dropout but adding (not multiplying) Gaussian noise with a given standard deviation. More...

#include <nnet-component.h>

Inheritance diagram for AdditiveNoiseComponent:
Collaboration diagram for AdditiveNoiseComponent:

Public Member Functions

void Init (int32 dim, BaseFloat noise_stddev)
 
 AdditiveNoiseComponent (int32 dim, BaseFloat stddev)
 
 AdditiveNoiseComponent ()
 
virtual int32 InputDim () const
 Get size of input vectors. More...
 
virtual int32 OutputDim () const
 Get size of output vectors. More...
 
virtual void InitFromString (std::string args)
 Initialize, typically from a line of a config file. More...
 
virtual void Read (std::istream &is, bool binary)
 
virtual void Write (std::ostream &os, bool binary) const
 Write component to stream. More...
 
virtual std::string Type () const
 
virtual bool BackpropNeedsInput () const
 
virtual bool BackpropNeedsOutput () const
 
virtual ComponentCopy () const
 Copy component (deep copy). More...
 
virtual void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out) const
 Perform forward pass propagation Input->Output. More...
 
virtual void Backprop (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in_value, const CuMatrixBase< BaseFloat > &out_value, const CuMatrixBase< BaseFloat > &out_deriv, Component *to_update, CuMatrix< BaseFloat > *in_deriv) const
 Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise). More...
 
- Public Member Functions inherited from RandomComponent
void ResetGenerator ()
 
- Public Member Functions inherited from Component
 Component ()
 
virtual int32 Index () const
 Returns the index in the sequence of layers in the neural net; intended only to be used in debugging information. More...
 
virtual void SetIndex (int32 index)
 
virtual std::vector< int32Context () const
 Return a vector describing the temporal context this component requires for each frame of output, as a sorted list. More...
 
void Propagate (const ChunkInfo &in_info, const ChunkInfo &out_info, const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out) const
 A non-virtual propagate function that first resizes output if necessary. More...
 
virtual std::string Info () const
 
virtual ~Component ()
 

Private Attributes

int32 dim_
 
BaseFloat stddev_
 

Additional Inherited Members

- Static Public Member Functions inherited from Component
static ComponentReadNew (std::istream &is, bool binary)
 Read component from stream. More...
 
static ComponentNewFromString (const std::string &initializer_line)
 Initialize the Component from one line that will contain first the type, e.g. More...
 
static ComponentNewComponentOfType (const std::string &type)
 Return a new Component of the given type e.g. More...
 
- Protected Attributes inherited from RandomComponent
CuRand< BaseFloatrandom_generator_
 

Detailed Description

This is a bit similar to dropout but adding (not multiplying) Gaussian noise with a given standard deviation.

Definition at line 1635 of file nnet-component.h.

Constructor & Destructor Documentation

◆ AdditiveNoiseComponent() [1/2]

AdditiveNoiseComponent ( int32  dim,
BaseFloat  stddev 
)
inline

Definition at line 1638 of file nnet-component.h.

1638 { Init(dim, stddev); }
void Init(int32 dim, BaseFloat noise_stddev)

◆ AdditiveNoiseComponent() [2/2]

Member Function Documentation

◆ Backprop()

virtual void Backprop ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in_value,
const CuMatrixBase< BaseFloat > &  out_value,
const CuMatrixBase< BaseFloat > &  out_deriv,
Component to_update,
CuMatrix< BaseFloat > *  in_deriv 
) const
inlinevirtual

Perform backward pass propagation of the derivative, and also either update the model (if to_update == this) or update another model or compute the model derivative (otherwise).

Note: in_value and out_value are the values of the input and output of the component, and these may be dummy variables if respectively BackpropNeedsInput() or BackpropNeedsOutput() return false for that component (not all components need these).

num_chunks lets us treat the input matrix as contiguous-in-time chunks of equal size; it only matters if splicing is involved.

Implements Component.

Definition at line 1660 of file nnet-component.h.

1666  { *in_deriv = out_deriv; }

◆ BackpropNeedsInput()

virtual bool BackpropNeedsInput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 1650 of file nnet-component.h.

1650 { return false; }

◆ BackpropNeedsOutput()

virtual bool BackpropNeedsOutput ( ) const
inlinevirtual

Reimplemented from Component.

Definition at line 1651 of file nnet-component.h.

1651 { return false; }

◆ Copy()

virtual Component* Copy ( ) const
inlinevirtual

Copy component (deep copy).

Implements Component.

Definition at line 1652 of file nnet-component.h.

References Component::Propagate().

◆ Init()

void Init ( int32  dim,
BaseFloat  noise_stddev 
)

Definition at line 3614 of file nnet-component.cc.

3614  {
3615  dim_ = dim;
3616  stddev_ = stddev;
3617 }

◆ InitFromString()

void InitFromString ( std::string  args)
virtual

Initialize, typically from a line of a config file.

The "args" will contain any parameters that need to be passed to the Component, e.g. dimensions.

Implements Component.

Definition at line 3584 of file nnet-component.cc.

References AffineComponentPreconditionedOnline::Init(), KALDI_ERR, and kaldi::nnet2::ParseFromString().

Referenced by kaldi::nnet2::UnitTestAdditiveNoiseComponent().

3584  {
3585  std::string orig_args(args);
3586  int32 dim;
3587  BaseFloat stddev = 1.0;
3588  bool ok = ParseFromString("dim", &args, &dim);
3589  ParseFromString("stddev", &args, &stddev);
3590 
3591  if (!ok || !args.empty() || dim <= 0)
3592  KALDI_ERR << "Invalid initializer for layer of type AdditiveNoiseComponent: \""
3593  << orig_args << "\"";
3594  Init(dim, stddev);
3595 }
kaldi::int32 int32
bool ParseFromString(const std::string &name, std::string *string, int32 *param)
Functions used in Init routines.
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ERR
Definition: kaldi-error.h:147
void Init(int32 dim, BaseFloat noise_stddev)

◆ InputDim()

virtual int32 InputDim ( ) const
inlinevirtual

Get size of input vectors.

Implements Component.

Definition at line 1640 of file nnet-component.h.

◆ OutputDim()

virtual int32 OutputDim ( ) const
inlinevirtual

Get size of output vectors.

Implements Component.

Definition at line 1641 of file nnet-component.h.

◆ Propagate()

void Propagate ( const ChunkInfo in_info,
const ChunkInfo out_info,
const CuMatrixBase< BaseFloat > &  in,
CuMatrixBase< BaseFloat > *  out 
) const
virtual

Perform forward pass propagation Input->Output.

Each row is one frame or training example. Interpreted as "num_chunks" equally sized chunks of frames; this only matters for layers that do things like context splicing. Typically this variable will either be 1 (when we're processing a single contiguous chunk of data) or will be the same as in.NumFrames(), but other values are possible if some layers do splicing.

Implements Component.

Definition at line 3619 of file nnet-component.cc.

References CuMatrixBase< Real >::AddMat(), CuMatrixBase< Real >::CopyFromMat(), AffineComponent::InputDim(), KALDI_ASSERT, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), and kaldi::RandUniform().

3622  {
3623  KALDI_ASSERT(in.NumCols() == this->InputDim());
3624  out->CopyFromMat(in);
3625  CuMatrix<BaseFloat> rand(in.NumRows(), in.NumCols());
3626  const_cast<CuRand<BaseFloat>&>(random_generator_).RandUniform(&rand);
3627  out->AddMat(stddev_, rand);
3628 }
float RandUniform(struct RandomState *state=NULL)
Returns a random number strictly between 0 and 1.
Definition: kaldi-math.h:151
virtual int32 InputDim() const
Get size of input vectors.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185
CuRand< BaseFloat > random_generator_

◆ Read()

void Read ( std::istream &  is,
bool  binary 
)
virtual

Implements Component.

Definition at line 3597 of file nnet-component.cc.

References kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), and kaldi::ReadBasicType().

3597  {
3598  ExpectOneOrTwoTokens(is, binary, "<AdditiveNoiseComponent>", "<Dim>");
3599  ReadBasicType(is, binary, &dim_);
3600  ExpectToken(is, binary, "<Stddev>");
3601  ReadBasicType(is, binary, &stddev_);
3602  ExpectToken(is, binary, "</AdditiveNoiseComponent>");
3603 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:191
static void ExpectOneOrTwoTokens(std::istream &is, bool binary, const std::string &token1, const std::string &token2)

◆ Type()

virtual std::string Type ( ) const
inlinevirtual

Implements Component.

Definition at line 1648 of file nnet-component.h.

1648 { return "AdditiveNoiseComponent"; }

◆ Write()

void Write ( std::ostream &  os,
bool  binary 
) const
virtual

Write component to stream.

Implements Component.

Definition at line 3605 of file nnet-component.cc.

References kaldi::WriteBasicType(), and kaldi::WriteToken().

3605  {
3606  WriteToken(os, binary, "<AdditiveNoiseComponent>");
3607  WriteToken(os, binary, "<Dim>");
3608  WriteBasicType(os, binary, dim_);
3609  WriteToken(os, binary, "<Stddev>");
3610  WriteBasicType(os, binary, stddev_);
3611  WriteToken(os, binary, "</AdditiveNoiseComponent>");
3612 }
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
void WriteBasicType(std::ostream &os, bool binary, T t)
WriteBasicType is the name of the write function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:34

Member Data Documentation

◆ dim_

int32 dim_
private

Definition at line 1668 of file nnet-component.h.

◆ stddev_

BaseFloat stddev_
private

Definition at line 1669 of file nnet-component.h.


The documentation for this class was generated from the following files: