RecurrentComponent Class Reference

Component with recurrent connections, 'tanh' non-linearity. More...

#include <nnet-recurrent.h>

Inheritance diagram for RecurrentComponent:

Collaboration diagram for RecurrentComponent:

Public Member Functions
	RecurrentComponent (int32 input_dim, int32 output_dim)
	~RecurrentComponent ()
Component *	Copy () const
	Copy component (deep copy),. More...
ComponentType	GetType () const
	Get Type Identification of the component,. More...
void	InitData (std::istream &is)
	Initialize the content of the component by the 'line' from the prototype,. More...
void	ReadData (std::istream &is, bool binary)
	Reads the component content. More...
void	WriteData (std::ostream &os, bool binary) const
	Writes the component content. More...
int32	NumParams () const
	Number of trainable parameters,. More...
void	GetGradient (VectorBase< BaseFloat > *gradient) const
	Get gradient reshaped as a vector,. More...
void	GetParams (VectorBase< BaseFloat > *params) const
	Get the trainable parameters reshaped as a vector,. More...
void	SetParams (const VectorBase< BaseFloat > &params)
	Set the trainable parameters from, reshaped as a vector,. More...
std::string	Info () const
	Print some additional info (after <ComponentName> and the dims),. More...
std::string	InfoGradient () const
	Print some additional info about gradient (after <...> and dims),. More...
void	PropagateFnc (const CuMatrixBase< BaseFloat > &in, CuMatrixBase< BaseFloat > *out)
	Abstract interface for propagation/backpropagation. More...
void	BackpropagateFnc (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrixBase< BaseFloat > *in_diff)
	Backward pass transformation (to be implemented by descending class...) More...
void	Update (const CuMatrixBase< BaseFloat > &input, const CuMatrixBase< BaseFloat > &diff)
	Compute gradient and update parameters,. More...
Public Member Functions inherited from MultistreamComponent
	MultistreamComponent (int32 input_dim, int32 output_dim)
bool	IsMultistream () const
	Check if component has 'Recurrent' interface (trainable and recurrent),. More...
virtual void	SetSeqLengths (const std::vector< int32 > &sequence_lengths)
int32	NumStreams () const
virtual void	ResetStreams (const std::vector< int32 > &stream_reset_flag)
	Optional function to reset the transfer of context (not used for BLSTMs. More...
Public Member Functions inherited from UpdatableComponent
	UpdatableComponent (int32 input_dim, int32 output_dim)
virtual	~UpdatableComponent ()
bool	IsUpdatable () const
	Check if contains trainable parameters,. More...
virtual void	SetTrainOptions (const NnetTrainOptions &opts)
	Set the training options to the component,. More...
const NnetTrainOptions &	GetTrainOptions () const
	Get the training options from the component,. More...
virtual void	SetLearnRateCoef (BaseFloat val)
	Set the learn-rate coefficient,. More...
virtual void	SetBiasLearnRateCoef (BaseFloat val)
	Set the learn-rate coefficient for bias,. More...
Public Member Functions inherited from Component
	Component (int32 input_dim, int32 output_dim)
	Generic interface of a component,. More...
virtual	~Component ()
int32	InputDim () const
	Get the dimension of the input,. More...
int32	OutputDim () const
	Get the dimension of the output,. More...
void	Propagate (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out)
	Perform forward-pass propagation 'in' -> 'out',. More...
void	Backpropagate (const CuMatrixBase< BaseFloat > &in, const CuMatrixBase< BaseFloat > &out, const CuMatrixBase< BaseFloat > &out_diff, CuMatrix< BaseFloat > *in_diff)
	Perform backward-pass propagation 'out_diff' -> 'in_diff'. More...
void	Write (std::ostream &os, bool binary) const
	Write the component to a stream,. More...

Private Attributes
BaseFloat	grad_clip_
	Clipping of the update,. More...
BaseFloat	diff_clip_
	Clipping in the BPTT loop,. More...
CuMatrix< BaseFloat >	w_forward_
CuMatrix< BaseFloat >	w_recurrent_
CuVector< BaseFloat >	bias_
CuMatrix< BaseFloat >	w_forward_corr_
CuMatrix< BaseFloat >	w_recurrent_corr_
CuVector< BaseFloat >	bias_corr_
CuMatrix< BaseFloat >	out_
CuMatrix< BaseFloat >	out_diff_bptt_

Additional Inherited Members
Public Types inherited from Component
enum	ComponentType {   kUnknown = 0x0, kUpdatableComponent = 0x0100, kAffineTransform, kLinearTransform,   kConvolutionalComponent, kLstmProjected, kBlstmProjected, kRecurrentComponent,   kActivationFunction = 0x0200, kSoftmax, kHiddenSoftmax, kBlockSoftmax,   kSigmoid, kTanh, kParametricRelu, kDropout,   kLengthNormComponent, kTranform = 0x0400, kRbm, kSplice,   kCopy, kTranspose, kBlockLinearity, kAddShift,   kRescale, kKlHmm = 0x0800, kSentenceAveragingComponent, kSimpleSentenceAveragingComponent,   kAveragePoolingComponent, kMaxPoolingComponent, kFramePoolingComponent, kParallelComponent,   kMultiBasisComponent }
	Component type identification mechanism,. More...
Static Public Member Functions inherited from Component
static const char *	TypeToMarker (ComponentType t)
	Converts component type to marker,. More...
static ComponentType	MarkerToType (const std::string &s)
	Converts marker to component type (case insensitive),. More...
static Component *	Init (const std::string &conf_line)
	Initialize component from a line in config file,. More...
static Component *	Read (std::istream &is, bool binary)
	Read the component from a stream (static method),. More...
Static Public Attributes inherited from Component
static const struct key_value	kMarkerMap []
	The table with pairs of Component types and markers (defined in nnet-component.cc),. More...
Protected Attributes inherited from MultistreamComponent
std::vector< int32 >	sequence_lengths_
Protected Attributes inherited from UpdatableComponent
NnetTrainOptions	opts_
	Option-class with training hyper-parameters,. More...
BaseFloat	learn_rate_coef_
	Scalar applied to learning rate for weight matrices (to be used in ::Update method),. More...
BaseFloat	bias_learn_rate_coef_
	Scalar applied to learning rate for bias (to be used in ::Update method),. More...
Protected Attributes inherited from Component
int32	input_dim_
	Data members,. More...
int32	output_dim_
	Dimension of the output of the Component,. More...

Detailed Description

Component with recurrent connections, 'tanh' non-linearity.

No internal state preserved, starting each sequence from zero vector.

Can be used in 'per-sentence' training and multi-stream training.

Definition at line 43 of file nnet-recurrent.h.

Constructor & Destructor Documentation

RecurrentComponent()

RecurrentComponent ( int32  input_dim, int32  output_dim  )

inline

Definition at line 45 of file nnet-recurrent.h.

Referenced by RecurrentComponent::Copy().

                                                       :
    MultistreamComponent(input_dim, output_dim)
  { }

~RecurrentComponent()

~RecurrentComponent ( )

inline

Definition at line 49 of file nnet-recurrent.h.

  { }

Member Function Documentation

BackpropagateFnc()

void BackpropagateFnc ( const CuMatrixBase< BaseFloat > &  in, const CuMatrixBase< BaseFloat > &  out, const CuMatrixBase< BaseFloat > &  out_diff, CuMatrixBase< BaseFloat > *  in_diff  )

inlinevirtual

Backward pass transformation (to be implemented by descending class...)

Implements Component.

Definition at line 243 of file nnet-recurrent.h.

References CuMatrixBase< Real >::AddMatMat(), CuMatrixBase< Real >::ApplyCeiling(), CuMatrixBase< Real >::ApplyFloor(), RecurrentComponent::diff_clip_, CuMatrixBase< Real >::DiffTanh(), kaldi::kNoTrans, CuMatrixBase< Real >::NumRows(), MultistreamComponent::NumStreams(), RecurrentComponent::out_diff_bptt_, CuMatrixBase< Real >::RowRange(), MultistreamComponent::sequence_lengths_, RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

                                                          {

    int32 T = in.NumRows() / NumStreams();
    int32 S = NumStreams();

    // Apply BPTT on 'out_diff',
    out_diff_bptt_ = out_diff;
    for (int32 t = T-1; t >= 1; t--) {
      // buffers,
      CuSubMatrix<BaseFloat> d_t = out_diff_bptt_.RowRange(t*S, S);
      CuSubMatrix<BaseFloat> d_t1 = out_diff_bptt_.RowRange((t-1)*S, S);
      const CuSubMatrix<BaseFloat> y_t = out.RowRange(t*S, S);

      // BPTT,
      d_t.DiffTanh(y_t, d_t);
      d_t1.AddMatMat(1.0, d_t, kNoTrans, w_recurrent_, kNoTrans, 1.0);

      // clipping,
      if (diff_clip_ > 0.0) {
        d_t1.ApplyFloor(-diff_clip_);
        d_t1.ApplyCeiling(diff_clip_);
      }

      // Zero diff for padded frames,
      if (sequence_lengths_.size() == S) {
        for (int32 s = 0; s < S; s++) {
          if (t >= sequence_lengths_[s]) {
            out_diff_bptt_.Row(t*S + s).SetZero();
          }
        }
      }
    }

    // Apply 'DiffTanh' on first block,
    CuSubMatrix<BaseFloat> d_t = out_diff_bptt_.RowRange(0, S);
    const CuSubMatrix<BaseFloat> y_t = out.RowRange(0, S);
    d_t.DiffTanh(y_t, d_t);

    // Transform diffs to 'in_diff',
    in_diff->AddMatMat(1.0, out_diff_bptt_, kNoTrans, w_forward_, kNoTrans, 0.0);

    // We are DONE ;)
  }

Copy()

Component* Copy ( ) const

inlinevirtual

Copy component (deep copy),.

Implements Component.

Definition at line 52 of file nnet-recurrent.h.

References RecurrentComponent::RecurrentComponent().

{ return new RecurrentComponent(*this); }

GetGradient()

void GetGradient ( VectorBase< BaseFloat > *  gradient ) const

inlinevirtual

Get gradient reshaped as a vector,.

Implements UpdatableComponent.

Definition at line 133 of file nnet-recurrent.h.

References RecurrentComponent::bias_corr_, VectorBase< Real >::Dim(), KALDI_ASSERT, RecurrentComponent::NumParams(), VectorBase< Real >::Range(), RecurrentComponent::w_forward_corr_, and RecurrentComponent::w_recurrent_corr_.

                                                          {
    KALDI_ASSERT(gradient->Dim() == NumParams());
    int32 offset, len;

    offset = 0;    len = w_forward_corr_.NumRows() * w_forward_corr_.NumCols();
    gradient->Range(offset, len).CopyRowsFromMat(w_forward_corr_);

    offset += len; len = w_recurrent_corr_.NumRows() * w_recurrent_corr_.NumCols();
    gradient->Range(offset, len).CopyRowsFromMat(w_recurrent_corr_);

    offset += len; len = bias_corr_.Dim();
    gradient->Range(offset, len).CopyFromVec(bias_corr_);

    offset += len;
    KALDI_ASSERT(offset == NumParams());
  }

GetParams()

void GetParams ( VectorBase< BaseFloat > *  params ) const

inlinevirtual

Get the trainable parameters reshaped as a vector,.

Implements UpdatableComponent.

Definition at line 150 of file nnet-recurrent.h.

References RecurrentComponent::bias_, VectorBase< Real >::Dim(), KALDI_ASSERT, RecurrentComponent::NumParams(), VectorBase< Real >::Range(), RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

                                                      {
    KALDI_ASSERT(params->Dim() == NumParams());
    int32 offset, len;

    offset = 0;    len = w_forward_.NumRows() * w_forward_.NumCols();
    params->Range(offset, len).CopyRowsFromMat(w_forward_);

    offset += len; len = w_recurrent_.NumRows() * w_recurrent_.NumCols();
    params->Range(offset, len).CopyRowsFromMat(w_recurrent_);

    offset += len; len = bias_.Dim();
    params->Range(offset, len).CopyFromVec(bias_);

    offset += len;
    KALDI_ASSERT(offset == NumParams());
  }

GetType()

ComponentType GetType ( ) const

inlinevirtual

Get Type Identification of the component,.

Implements Component.

Definition at line 53 of file nnet-recurrent.h.

References Component::kRecurrentComponent.

{ return kRecurrentComponent; }

Info()

std::string Info ( ) const

inlinevirtual

Print some additional info (after <ComponentName> and the dims),.

Reimplemented from Component.

Definition at line 184 of file nnet-recurrent.h.

References RecurrentComponent::bias_, kaldi::nnet1::MomentStatistics(), RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

                         {
    return std::string("  ") +
      "\n  w_forward_  "   + MomentStatistics(w_forward_) +
      "\n  w_recurrent_  " + MomentStatistics(w_recurrent_) +
      "\n  bias_  "        + MomentStatistics(bias_);
  }

InfoGradient()

std::string InfoGradient ( ) const

inlinevirtual

Print some additional info about gradient (after <...> and dims),.

Reimplemented from Component.

Definition at line 191 of file nnet-recurrent.h.

References RecurrentComponent::bias_corr_, UpdatableComponent::bias_learn_rate_coef_, RecurrentComponent::diff_clip_, RecurrentComponent::grad_clip_, UpdatableComponent::learn_rate_coef_, kaldi::nnet1::MomentStatistics(), RecurrentComponent::out_, RecurrentComponent::out_diff_bptt_, kaldi::nnet1::ToString(), RecurrentComponent::w_forward_corr_, and RecurrentComponent::w_recurrent_corr_.

                                 {
    return std::string("") +
      "( learn_rate_coef " + ToString(learn_rate_coef_) +
      ", bias_learn_rate_coef " + ToString(bias_learn_rate_coef_) +
      ", grad-clip " + ToString(grad_clip_) +
      ", diff-clip " + ToString(diff_clip_) + " )" +
      "\n  Gradients:" +
      "\n  w_forward_corr_  "   + MomentStatistics(w_forward_corr_) +
      "\n  w_recurrent_corr_  "   + MomentStatistics(w_recurrent_corr_) +
      "\n  bias_corr_  "     + MomentStatistics(bias_corr_) +
      "\n  Forward-pass:" +
      "\n  out_  " + MomentStatistics(out_) +
      "\n  Backward-pass:" +
      "\n  out_diff_bptt_  " + MomentStatistics(out_diff_bptt_);
  }

InitData()

void InitData ( std::istream &  is )

inlinevirtual

Initialize the content of the component by the 'line' from the prototype,.

Implements UpdatableComponent.

Definition at line 55 of file nnet-recurrent.h.

References RecurrentComponent::bias_, UpdatableComponent::bias_learn_rate_coef_, RecurrentComponent::diff_clip_, RecurrentComponent::grad_clip_, Component::input_dim_, KALDI_ERR, UpdatableComponent::learn_rate_coef_, Component::output_dim_, kaldi::nnet1::RandUniform(), kaldi::ReadBasicType(), kaldi::ReadToken(), RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

                                {
    // define options,
    float param_scale = 0.02;
    // parse the line from prototype,
    std::string token;
    while (is >> std::ws, !is.eof()) {
      ReadToken(is, false, &token);
       if (token == "<GradClip>") ReadBasicType(is, false, &grad_clip_);
      else if (token == "<DiffClip>") ReadBasicType(is, false, &diff_clip_);
      else if (token == "<LearnRateCoef>") ReadBasicType(is, false, &learn_rate_coef_);
      else if (token == "<BiasLearnRateCoef>") ReadBasicType(is, false, &bias_learn_rate_coef_);
      else if (token == "<ParamScale>") ReadBasicType(is, false, &param_scale);
      else KALDI_ERR << "Unknown token " << token << ", a typo in config?"
                     << " (GradClip|DiffClip|LearnRateCoef|BiasLearnRateCoef|ParamScale)";
    }

    // init the weights and biases (from uniform dist.),
    w_forward_.Resize(output_dim_, input_dim_);
    w_recurrent_.Resize(output_dim_, output_dim_);
    bias_.Resize(output_dim_);

    RandUniform(0.0, 2.0 * param_scale, &w_forward_);
    RandUniform(0.0, 2.0 * param_scale, &w_recurrent_);
    RandUniform(0.0, 2.0 * param_scale, &bias_);
  }

NumParams()

int32 NumParams ( ) const

inlinevirtual

Number of trainable parameters,.

Implements UpdatableComponent.

Definition at line 127 of file nnet-recurrent.h.

References RecurrentComponent::bias_, RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

Referenced by RecurrentComponent::GetGradient(), RecurrentComponent::GetParams(), and RecurrentComponent::SetParams().

                          {
    return w_forward_.NumRows() * w_forward_.NumCols() +
      w_recurrent_.NumRows() * w_recurrent_.NumCols() +
      bias_.Dim();
  }

PropagateFnc()

void PropagateFnc ( const CuMatrixBase< BaseFloat > &  in, CuMatrixBase< BaseFloat > *  out  )

inlinevirtual

Abstract interface for propagation/backpropagation.

Forward pass transformation (to be implemented by descending class...)

Implements Component.

Definition at line 207 of file nnet-recurrent.h.

References CuMatrixBase< Real >::AddMatMat(), CuMatrixBase< Real >::AddVecToRows(), RecurrentComponent::bias_, KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, CuMatrixBase< Real >::NumRows(), MultistreamComponent::NumStreams(), RecurrentComponent::out_, CuMatrixBase< Real >::Row(), CuMatrixBase< Real >::RowRange(), MultistreamComponent::sequence_lengths_, RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

                                                  {


    KALDI_ASSERT(in.NumRows() % NumStreams() == 0);
    int32 T = in.NumRows() / NumStreams();
    int32 S = NumStreams();

    // Precopy bias,
    out->AddVecToRows(1.0, bias_, 0.0);
    // Apply 'forward' connections,
    out->AddMatMat(1.0, in, kNoTrans, w_forward_, kTrans, 1.0);

    // First line of 'out' w/o recurrent signal, apply 'tanh' directly,
    out->RowRange(0, S).Tanh(out->RowRange(0, S));

    // Apply 'recurrent' connections,
    for (int32 t = 1; t < T; t++) {
      out->RowRange(t*S, S).AddMatMat(1.0, out->RowRange((t-1)*S, S), kNoTrans, w_recurrent_, kTrans, 1.0);
      out->RowRange(t*S, S).Tanh(out->RowRange(t*S, S));
      // Zero output for padded frames,
      if (sequence_lengths_.size() == S) {
        for (int32 s = 0; s < S; s++) {
          if (t >= sequence_lengths_[s]) {
            out->Row(t*S + s).SetZero();
          }
        }
      }
      //
    }

    out_ = (*out);  // We'll need a copy for updating the recurrent weights!

    // We are DONE ;)
  }

ReadData()

void ReadData ( std::istream &  is, bool  binary  )

inlinevirtual

Reads the component content.

Reimplemented from Component.

Definition at line 81 of file nnet-recurrent.h.

References RecurrentComponent::bias_, UpdatableComponent::bias_learn_rate_coef_, RecurrentComponent::diff_clip_, kaldi::ExpectToken(), RecurrentComponent::grad_clip_, KALDI_ERR, UpdatableComponent::learn_rate_coef_, kaldi::Peek(), kaldi::PeekToken(), kaldi::ReadBasicType(), kaldi::ReadToken(), RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

                                             {
    // Read all the '<Tokens>' in arbitrary order,
    while ('<' == Peek(is, binary)) {
      std::string token;
      int first_char = PeekToken(is, binary);
      switch (first_char) {
        case 'G': ExpectToken(is, binary, "<GradClip>");
          ReadBasicType(is, binary, &grad_clip_);
          break;
        case 'D': ExpectToken(is, binary, "<DiffClip>");
          ReadBasicType(is, binary, &diff_clip_);
          break;
        case 'L': ExpectToken(is, binary, "<LearnRateCoef>");
          ReadBasicType(is, binary, &learn_rate_coef_);
          break;
        case 'B': ExpectToken(is, binary, "<BiasLearnRateCoef>");
          ReadBasicType(is, binary, &bias_learn_rate_coef_);
          break;
        default: ReadToken(is, false, &token);
          KALDI_ERR << "Unknown token: " << token;
      }
    }

    // Read the data (data follow the tokens),
    w_forward_.Read(is, binary);
    w_recurrent_.Read(is, binary);
    bias_.Read(is, binary);
  }

SetParams()

void SetParams ( const VectorBase< BaseFloat > &  params )

inlinevirtual

Set the trainable parameters from, reshaped as a vector,.

Implements UpdatableComponent.

Definition at line 167 of file nnet-recurrent.h.

References RecurrentComponent::bias_, VectorBase< Real >::Dim(), KALDI_ASSERT, RecurrentComponent::NumParams(), VectorBase< Real >::Range(), RecurrentComponent::w_forward_, and RecurrentComponent::w_recurrent_.

                                                      {
    KALDI_ASSERT(params.Dim() == NumParams());
    int32 offset, len;

    offset = 0;    len = w_forward_.NumRows() * w_forward_.NumCols();
    w_forward_.CopyRowsFromVec(params.Range(offset, len));

    offset += len; len = w_recurrent_.NumRows() * w_recurrent_.NumCols();
    w_recurrent_.CopyRowsFromVec(params.Range(offset, len));

    offset += len; len = bias_.Dim();
    bias_.CopyFromVec(params.Range(offset, len));

    offset += len;
    KALDI_ASSERT(offset == NumParams());
  }

Update()

void Update ( const CuMatrixBase< BaseFloat > &  input, const CuMatrixBase< BaseFloat > &  diff  )

inlinevirtual

Compute gradient and update parameters,.

Implements UpdatableComponent.

Definition at line 290 of file nnet-recurrent.h.

References RecurrentComponent::bias_, RecurrentComponent::bias_corr_, UpdatableComponent::bias_learn_rate_coef_, kaldi::kNoTrans, kaldi::kSetZero, kaldi::kTrans, NnetTrainOptions::learn_rate, UpdatableComponent::learn_rate_coef_, NnetTrainOptions::momentum, CuMatrixBase< Real >::NumRows(), MultistreamComponent::NumStreams(), UpdatableComponent::opts_, RecurrentComponent::out_, RecurrentComponent::out_diff_bptt_, Component::OutputDim(), RecurrentComponent::w_forward_, RecurrentComponent::w_forward_corr_, RecurrentComponent::w_recurrent_, and RecurrentComponent::w_recurrent_corr_.

                                                   {
    int32 T = input.NumRows() / NumStreams();
    int32 S = NumStreams();

    // getting the learning rate,
    const BaseFloat lr  = opts_.learn_rate;
    const BaseFloat mmt = opts_.momentum;

    if (bias_corr_.Dim() != OutputDim()) {
      w_forward_corr_.Resize(w_forward_.NumRows(), w_forward_.NumCols(), kSetZero);
      w_recurrent_corr_.Resize(w_recurrent_.NumRows(), w_recurrent_.NumCols(), kSetZero);
      bias_corr_.Resize(OutputDim(), kSetZero);
    }

    // getting the gradients,
    w_forward_corr_.AddMatMat(1.0, out_diff_bptt_, kTrans, input, kNoTrans, mmt);


    w_recurrent_corr_.AddMatMat(1.0, out_diff_bptt_.RowRange(S, (T-1)*S), kTrans,
                                               out_.RowRange(0, (T-1)*S), kNoTrans, mmt);

    bias_corr_.AddRowSumMat(1.0, out_diff_bptt_, mmt);

    // updating,
    w_forward_.AddMat(-lr * learn_rate_coef_, w_forward_corr_);
    w_recurrent_.AddMat(-lr * learn_rate_coef_, w_recurrent_corr_);
    bias_.AddVec(-lr * bias_learn_rate_coef_, bias_corr_);
  }

WriteData()

void WriteData ( std::ostream &  os, bool  binary  ) const

inlinevirtual

Writes the component content.

Reimplemented from Component.

Definition at line 110 of file nnet-recurrent.h.

References RecurrentComponent::bias_, UpdatableComponent::bias_learn_rate_coef_, RecurrentComponent::diff_clip_, RecurrentComponent::grad_clip_, UpdatableComponent::learn_rate_coef_, RecurrentComponent::w_forward_, RecurrentComponent::w_recurrent_, kaldi::WriteBasicType(), and kaldi::WriteToken().

                                                    {
    WriteToken(os, binary, "<GradClip>");
    WriteBasicType(os, binary, grad_clip_);
    WriteToken(os, binary, "<DiffClip>");
    WriteBasicType(os, binary, diff_clip_);

    WriteToken(os, binary, "<LearnRateCoef>");
    WriteBasicType(os, binary, learn_rate_coef_);
    WriteToken(os, binary, "<BiasLearnRateCoef>");
    WriteBasicType(os, binary, bias_learn_rate_coef_);

    if (!binary) os << "\n";
    w_forward_.Write(os, binary);
    w_recurrent_.Write(os, binary);
    bias_.Write(os, binary);
  }

Member Data Documentation

bias_

CuVector<BaseFloat> bias_

private

Definition at line 328 of file nnet-recurrent.h.

Referenced by RecurrentComponent::GetParams(), RecurrentComponent::Info(), RecurrentComponent::InitData(), RecurrentComponent::NumParams(), RecurrentComponent::PropagateFnc(), RecurrentComponent::ReadData(), RecurrentComponent::SetParams(), RecurrentComponent::Update(), and RecurrentComponent::WriteData().

bias_corr_

CuVector<BaseFloat> bias_corr_

private

Definition at line 333 of file nnet-recurrent.h.

Referenced by RecurrentComponent::GetGradient(), RecurrentComponent::InfoGradient(), and RecurrentComponent::Update().

diff_clip_

BaseFloat diff_clip_

private

Clipping in the BPTT loop,.

Definition at line 323 of file nnet-recurrent.h.

Referenced by RecurrentComponent::BackpropagateFnc(), RecurrentComponent::InfoGradient(), RecurrentComponent::InitData(), RecurrentComponent::ReadData(), and RecurrentComponent::WriteData().

grad_clip_

BaseFloat grad_clip_

private

Clipping of the update,.

Definition at line 322 of file nnet-recurrent.h.

Referenced by RecurrentComponent::InfoGradient(), RecurrentComponent::InitData(), RecurrentComponent::ReadData(), and RecurrentComponent::WriteData().

out_

CuMatrix<BaseFloat> out_

private

Definition at line 336 of file nnet-recurrent.h.

Referenced by RecurrentComponent::InfoGradient(), RecurrentComponent::PropagateFnc(), and RecurrentComponent::Update().

out_diff_bptt_

CuMatrix<BaseFloat> out_diff_bptt_

private

Definition at line 339 of file nnet-recurrent.h.

Referenced by RecurrentComponent::BackpropagateFnc(), RecurrentComponent::InfoGradient(), and RecurrentComponent::Update().

w_forward_

CuMatrix<BaseFloat> w_forward_

private

Definition at line 326 of file nnet-recurrent.h.

Referenced by RecurrentComponent::BackpropagateFnc(), RecurrentComponent::GetParams(), RecurrentComponent::Info(), RecurrentComponent::InitData(), RecurrentComponent::NumParams(), RecurrentComponent::PropagateFnc(), RecurrentComponent::ReadData(), RecurrentComponent::SetParams(), RecurrentComponent::Update(), and RecurrentComponent::WriteData().

w_forward_corr_

CuMatrix<BaseFloat> w_forward_corr_

private

Definition at line 331 of file nnet-recurrent.h.

Referenced by RecurrentComponent::GetGradient(), RecurrentComponent::InfoGradient(), and RecurrentComponent::Update().

w_recurrent_

CuMatrix<BaseFloat> w_recurrent_

private

Definition at line 327 of file nnet-recurrent.h.

Referenced by RecurrentComponent::BackpropagateFnc(), RecurrentComponent::GetParams(), RecurrentComponent::Info(), RecurrentComponent::InitData(), RecurrentComponent::NumParams(), RecurrentComponent::PropagateFnc(), RecurrentComponent::ReadData(), RecurrentComponent::SetParams(), RecurrentComponent::Update(), and RecurrentComponent::WriteData().

w_recurrent_corr_

CuMatrix<BaseFloat> w_recurrent_corr_

private

Definition at line 332 of file nnet-recurrent.h.

Referenced by RecurrentComponent::GetGradient(), RecurrentComponent::InfoGradient(), and RecurrentComponent::Update().

---

The documentation for this class was generated from the following file:

• nnet/nnet-recurrent.h