Nnet Class Reference

#include <nnet-nnet.h>

Collaboration diagram for Nnet:

Public Member Functions

 Nnet ()
 
 ~Nnet ()
 
 Nnet (const Nnet &other)
 
Nnetoperator= (const Nnet &other)
 
void Propagate (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out)
 Perform forward pass through the network,. More...
 
void Backpropagate (const CuMatrixBase< BaseFloat > &out_diff, CuMatrix< BaseFloat > *in_diff)
 Perform backward pass through the network,. More...
 
void Feedforward (const CuMatrixBase< BaseFloat > &in, CuMatrix< BaseFloat > *out)
 Perform forward pass through the network (with 2 swapping buffers),. More...
 
int32 InputDim () const
 Dimensionality on network input (input feature dim.),. More...
 
int32 OutputDim () const
 Dimensionality of network outputs (posteriors | bn-features | etc.),. More...
 
int32 NumComponents () const
 Returns the number of 'Components' which form the NN. More...
 
const ComponentGetComponent (int32 c) const
 Component accessor,. More...
 
ComponentGetComponent (int32 c)
 Component accessor,. More...
 
const ComponentGetLastComponent () const
 LastComponent accessor,. More...
 
ComponentGetLastComponent ()
 LastComponent accessor,. More...
 
void ReplaceComponent (int32 c, const Component &comp)
 Replace c'th component in 'this' Nnet (deep copy),. More...
 
void SwapComponent (int32 c, Component **comp)
 Swap c'th component with the pointer,. More...
 
void AppendComponent (const Component &comp)
 Append Component to 'this' instance of Nnet (deep copy),. More...
 
void AppendComponentPointer (Component *dynamically_allocated_comp)
 Append Component* to 'this' instance of Nnet by a shallow copy ('this' instance of Nnet over-takes the ownership of the pointer). More...
 
void AppendNnet (const Nnet &nnet_to_append)
 Append other Nnet to the 'this' Nnet (copy all its components),. More...
 
void RemoveComponent (int32 c)
 Remove c'th component,. More...
 
void RemoveLastComponent ()
 Remove the last of the Components,. More...
 
const std::vector< CuMatrix< BaseFloat > > & PropagateBuffer () const
 Access to the forward-pass buffers. More...
 
const std::vector< CuMatrix< BaseFloat > > & BackpropagateBuffer () const
 Access to the backward-pass buffers. More...
 
int32 NumParams () const
 Get the number of parameters in the network,. More...
 
void GetGradient (Vector< BaseFloat > *gradient) const
 Get the gradient stored in the network,. More...
 
void GetParams (Vector< BaseFloat > *params) const
 Get the network weights in a supervector,. More...
 
void SetParams (const VectorBase< BaseFloat > &params)
 Set the network weights from a supervector,. More...
 
void SetDropoutRate (BaseFloat r)
 Set the dropout rate. More...
 
void ResetStreams (const std::vector< int32 > &stream_reset_flag)
 Reset streams in multi-stream training,. More...
 
void SetSeqLengths (const std::vector< int32 > &sequence_lengths)
 Set sequence length in LSTM multi-stream training,. More...
 
void Init (const std::string &proto_file)
 Initialize the Nnet from the prototype,. More...
 
void Read (const std::string &rxfilename)
 Read Nnet from 'rxfilename',. More...
 
void Read (std::istream &in, bool binary)
 Read Nnet from 'istream',. More...
 
void Write (const std::string &wxfilename, bool binary) const
 Write Nnet to 'wxfilename',. More...
 
void Write (std::ostream &out, bool binary) const
 Write Nnet to 'ostream',. More...
 
std::string Info () const
 Create string with human readable description of the nnet,. More...
 
std::string InfoGradient (bool header=true) const
 Create string with per-component gradient statistics,. More...
 
std::string InfoPropagate (bool header=true) const
 Create string with propagation-buffer statistics,. More...
 
std::string InfoBackPropagate (bool header=true) const
 Create string with back-propagation-buffer statistics,. More...
 
void Check () const
 Consistency check,. More...
 
void Destroy ()
 Relese the memory,. More...
 
void SetTrainOptions (const NnetTrainOptions &opts)
 Set hyper-parameters of the training (pushes to all UpdatableComponents),. More...
 
const NnetTrainOptionsGetTrainOptions () const
 Get training hyper-parameters from the network,. More...
 

Private Attributes

std::vector< Component * > components_
 Vector which contains all the components composing the neural network, the components are for example: AffineTransform, Sigmoid, Softmax. More...
 
std::vector< CuMatrix< BaseFloat > > propagate_buf_
 Buffers for forward pass (on demand initialization),. More...
 
std::vector< CuMatrix< BaseFloat > > backpropagate_buf_
 Buffers for backward pass (on demand initialization),. More...
 
NnetTrainOptions opts_
 Option class with hyper-parameters passed to UpdatableComponent(s) More...
 

Detailed Description

Definition at line 37 of file nnet-nnet.h.

Constructor & Destructor Documentation

◆ Nnet() [1/2]

Nnet ( )

Definition at line 31 of file nnet-nnet.cc.

31  {
32 }

◆ ~Nnet()

~Nnet ( )

Definition at line 34 of file nnet-nnet.cc.

References Nnet::Destroy().

34  {
35  Destroy();
36 }
void Destroy()
Relese the memory,.
Definition: nnet-nnet.cc:498

◆ Nnet() [2/2]

Nnet ( const Nnet other)

Definition at line 38 of file nnet-nnet.cc.

References Nnet::backpropagate_buf_, Nnet::Check(), Nnet::components_, Component::Copy(), Nnet::GetComponent(), rnnlm::i, Nnet::NumComponents(), Nnet::opts_, Nnet::propagate_buf_, and Nnet::SetTrainOptions().

38  {
39  // copy the components
40  for (int32 i = 0; i < other.NumComponents(); i++) {
41  components_.push_back(other.GetComponent(i).Copy());
42  }
43  // create empty buffers
44  propagate_buf_.resize(NumComponents()+1);
46  // copy train opts
47  SetTrainOptions(other.opts_);
48  Check();
49 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
std::vector< CuMatrix< BaseFloat > > propagate_buf_
Buffers for forward pass (on demand initialization),.
Definition: nnet-nnet.h:173
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
void SetTrainOptions(const NnetTrainOptions &opts)
Set hyper-parameters of the training (pushes to all UpdatableComponents),.
Definition: nnet-nnet.cc:508
std::vector< CuMatrix< BaseFloat > > backpropagate_buf_
Buffers for backward pass (on demand initialization),.
Definition: nnet-nnet.h:175

Member Function Documentation

◆ AppendComponent()

void AppendComponent ( const Component comp)

Append Component to 'this' instance of Nnet (deep copy),.

Definition at line 182 of file nnet-nnet.cc.

References Nnet::Check(), Nnet::components_, and Component::Copy().

Referenced by Nnet::AppendNnet(), main(), and Nnet::NumComponents().

182  {
183  components_.push_back(comp.Copy()); // append,
184  Check();
185 }
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ AppendComponentPointer()

void AppendComponentPointer ( Component dynamically_allocated_comp)

Append Component* to 'this' instance of Nnet by a shallow copy ('this' instance of Nnet over-takes the ownership of the pointer).

Definition at line 187 of file nnet-nnet.cc.

References Nnet::Check(), and Nnet::components_.

Referenced by Nnet::Init(), main(), Nnet::NumComponents(), and Nnet::Read().

187  {
188  components_.push_back(dynamically_allocated_comp); // append,
189  Check();
190 }
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ AppendNnet()

void AppendNnet ( const Nnet nnet_to_append)

Append other Nnet to the 'this' Nnet (copy all its components),.

Definition at line 192 of file nnet-nnet.cc.

References Nnet::AppendComponent(), Nnet::Check(), Nnet::GetComponent(), rnnlm::i, and Nnet::NumComponents().

Referenced by main(), and Nnet::NumComponents().

192  {
193  for (int32 i = 0; i < other.NumComponents(); i++) {
194  AppendComponent(other.GetComponent(i));
195  }
196  Check();
197 }
kaldi::int32 int32
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
void AppendComponent(const Component &comp)
Append Component to &#39;this&#39; instance of Nnet (deep copy),.
Definition: nnet-nnet.cc:182

◆ Backpropagate()

void Backpropagate ( const CuMatrixBase< BaseFloat > &  out_diff,
CuMatrix< BaseFloat > *  in_diff 
)

Perform backward pass through the network,.

Error back-propagation through the network, (from last component to first).

Definition at line 96 of file nnet-nnet.cc.

References Nnet::backpropagate_buf_, Nnet::components_, rnnlm::i, KALDI_ASSERT, Nnet::NumComponents(), Nnet::propagate_buf_, and UpdatableComponent::Update().

Referenced by MultiBasisComponent::BackpropagateFnc(), and main().

97  {
98  // Copy the derivative in case of empty network,
99  if (NumComponents() == 0) {
100  (*in_diff) = out_diff; // copy,
101  return;
102  }
103  // We need C+1 buffers,
104  KALDI_ASSERT(static_cast<int32>(propagate_buf_.size()) == NumComponents()+1);
105  if (backpropagate_buf_.size() != NumComponents()+1) {
106  backpropagate_buf_.resize(NumComponents()+1);
107  }
108  // Copy 'out_diff' to last buffer,
109  backpropagate_buf_[NumComponents()] = out_diff;
110  // Loop from last Component to the first,
111  for (int32 i = NumComponents()-1; i >= 0; i--) {
112  // Backpropagate through 'Component',
113  components_[i]->Backpropagate(propagate_buf_[i],
114  propagate_buf_[i+1],
115  backpropagate_buf_[i+1],
116  &backpropagate_buf_[i]);
117  // Update 'Component' (if applicable),
118  if (components_[i]->IsUpdatable()) {
119  UpdatableComponent* uc =
120  dynamic_cast<UpdatableComponent*>(components_[i]);
121  uc->Update(propagate_buf_[i], backpropagate_buf_[i+1]);
122  }
123  }
124  // Export the derivative (if applicable),
125  if (NULL != in_diff) {
126  (*in_diff) = backpropagate_buf_[0];
127  }
128 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
std::vector< CuMatrix< BaseFloat > > propagate_buf_
Buffers for forward pass (on demand initialization),.
Definition: nnet-nnet.h:173
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185
std::vector< CuMatrix< BaseFloat > > backpropagate_buf_
Buffers for backward pass (on demand initialization),.
Definition: nnet-nnet.h:175

◆ BackpropagateBuffer()

const std::vector<CuMatrix<BaseFloat> >& BackpropagateBuffer ( ) const
inline

Access to the backward-pass buffers.

Definition at line 109 of file nnet-nnet.h.

References Nnet::backpropagate_buf_, Nnet::Check(), Nnet::Destroy(), Nnet::GetGradient(), Nnet::GetParams(), Nnet::Info(), Nnet::InfoBackPropagate(), Nnet::InfoGradient(), Nnet::InfoPropagate(), Nnet::Init(), Nnet::NumParams(), Nnet::Read(), Nnet::ResetStreams(), Nnet::SetDropoutRate(), Nnet::SetParams(), Nnet::SetSeqLengths(), Nnet::SetTrainOptions(), and Nnet::Write().

109  {
110  return backpropagate_buf_;
111  }
std::vector< CuMatrix< BaseFloat > > backpropagate_buf_
Buffers for backward pass (on demand initialization),.
Definition: nnet-nnet.h:175

◆ Check()

void Check ( ) const

Consistency check,.

Definition at line 467 of file nnet-nnet.cc.

References Nnet::components_, Nnet::GetParams(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, KALDI_ISINF, KALDI_ISNAN, VectorBase< Real >::Sum(), and Component::TypeToMarker().

Referenced by Nnet::AppendComponent(), Nnet::AppendComponentPointer(), Nnet::AppendNnet(), Nnet::BackpropagateBuffer(), Nnet::Init(), Nnet::Nnet(), Nnet::operator=(), Nnet::Read(), Nnet::RemoveComponent(), Nnet::ReplaceComponent(), Nnet::SwapComponent(), and Nnet::Write().

467  {
468  // check dims,
469  for (size_t i = 0; i + 1 < components_.size(); i++) {
470  KALDI_ASSERT(components_[i] != NULL);
471  int32 output_dim = components_[i]->OutputDim(),
472  next_input_dim = components_[i+1]->InputDim();
473  // show error message,
474  if (output_dim != next_input_dim) {
475  KALDI_ERR << "Component dimension mismatch!"
476  << " Output dim of [" << i << "] "
477  << Component::TypeToMarker(components_[i]->GetType())
478  << " is " << output_dim << ". "
479  << "Input dim of next [" << i+1 << "] "
480  << Component::TypeToMarker(components_[i+1]->GetType())
481  << " is " << next_input_dim << ".";
482  }
483  }
484  // check for nan/inf in network weights,
485  Vector<BaseFloat> weights;
486  GetParams(&weights);
487  BaseFloat sum = weights.Sum();
488  if (KALDI_ISINF(sum)) {
489  KALDI_ERR << "'inf' in network parameters "
490  << "(weight explosion, need lower learning rate?)";
491  }
492  if (KALDI_ISNAN(sum)) {
493  KALDI_ERR << "'nan' in network parameters (need lower learning rate?)";
494  }
495 }
void GetParams(Vector< BaseFloat > *params) const
Get the network weights in a supervector,.
Definition: nnet-nnet.cc:237
#define KALDI_ISINF
Definition: kaldi-math.h:73
kaldi::int32 int32
static const char * TypeToMarker(ComponentType t)
Converts component type to marker,.
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ERR
Definition: kaldi-error.h:147
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ISNAN
Definition: kaldi-math.h:72
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ Destroy()

void Destroy ( )

Relese the memory,.

Definition at line 498 of file nnet-nnet.cc.

References Nnet::backpropagate_buf_, Nnet::components_, rnnlm::i, Nnet::NumComponents(), and Nnet::propagate_buf_.

Referenced by Nnet::BackpropagateBuffer(), Nnet::operator=(), and Nnet::~Nnet().

498  {
499  for (int32 i = 0; i < NumComponents(); i++) {
500  delete components_[i];
501  }
502  components_.resize(0);
503  propagate_buf_.resize(0);
504  backpropagate_buf_.resize(0);
505 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
std::vector< CuMatrix< BaseFloat > > propagate_buf_
Buffers for forward pass (on demand initialization),.
Definition: nnet-nnet.h:173
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
std::vector< CuMatrix< BaseFloat > > backpropagate_buf_
Buffers for backward pass (on demand initialization),.
Definition: nnet-nnet.h:175

◆ Feedforward()

void Feedforward ( const CuMatrixBase< BaseFloat > &  in,
CuMatrix< BaseFloat > *  out 
)

Perform forward pass through the network (with 2 swapping buffers),.

Definition at line 131 of file nnet-nnet.cc.

References Nnet::components_, rnnlm::i, KALDI_ASSERT, Nnet::NumComponents(), and CuMatrix< Real >::Swap().

Referenced by main().

132  {
133  KALDI_ASSERT(NULL != out);
134  (*out) = in; // works even with 0 components,
135  CuMatrix<BaseFloat> tmp_in;
136  for (int32 i = 0; i < NumComponents(); i++) {
137  out->Swap(&tmp_in);
138  components_[i]->Propagate(tmp_in, out);
139  }
140 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ GetComponent() [1/2]

const Component & GetComponent ( int32  c) const

Component accessor,.

Definition at line 153 of file nnet-nnet.cc.

References Nnet::components_.

Referenced by Nnet::AppendNnet(), kaldi::ConvertNnet1ToNnet2(), Rbm::InitData(), main(), Nnet::Nnet(), Nnet::NumComponents(), Nnet::operator=(), Nnet::ResetStreams(), Nnet::SetDropoutRate(), Nnet::SetSeqLengths(), and Nnet::SetTrainOptions().

153  {
154  return *(components_.at(c));
155 }
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ GetComponent() [2/2]

Component & GetComponent ( int32  c)

Component accessor,.

Definition at line 157 of file nnet-nnet.cc.

References Nnet::components_.

157  {
158  return *(components_.at(c));
159 }
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ GetGradient()

void GetGradient ( Vector< BaseFloat > *  gradient) const

Get the gradient stored in the network,.

Definition at line 221 of file nnet-nnet.cc.

References Nnet::components_, UpdatableComponent::GetGradient(), rnnlm::i, KALDI_ASSERT, Nnet::NumParams(), UpdatableComponent::NumParams(), VectorBase< Real >::Range(), and Vector< Real >::Resize().

Referenced by Nnet::BackpropagateBuffer().

221  {
222  gradient->Resize(NumParams());
223  int32 pos = 0;
224  // loop over Components,
225  for (int32 i = 0; i < components_.size(); i++) {
226  if (components_[i]->IsUpdatable()) {
227  UpdatableComponent& c =
228  dynamic_cast<UpdatableComponent&>(*components_[i]);
229  SubVector<BaseFloat> grad_range(gradient->Range(pos, c.NumParams()));
230  c.GetGradient(&grad_range); // getting gradient,
231  pos += c.NumParams();
232  }
233  }
234  KALDI_ASSERT(pos == NumParams());
235 }
int32 NumParams() const
Get the number of parameters in the network,.
Definition: nnet-nnet.cc:210
kaldi::int32 int32
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ GetLastComponent() [1/2]

const Component & GetLastComponent ( ) const

LastComponent accessor,.

Definition at line 161 of file nnet-nnet.cc.

References Nnet::components_, and Nnet::NumComponents().

Referenced by main(), and Nnet::NumComponents().

161  {
162  return *(components_.at(NumComponents()-1));
163 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ GetLastComponent() [2/2]

Component & GetLastComponent ( )

LastComponent accessor,.

Definition at line 165 of file nnet-nnet.cc.

References Nnet::components_, and Nnet::NumComponents().

165  {
166  return *(components_.at(NumComponents()-1));
167 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ GetParams()

void GetParams ( Vector< BaseFloat > *  params) const

Get the network weights in a supervector,.

Definition at line 237 of file nnet-nnet.cc.

References Nnet::components_, UpdatableComponent::GetParams(), rnnlm::i, KALDI_ASSERT, Nnet::NumParams(), UpdatableComponent::NumParams(), VectorBase< Real >::Range(), and Vector< Real >::Resize().

Referenced by Nnet::BackpropagateBuffer(), Nnet::Check(), and MultiBasisComponent::GetParams().

237  {
238  params->Resize(NumParams());
239  int32 pos = 0;
240  // loop over Components,
241  for (int32 i = 0; i < components_.size(); i++) {
242  if (components_[i]->IsUpdatable()) {
243  UpdatableComponent& c =
244  dynamic_cast<UpdatableComponent&>(*components_[i]);
245  SubVector<BaseFloat> params_range(params->Range(pos, c.NumParams()));
246  c.GetParams(&params_range); // getting params,
247  pos += c.NumParams();
248  }
249  }
250  KALDI_ASSERT(pos == NumParams());
251 }
int32 NumParams() const
Get the number of parameters in the network,.
Definition: nnet-nnet.cc:210
kaldi::int32 int32
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ GetTrainOptions()

const NnetTrainOptions& GetTrainOptions ( ) const
inline

Get training hyper-parameters from the network,.

Definition at line 163 of file nnet-nnet.h.

References Nnet::opts_.

163  {
164  return opts_;
165  }
NnetTrainOptions opts_
Option class with hyper-parameters passed to UpdatableComponent(s)
Definition: nnet-nnet.h:178

◆ Info()

std::string Info ( ) const

Create string with human readable description of the nnet,.

Definition at line 386 of file nnet-nnet.cc.

References Nnet::components_, rnnlm::i, Nnet::InputDim(), Nnet::NumComponents(), Nnet::NumParams(), Nnet::OutputDim(), and Component::TypeToMarker().

Referenced by Nnet::BackpropagateBuffer(), MultiBasisComponent::Info(), and main().

386  {
387  // global info
388  std::ostringstream ostr;
389  ostr << "num-components " << NumComponents() << std::endl;
390  if (NumComponents() == 0)
391  return ostr.str();
392  ostr << "input-dim " << InputDim() << std::endl;
393  ostr << "output-dim " << OutputDim() << std::endl;
394  ostr << "number-of-parameters " << static_cast<float>(NumParams())/1e6
395  << " millions" << std::endl;
396  // topology & weight stats
397  for (int32 i = 0; i < NumComponents(); i++) {
398  ostr << "component " << i+1 << " : "
399  << Component::TypeToMarker(components_[i]->GetType())
400  << ", input-dim " << components_[i]->InputDim()
401  << ", output-dim " << components_[i]->OutputDim()
402  << ", " << components_[i]->Info() << std::endl;
403  }
404  return ostr.str();
405 }
int32 NumParams() const
Get the number of parameters in the network,.
Definition: nnet-nnet.cc:210
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
int32 InputDim() const
Dimensionality on network input (input feature dim.),.
Definition: nnet-nnet.cc:148
kaldi::int32 int32
static const char * TypeToMarker(ComponentType t)
Converts component type to marker,.
int32 OutputDim() const
Dimensionality of network outputs (posteriors | bn-features | etc.),.
Definition: nnet-nnet.cc:143
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ InfoBackPropagate()

std::string InfoBackPropagate ( bool  header = true) const

Create string with back-propagation-buffer statistics,.

Definition at line 443 of file nnet-nnet.cc.

References Nnet::backpropagate_buf_, Nnet::components_, rnnlm::i, Component::kMultiBasisComponent, Component::kParallelComponent, kaldi::nnet1::MomentStatistics(), Nnet::NumComponents(), and Component::TypeToMarker().

Referenced by Nnet::BackpropagateBuffer(), MultiBasisComponent::InfoBackPropagate(), and main().

443  {
444  std::ostringstream ostr;
445  // forward-pass buffer stats
446  if (header) ostr << "\n### BACKWARD PROPAGATION BUFFER CONTENT :\n";
447  ostr << "[0] diff of <Input> " << MomentStatistics(backpropagate_buf_[0])
448  << std::endl;
449  for (int32 i = 0; i < NumComponents(); i++) {
450  ostr << "["<<1+i<< "] diff-output of "
451  << Component::TypeToMarker(components_[i]->GetType())
452  << MomentStatistics(backpropagate_buf_[i+1]) << std::endl;
453  // nested networks too...
454  if (Component::kParallelComponent == components_[i]->GetType()) {
455  ostr <<
456  dynamic_cast<ParallelComponent*>(components_[i])->InfoBackPropagate();
457  }
458  if (Component::kMultiBasisComponent == components_[i]->GetType()) {
459  ostr << dynamic_cast<MultiBasisComponent*>(components_[i])->InfoBackPropagate();
460  }
461  }
462  if (header) ostr << "### END BACKWARD\n\n";
463  return ostr.str();
464 }
std::string MomentStatistics(const VectorBase< Real > &vec)
Get a string with statistics of the data in a vector, so we can print them easily.
Definition: nnet-utils.h:63
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
static const char * TypeToMarker(ComponentType t)
Converts component type to marker,.
std::string InfoBackPropagate(bool header=true) const
Create string with back-propagation-buffer statistics,.
Definition: nnet-nnet.cc:443
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
std::vector< CuMatrix< BaseFloat > > backpropagate_buf_
Buffers for backward pass (on demand initialization),.
Definition: nnet-nnet.h:175

◆ InfoGradient()

std::string InfoGradient ( bool  header = true) const

Create string with per-component gradient statistics,.

Definition at line 407 of file nnet-nnet.cc.

References Nnet::components_, rnnlm::i, Nnet::NumComponents(), and Component::TypeToMarker().

Referenced by Nnet::BackpropagateBuffer(), MultiBasisComponent::InfoGradient(), and main().

407  {
408  std::ostringstream ostr;
409  // gradient stats
410  if (header) ostr << "\n### GRADIENT STATS :\n";
411  for (int32 i = 0; i < NumComponents(); i++) {
412  ostr << "Component " << i+1 << " : "
413  << Component::TypeToMarker(components_[i]->GetType())
414  << ", " << components_[i]->InfoGradient() << std::endl;
415  }
416  if (header) ostr << "### END GRADIENT\n";
417  return ostr.str();
418 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
static const char * TypeToMarker(ComponentType t)
Converts component type to marker,.
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ InfoPropagate()

std::string InfoPropagate ( bool  header = true) const

Create string with propagation-buffer statistics,.

Definition at line 420 of file nnet-nnet.cc.

References Nnet::components_, rnnlm::i, Component::kMultiBasisComponent, Component::kParallelComponent, kaldi::nnet1::MomentStatistics(), Nnet::NumComponents(), Nnet::propagate_buf_, and Component::TypeToMarker().

Referenced by Nnet::BackpropagateBuffer(), MultiBasisComponent::InfoPropagate(), and main().

420  {
421  std::ostringstream ostr;
422  // forward-pass buffer stats
423  if (header) ostr << "\n### FORWARD PROPAGATION BUFFER CONTENT :\n";
424  ostr << "[0] output of <Input> " << MomentStatistics(propagate_buf_[0])
425  << std::endl;
426  for (int32 i = 0; i < NumComponents(); i++) {
427  ostr << "[" << 1+i << "] output of "
428  << Component::TypeToMarker(components_[i]->GetType())
429  << MomentStatistics(propagate_buf_[i+1]) << std::endl;
430  // nested networks too...
431  if (Component::kParallelComponent == components_[i]->GetType()) {
432  ostr <<
433  dynamic_cast<ParallelComponent*>(components_[i])->InfoPropagate();
434  }
435  if (Component::kMultiBasisComponent == components_[i]->GetType()) {
436  ostr << dynamic_cast<MultiBasisComponent*>(components_[i])->InfoPropagate();
437  }
438  }
439  if (header) ostr << "### END FORWARD\n";
440  return ostr.str();
441 }
std::string MomentStatistics(const VectorBase< Real > &vec)
Get a string with statistics of the data in a vector, so we can print them easily.
Definition: nnet-utils.h:63
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
static const char * TypeToMarker(ComponentType t)
Converts component type to marker,.
std::vector< CuMatrix< BaseFloat > > propagate_buf_
Buffers for forward pass (on demand initialization),.
Definition: nnet-nnet.h:173
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
std::string InfoPropagate(bool header=true) const
Create string with propagation-buffer statistics,.
Definition: nnet-nnet.cc:420

◆ Init()

void Init ( const std::string &  proto_file)

Initialize the Nnet from the prototype,.

Definition at line 301 of file nnet-nnet.cc.

References Nnet::AppendComponentPointer(), Nnet::Check(), Input::Close(), Component::Init(), KALDI_ASSERT, KALDI_VLOG, and Input::Stream().

Referenced by Nnet::BackpropagateBuffer(), MultiBasisComponent::InitData(), ParallelComponent::InitData(), and main().

301  {
302  Input in(proto_file);
303  std::istream &is = in.Stream();
304  std::string proto_line, token;
305 
306  // Initialize from the prototype, where each line
307  // contains the description for one component.
308  while (is >> std::ws, !is.eof()) {
309  KALDI_ASSERT(is.good());
310 
311  // get a line from the proto file,
312  std::getline(is, proto_line);
313  if (proto_line == "") continue;
314  KALDI_VLOG(1) << proto_line;
315 
316  // get the 1st token from the line,
317  std::istringstream(proto_line) >> std::ws >> token;
318  // ignore these tokens:
319  if (token == "<NnetProto>" || token == "</NnetProto>") continue;
320 
321  // create new component, append to Nnet,
322  this->AppendComponentPointer(Component::Init(proto_line+"\n"));
323  }
324  // cleanup
325  in.Close();
326  Check();
327 }
void AppendComponentPointer(Component *dynamically_allocated_comp)
Append Component* to &#39;this&#39; instance of Nnet by a shallow copy (&#39;this&#39; instance of Nnet over-takes th...
Definition: nnet-nnet.cc:187
static Component * Init(const std::string &conf_line)
Initialize component from a line in config file,.
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185
#define KALDI_VLOG(v)
Definition: kaldi-error.h:156

◆ InputDim()

int32 InputDim ( ) const

Dimensionality on network input (input feature dim.),.

Definition at line 148 of file nnet-nnet.cc.

References Nnet::components_, and KALDI_ASSERT.

Referenced by MultiBasisComponent::BackpropagateFnc(), Nnet::Info(), MultiBasisComponent::InitData(), Rbm::InitData(), main(), MultiBasisComponent::PropagateFnc(), and MultiBasisComponent::ReadData().

148  {
149  KALDI_ASSERT(!components_.empty());
150  return components_.front()->InputDim();
151 }
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ NumComponents()

int32 NumComponents ( ) const
inline

Returns the number of 'Components' which form the NN.

Typically a NN layer is composed of 2 components: the <AffineTransform> with trainable parameters and a non-linearity like <Sigmoid> or <Softmax>. Usually there are 2x more Components than the NN layers.

Definition at line 66 of file nnet-nnet.h.

References Nnet::AppendComponent(), Nnet::AppendComponentPointer(), Nnet::AppendNnet(), Nnet::components_, Nnet::GetComponent(), Nnet::GetLastComponent(), Nnet::RemoveComponent(), Nnet::RemoveLastComponent(), Nnet::ReplaceComponent(), and Nnet::SwapComponent().

Referenced by Nnet::AppendNnet(), Nnet::Backpropagate(), kaldi::ConvertNnet1ToNnet2(), Nnet::Destroy(), Nnet::Feedforward(), Nnet::GetLastComponent(), Nnet::Info(), Nnet::InfoBackPropagate(), Nnet::InfoGradient(), Nnet::InfoPropagate(), main(), Nnet::Nnet(), Nnet::operator=(), Nnet::Propagate(), Nnet::Read(), Nnet::RemoveLastComponent(), Nnet::ResetStreams(), Nnet::SetDropoutRate(), Nnet::SetSeqLengths(), Nnet::SetTrainOptions(), and Nnet::Write().

66  {
67  return components_.size();
68  }
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ NumParams()

int32 NumParams ( ) const

Get the number of parameters in the network,.

Definition at line 210 of file nnet-nnet.cc.

References Nnet::components_, and rnnlm::n.

Referenced by Nnet::BackpropagateBuffer(), Nnet::GetGradient(), Nnet::GetParams(), Nnet::Info(), MultiBasisComponent::NumParams(), Nnet::SetParams(), and MultiBasisComponent::SetParams().

210  {
211  int32 n_params = 0;
212  for (int32 n = 0; n < components_.size(); n++) {
213  if (components_[n]->IsUpdatable()) {
214  n_params +=
215  dynamic_cast<UpdatableComponent*>(components_[n])->NumParams();
216  }
217  }
218  return n_params;
219 }
int32 NumParams() const
Get the number of parameters in the network,.
Definition: nnet-nnet.cc:210
kaldi::int32 int32
struct rnnlm::@11::@12 n
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ operator=()

Nnet & operator= ( const Nnet other)

Definition at line 51 of file nnet-nnet.cc.

References Nnet::backpropagate_buf_, Nnet::Check(), Nnet::components_, Component::Copy(), Nnet::Destroy(), Nnet::GetComponent(), rnnlm::i, Nnet::NumComponents(), Nnet::opts_, Nnet::propagate_buf_, and Nnet::SetTrainOptions().

51  {
52  Destroy();
53  // copy the components
54  for (int32 i = 0; i < other.NumComponents(); i++) {
55  components_.push_back(other.GetComponent(i).Copy());
56  }
57  // create empty buffers
58  propagate_buf_.resize(NumComponents()+1);
60  // copy train opts
61  SetTrainOptions(other.opts_);
62  Check();
63  return *this;
64 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
void Destroy()
Relese the memory,.
Definition: nnet-nnet.cc:498
std::vector< CuMatrix< BaseFloat > > propagate_buf_
Buffers for forward pass (on demand initialization),.
Definition: nnet-nnet.h:173
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
void SetTrainOptions(const NnetTrainOptions &opts)
Set hyper-parameters of the training (pushes to all UpdatableComponents),.
Definition: nnet-nnet.cc:508
std::vector< CuMatrix< BaseFloat > > backpropagate_buf_
Buffers for backward pass (on demand initialization),.
Definition: nnet-nnet.h:175

◆ OutputDim()

int32 OutputDim ( ) const

Dimensionality of network outputs (posteriors | bn-features | etc.),.

Definition at line 143 of file nnet-nnet.cc.

References Nnet::components_, and KALDI_ASSERT.

Referenced by Nnet::Info(), MultiBasisComponent::InitData(), main(), and MultiBasisComponent::ReadData().

143  {
144  KALDI_ASSERT(!components_.empty());
145  return components_.back()->OutputDim();
146 }
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ Propagate()

void Propagate ( const CuMatrixBase< BaseFloat > &  in,
CuMatrix< BaseFloat > *  out 
)

Perform forward pass through the network,.

Forward propagation through the network, (from first component to last).

Definition at line 70 of file nnet-nnet.cc.

References Nnet::components_, rnnlm::i, Nnet::NumComponents(), and Nnet::propagate_buf_.

Referenced by main(), and MultiBasisComponent::PropagateFnc().

71  {
72  // In case of empty network copy input to output,
73  if (NumComponents() == 0) {
74  (*out) = in; // copy,
75  return;
76  }
77  // We need C+1 buffers,
78  if (propagate_buf_.size() != NumComponents()+1) {
79  propagate_buf_.resize(NumComponents()+1);
80  }
81  // Copy input to first buffer,
82  propagate_buf_[0] = in;
83  // Propagate through all the components,
84  for (int32 i = 0; i < static_cast<int32>(components_.size()); i++) {
85  components_[i]->Propagate(propagate_buf_[i], &propagate_buf_[i+1]);
86  }
87  // Copy the output from the last buffer,
88  (*out) = propagate_buf_[NumComponents()];
89 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
std::vector< CuMatrix< BaseFloat > > propagate_buf_
Buffers for forward pass (on demand initialization),.
Definition: nnet-nnet.h:173
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ PropagateBuffer()

const std::vector<CuMatrix<BaseFloat> >& PropagateBuffer ( ) const
inline

Access to the forward-pass buffers.

Definition at line 105 of file nnet-nnet.h.

References Nnet::propagate_buf_.

105  {
106  return propagate_buf_;
107  }
std::vector< CuMatrix< BaseFloat > > propagate_buf_
Buffers for forward pass (on demand initialization),.
Definition: nnet-nnet.h:173

◆ Read() [1/2]

void Read ( const std::string &  rxfilename)

Read Nnet from 'rxfilename',.

I/O wrapper for converting 'rxfilename' to 'istream',.

Definition at line 333 of file nnet-nnet.cc.

References Input::Close(), KALDI_WARN, Nnet::NumComponents(), and Input::Stream().

Referenced by Nnet::BackpropagateBuffer(), MultiBasisComponent::InitData(), ParallelComponent::InitData(), Rbm::InitData(), main(), ParallelComponent::ReadData(), and MultiBasisComponent::ReadData().

333  {
334  bool binary;
335  Input in(rxfilename, &binary);
336  Read(in.Stream(), binary);
337  in.Close();
338  // Warn if the NN is empty
339  if (NumComponents() == 0) {
340  KALDI_WARN << "The network '" << rxfilename << "' is empty.";
341  }
342 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
void Read(const std::string &rxfilename)
Read Nnet from &#39;rxfilename&#39;,.
Definition: nnet-nnet.cc:333
#define KALDI_WARN
Definition: kaldi-error.h:150

◆ Read() [2/2]

void Read ( std::istream &  in,
bool  binary 
)

Read Nnet from 'istream',.

Definition at line 345 of file nnet-nnet.cc.

References Nnet::AppendComponentPointer(), Nnet::Check(), Nnet::components_, Component::InputDim(), KALDI_ERR, Nnet::NumComponents(), and Component::Read().

345  {
346  // Read the Components through the 'factory' Component::Read(...),
347  Component* comp(NULL);
348  while (comp = Component::Read(is, binary), comp != NULL) {
349  // Check dims,
350  if (NumComponents() > 0) {
351  if (components_.back()->OutputDim() != comp->InputDim()) {
352  KALDI_ERR << "Dimensionality mismatch!"
353  << " Previous layer output:" << components_.back()->OutputDim()
354  << " Current layer input:" << comp->InputDim();
355  }
356  }
357  // Append to 'this' Nnet,
359  }
360  Check();
361 }
void AppendComponentPointer(Component *dynamically_allocated_comp)
Append Component* to &#39;this&#39; instance of Nnet by a shallow copy (&#39;this&#39; instance of Nnet over-takes th...
Definition: nnet-nnet.cc:187
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
static Component * Read(std::istream &is, bool binary)
Read the component from a stream (static method),.
#define KALDI_ERR
Definition: kaldi-error.h:147
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ RemoveComponent()

void RemoveComponent ( int32  c)

Remove c'th component,.

Definition at line 199 of file nnet-nnet.cc.

References Nnet::Check(), and Nnet::components_.

Referenced by main(), Nnet::NumComponents(), and Nnet::RemoveLastComponent().

199  {
200  Component* ptr = components_.at(c);
201  components_.erase(components_.begin()+c);
202  delete ptr;
203  Check();
204 }
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ RemoveLastComponent()

void RemoveLastComponent ( )

Remove the last of the Components,.

Definition at line 206 of file nnet-nnet.cc.

References Nnet::NumComponents(), and Nnet::RemoveComponent().

Referenced by main(), and Nnet::NumComponents().

206  {
208 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
void RemoveComponent(int32 c)
Remove c&#39;th component,.
Definition: nnet-nnet.cc:199

◆ ReplaceComponent()

void ReplaceComponent ( int32  c,
const Component comp 
)

Replace c'th component in 'this' Nnet (deep copy),.

Definition at line 169 of file nnet-nnet.cc.

References Nnet::Check(), Nnet::components_, and Component::Copy().

Referenced by Nnet::NumComponents().

169  {
170  delete components_.at(c);
171  components_.at(c) = comp.Copy(); // deep copy,
172  Check();
173 }
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ ResetStreams()

void ResetStreams ( const std::vector< int32 > &  stream_reset_flag)

Reset streams in multi-stream training,.

Definition at line 281 of file nnet-nnet.cc.

References Nnet::GetComponent(), Component::IsMultistream(), Nnet::NumComponents(), and MultistreamComponent::ResetStreams().

Referenced by Nnet::BackpropagateBuffer(), and main().

281  {
282  for (int32 c = 0; c < NumComponents(); c++) {
283  if (GetComponent(c).IsMultistream()) {
284  MultistreamComponent& comp =
285  dynamic_cast<MultistreamComponent&>(GetComponent(c));
286  comp.ResetStreams(stream_reset_flag);
287  }
288  }
289 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
virtual bool IsMultistream() const
Check if component has &#39;Recurrent&#39; interface (trainable and recurrent),.
const Component & GetComponent(int32 c) const
Component accessor,.
Definition: nnet-nnet.cc:153

◆ SetDropoutRate()

void SetDropoutRate ( BaseFloat  r)

Set the dropout rate.

Definition at line 268 of file nnet-nnet.cc.

References Nnet::GetComponent(), Dropout::GetDropoutRate(), Component::GetType(), KALDI_LOG, Component::kDropout, Nnet::NumComponents(), and Dropout::SetDropoutRate().

Referenced by Nnet::BackpropagateBuffer(), and main().

268  {
269  for (int32 c = 0; c < NumComponents(); c++) {
271  Dropout& comp = dynamic_cast<Dropout&>(GetComponent(c));
272  BaseFloat r_old = comp.GetDropoutRate();
273  comp.SetDropoutRate(r);
274  KALDI_LOG << "Setting dropout-rate in component " << c
275  << " from " << r_old << " to " << r;
276  }
277  }
278 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
float BaseFloat
Definition: kaldi-types.h:29
const Component & GetComponent(int32 c) const
Component accessor,.
Definition: nnet-nnet.cc:153
virtual ComponentType GetType() const =0
Get Type Identification of the component,.
#define KALDI_LOG
Definition: kaldi-error.h:153

◆ SetParams()

void SetParams ( const VectorBase< BaseFloat > &  params)

Set the network weights from a supervector,.

Definition at line 253 of file nnet-nnet.cc.

References Nnet::components_, VectorBase< Real >::Dim(), rnnlm::i, KALDI_ASSERT, Nnet::NumParams(), UpdatableComponent::NumParams(), VectorBase< Real >::Range(), and UpdatableComponent::SetParams().

Referenced by Nnet::BackpropagateBuffer(), and MultiBasisComponent::SetParams().

253  {
254  KALDI_ASSERT(params.Dim() == NumParams());
255  int32 pos = 0;
256  // loop over Components,
257  for (int32 i = 0; i < components_.size(); i++) {
258  if (components_[i]->IsUpdatable()) {
259  UpdatableComponent& c =
260  dynamic_cast<UpdatableComponent&>(*components_[i]);
261  c.SetParams(params.Range(pos, c.NumParams())); // setting params,
262  pos += c.NumParams();
263  }
264  }
265  KALDI_ASSERT(pos == NumParams());
266 }
int32 NumParams() const
Get the number of parameters in the network,.
Definition: nnet-nnet.cc:210
kaldi::int32 int32
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ SetSeqLengths()

void SetSeqLengths ( const std::vector< int32 > &  sequence_lengths)

Set sequence length in LSTM multi-stream training,.

Definition at line 291 of file nnet-nnet.cc.

References Nnet::GetComponent(), Component::IsMultistream(), Nnet::NumComponents(), and MultistreamComponent::SetSeqLengths().

Referenced by Nnet::BackpropagateBuffer(), and main().

291  {
292  for (int32 c = 0; c < NumComponents(); c++) {
293  if (GetComponent(c).IsMultistream()) {
294  MultistreamComponent& comp =
295  dynamic_cast<MultistreamComponent&>(GetComponent(c));
296  comp.SetSeqLengths(sequence_lengths);
297  }
298  }
299 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
virtual bool IsMultistream() const
Check if component has &#39;Recurrent&#39; interface (trainable and recurrent),.
const Component & GetComponent(int32 c) const
Component accessor,.
Definition: nnet-nnet.cc:153

◆ SetTrainOptions()

void SetTrainOptions ( const NnetTrainOptions opts)

Set hyper-parameters of the training (pushes to all UpdatableComponents),.

Definition at line 508 of file nnet-nnet.cc.

References Nnet::GetComponent(), Component::IsUpdatable(), Nnet::NumComponents(), and Nnet::opts_.

Referenced by Nnet::BackpropagateBuffer(), main(), Nnet::Nnet(), Nnet::operator=(), and MultiBasisComponent::SetTrainOptions().

508  {
509  opts_ = opts;
510  // set values to individual components,
511  for (int32 l = 0; l < NumComponents(); l++) {
512  if (GetComponent(l).IsUpdatable()) {
513  dynamic_cast<UpdatableComponent&>(GetComponent(l)).SetTrainOptions(opts_);
514  }
515  }
516 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
virtual bool IsUpdatable() const
Check if componeny has &#39;Updatable&#39; interface (trainable components),.
const Component & GetComponent(int32 c) const
Component accessor,.
Definition: nnet-nnet.cc:153
void SetTrainOptions(const NnetTrainOptions &opts)
Set hyper-parameters of the training (pushes to all UpdatableComponents),.
Definition: nnet-nnet.cc:508
NnetTrainOptions opts_
Option class with hyper-parameters passed to UpdatableComponent(s)
Definition: nnet-nnet.h:178

◆ SwapComponent()

void SwapComponent ( int32  c,
Component **  comp 
)

Swap c'th component with the pointer,.

Definition at line 175 of file nnet-nnet.cc.

References Nnet::Check(), and Nnet::components_.

Referenced by Nnet::NumComponents().

175  {
176  Component* tmp = components_.at(c);
177  components_.at(c) = *comp;
178  (*comp) = tmp;
179  Check();
180 }
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

◆ Write() [1/2]

void Write ( const std::string &  wxfilename,
bool  binary 
) const

Write Nnet to 'wxfilename',.

I/O wrapper for converting 'wxfilename' to 'ostream',.

Definition at line 367 of file nnet-nnet.cc.

References Output::Close(), and Output::Stream().

Referenced by Nnet::BackpropagateBuffer(), main(), and MultiBasisComponent::WriteData().

367  {
368  Output out(wxfilename, binary, true);
369  Write(out.Stream(), binary);
370  out.Close();
371 }
void Write(const std::string &wxfilename, bool binary) const
Write Nnet to &#39;wxfilename&#39;,.
Definition: nnet-nnet.cc:367

◆ Write() [2/2]

void Write ( std::ostream &  out,
bool  binary 
) const

Write Nnet to 'ostream',.

Definition at line 374 of file nnet-nnet.cc.

References Nnet::Check(), Nnet::components_, rnnlm::i, Nnet::NumComponents(), and kaldi::WriteToken().

374  {
375  Check();
376  WriteToken(os, binary, "<Nnet>");
377  if (binary == false) os << std::endl;
378  for (int32 i = 0; i < NumComponents(); i++) {
379  components_[i]->Write(os, binary);
380  }
381  WriteToken(os, binary, "</Nnet>");
382  if (binary == false) os << std::endl;
383 }
int32 NumComponents() const
Returns the number of &#39;Components&#39; which form the NN.
Definition: nnet-nnet.h:66
kaldi::int32 int32
void Check() const
Consistency check,.
Definition: nnet-nnet.cc:467
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
std::vector< Component * > components_
Vector which contains all the components composing the neural network, the components are for example...
Definition: nnet-nnet.h:170

Member Data Documentation

◆ backpropagate_buf_

std::vector<CuMatrix<BaseFloat> > backpropagate_buf_
private

Buffers for backward pass (on demand initialization),.

Definition at line 175 of file nnet-nnet.h.

Referenced by Nnet::Backpropagate(), Nnet::BackpropagateBuffer(), Nnet::Destroy(), Nnet::InfoBackPropagate(), Nnet::Nnet(), and Nnet::operator=().

◆ components_

◆ opts_

NnetTrainOptions opts_
private

Option class with hyper-parameters passed to UpdatableComponent(s)

Definition at line 178 of file nnet-nnet.h.

Referenced by Nnet::GetTrainOptions(), Nnet::Nnet(), Nnet::operator=(), and Nnet::SetTrainOptions().

◆ propagate_buf_

std::vector<CuMatrix<BaseFloat> > propagate_buf_
private

Buffers for forward pass (on demand initialization),.

Definition at line 173 of file nnet-nnet.h.

Referenced by Nnet::Backpropagate(), Nnet::Destroy(), Nnet::InfoPropagate(), Nnet::Nnet(), Nnet::operator=(), Nnet::Propagate(), and Nnet::PropagateBuffer().


The documentation for this class was generated from the following files: