ModelUpdateConsolidator Class Reference

This class is responsible for consolidating the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices. More...

Collaboration diagram for ModelUpdateConsolidator:

Public Member Functions
	ModelUpdateConsolidator (const Nnet &nnet, NnetComputation *computation)
void	ConsolidateModelUpdate ()

Private Member Functions
void	ConsolidateUpdateForComponent (int32 component, const std::vector< int32 > &backprop_commands)
	This function, called from ConsolidateModelUpdate, is passed a list of commands that are all backprops for the same component, and it consolidates them into a single model-update command. More...
void	AddCommandsToComputation ()
	This function, called at the end of ConsolidateModelUpdate(), takes the commands that we have put in extra_commands_, final_commands_ and final_deallocate_commands_, and puts them in the appropriate place in computation->commands_. More...
int32	ConsolidateSubmatrices (const std::vector< int32 > &commands, const std::vector< int32 > &submatrices)
	You call this function when you want to consolidate the values of a list of submatrices taken just prior to particular commands. More...
void	AppendDebugInfoForSubmatrix (int32 submatrix_index, NnetComputation::MatrixDebugInfo *debug_info) const
	This function, called from ConsolidateSubmatrices, will update 'debug_info' by appending the corresponding 'indexes' from the existing debug info for this submatrix. More...

Private Attributes
const Nnet &	nnet_
NnetComputation *	computation_
std::vector< std::vector< NnetComputation::Command > >	extra_commands_
std::vector< NnetComputation::Command >	final_commands_
std::vector< NnetComputation::Command >	final_deallocate_commands_

Detailed Description

This class is responsible for consolidating the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices.

This is useful for recurrent networks.

Definition at line 1282 of file nnet-optimize-utils.cc.

Constructor & Destructor Documentation

ModelUpdateConsolidator()

ModelUpdateConsolidator	(	const Nnet &	nnet,
		NnetComputation *	computation
	)

Definition at line 1509 of file nnet-optimize-utils.cc.

                                 :
    nnet_(nnet), computation_(computation),
    extra_commands_(computation->commands.size()) { }

Member Function Documentation

AddCommandsToComputation()

void AddCommandsToComputation ( )

private

This function, called at the end of ConsolidateModelUpdate(), takes the commands that we have put in extra_commands_, final_commands_ and final_deallocate_commands_, and puts them in the appropriate place in computation->commands_.

Definition at line 1434 of file nnet-optimize-utils.cc.

References NnetComputation::commands, MatrixExtender::computation_, rnnlm::i, and KALDI_ASSERT.

Referenced by ModelUpdateConsolidator::ConsolidateModelUpdate().

                                                       {
  KALDI_ASSERT(computation_->commands.size() == extra_commands_.size());
  int32 old_num_commands = computation_->commands.size(),
      new_num_commands = old_num_commands +
      static_cast<int32>(final_commands_.size() +
                         final_deallocate_commands_.size());
  for (size_t i = 0; i < extra_commands_.size(); i++)
    new_num_commands += static_cast<int32>(extra_commands_[i].size());
  std::vector<NnetComputation::Command> new_commands;
  new_commands.reserve(new_num_commands);
  for (int32 c = 0; c < old_num_commands; c++) {
    new_commands.insert(new_commands.end(),
                        extra_commands_[c].begin(), extra_commands_[c].end());
    new_commands.push_back(computation_->commands[c]);
  }
  new_commands.insert(new_commands.end(),
                      final_commands_.begin(), final_commands_.end());
  new_commands.insert(new_commands.end(),
                      final_deallocate_commands_.begin(),
                      final_deallocate_commands_.end());
  computation_->commands.swap(new_commands);
}

AppendDebugInfoForSubmatrix()

void AppendDebugInfoForSubmatrix ( int32  submatrix_index, NnetComputation::MatrixDebugInfo *  debug_info  ) const

private

This function, called from ConsolidateSubmatrices, will update 'debug_info' by appending the corresponding 'indexes' from the existing debug info for this submatrix.

It will also set the 'is_deriv' of '*debug_info' to the same value as the debug info for 'submatrix_index', and set the 'node_index' to the 'node_index' in the debug info for that submatrix-index. It requires that computation_->matrix_debug_info be nonempty.

Definition at line 1347 of file nnet-optimize-utils.cc.

References NnetComputation::MatrixDebugInfo::cindexes, MatrixExtender::computation_, NnetComputation::MatrixDebugInfo::is_deriv, KALDI_ASSERT, NnetComputation::matrices, NnetComputation::matrix_debug_info, and NnetComputation::submatrices.

                                                      {
  KALDI_ASSERT(!computation_->matrix_debug_info.empty());
  KALDI_ASSERT(static_cast<size_t>(submatrix_index) <
               computation_->submatrices.size());
  NnetComputation::SubMatrixInfo submatrix_info =
      computation_->submatrices[submatrix_index];
  int32 matrix_index = submatrix_info.matrix_index;
  KALDI_ASSERT(matrix_index > 0 && static_cast<size_t>(matrix_index) <
               computation_->matrix_debug_info.size());
  const NnetComputation::MatrixDebugInfo &src_info =
      computation_->matrix_debug_info[matrix_index];
  debug_info->is_deriv = src_info.is_deriv;
  KALDI_ASSERT(src_info.cindexes.size() ==
               computation_->matrices[matrix_index].num_rows);
  int32 row_begin = submatrix_info.row_offset,
      row_end = row_begin + submatrix_info.num_rows;
  debug_info->cindexes.insert(debug_info->cindexes.end(),
                             src_info.cindexes.begin() + row_begin,
                             src_info.cindexes.begin() + row_end);
}

ConsolidateModelUpdate()

void ConsolidateModelUpdate

(

)

Definition at line 1515 of file nnet-optimize-utils.cc.

References ModelUpdateConsolidator::AddCommandsToComputation(), NnetComputation::Command::arg1, NnetComputation::Command::command_type, NnetComputation::commands, ModelUpdateConsolidator::computation_, ModelUpdateConsolidator::ConsolidateUpdateForComponent(), Nnet::GetComponent(), kaldi::nnet3::kBackprop, kaldi::nnet3::kSimpleComponent, kaldi::nnet3::kUpdatableComponent, kaldi::nnet3::kUsesMemo, ModelUpdateConsolidator::nnet_, Nnet::NumComponents(), and Component::Properties().

Referenced by kaldi::nnet3::ConsolidateModelUpdate().

                                                     {
  int32 num_components = nnet_.NumComponents(),
      num_commands = computation_->commands.size();
  // 'backprop_commands' is a list, for each component (but nonempty only for
  // updatable simple components), of the command indexes for the backprop
  // commands.
  std::vector<std::vector<int32> > backprop_commands(num_components);
  for (int32 command_index = 0;
       command_index < num_commands; command_index++) {
    const NnetComputation::Command &c = computation_->commands[command_index];
    if (c.command_type == kBackprop) {
      int32 component_index = c.arg1;
      const Component *component = nnet_.GetComponent(component_index);
      int32 properties = component->Properties();
      if ((properties & kUpdatableComponent) &&
          (properties & kSimpleComponent) &&
          !(properties & kUsesMemo))
        backprop_commands[component_index].push_back(command_index);
    }
  }
  bool consolidated = false;
  for (int32 component = 0; component < num_components; component++) {
    if (backprop_commands[component].size() > 1) {
      ConsolidateUpdateForComponent(component,
                                    backprop_commands[component]);
      consolidated = true;
    }
  }
  if (!consolidated)  // This is an optimization to avoid redundant computation
    return;           // if there is nothing to do.
  // the following function call commits all the commands we stored in member
  // variables, to computation_->commands.
  AddCommandsToComputation();
}

ConsolidateSubmatrices()

int32 ConsolidateSubmatrices ( const std::vector< int32 > &  commands, const std::vector< int32 > &  submatrices  )

private

You call this function when you want to consolidate the values of a list of submatrices taken just prior to particular commands.

The input 'commands' and 'submatrices' lists must be the same size, and size must be > 1. This function will create a new matrix that is the row-wise concatentation of all these submatrices, with values taken just prior to the respective command indexes. This function will will add to extra_commands_ the commands to do the copying at the appropriate places (at the supplied command indexes; they will be inserted just before). The return value is the submatrix index of a submatrix that represents the whole of the consolidated matrix. This command will insert, at the beginning of the computation (in extra_commands_[0]), a command to initialize the matrix; and will append to final_deallocate_commands_ the commands to deallocate the matrix. If computation_->matrix_debug_info is nonempty, this function will also update computation_->matrix_debug_info with suitable values for the newly added matrix

Definition at line 1371 of file nnet-optimize-utils.cc.

References MatrixExtender::computation_, rnnlm::i, NnetComputation::IsWholeMatrix(), KALDI_ASSERT, kaldi::nnet3::kAllocMatrix, kaldi::nnet3::kDeallocMatrix, kaldi::kDefaultStride, kaldi::nnet3::kMatrixCopy, kaldi::nnet3::kSetConst, kaldi::kStrideEqualNumCols, NnetComputation::matrices, NnetComputation::matrix_debug_info, NnetComputation::NewMatrix(), NnetComputation::NewSubMatrix(), and NnetComputation::submatrices.

                                         {
  int32 num_submatrices = submatrices.size();
  KALDI_ASSERT(num_submatrices > 1 && commands.size() == submatrices.size());
  int32 first_submatrix = submatrices[0];
  int32 num_cols = computation_->submatrices[first_submatrix].num_cols,
      num_rows = 0;
  MatrixStrideType stride_type = kDefaultStride;
  NnetComputation::MatrixDebugInfo debug_info;
  for (int32 i = 0; i < num_submatrices; i++) {
    int32 submatrix = submatrices[i];
    num_rows += computation_->submatrices[submatrix].num_rows;
    KALDI_ASSERT(computation_->submatrices[submatrix].num_cols == num_cols);
    if (!computation_->matrix_debug_info.empty())
      AppendDebugInfoForSubmatrix(submatrix, &debug_info);
    if (computation_->IsWholeMatrix(submatrix)) {
      int32 matrix = computation_->submatrices[submatrix].matrix_index;
      if (computation_->matrices[matrix].stride_type == kStrideEqualNumCols)
        stride_type = kStrideEqualNumCols;
    }
  }
  // new_whole_submatrix is a new submatrix index corresponding to the whole
  // of a new matrix that we are creating.
  int32 new_whole_submatrix = computation_->NewMatrix(num_rows, num_cols,
                                                      stride_type);
  // Add commands at the very start, to initialize and then zero this new
  // matrix.  we can later on remove the zeroing if it is not necessary.
  extra_commands_[0].push_back(
      NnetComputation::Command(kAllocMatrix, new_whole_submatrix));
  extra_commands_[0].push_back(
      NnetComputation::Command(0.0, kSetConst, new_whole_submatrix));

  final_deallocate_commands_.push_back(
      NnetComputation::Command(kDeallocMatrix, new_whole_submatrix));
  int32 new_matrix_index =
      computation_->submatrices[new_whole_submatrix].matrix_index;
  if (!computation_->matrix_debug_info.empty())
    computation_->matrix_debug_info[new_matrix_index].Swap(&debug_info);

  int32 row_offset = 0;
  for (int32 i = 0; i < num_submatrices; i++) {
    int32 submatrix_index = submatrices[i];
    int32 this_num_rows = computation_->submatrices[submatrix_index].num_rows;
    // submatrix corresponding to the part of the new matrix corresponding
    // to 'submatrices[i]'.
    int32 new_submatrix = computation_->NewSubMatrix(new_whole_submatrix,
                                                     row_offset, this_num_rows,
                                                     0, num_cols);
    // Just before command 'commands[i]', add a command that assigns to the
    // submatrix numbered 'new_submatrix' the contents of the submatrix numbered
    // 'submatrices[i]'.  Note: we hope that a later pass of optimization
    // (VariableMergingOptimization) will remove this redundant copy by
    // having the operation that created it write directly to the location
    // we want it to be.
    NnetComputation::Command c(kMatrixCopy, new_submatrix, submatrices[i]);
    extra_commands_[commands[i]].push_back(c);
    row_offset += this_num_rows;
  }
  KALDI_ASSERT(row_offset == num_rows);
  return new_whole_submatrix;
}

ConsolidateUpdateForComponent()

void ConsolidateUpdateForComponent ( int32  component_index, const std::vector< int32 > &  backprop_commands  )

private

This function, called from ConsolidateModelUpdate, is passed a list of commands that are all backprops for the same component, and it consolidates them into a single model-update command.

Definition at line 1460 of file nnet-optimize-utils.cc.

References NnetComputation::Command::arg2, NnetComputation::Command::arg3, NnetComputation::Command::arg4, NnetComputation::Command::arg5, NnetComputation::Command::command_type, NnetComputation::commands, MatrixExtender::computation_, rnnlm::i, KALDI_ASSERT, kaldi::nnet3::kBackprop, kaldi::nnet3::kBackpropNeedsInput, kaldi::nnet3::kBackpropNeedsOutput, kaldi::nnet3::kBackpropNoModelUpdate, and Component::Properties().

Referenced by ModelUpdateConsolidator::ConsolidateModelUpdate().

                                               {
  const Component *component = nnet_.GetComponent(component_index);
  int32 num_backprop_commands = backprop_commands.size();

  bool need_input = (component->Properties() & kBackpropNeedsInput) != 0,
      need_output = (component->Properties() & kBackpropNeedsOutput) != 0;

  std::vector<int32>  input_submatrices(num_backprop_commands),
      output_submatrices(num_backprop_commands),
      output_deriv_submatrices(num_backprop_commands);

  for (int32 i = 0; i < num_backprop_commands; i++) {
    int32 command_index = backprop_commands[i];
    NnetComputation::Command &command =
        computation_->commands[command_index];
    // arg2 must be 0 because simple components don't use precomputed indexes.
    KALDI_ASSERT(command.command_type == kBackprop && command.arg2 == 0);
    command.command_type = kBackpropNoModelUpdate;
    int32 input_submatrix = command.arg3,
        output_submatrix = command.arg4,
        output_deriv_submatrix = command.arg5;
    KALDI_ASSERT((input_submatrix != 0) == need_input &&
                 (output_submatrix != 0) == need_output);
    input_submatrices[i] = input_submatrix;
    output_submatrices[i] = output_submatrix;
    output_deriv_submatrices[i] = output_deriv_submatrix;
  }
  // Get the sub-matrix indexes of whichever of the consolidated matrices we
  // need (will usually be input_submatrix and output_deriv_submatrix).
  int32 input_submatrix = (need_input ?
                           ConsolidateSubmatrices(backprop_commands,
                                                  input_submatrices) : 0),
      output_submatrix = (need_output ?
                         ConsolidateSubmatrices(backprop_commands,
                                                output_submatrices) : 0),
      output_deriv_submatrix = ConsolidateSubmatrices(backprop_commands,
                                                      output_deriv_submatrices);
  int32 precomputed_indexes_index = 0,  // unused since simple component
      input_deriv_submatrix = 0,  // we don't need the input-deriv.
      memo_index = 0;  // we checked that no memos were used.
  NnetComputation::Command c(kBackprop, component_index, precomputed_indexes_index,
                             input_submatrix, output_submatrix,
                             output_deriv_submatrix, input_deriv_submatrix,
                             memo_index);
  final_commands_.push_back(c);
}

Member Data Documentation

computation_

NnetComputation* computation_

private

Definition at line 1329 of file nnet-optimize-utils.cc.

Referenced by ModelUpdateConsolidator::ConsolidateModelUpdate(), DerivativeTimeLimiter::GetPruneValues(), DerivativeTimeLimiter::MapAddRowRangesCommand(), DerivativeTimeLimiter::MapIndexesCommand(), DerivativeTimeLimiter::MapIndexesMultiCommand(), DerivativeTimeLimiter::MapSimpleMatrixCommand(), and DerivativeTimeLimiter::RowIsKept().

extra_commands_

std::vector<std::vector<NnetComputation::Command> > extra_commands_

private

Definition at line 1335 of file nnet-optimize-utils.cc.

final_commands_

std::vector<NnetComputation::Command> final_commands_

private

Definition at line 1340 of file nnet-optimize-utils.cc.

final_deallocate_commands_

std::vector<NnetComputation::Command> final_deallocate_commands_

private

Definition at line 1343 of file nnet-optimize-utils.cc.

nnet_

const Nnet& nnet_

private

Definition at line 1328 of file nnet-optimize-utils.cc.

Referenced by ModelUpdateConsolidator::ConsolidateModelUpdate(), and DerivativeTimeLimiter::ModifyCommand().

---

The documentation for this class was generated from the following file:

• nnet3/nnet-optimize-utils.cc