ModelUpdateConsolidator Class Reference

This class is responsible for consolidating the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices. More...

Collaboration diagram for ModelUpdateConsolidator:

Public Member Functions

 ModelUpdateConsolidator (const Nnet &nnet, NnetComputation *computation)
 
void ConsolidateModelUpdate ()
 

Private Member Functions

void ConsolidateUpdateForComponent (int32 component, const std::vector< int32 > &backprop_commands)
 This function, called from ConsolidateModelUpdate, is passed a list of commands that are all backprops for the same component, and it consolidates them into a single model-update command. More...
 
void AddCommandsToComputation ()
 This function, called at the end of ConsolidateModelUpdate(), takes the commands that we have put in extra_commands_, final_commands_ and final_deallocate_commands_, and puts them in the appropriate place in computation->commands_. More...
 
int32 ConsolidateSubmatrices (const std::vector< int32 > &commands, const std::vector< int32 > &submatrices)
 You call this function when you want to consolidate the values of a list of submatrices taken just prior to particular commands. More...
 
void AppendDebugInfoForSubmatrix (int32 submatrix_index, NnetComputation::MatrixDebugInfo *debug_info) const
 This function, called from ConsolidateSubmatrices, will update 'debug_info' by appending the corresponding 'indexes' from the existing debug info for this submatrix. More...
 

Private Attributes

const Nnetnnet_
 
NnetComputationcomputation_
 
std::vector< std::vector< NnetComputation::Command > > extra_commands_
 
std::vector< NnetComputation::Commandfinal_commands_
 
std::vector< NnetComputation::Commandfinal_deallocate_commands_
 

Detailed Description

This class is responsible for consolidating the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices.

This is useful for recurrent networks.

Definition at line 1282 of file nnet-optimize-utils.cc.

Constructor & Destructor Documentation

◆ ModelUpdateConsolidator()

ModelUpdateConsolidator ( const Nnet nnet,
NnetComputation computation 
)

Definition at line 1509 of file nnet-optimize-utils.cc.

1511  :
1512  nnet_(nnet), computation_(computation),
1513  extra_commands_(computation->commands.size()) { }
std::vector< std::vector< NnetComputation::Command > > extra_commands_

Member Function Documentation

◆ AddCommandsToComputation()

void AddCommandsToComputation ( )
private

This function, called at the end of ConsolidateModelUpdate(), takes the commands that we have put in extra_commands_, final_commands_ and final_deallocate_commands_, and puts them in the appropriate place in computation->commands_.

Definition at line 1434 of file nnet-optimize-utils.cc.

References NnetComputation::commands, MatrixExtender::computation_, rnnlm::i, and KALDI_ASSERT.

Referenced by ModelUpdateConsolidator::ConsolidateModelUpdate().

1434  {
1436  int32 old_num_commands = computation_->commands.size(),
1437  new_num_commands = old_num_commands +
1438  static_cast<int32>(final_commands_.size() +
1440  for (size_t i = 0; i < extra_commands_.size(); i++)
1441  new_num_commands += static_cast<int32>(extra_commands_[i].size());
1442  std::vector<NnetComputation::Command> new_commands;
1443  new_commands.reserve(new_num_commands);
1444  for (int32 c = 0; c < old_num_commands; c++) {
1445  new_commands.insert(new_commands.end(),
1446  extra_commands_[c].begin(), extra_commands_[c].end());
1447  new_commands.push_back(computation_->commands[c]);
1448  }
1449  new_commands.insert(new_commands.end(),
1450  final_commands_.begin(), final_commands_.end());
1451  new_commands.insert(new_commands.end(),
1454  computation_->commands.swap(new_commands);
1455 }
std::vector< NnetComputation::Command > final_commands_
std::vector< std::vector< NnetComputation::Command > > extra_commands_
kaldi::int32 int32
std::vector< Command > commands
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185
std::vector< NnetComputation::Command > final_deallocate_commands_

◆ AppendDebugInfoForSubmatrix()

void AppendDebugInfoForSubmatrix ( int32  submatrix_index,
NnetComputation::MatrixDebugInfo debug_info 
) const
private

This function, called from ConsolidateSubmatrices, will update 'debug_info' by appending the corresponding 'indexes' from the existing debug info for this submatrix.

It will also set the 'is_deriv' of '*debug_info' to the same value as the debug info for 'submatrix_index', and set the 'node_index' to the 'node_index' in the debug info for that submatrix-index. It requires that computation_->matrix_debug_info be nonempty.

Definition at line 1347 of file nnet-optimize-utils.cc.

References NnetComputation::MatrixDebugInfo::cindexes, MatrixExtender::computation_, NnetComputation::MatrixDebugInfo::is_deriv, KALDI_ASSERT, NnetComputation::matrices, NnetComputation::matrix_debug_info, and NnetComputation::submatrices.

1349  {
1351  KALDI_ASSERT(static_cast<size_t>(submatrix_index) <
1352  computation_->submatrices.size());
1353  NnetComputation::SubMatrixInfo submatrix_info =
1354  computation_->submatrices[submatrix_index];
1355  int32 matrix_index = submatrix_info.matrix_index;
1356  KALDI_ASSERT(matrix_index > 0 && static_cast<size_t>(matrix_index) <
1358  const NnetComputation::MatrixDebugInfo &src_info =
1359  computation_->matrix_debug_info[matrix_index];
1360  debug_info->is_deriv = src_info.is_deriv;
1361  KALDI_ASSERT(src_info.cindexes.size() ==
1362  computation_->matrices[matrix_index].num_rows);
1363  int32 row_begin = submatrix_info.row_offset,
1364  row_end = row_begin + submatrix_info.num_rows;
1365  debug_info->cindexes.insert(debug_info->cindexes.end(),
1366  src_info.cindexes.begin() + row_begin,
1367  src_info.cindexes.begin() + row_end);
1368 }
std::vector< MatrixDebugInfo > matrix_debug_info
kaldi::int32 int32
std::vector< MatrixInfo > matrices
std::vector< SubMatrixInfo > submatrices
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ ConsolidateModelUpdate()

void ConsolidateModelUpdate ( )

Definition at line 1515 of file nnet-optimize-utils.cc.

References ModelUpdateConsolidator::AddCommandsToComputation(), NnetComputation::Command::arg1, NnetComputation::Command::command_type, NnetComputation::commands, ModelUpdateConsolidator::computation_, ModelUpdateConsolidator::ConsolidateUpdateForComponent(), Nnet::GetComponent(), kaldi::nnet3::kBackprop, kaldi::nnet3::kSimpleComponent, kaldi::nnet3::kUpdatableComponent, kaldi::nnet3::kUsesMemo, ModelUpdateConsolidator::nnet_, Nnet::NumComponents(), and Component::Properties().

Referenced by kaldi::nnet3::ConsolidateModelUpdate().

1515  {
1516  int32 num_components = nnet_.NumComponents(),
1517  num_commands = computation_->commands.size();
1518  // 'backprop_commands' is a list, for each component (but nonempty only for
1519  // updatable simple components), of the command indexes for the backprop
1520  // commands.
1521  std::vector<std::vector<int32> > backprop_commands(num_components);
1522  for (int32 command_index = 0;
1523  command_index < num_commands; command_index++) {
1524  const NnetComputation::Command &c = computation_->commands[command_index];
1525  if (c.command_type == kBackprop) {
1526  int32 component_index = c.arg1;
1527  const Component *component = nnet_.GetComponent(component_index);
1528  int32 properties = component->Properties();
1529  if ((properties & kUpdatableComponent) &&
1530  (properties & kSimpleComponent) &&
1531  !(properties & kUsesMemo))
1532  backprop_commands[component_index].push_back(command_index);
1533  }
1534  }
1535  bool consolidated = false;
1536  for (int32 component = 0; component < num_components; component++) {
1537  if (backprop_commands[component].size() > 1) {
1539  backprop_commands[component]);
1540  consolidated = true;
1541  }
1542  }
1543  if (!consolidated) // This is an optimization to avoid redundant computation
1544  return; // if there is nothing to do.
1545  // the following function call commits all the commands we stored in member
1546  // variables, to computation_->commands.
1548 }
kaldi::int32 int32
std::vector< Command > commands
void AddCommandsToComputation()
This function, called at the end of ConsolidateModelUpdate(), takes the commands that we have put in ...
void ConsolidateUpdateForComponent(int32 component, const std::vector< int32 > &backprop_commands)
This function, called from ConsolidateModelUpdate, is passed a list of commands that are all backprop...
Component * GetComponent(int32 c)
Return component indexed c. Not a copy; not owned by caller.
Definition: nnet-nnet.cc:150
int32 NumComponents() const
Definition: nnet-nnet.h:124

◆ ConsolidateSubmatrices()

int32 ConsolidateSubmatrices ( const std::vector< int32 > &  commands,
const std::vector< int32 > &  submatrices 
)
private

You call this function when you want to consolidate the values of a list of submatrices taken just prior to particular commands.

The input 'commands' and 'submatrices' lists must be the same size, and size must be > 1. This function will create a new matrix that is the row-wise concatentation of all these submatrices, with values taken just prior to the respective command indexes. This function will will add to extra_commands_ the commands to do the copying at the appropriate places (at the supplied command indexes; they will be inserted just before). The return value is the submatrix index of a submatrix that represents the whole of the consolidated matrix. This command will insert, at the beginning of the computation (in extra_commands_[0]), a command to initialize the matrix; and will append to final_deallocate_commands_ the commands to deallocate the matrix. If computation_->matrix_debug_info is nonempty, this function will also update computation_->matrix_debug_info with suitable values for the newly added matrix

Definition at line 1371 of file nnet-optimize-utils.cc.

References MatrixExtender::computation_, rnnlm::i, NnetComputation::IsWholeMatrix(), KALDI_ASSERT, kaldi::nnet3::kAllocMatrix, kaldi::nnet3::kDeallocMatrix, kaldi::kDefaultStride, kaldi::nnet3::kMatrixCopy, kaldi::nnet3::kSetConst, kaldi::kStrideEqualNumCols, NnetComputation::matrices, NnetComputation::matrix_debug_info, NnetComputation::NewMatrix(), NnetComputation::NewSubMatrix(), and NnetComputation::submatrices.

1373  {
1374  int32 num_submatrices = submatrices.size();
1375  KALDI_ASSERT(num_submatrices > 1 && commands.size() == submatrices.size());
1376  int32 first_submatrix = submatrices[0];
1377  int32 num_cols = computation_->submatrices[first_submatrix].num_cols,
1378  num_rows = 0;
1379  MatrixStrideType stride_type = kDefaultStride;
1380  NnetComputation::MatrixDebugInfo debug_info;
1381  for (int32 i = 0; i < num_submatrices; i++) {
1382  int32 submatrix = submatrices[i];
1383  num_rows += computation_->submatrices[submatrix].num_rows;
1384  KALDI_ASSERT(computation_->submatrices[submatrix].num_cols == num_cols);
1385  if (!computation_->matrix_debug_info.empty())
1386  AppendDebugInfoForSubmatrix(submatrix, &debug_info);
1387  if (computation_->IsWholeMatrix(submatrix)) {
1388  int32 matrix = computation_->submatrices[submatrix].matrix_index;
1389  if (computation_->matrices[matrix].stride_type == kStrideEqualNumCols)
1390  stride_type = kStrideEqualNumCols;
1391  }
1392  }
1393  // new_whole_submatrix is a new submatrix index corresponding to the whole
1394  // of a new matrix that we are creating.
1395  int32 new_whole_submatrix = computation_->NewMatrix(num_rows, num_cols,
1396  stride_type);
1397  // Add commands at the very start, to initialize and then zero this new
1398  // matrix. we can later on remove the zeroing if it is not necessary.
1399  extra_commands_[0].push_back(
1400  NnetComputation::Command(kAllocMatrix, new_whole_submatrix));
1401  extra_commands_[0].push_back(
1402  NnetComputation::Command(0.0, kSetConst, new_whole_submatrix));
1403 
1404  final_deallocate_commands_.push_back(
1405  NnetComputation::Command(kDeallocMatrix, new_whole_submatrix));
1406  int32 new_matrix_index =
1407  computation_->submatrices[new_whole_submatrix].matrix_index;
1408  if (!computation_->matrix_debug_info.empty())
1409  computation_->matrix_debug_info[new_matrix_index].Swap(&debug_info);
1410 
1411  int32 row_offset = 0;
1412  for (int32 i = 0; i < num_submatrices; i++) {
1413  int32 submatrix_index = submatrices[i];
1414  int32 this_num_rows = computation_->submatrices[submatrix_index].num_rows;
1415  // submatrix corresponding to the part of the new matrix corresponding
1416  // to 'submatrices[i]'.
1417  int32 new_submatrix = computation_->NewSubMatrix(new_whole_submatrix,
1418  row_offset, this_num_rows,
1419  0, num_cols);
1420  // Just before command 'commands[i]', add a command that assigns to the
1421  // submatrix numbered 'new_submatrix' the contents of the submatrix numbered
1422  // 'submatrices[i]'. Note: we hope that a later pass of optimization
1423  // (VariableMergingOptimization) will remove this redundant copy by
1424  // having the operation that created it write directly to the location
1425  // we want it to be.
1426  NnetComputation::Command c(kMatrixCopy, new_submatrix, submatrices[i]);
1427  extra_commands_[commands[i]].push_back(c);
1428  row_offset += this_num_rows;
1429  }
1430  KALDI_ASSERT(row_offset == num_rows);
1431  return new_whole_submatrix;
1432 }
std::vector< MatrixDebugInfo > matrix_debug_info
std::vector< std::vector< NnetComputation::Command > > extra_commands_
int32 NewMatrix(int32 num_rows, int32 num_cols, MatrixStrideType stride_type)
Convenience function used when adding new matrices.
kaldi::int32 int32
std::vector< MatrixInfo > matrices
int32 NewSubMatrix(int32 base_submatrix, int32 row_offset, int32 num_rows, int32 col_offset, int32 num_cols)
Convenience function used when adding new sub-matrices.
std::vector< SubMatrixInfo > submatrices
MatrixStrideType
Definition: matrix-common.h:44
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185
void AppendDebugInfoForSubmatrix(int32 submatrix_index, NnetComputation::MatrixDebugInfo *debug_info) const
This function, called from ConsolidateSubmatrices, will update &#39;debug_info&#39; by appending the correspo...
std::vector< NnetComputation::Command > final_deallocate_commands_
bool IsWholeMatrix(int32 submatrix_index) const

◆ ConsolidateUpdateForComponent()

void ConsolidateUpdateForComponent ( int32  component_index,
const std::vector< int32 > &  backprop_commands 
)
private

This function, called from ConsolidateModelUpdate, is passed a list of commands that are all backprops for the same component, and it consolidates them into a single model-update command.

Definition at line 1460 of file nnet-optimize-utils.cc.

References NnetComputation::Command::arg2, NnetComputation::Command::arg3, NnetComputation::Command::arg4, NnetComputation::Command::arg5, NnetComputation::Command::command_type, NnetComputation::commands, MatrixExtender::computation_, rnnlm::i, KALDI_ASSERT, kaldi::nnet3::kBackprop, kaldi::nnet3::kBackpropNeedsInput, kaldi::nnet3::kBackpropNeedsOutput, kaldi::nnet3::kBackpropNoModelUpdate, and Component::Properties().

Referenced by ModelUpdateConsolidator::ConsolidateModelUpdate().

1462  {
1463  const Component *component = nnet_.GetComponent(component_index);
1464  int32 num_backprop_commands = backprop_commands.size();
1465 
1466  bool need_input = (component->Properties() & kBackpropNeedsInput) != 0,
1467  need_output = (component->Properties() & kBackpropNeedsOutput) != 0;
1468 
1469  std::vector<int32> input_submatrices(num_backprop_commands),
1470  output_submatrices(num_backprop_commands),
1471  output_deriv_submatrices(num_backprop_commands);
1472 
1473  for (int32 i = 0; i < num_backprop_commands; i++) {
1474  int32 command_index = backprop_commands[i];
1475  NnetComputation::Command &command =
1476  computation_->commands[command_index];
1477  // arg2 must be 0 because simple components don't use precomputed indexes.
1478  KALDI_ASSERT(command.command_type == kBackprop && command.arg2 == 0);
1479  command.command_type = kBackpropNoModelUpdate;
1480  int32 input_submatrix = command.arg3,
1481  output_submatrix = command.arg4,
1482  output_deriv_submatrix = command.arg5;
1483  KALDI_ASSERT((input_submatrix != 0) == need_input &&
1484  (output_submatrix != 0) == need_output);
1485  input_submatrices[i] = input_submatrix;
1486  output_submatrices[i] = output_submatrix;
1487  output_deriv_submatrices[i] = output_deriv_submatrix;
1488  }
1489  // Get the sub-matrix indexes of whichever of the consolidated matrices we
1490  // need (will usually be input_submatrix and output_deriv_submatrix).
1491  int32 input_submatrix = (need_input ?
1492  ConsolidateSubmatrices(backprop_commands,
1493  input_submatrices) : 0),
1494  output_submatrix = (need_output ?
1495  ConsolidateSubmatrices(backprop_commands,
1496  output_submatrices) : 0),
1497  output_deriv_submatrix = ConsolidateSubmatrices(backprop_commands,
1498  output_deriv_submatrices);
1499  int32 precomputed_indexes_index = 0, // unused since simple component
1500  input_deriv_submatrix = 0, // we don't need the input-deriv.
1501  memo_index = 0; // we checked that no memos were used.
1502  NnetComputation::Command c(kBackprop, component_index, precomputed_indexes_index,
1503  input_submatrix, output_submatrix,
1504  output_deriv_submatrix, input_deriv_submatrix,
1505  memo_index);
1506  final_commands_.push_back(c);
1507 }
std::vector< NnetComputation::Command > final_commands_
kaldi::int32 int32
std::vector< Command > commands
int32 ConsolidateSubmatrices(const std::vector< int32 > &commands, const std::vector< int32 > &submatrices)
You call this function when you want to consolidate the values of a list of submatrices taken just pr...
Component * GetComponent(int32 c)
Return component indexed c. Not a copy; not owned by caller.
Definition: nnet-nnet.cc:150
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

Member Data Documentation

◆ computation_

◆ extra_commands_

std::vector<std::vector<NnetComputation::Command> > extra_commands_
private

Definition at line 1335 of file nnet-optimize-utils.cc.

◆ final_commands_

std::vector<NnetComputation::Command> final_commands_
private

Definition at line 1340 of file nnet-optimize-utils.cc.

◆ final_deallocate_commands_

std::vector<NnetComputation::Command> final_deallocate_commands_
private

Definition at line 1343 of file nnet-optimize-utils.cc.

◆ nnet_


The documentation for this class was generated from the following file: