kaldi::nnet3 Namespace Reference

Namespaces

 attention
 
 computation_graph
 
 time_height_convolution
 

Classes

struct  Access
 
class  AffineComponent
 
class  AmNnetSimple
 
struct  Analyzer
 This struct exists to set up various pieces of analysis; it helps avoid the repetition of code where we compute all these things in sequence. More...
 
class  BackpropTruncationComponent
 
class  BackpropTruncationComponentPrecomputedIndexes
 
class  BatchNormComponent
 
class  BinarySumDescriptor
 BinarySumDescriptor can represent either A + B, or (A if defined, else B). More...
 
class  BlockAffineComponent
 This class implements an affine transform using a block diagonal matrix e.g., one whose weight matrix is all zeros except for blocks on the diagonal. More...
 
class  CachingOptimizingCompiler
 This class enables you to do the compilation and optimization in one call, and also ensures that if the ComputationRequest is identical to the previous one, the compilation process is not repeated. More...
 
struct  CachingOptimizingCompilerOptions
 
class  ChainExampleMerger
 This class is responsible for arranging examples in groups that have the same strucure (i.e. More...
 
struct  ChainObjectiveInfo
 
class  ChainTrainerMemoryHolder
 
struct  CheckComputationOptions
 
struct  ChunkInfo
 
struct  ChunkTimeInfo
 struct ChunkTimeInfo is used by class UtteranceSplitter to output information about how we split an utterance into chunks. More...
 
struct  CindexHasher
 
class  CindexSet
 
struct  CindexVectorHasher
 
class  ClipGradientComponent
 
struct  CollapseModelConfig
 Config class for the CollapseModel function. More...
 
struct  CommandAttributes
 
struct  CommandPairComparator
 
struct  ComparePair
 
class  Compiler
 This class creates an initial version of the NnetComputation, without any optimization or sharing of matrices. More...
 
struct  CompilerOptions
 
class  Component
 Abstract base-class for neural-net components. More...
 
class  ComponentPrecomputedIndexes
 
class  CompositeComponent
 CompositeComponent is a component representing a sequence of [simple] components. More...
 
class  ComputationAnalysis
 This class performs various kinds of specific analysis on top of what class Analyzer gives you immediately. More...
 
class  ComputationCache
 Class ComputationCache is used inside class CachingOptimizingCompiler to cache previously computed computations. More...
 
class  ComputationChecker
 
class  ComputationExpander
 
struct  ComputationGraph
 The first step in compilation is to turn the ComputationSpecification into a ComputationGraph, where for each Cindex we have a list of other Cindexes that it depends on. More...
 
class  ComputationGraphBuilder
 An abstract representation of a set of Cindexes. More...
 
class  ComputationLoopedOptimizer
 
class  ComputationRenumberer
 
struct  ComputationRequest
 
struct  ComputationRequestHasher
 
struct  ComputationRequestPtrEqual
 
class  ComputationStepsComputer
 This class arranges the cindex_ids of the computation into a sequence of lists called "steps", which will correspond roughly to the commands in the compiled computation. More...
 
class  ComputationVariables
 This class relates the matrices and sub-matrices in the computation to imaginary "variables", such that we can think of the operations as operating on sets of individual variables, and we can then do analysis that lets us do optimization. More...
 
class  ConfigLine
 This class is responsible for parsing input like hi-there xx=yyy a=b c empty= f-oo=Append(bar, sss) ba_z=123 bing='a b c' baz="a b c d='a b' e" and giving you access to the fields, in this case. More...
 
class  ConstantComponent
 
class  ConstantFunctionComponent
 
class  ConstantSumDescriptor
 This is an alternative base-case of SumDescriptor (an alternative to SimpleSumDescriptor) which represents a constant term, e.g. More...
 
class  ConvolutionComponent
 WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted. More...
 
class  DecodableAmNnetLoopedOnline
 
class  DecodableAmNnetSimple
 
class  DecodableAmNnetSimpleLooped
 
class  DecodableAmNnetSimpleParallel
 
class  DecodableNnetLoopedOnline
 
class  DecodableNnetLoopedOnlineBase
 
class  DecodableNnetSimple
 
class  DecodableNnetSimpleLooped
 
class  DecodableNnetSimpleLoopedInfo
 When you instantiate class DecodableNnetSimpleLooped, you should give it a const reference to this class, that has been previously initialized. More...
 
class  DerivativeTimeLimiter
 
class  Descriptor
 
class  DiscriminativeExampleMerger
 This class is responsible for arranging examples in groups that have the same strucure (i.e. More...
 
struct  DiscriminativeObjectiveFunctionInfo
 
class  DistributeComponent
 This Component takes a larger input-dim than output-dim, where the input-dim must be a multiple of the output-dim, and distributes different blocks of the input dimension to different 'x' values. More...
 
class  DistributeComponentPrecomputedIndexes
 
class  DropoutComponent
 
class  DropoutMaskComponent
 
class  ElementwiseProductComponent
 
struct  ExampleGenerationConfig
 
class  ExampleMerger
 This class is responsible for arranging examples in groups that have the same strucure (i.e. More...
 
class  ExampleMergingConfig
 
class  ExampleMergingStats
 This class is responsible for storing, and displaying in log messages, statistics about how examples of different sizes (c.f. More...
 
struct  FirstElementComparator
 
struct  FirstElementIsEqualComparator
 
class  FixedAffineComponent
 FixedAffineComponent is an affine transform that is supplied at network initialization time and is not trainable. More...
 
class  FixedBiasComponent
 FixedBiasComponent applies a fixed per-element bias; it's similar to the AddShift component in the nnet1 setup (and only needed for nnet1 model conversion. More...
 
class  FixedScaleComponent
 FixedScaleComponent applies a fixed per-element scale; it's similar to the Rescale component in the nnet1 setup (and only needed for nnet1 model conversion). More...
 
class  ForwardingDescriptor
 A ForwardingDescriptor describes how we copy data from another NetworkNode, or from multiple other NetworkNodes, possibly with a scalar weight. More...
 
struct  GeneralDescriptor
 This class is only used when parsing Descriptors. More...
 
class  GeneralDropoutComponent
 GeneralDropoutComponent implements dropout, including a continuous variant where the thing we multiply is not just zero or one, but may be a continuous value. More...
 
class  GeneralDropoutComponentPrecomputedIndexes
 
struct  ImageAugmentationConfig
 
struct  Index
 struct Index is intended to represent the various indexes by which we number the rows of the matrices that the Components process: mainly 'n', the index of the member of the minibatch, 't', used for the frame index in speech recognition, and 'x', which is a catch-all extra index which we might use in convolutional setups or for other reasons. More...
 
struct  IndexHasher
 
struct  IndexLessNxt
 
class  IndexSet
 An abstract representation of a set of Indexes. More...
 
struct  IndexVectorHasher
 
struct  IoSpecification
 
struct  IoSpecificationHasher
 
class  LinearComponent
 
class  LogSoftmaxComponent
 
class  LstmNonlinearityComponent
 
struct  MatrixAccesses
 
class  MatrixExtender
 
class  MaxpoolingComponent
 
class  MemoryCompressionOptimizer
 This class is used in the function OptimizeMemoryCompression(), once we determine that there is some potential to do memory compression for this computation. More...
 
struct  MiscComputationInfo
 
class  ModelCollapser
 
class  ModelUpdateConsolidator
 This class is responsible for consolidating the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices. More...
 
class  NaturalGradientAffineComponent
 
class  NaturalGradientPerElementScaleComponent
 NaturalGradientPerElementScaleComponent is like PerElementScaleComponent but it uses a natural gradient update for the per-element scales. More...
 
class  NaturalGradientRepeatedAffineComponent
 
struct  NetworkNode
 NetworkNode is used to represent, three types of thing: either an input of the network (which pretty much just states the dimension of the input vector); a Component (e.g. More...
 
class  Nnet
 
class  NnetChainComputeProb
 This class is for computing objective-function values in a nnet3+chain setup, for diagnostics. More...
 
struct  NnetChainExample
 NnetChainExample is like NnetExample, but specialized for lattice-free (chain) training. More...
 
struct  NnetChainExampleStructureCompare
 This comparator object compares just the structural aspects of the NnetChainExample without looking at the value of the features. More...
 
struct  NnetChainExampleStructureHasher
 This hashing object hashes just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetChainSupervision
 
class  NnetChainTrainer
 This class is for single-threaded training of neural nets using the 'chain' model. More...
 
struct  NnetChainTrainingOptions
 
struct  NnetComputation
 
struct  NnetComputeOptions
 
class  NnetComputeProb
 This class is for computing cross-entropy and accuracy values in a neural network, for diagnostics. More...
 
struct  NnetComputeProbOptions
 
class  NnetComputer
 class NnetComputer is responsible for executing the computation described in the "computation" object. More...
 
class  NnetComputerFromEg
 
class  NnetDiscriminativeComputeObjf
 This class is for computing objective-function values in a nnet3 discriminative training, for diagnostics. More...
 
struct  NnetDiscriminativeExample
 NnetDiscriminativeExample is like NnetExample, but specialized for sequence training. More...
 
struct  NnetDiscriminativeExampleStructureCompare
 This comparator object compares just the structural aspects of the NnetDiscriminativeExample without looking at the value of the features. More...
 
struct  NnetDiscriminativeExampleStructureHasher
 This hashing object hashes just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetDiscriminativeOptions
 
struct  NnetDiscriminativeSupervision
 
class  NnetDiscriminativeTrainer
 This class is for single-threaded discriminative training of neural nets. More...
 
struct  NnetExample
 NnetExample is the input data and corresponding label (or labels) for one or more frames of input, used for standard cross-entropy training of neural nets (and possibly for other objective functions). More...
 
struct  NnetExampleStructureCompare
 This comparator object compares just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetExampleStructureHasher
 This hashing object hashes just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetGenerationOptions
 
struct  NnetIo
 
struct  NnetIoStructureCompare
 This comparison object compares just the structural aspects of the NnetIo object (name, indexes, feature dimension) without looking at the value of features. More...
 
struct  NnetIoStructureHasher
 This hashing object hashes just the structural aspects of the NnetIo object (name, indexes, feature dimension) without looking at the value of features. More...
 
class  NnetLdaStatsAccumulator
 
struct  NnetOptimizeOptions
 
struct  NnetSimpleComputationOptions
 
struct  NnetSimpleLoopedComputationOptions
 
class  NnetTrainer
 This class is for single-threaded training of neural nets using standard objective functions such as cross-entropy (implemented with logsoftmax nonlinearity and a linear objective function) and quadratic loss. More...
 
struct  NnetTrainerOptions
 
class  NonlinearComponent
 
class  NoOpComponent
 NoOpComponent just duplicates its input. More...
 
class  NormalizeComponent
 
struct  ObjectiveFunctionInfo
 
class  OffsetForwardingDescriptor
 Offsets in 't' and 'x' values of other ForwardingDescriptors. More...
 
class  OnlineNaturalGradient
 Keywords for search: natural gradient, naturalgradient, NG-SGD. More...
 
class  OnlineNaturalGradientSimple
 
class  OptionalSumDescriptor
 This is the case of class SumDescriptor, in which we contain just one term, and that term is optional (an IfDefined() expression). More...
 
struct  PairIsEqualComparator
 
struct  PerDimObjectiveInfo
 
class  PerElementOffsetComponent
 
class  PerElementScaleComponent
 PerElementScaleComponent scales each dimension of its input with a separate trainable scale; it's like a linear component with a diagonal matrix. More...
 
class  PermuteComponent
 PermuteComponent changes the order of the columns (i.e. More...
 
class  PnormComponent
 
class  RandomComponent
 
class  RectifiedLinearComponent
 
class  RepeatedAffineComponent
 
class  ReplaceIndexForwardingDescriptor
 This ForwardingDescriptor modifies the indexes (n, t, x) by replacing one of them (normally t) with a constant value and keeping the rest. More...
 
class  RestrictedAttentionComponent
 RestrictedAttentionComponent implements an attention model with restricted temporal context. More...
 
class  RoundingForwardingDescriptor
 For use in clockwork RNNs and the like, this forwarding-descriptor rounds the time-index t down to the the closest t' <= t that is an exact multiple of t_modulus_. More...
 
class  RowOpsSplitter
 
class  ScaleAndOffsetComponent
 
class  SigmoidComponent
 
class  SimpleForwardingDescriptor
 SimpleForwardingDescriptor is the base-case of ForwardingDescriptor, consisting of a source node in the graph with a given scalar weight (which will in the normal case be 1.0). More...
 
struct  SimpleObjectiveInfo
 
class  SimpleSumDescriptor
 This is the normal base-case of SumDescriptor which just wraps a ForwardingDescriptor. More...
 
class  SoftmaxComponent
 
class  StatisticsExtractionComponent
 
class  StatisticsExtractionComponentPrecomputedIndexes
 
class  StatisticsPoolingComponent
 
class  StatisticsPoolingComponentPrecomputedIndexes
 
class  SumBlockComponent
 SumBlockComponent sums over blocks of its input: for instance, if you create one with the config "input-dim=400 output-dim=100", its output will be the sum over the 4 100-dimensional blocks of the input. More...
 
class  SumDescriptor
 This is an abstract base-class. More...
 
class  SumGroupComponent
 SumGroupComponent is used to sum up groups of posteriors. More...
 
class  SvdApplier
 
class  SwitchingForwardingDescriptor
 Chooses from different inputs based on the the time index modulo (the number of ForwardingDescriptors given as inputs). More...
 
class  TanhComponent
 
struct  TarjanNode
 
class  TdnnComponent
 TdnnComponent is a more memory-efficient alternative to manually splicing several frames of input and then using a NaturalGradientAffineComponent or a LinearComponent. More...
 
class  TimeHeightConvolutionComponent
 TimeHeightConvolutionComponent implements 2-dimensional convolution where one of the dimensions of convolution (which traditionally would be called the width axis) is identified with time (i.e. More...
 
class  UpdatableComponent
 Class UpdatableComponent is a Component which has trainable parameters; it extends the interface of Component. More...
 
class  UtteranceSplitter
 
class  VariableMergingOptimizer
 This class is responsible for merging matrices, although you probably want to access it via the the function VariableMergingOptimization(). More...
 

Typedefs

typedef TableWriter< KaldiObjectHolder< NnetChainExample > > NnetChainExampleWriter
 
typedef SequentialTableReader< KaldiObjectHolder< NnetChainExample > > SequentialNnetChainExampleReader
 
typedef RandomAccessTableReader< KaldiObjectHolder< NnetChainExample > > RandomAccessNnetChainExampleReader
 
typedef std::pair< int32, IndexCindex
 
typedef TableWriter< KaldiObjectHolder< NnetDiscriminativeExample > > NnetDiscriminativeExampleWriter
 
typedef SequentialTableReader< KaldiObjectHolder< NnetDiscriminativeExample > > SequentialNnetDiscriminativeExampleReader
 
typedef RandomAccessTableReader< KaldiObjectHolder< NnetDiscriminativeExample > > RandomAccessNnetDiscriminativeExampleReader
 
typedef TableWriter< KaldiObjectHolder< NnetExample > > NnetExampleWriter
 
typedef SequentialTableReader< KaldiObjectHolder< NnetExample > > SequentialNnetExampleReader
 
typedef RandomAccessTableReader< KaldiObjectHolder< NnetExample > > RandomAccessNnetExampleReader
 

Enumerations

enum  AccessType { kReadAccess, kWriteAccess, kReadWriteAccess }
 
enum  ComponentProperties {
  kSimpleComponent = 0x001, kUpdatableComponent = 0x002, kPropagateInPlace = 0x004, kPropagateAdds = 0x008,
  kReordersIndexes = 0x010, kBackpropAdds = 0x020, kBackpropNeedsInput = 0x040, kBackpropNeedsOutput = 0x080,
  kBackpropInPlace = 0x100, kStoresStats = 0x200, kInputContiguous = 0x400, kOutputContiguous = 0x800,
  kUsesMemo = 0x1000, kRandomComponent = 0x2000
}
 
enum  CommandType {
  kAllocMatrix, kDeallocMatrix, kSwapMatrix, kSetConst,
  kPropagate, kBackprop, kBackpropNoModelUpdate, kMatrixCopy,
  kMatrixAdd, kCopyRows, kAddRows, kCopyRowsMulti,
  kCopyToRowsMulti, kAddRowsMulti, kAddToRowsMulti, kAddRowRanges,
  kCompressMatrix, kDecompressMatrix, kAcceptInput, kProvideOutput,
  kNoOperation, kNoOperationPermanent, kNoOperationMarker, kNoOperationLabel,
  kGotoLabel
}
 CommandType is an enum that describes the category of the command used in the NnetComputation. More...
 
enum  ObjectiveType { kLinear, kQuadratic }
 This enum is for a kind of annotation we associate with output nodes of the network; it's for the convenience of calling code so that if the objective is one of a few standard types, we can compute it directly and know how to interpret the supervision labels. More...
 
enum  NodeType {
  kInput, kDescriptor, kComponent, kDimRange,
  kNone
}
 
enum  FillMode { kNearest, kReflect }
 

Functions

void UnitTestPreconditionDirectionsOnline ()
 
std::string PrintCommand (int32 num_commands, int32 command)
 
void UnitTestNnetAnalyze ()
 
static void IndexesMultiToSubmatrixIndexes (const std::vector< std::pair< int32, int32 > > &indexes_multi, std::vector< int32 > *submatrix_indexes)
 given a vector of pairs from computation.indexes_multi_indexes containing paris (submatrix-index, row-index), this function outputs to "submatrix_indexes" all (unique) submatrix indexes that appear; and it outputs to "contains_null_marker" true if the pair (-1, -1) appears anywhere in indexes_multi, and false otherwise. More...
 
void ComputeCommandAttributes (const Nnet &nnet, const NnetComputation &computation, const ComputationVariables &vars, std::vector< CommandAttributes > *attributes)
 
void ComputeVariableAccesses (const ComputationVariables &variables, const std::vector< CommandAttributes > &command_attributes, std::vector< std::vector< Access > > *variable_accesses)
 After the command-level attributes have been computed, this function organizes them per variable (see class ComputationVariables for how a variable is defined; it is part of a matrix). More...
 
void ComputeMatrixAccesses (const Nnet &nnet, const NnetComputation &computation, const ComputationVariables &variables, const std::vector< CommandAttributes > &command_attributes, std::vector< MatrixAccesses > *matrix_accesses)
 This function organizes information in the CommandAttributes in a way that is convenient to access per matrix. More...
 
static void CheckComputationOnline (const Nnet &nnet, NnetComputation computation, bool check_rewrite)
 
void CheckComputation (const Nnet &nnet, const NnetComputation &computation, bool check_rewrite=false)
 This is a convenience interface for class ComputationChecker. More...
 
void ComputeMatrixToSubmatrix (const NnetComputation &computation, std::vector< std::vector< int32 > > *mat_to_submat)
 This function computes a vector 'mat_to_submat', indexed by matrix index, such that (*mat_to_submat)[m] is a list of all the submatrix indexes that refer to matrix m. More...
 
void PrintMatrixAccesses (std::ostream &os, const std::vector< MatrixAccesses > &matrix_accesses)
 This function is to be used in debugging; it produces human-readable output. More...
 
void PrintCommandAttributes (std::ostream &os, const std::vector< CommandAttributes > &attributes)
 This function is to be used in debugging; it produces human-readable output. More...
 
void GetCommandsOfType (const NnetComputation &computation, CommandType t, std::vector< int32 > *command_indexes)
 This utility function works out from a computation, the command-indexes of the commands of the given type. More...
 
int64 GetMaxMemoryUse (const NnetComputation &computation)
 
int32 MaxMemoryUsage (const NnetComputation &computation)
 Returns the total memory, in bytes, used by the computation (just the temporary memory, not counting the memory used by the nnet itself). More...
 
static bool HasXentOutputs (const Nnet &nnet)
 
void RecomputeStats (const std::vector< NnetChainExample > &egs, const chain::ChainTrainingOptions &chain_config, const fst::StdVectorFst &den_fst, Nnet *nnet)
 This function zeros the stored component-level stats in the nnet using ZeroComponentStats(), then recomputes them with the supplied egs. More...
 
static void MergeSupervision (const std::vector< const NnetChainSupervision *> &inputs, NnetChainSupervision *output)
 
void MergeChainExamples (bool compress, std::vector< NnetChainExample > *input, NnetChainExample *output)
 This function merges a list of NnetChainExample objects into a single one– intended to be used when forming minibatches for neural net training. More...
 
void GetChainComputationRequest (const Nnet &nnet, const NnetChainExample &eg, bool need_model_derivative, bool store_component_stats, bool use_xent_regularization, bool use_xent_derivative, ComputationRequest *computation_request)
 This function takes a NnetChainExample and produces a ComputationRequest. More...
 
void ShiftChainExampleTimes (int32 frame_shift, const std::vector< std::string > &exclude_names, NnetChainExample *eg)
 Shifts the time-index t of everything in the input of "eg" by adding "t_offset" to all "t" values– but excluding those with names listed in "exclude_names", e.g. More...
 
int32 GetNnetChainExampleSize (const NnetChainExample &a)
 
int32 GetChainNnetExampleSize (const NnetChainExample &a)
 This function returns the 'size' of a chain example as defined for purposes of merging egs, which is defined as the largest number of Indexes in any of the inputs or outputs of the example. More...
 
void UnitTestIndexIo ()
 
void UnitTestCindexIo ()
 
static void WriteIndexVectorElementBinary (std::ostream &os, const std::vector< Index > &vec, int32 i)
 
static void ReadIndexVectorElementBinary (std::istream &is, int32 i, std::vector< Index > *vec)
 
void WriteIndexVector (std::ostream &os, bool binary, const std::vector< Index > &vec)
 
void ReadIndexVector (std::istream &is, bool binary, std::vector< Index > *vec)
 
static void WriteCindexVectorElementBinary (std::ostream &os, const std::vector< Cindex > &vec, int32 i)
 
static void ReadCindexVectorElementBinary (std::istream &is, int32 i, std::vector< Cindex > *vec)
 
void WriteCindexVector (std::ostream &os, bool binary, const std::vector< Cindex > &vec)
 
void ReadCindexVector (std::istream &is, bool binary, std::vector< Cindex > *vec)
 
std::ostream & operator<< (std::ostream &ostream, const Index &index)
 
std::ostream & operator<< (std::ostream &ostream, const Cindex &cindex)
 
void PrintCindex (std::ostream &os, const Cindex &cindex, const std::vector< std::string > &node_names)
 
void PrintIndexes (std::ostream &ostream, const std::vector< Index > &indexes)
 this will only be used for pretty-printing. More...
 
void PrintCindexes (std::ostream &ostream, const std::vector< Cindex > &cindexes, const std::vector< std::string > &node_names)
 this will only be used for pretty-printing. More...
 
void PrintIntegerVector (std::ostream &os, const std::vector< int32 > &ints)
 
void AppendCindexes (int32 node, const std::vector< Index > &indexes, std::vector< Cindex > *out)
 Appends to 'out' the pairs (node, indexes[0]), (node, indexes[1]), ... More...
 
void ModifyNnetIvectorPeriod (int32 ivector_period, Nnet *nnet)
 This function modifies the descriptors in the neural network to change the periodicity with which it expects to read an iVector at its input. More...
 
int32 GetChunkSize (const Nnet &nnet, int32 frame_subsampling_factor, int32 advised_chunk_size)
 
template<class I >
Mod (I m, I n)
 Mod(m, n), defined for integers m and n where n > 0, returns the modulus m % n, defined as the integer 0 <= i < n such that i and m are congruent modulo n; for instance, Mod(13, 10) = 3. More...
 
static void CreateComputationRequestInternal (int32 begin_input_t, int32 end_input_t, int32 begin_output_t, int32 end_output_t, int32 num_sequences, int32 frame_subsampling_factor, const std::set< int32 > &ivector_times, ComputationRequest *request)
 
void CreateLoopedComputationRequest (const Nnet &nnet, int32 chunk_size, int32 frame_subsampling_factor, int32 ivector_period, int32 left_context_begin, int32 right_context, int32 num_sequences, ComputationRequest *request1, ComputationRequest *request2, ComputationRequest *request3)
 This function creates computation request suitable for giving to ComputeLooped(). More...
 
void AddTimeOffsetToComputationRequest (int32 t_offset, ComputationRequest *request)
 
static bool ExtrapolateComputationRequest (const ComputationRequest &request1, const ComputationRequest &request2, ComputationRequest *request3)
 
static bool CompileLoopedInternal (const Nnet &nnet, NnetOptimizeOptions optimize_opts, const ComputationRequest &request1, const ComputationRequest &request2, const ComputationRequest &request3, int32 num_requests, NnetComputation *computation)
 
void CompileLooped (const Nnet &nnet, const NnetOptimizeOptions &optimize_opts, const ComputationRequest &request1, const ComputationRequest &request2, const ComputationRequest &request3, NnetComputation *computation)
 CompileLooped() provides an internal interface for 'looped' computation. More...
 
void CreateLoopedComputationRequestSimple (const Nnet &nnet, int32 chunk_size, int32 frame_subsampling_factor, int32 ivector_period, int32 extra_left_context_begin, int32 extra_right_context, int32 num_sequences, ComputationRequest *request1, ComputationRequest *request2, ComputationRequest *request3)
 This function is deprecated. More...
 
void UnitTestNnetCompile ()
 
void UnitTestNnetCompileMulti ()
 
void UnitTestNnetCompileLooped ()
 
void PrintVectorVectorPair (std::vector< std::vector< std::pair< int32, int32 > > > vec_vec_pair)
 
void UnitTestSplitLocationsBackward (bool verbose)
 
void UnitTestHasContiguousProperty ()
 
void UnitTestEnsureContiguousProperty ()
 
void UnitTestSplitLocations (bool verbose)
 
bool SecondElementComparator (const std::pair< int32, int32 > &first_pair, const std::pair< int32, int32 > &second_pair)
 
void SortSubmatLists (const std::vector< std::vector< std::pair< int32, int32 > > > submat_lists, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, int32 *max_submat_list_size)
 
void ComputeSubmatIndexHistogram (const std::vector< std::vector< std::pair< int32, int32 > > > sorted_submat_lists, unordered_map< int32, std::vector< int32 > > *submat_histogram)
 
void FindSubmatIndexInSubmatLists (int32 submat_index, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::vector< std::pair< int32, int32 > >::iterator > *output_iterator_list, int32 *max_remaining_submat_list_size)
 
void ExtractGivenPairsFromSubmatLists (std::vector< std::vector< std::pair< int32, int32 > >::iterator > &input_iterator_list, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::pair< int32, int32 > > *list_of_pairs)
 
static void ExtractLastPairFromSubmatLists (std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::pair< int32, int32 > > *list_of_pairs)
 
static void SplitLocationsUsingSubmatHistogram (int32 max_submat_list_size, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::pair< int32, int32 > > *submat_histogram_vector, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 
void SplitLocations (const std::vector< std::vector< std::pair< int32, int32 > > > &submat_lists, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 The input to this function is a vector of lists of pairs, and this function splits it up into a list of vectors of pairs. More...
 
bool ConvertToIndexes (const std::vector< std::pair< int32, int32 > > &location_vector, int32 *first_value, std::vector< int32 > *second_values)
 If it is the case for some i >= 0 that all the .first elements of "location_vector" are either i or -1, then output i to first_value and the .second elements into "second_values", and return true. More...
 
void EnsureContiguousProperty (const std::vector< int32 > &indexes, std::vector< std::vector< int32 > > *indexes_out)
 This function takes a vector of indexes and splits it up into as separate vectors of the same size, as needed to ensure that the 'contiguous property' holds. More...
 
void SplitPairList (std::vector< std::pair< int32, int32 > > &list, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 This function splits a vector of pairs into a list of vectors of pairs. More...
 
void SplitLocationsBackward (const std::vector< std::vector< std::pair< int32, int32 > > > &submat_lists, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 This function has the same interface as SplitLocations(); however, it ensures certain additional properties of the output "split_lists", which are necessary because of the way it is used in backprop code. More...
 
bool HasContiguousProperty (const std::vector< int32 > &indexes, std::vector< std::pair< int32, int32 > > *reverse_indexes)
 This function returns true if for each integer i != -1, all the indexes j at which indexes[j] == i are consecutive with no gaps (more formally: if j1 < j2 < j3 and indexes[j1] != -1 and indexes[j1] == indexes[j3], then indexes[j1] == indexes[j2]). More...
 
void GetNxList (const std::vector< Index > &indexes, std::vector< std::pair< int32, int32 > > *pairs)
 This function outputs a unique, lexicographically sorted list of the pairs of (n, x) values that are encountered in the provided list of Indexes. More...
 
void GetTList (const std::vector< Index > &indexes, std::vector< int32 > *t_values)
 This function outputs a sorted, unique list of the 't' values that are encountered in the provided list of Indexes If 't' values equal to kNoTime are encountered, they are ignored and are not output. More...
 
static void ResetSeed (int32 rand_seed, const Component &c)
 
bool CheckStringsApproxEqual (const std::string &a, const std::string &b, int32 tolerance=3)
 
void TestNnetComponentIo (Component *c)
 
void TestNnetComponentCopy (Component *c)
 
void TestNnetComponentAddScale (Component *c)
 
void TestNnetComponentVectorizeUnVectorize (Component *c)
 
void TestNnetComponentUpdatable (Component *c)
 
ComponentPrecomputedIndexesGetPrecomputedIndexes (const Component &c, int32 num_rows)
 
void TestSimpleComponentPropagateProperties (const Component &c)
 
bool TestSimpleComponentDataDerivative (const Component &c, BaseFloat perturb_delta)
 
bool TestSimpleComponentModelDerivative (const Component &c, BaseFloat perturb_delta, bool test_derivative)
 
void UnitTestNnetComponent ()
 
std::ostream & operator<< (std::ostream &os, const ComputationGraphBuilder::ComputableInfo &info)
 This is to be used in logging only. More...
 
void ComputeComputationGraph (const Nnet &nnet, const ComputationRequest &request, ComputationGraph *graph)
 
static int32 SumVectorSizes (const std::vector< std::vector< int32 > > &vec)
 
static int32 SumVectorSizes (const std::vector< std::vector< std::vector< int32 > > > &vec)
 
static void ComputeComputationPhasesForEpoch (const Nnet &nnet, const ComputationGraph &graph, const std::vector< int32 > &this_epoch, const std::vector< std::vector< int32 > > &dependencies_subset, const std::vector< std::vector< int32 > > &depend_on_subset, bool epoch_is_trivial, std::vector< int32 > *phase_indexes, std::vector< std::vector< int32 > > *phases)
 
void ComputeComputationPhases (const Nnet &nnet, const ComputationGraph &computation_graph, std::vector< std::vector< std::vector< int32 > > > *phases_per_segment)
 This function divides a computation into 'phases', where a 'phase' is a collection of cindexes which can (as far as the computation graph is concerned) all be computed at the same time, and depend only on cindexes previously computed in earlier phases. More...
 
static void GetIndexesStrings (const Nnet &nnet, const NnetComputation &computation, std::vector< std::string > *indexes_strings)
 
static void GetIndexesMultiStrings (const Nnet &nnet, const NnetComputation &computation, std::vector< std::string > *indexes_multi_strings)
 
static void PrintCommand (std::ostream &os_out, const Nnet &nnet, const NnetComputation &computation, int32 command_index, const std::vector< std::string > &submatrix_strings, const std::vector< std::string > &indexes_strings, const std::vector< std::string > &indexes_multi_strings)
 
static void PrintComputationPreamble (std::ostream &os, const NnetComputation &c, const Nnet &nnet, const std::vector< std::string > &submatrix_strings, const std::vector< std::string > &indexes_strings, const std::vector< std::string > &indexes_multi_strings)
 
std::ostream & operator<< (std::ostream &os, NnetComputation &computation)
 
void UnitTestNnetComputationIo (NnetComputation *computation)
 
void UnitTestComputationRequestIo (ComputationRequest *request)
 
void TestNnetDecodable (Nnet *nnet)
 
void UnitTestNnetCompute ()
 
void ComputeMinAndMaxTimes (const std::vector< Index > &indexes, int32 *min_t, int32 *max_t)
 
void SetDerivTimesOptions (const ComputationRequest &request, NnetOptimizeOptions *opt_config)
 
void UnitTestNnetModelDerivatives ()
 
void UnitTestNnetInputDerivatives ()
 
ForwardingDescriptorGenRandForwardingDescriptor (int32 num_nodes)
 
SumDescriptorGenRandSumDescriptor (int32 num_nodes)
 
void GenRandDescriptor (int32 num_nodes, Descriptor *desc)
 
void UnitTestDescriptorIo ()
 
void UnitTestGeneralDescriptor ()
 
std::string NormalizeTextDescriptor (const std::vector< std::string > &node_names, const std::string &desc_str)
 
void UnitTestGeneralDescriptorSpecial ()
 
static std::string ParsingContext (const std::string *token_ptr)
 
static void ExpectToken (const std::string &token, const std::string &what_we_are_parsing, const std::string **next_token)
 
static int32 ReadIntegerToken (const std::string &what_we_are_parsing, const std::string **next_token)
 
void ComputeAccuracy (const GeneralMatrix &supervision, const CuMatrixBase< BaseFloat > &nnet_output, BaseFloat *tot_weight, BaseFloat *tot_accuracy, VectorBase< BaseFloat > *tot_weight_vec=NULL, VectorBase< BaseFloat > *tot_accuracy_vec=NULL)
 This function computes the frame accuracy for this minibatch. More...
 
void MergeSupervision (const std::vector< const NnetDiscriminativeSupervision *> &inputs, NnetDiscriminativeSupervision *output)
 
void MergeDiscriminativeExamples (bool compress, std::vector< NnetDiscriminativeExample > *input, NnetDiscriminativeExample *output)
 
void GetDiscriminativeComputationRequest (const Nnet &nnet, const NnetDiscriminativeExample &eg, bool need_model_derivative, bool store_component_stats, bool use_xent_regularization, bool use_xent_derivative, ComputationRequest *computation_request)
 This function takes a NnetDiscriminativeExample and produces a ComputationRequest. More...
 
void ShiftDiscriminativeExampleTimes (int32 frame_shift, const std::vector< std::string > &exclude_names, NnetDiscriminativeExample *eg)
 Shifts the time-index t of everything in the input of "eg" by adding "t_offset" to all "t" values– but excluding those with names listed in "exclude_names", e.g. More...
 
int32 GetNnetDiscriminativeExampleSize (const NnetDiscriminativeExample &a)
 
void MergeDiscriminativeExamples (std::vector< NnetDiscriminativeExample > *input, bool compress, NnetDiscriminativeExample *output)
 Appends the given vector of examples (which must be non-empty) into a single output example. More...
 
int32 GetDiscriminativeNnetExampleSize (const NnetDiscriminativeExample &a)
 This function returns the 'size' of a discriminative example as defined for purposes of merging egs, which is defined as the largest number of Indexes in any of the inputs or outputs of the example. More...
 
void UnitTestNnetExample ()
 
void UnitTestNnetMergeExamples ()
 
static void GetIoNames (const std::vector< NnetExample > &src, std::vector< std::string > *names_vec)
 
static void GetIoSizes (const std::vector< NnetExample > &src, const std::vector< std::string > &names, std::vector< int32 > *sizes)
 
static void MergeIo (const std::vector< NnetExample > &src, const std::vector< std::string > &names, const std::vector< int32 > &sizes, bool compress, NnetExample *merged_eg)
 
void MergeExamples (const std::vector< NnetExample > &src, bool compress, NnetExample *dest)
 Merge a set of input examples into a single example (typically the size of "src" will be the minibatch size). More...
 
void ShiftExampleTimes (int32 t_offset, const std::vector< std::string > &exclude_names, NnetExample *eg)
 Shifts the time-index t of everything in the "eg" by adding "t_offset" to all "t" values. More...
 
void GetComputationRequest (const Nnet &nnet, const NnetExample &eg, bool need_model_derivative, bool store_component_stats, ComputationRequest *computation_request)
 This function takes a NnetExample (which should already have been frame-selected, if desired, and merged into a minibatch) and produces a ComputationRequest. More...
 
void WriteVectorAsChar (std::ostream &os, bool binary, const VectorBase< BaseFloat > &vec)
 
void ReadVectorAsChar (std::istream &is, bool binary, Vector< BaseFloat > *vec)
 
void RoundUpNumFrames (int32 frame_subsampling_factor, int32 *num_frames, int32 *num_frames_overlap)
 
int32 GetNnetExampleSize (const NnetExample &a)
 This function returns the 'size' of a nnet-example as defined for purposes of merging egs, which is defined as the largest number of Indexes in any of the inputs or outputs of the example. More...
 
static void CopyPairVector (const CuArray< Int32Pair > &in, std::vector< std::pair< int32, int32 > > *out)
 
static void CopyPairVector (const std::vector< std::pair< int32, int32 > > &in, CuArray< Int32Pair > *out)
 
bool AssertGraphEqual (const std::vector< std::vector< int32 > > &graph1, const std::vector< std::vector< int32 > > &graph2)
 
bool AssertVectorEqual (const std::vector< int32 > &vec1, const std::vector< int32 > &vec2)
 
void BuildTestGraph (std::vector< std::vector< int32 > > *graph)
 
void BuildTestGraphTranspose (std::vector< std::vector< int32 > > *graph)
 
void BuildTestSccs (std::vector< std::vector< int32 > > *sccs)
 
void BuildTestSccGraph (std::vector< std::vector< int32 > > *scc_graph)
 
void BuildTestTopSortOrder (std::vector< int32 > *node_to_order)
 
void UnitTestComputeGraphTranspose ()
 
void UnitTestFindSccs ()
 
void UnitTestMakeSccGraph ()
 
void UnitTestComputeTopSortOrder ()
 
void UnitTestComputeTopSortOrder2 ()
 
void NnetToDirectedGraph (const Nnet &nnet, std::vector< std::vector< int32 > > *graph)
 This function takes an nnet and turns it to a directed graph on nodes. More...
 
void ComputeGraphTranspose (const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *graph_transpose)
 Outputs a graph in which the order of arcs is reversed. More...
 
void TarjanSccRecursive (int32 node, const std::vector< std::vector< int32 > > &graph, int32 *global_index, std::vector< TarjanNode > *tarjan_nodes, std::vector< int32 > *tarjan_stack, std::vector< std::vector< int32 > > *sccs)
 
void FindSccsTarjan (const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *sccs)
 
void FindSccs (const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *sccs)
 Given a directed graph (where each std::vector<int32> is a list of destination-nodes of arcs coming from the current node), partition it into strongly connected components (i.e. More...
 
void MakeSccGraph (const std::vector< std::vector< int32 > > &graph, const std::vector< std::vector< int32 > > &sccs, std::vector< std::vector< int32 > > *scc_graph)
 Given a list of sccs of a graph (e.g. More...
 
void ComputeTopSortOrderRecursive (int32 node, const std::vector< std::vector< int32 > > &graph, std::vector< bool > *cycle_detector, std::vector< bool > *is_visited, std::vector< int32 > *reversed_orders)
 
void ComputeTopSortOrder (const std::vector< std::vector< int32 > > &graph, std::vector< int32 > *node_to_order)
 Given an acyclic graph (where each std::vector<int32> is a list of destination-nodes of arcs coming from the current node), compute a topological ordering of the graph nodes. More...
 
std::string PrintGraphToString (const std::vector< std::vector< int32 > > &graph)
 Prints a graph to a string in a pretty way for human readability, e.g. More...
 
void ComputeNnetComputationEpochs (const Nnet &nnet, std::vector< int32 > *node_to_epoch)
 This function computes the order in which we need to compute each node in the graph, where each node-index n maps to an epoch-index t = 0, 1, ... More...
 
bool GraphHasCycles (const std::vector< std::vector< int32 > > &graph)
 This function returns 'true' if the graph represented in 'graph' contains cycles (including cycles where a single node has an arc to itself). More...
 
void UnitTestNnetIo ()
 
static bool UnitTestNnetOptimizeWithOptions (int32 srand_seed, NnetOptimizeOptions opt_config, CachingOptimizingCompilerOptions compiler_config)
 
static void UnitTestNnetOptimizeInternal (int32 srand_seed)
 
static void UnitTestNnetOptimize ()
 
void IdentifySubmatrixArgs (NnetComputation::Command *command, std::vector< int32 * > *submatrix_args)
 This function outputs to "submatrix_args" the addresses of a subset of arguments arg1 through arg6 in "command", that correspond to the indexes of submatrices. More...
 
void IdentifySubmatrixArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *submatrix_args)
 This function outputs to "submatrix_args" the addresses of the args (arguments arg1 through arg6) in the vector "commands", that correspond to the indexes of submatrices. More...
 
void IdentifyMatrixArgsInComputation (NnetComputation *computation, std::vector< int32 *> *matrix_args)
 
void IdentifyIndexesMultiArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *indexes_multi_args)
 Identifies in the vector of commands, arguments that correspond to indexes into the computation's indexes_multi array, and outputs a list of pointers to those arguments to 'indexes_multi_args'. More...
 
void IdentifyIndexesRangesArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *indexes_ranges_args)
 Identifies in the vector of commands, arguments that correspond to indexes into the computation's 'indexes_ranges' array, and outputs a list of pointers to those arguments to 'indexes_ranges_args'. More...
 
void IdentifyIndexesArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *indexes_args)
 Identifies in the vector of commands, arguments that correspond to indexes into the computation's 'indexes' array, and outputs a list of pointers to those arguments to 'indexes_args'. More...
 
void IdentifySubmatrixArgsInComputation (NnetComputation *computation, std::vector< int32 * > *submatrix_args)
 This function outputs to "submatrix_args" the addresses of integers in 'computation' that correspond to submatrices. More...
 
void RenumberComputation (NnetComputation *computation)
 This function detects submatrices and matrices that are never used (e.g. More...
 
static bool IsNoop (const NnetComputation::Command &command)
 
void RemoveNoOps (NnetComputation *computation)
 Removes commands of type kNoOperation in the computation. More...
 
static NnetComputation::SubMatrixInfo GetSubMatrixOfSubMatrix (const NnetComputation &computation, int32 submat_a, int32 submat_b)
 This static function returns a SubMatrixInfo corresponding to replacing the matrix-index in a's "matrix_index" with, essentially, sub-matrix b. More...
 
void ExtendMatrices (NnetComputation *computation)
 This is not really an optimization in itself but it can make things easier for class VariableMergingOptimizer (usually called by its wrapper VariableMergingOptimization()). More...
 
void ConsolidateModelUpdate (const Nnet &nnet, NnetComputation *computation)
 This optimization consolidates the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices. More...
 
void LimitDerivativeTimes (const Nnet &nnet, int32 min_deriv_time, int32 max_deriv_time, NnetComputation *computation)
 
static bool IndexesHaveSpecialStructure (const std::vector< int32 > &indexes, int32 *first_nonnegative_pos, int32 *first_nonnegative_value, int32 *num_nonnegative_indexes)
 
bool ReplaceRowWithMatrixOps (NnetComputation *computation)
 This function detects cases where commands of type kCopyRows, kAddRows or kAddToRows can be converted to commands of type kMatrixCopy or kMatrixAdd, and converts them (this may involve adding submatrices). More...
 
static void FindNumLeadingAndTrailingNegatives (const std::vector< int32 > &vec, int32 *num_leading_negatives, int32 *num_trailing_negatives)
 
static bool SnipSingleRowOp (NnetComputation *computation, int32 command_index)
 
static void FindNumLeadingAndTrailingNegatives (const std::vector< std::pair< int32, int32 > > &vec, int32 *num_leading_negatives, int32 *num_trailing_negatives)
 
static bool SnipMultiRowOp (NnetComputation *computation, int32 command_index)
 
static void FindNumLeadingAndTrailingIdenticals (const std::vector< std::pair< int32, int32 > > &vec, int32 *num_leading_identicals, int32 *num_trailing_identicals)
 
static bool SnipRangesRowOp (NnetComputation *computation, int32 command_index)
 
bool SnipRowOps (NnetComputation *computation)
 This function detects cases where commands of type kCopyRows, kAddRows, kAddRowsMulti, kAddToRowsMulti, kCopyRowsMulti, kCopyToRowsMulti or kAddRowRanges use indexes that start or end with -1's or equivalents, and replace them with similar commands that act on a sub-matrix of the matrices they are currently acting on. More...
 
bool SplitRowOps (NnetComputation *computation)
 This function detects cases where commands of type kAddRowsMulti, kAddToRowsMulti, kCopyRowsMulti, kCopyToRowsMulti use indexes that correspond to at most two submatrices, in two distinct ranges without gaps filled by -1's, and could be converted to at most two commands of type kMatrixAdd, kMatrixCopy, kAddRows or kCopyRows. More...
 
static int32 FindNStride (const std::vector< Index > &indexes, bool full_check)
 
static int32 FindNStride (const std::vector< Cindex > &cindexes, bool full_check)
 
static void ConvertNumNValues (int32 n_stride, int32 old_N, int32 new_N, const std::vector< Index > &indexes_in, std::vector< Index > *indexes_out)
 
void ExpandComputation (const Nnet &nnet, const MiscComputationInfo &misc_info, const NnetComputation &computation, bool need_debug_info, int32 num_n_values, NnetComputation *expanded_computation)
 This function is used in 'shortcut' compilation to expand a computation that has been compiled for exactly 2 'n' values, to one that is suitable for some num_n_values > 2. More...
 
static bool IoSpecificationIsDecomposable (const IoSpecification &io_spec, IoSpecification *mini_io_spec, int32 *num_n_values_out)
 
bool RequestIsDecomposable (const ComputationRequest &request, ComputationRequest *mini_request, int32 *num_n_values)
 This function, used in 'shortcut' compilation where we first compile a smaller computation with the same structure but only 2 distinct 'n' values, works out whether a computation is 'decomposable'; if so, it returns true and outputs the 'mini_request' with the same structure, and the number of 'n' values. More...
 
void OptimizeLoopedComputation (const Nnet &nnet, NnetComputation *computation)
 This function tries to optimize computation 'computation' for an 'looped' computation. More...
 
void FixGotoLabel (NnetComputation *computation)
 This function ensures that the arg1 of a final command of type kGotoLabel is the same as the command with type kNoOperationLabel. More...
 
bool MatrixIsUnused (const Analyzer &analyzer, const NnetComputation &computation, int32 m)
 This function returns true if matrix 1 <= m < computation->matrices.size() is unused, defined as: it is not an input or an output, and is not accessed other than via commands of type kAllocMatrix, kDeallocMatrix, and kSetConst. More...
 
void RemoveCommandsForUnusedMatrix (const Analyzer &analyzer, int32 m, NnetComputation *computation)
 This function removes from 'computation' the commands accessing matrix 'm', which is assumed to be unused according to the MatrixIsUnused() command above. More...
 
void InsertCommands (std::vector< std::pair< int32, NnetComputation::Command > > *commands, NnetComputation *computation)
 Inserts commands into the computation at the requested places. More...
 
void OptimizeMemoryCompression (const Nnet &nnet, int32 memory_compression_level, NnetComputation *computation)
 Performs optimization to reduce memory usage where possible, making use of the kCompressMatrix and kDecompressMatrix commands. More...
 
int32 MaxOutputTimeInRequest (const ComputationRequest &request)
 
void MoveSizingCommands (const Nnet &nnet, NnetComputation *computation)
 This optimization moves commands that allocate and zero matrices to as late as possible, and moves commands that deallocate matrices to as early as possible. More...
 
void RemoveUnnecessaryZeroing (const Nnet &nnet, NnetComputation *computation)
 This optimization function removes, where possible, commands of type type kSetConst. More...
 
static void ComputeCommandPairs (const std::pair< std::vector< int32 >, std::vector< int32 > > &lists, std::vector< std::pair< int32, int32 > > *pairs)
 
void RemoveUnnecessaryAllocation (const Nnet &nnet, NnetComputation *computation)
 This optimization detects cases where we deallocate a matrix, and then later allocate another matrix of the same size; and replaces them with commands of type kAllocFromOther or kAllocFromOtherZeroed. More...
 
void VariableMergingOptimization (const NnetOptimizeOptions &config, const Nnet &nnet, NnetComputation *computation)
 This wraps class VariableMergingOptimizer in a simplified interface. More...
 
void ConvertAdditionToAssignment (const Nnet &nnet, NnetComputation *computation)
 This converts addition operations (things with Add in their names) to copy operations (things with Copy in their names). More...
 
void Optimize (const NnetOptimizeOptions &config, const Nnet &nnet, int32 max_output_time_in_request, NnetComputation *computation)
 This is the top-level function for optimizing a computation. More...
 
static void SplitComputationIntoSegments (const NnetComputation &computation, std::vector< std::pair< int32, int32 > > *segments)
 Split the computation up into segments bounded by kNoOperationMarker. More...
 
void ConsolidateIoOperations (const Nnet &nnet, NnetComputation *computation)
 This optimization puts the input operations (kAcceptInput) and output operations (kProvideOutput) at the very beginning or end of segments of computation, respectively. More...
 
void LimitDerivativeTimes (const Nnet &nnet, const ComputationRequest &request, const NnetOptimizeOptions &opts, NnetComputation *computation)
 This optimization, which has no effect unless you set –min-deriv-time or –max-deriv-time, modifies the backprop operations for efficiency based on the assumption that derivatives for any Cindex with t < min_deriv_time or t > max_deriv_time are zero. More...
 
void UnitTestConfigLineParse ()
 
void UnitTestReadConfig ()
 
void UnitTestDescriptorTokenize ()
 
void UnitTestSummarizeVector ()
 
void UnitTestNameMatchesPattern ()
 
void ExpectOneOrTwoTokens (std::istream &is, bool binary, const std::string &token1, const std::string &token2)
 This function is like ExpectToken but for two tokens, and it will either accept token1 and then token2, or just token2. More...
 
bool ParseFromString (const std::string &name, std::string *string, int32 *param)
 Function used in Init routines. More...
 
bool ParseFromString (const std::string &name, std::string *string, bool *param)
 This version of ParseFromString is for parameters of type bool, which can appear as any string beginning with f, F, t or T. More...
 
bool ParseFromString (const std::string &name, std::string *string, BaseFloat *param)
 This version of ParseFromString is for parameters of type BaseFloat. More...
 
bool ParseFromString (const std::string &name, std::string *string, std::string *param)
 This version of ParseFromString is for parsing strings. More...
 
bool ParseFromString (const std::string &name, std::string *string, std::vector< int32 > *param)
 This version of ParseFromString handles colon-separated or comma-separated lists of integers. More...
 
bool DescriptorTokenize (const std::string &input, std::vector< std::string > *tokens)
 This function tokenizes input when parsing Descriptor configuration values. More...
 
bool IsValidName (const std::string &name)
 Returns true if 'name' would be a valid name for a component or node in a Nnet. More...
 
void ReadConfigLines (std::istream &is, std::vector< std::string > *lines)
 This function reads in a config file and *appends* its contents to a vector of lines; it is responsible for removing comments (anything after '#') and stripping out any lines that contain only whitespace after comment removal. More...
 
std::string ErrorContext (std::istream &is)
 Return a string used in error messages. More...
 
std::string ErrorContext (const std::string &str)
 
static void PrintFloatSuccinctly (std::ostream &os, BaseFloat f)
 
std::string SummarizeVector (const VectorBase< float > &vec)
 Returns a string that summarizes a vector fairly succintly, for printing stats in info lines. More...
 
std::string SummarizeVector (const VectorBase< double > &vec)
 
std::string SummarizeVector (const CuVectorBase< BaseFloat > &cu_vec)
 
void PrintParameterStats (std::ostringstream &os, const std::string &name, const CuVectorBase< BaseFloat > &params, bool include_mean=false)
 Print to 'os' some information about the mean and standard deviation of some parameters, used in Info() functions in nnet-simple-component.cc. More...
 
void PrintParameterStats (std::ostringstream &os, const std::string &name, const CuMatrix< BaseFloat > &params, bool include_mean=false, bool include_row_norms=false, bool include_column_norms=false, bool include_singular_values=false)
 Print to 'os' some information about the mean and standard deviation of some parameters, used in Info() functions in nnet-simple-component.cc. More...
 
void ParseConfigLines (const std::vector< std::string > &lines, std::vector< ConfigLine > *config_lines)
 This function converts config-lines from a simple sequence of strings as output by ReadConfigLines(), into a sequence of first-tokens and name-value pairs. More...
 
bool NameMatchesPattern (const char *name, const char *pattern)
 
int32 YzxVectorIndex (int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
 
int32 ZyxVectorIndex (int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
 
void RearrangeIndexes (const std::vector< std::vector< int32 > > &in, std::vector< std::vector< int32 > > *out)
 
void GenerateConfigSequenceSimplest (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceSimpleContext (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceSimple (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceStatistics (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceRnn (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceRnnClockwork (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceLstm (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceLstmWithTruncation (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceLstmType2 (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceCnn (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceCnnNew (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceRestrictedAttention (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceDistribute (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceCompositeBlock (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 Generate a config string with a composite component composed only of block affine, repeated affine, and natural gradient repeated affine components. More...
 
void GenerateConfigSequence (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 Generates a sequence of at least one config files, output as strings, where the first in the sequence is the initial nnet, and the remaining ones may do things like add layers. More...
 
void ComputeExampleComputationRequestSimple (const Nnet &nnet, ComputationRequest *request, std::vector< Matrix< BaseFloat > > *inputs)
 This function computes an example computation request, for testing purposes. More...
 
static void GenerateRandomComponentConfig (std::string *component_type, std::string *config)
 
ComponentGenerateRandomSimpleComponent ()
 Generates random simple component for testing. More...
 
bool NnetParametersAreIdentical (const Nnet &nnet1, const Nnet &nnet2, BaseFloat threshold)
 Used for testing that the updatable parameters in two networks are the same. More...
 
void GenerateSimpleNnetTrainingExample (int32 num_supervised_frames, int32 left_context, int32 right_context, int32 input_dim, int32 output_dim, int32 ivector_dim, NnetExample *example)
 Low-level function that generates an nnet training example. More...
 
bool ExampleApproxEqual (const NnetExample &eg1, const NnetExample &eg2, BaseFloat delta)
 Returns true if the examples are approximately equal (only intended to be used in testing). More...
 
void ComputeObjectiveFunction (const GeneralMatrix &supervision, ObjectiveType objective_type, const std::string &output_name, bool supply_deriv, NnetComputer *computer, BaseFloat *tot_weight, BaseFloat *tot_objf)
 This function computes the objective function, and if supply_deriv = true, supplies its derivative to the NnetComputation object. More...
 
void UnitTestNnetContext ()
 
void UnitTestConvertRepeatedToBlockAffine ()
 
void UnitTestConvertRepeatedToBlockAffineComposite ()
 
int32 NumOutputNodes (const Nnet &nnet)
 returns the number of output nodes of this nnet. More...
 
int32 NumInputNodes (const Nnet &nnet)
 returns the number of input nodes of this nnet. More...
 
bool IsSimpleNnet (const Nnet &nnet)
 This function returns true if the nnet has the following properties: It has an output called "output" (other outputs are allowed but may be ignored). More...
 
void EvaluateComputationRequest (const Nnet &nnet, const ComputationRequest &request, std::vector< std::vector< bool > > *is_computable)
 Given an nnet and a computation request, this function works out which requested outputs in the computation request are computable; it outputs this information as a vector "is_computable" indexed by the same indexes as request.outputs. More...
 
static bool ComputeSimpleNnetContextForShift (const Nnet &nnet, int32 input_start, int32 window_size, int32 *left_context, int32 *right_context)
 
void ComputeSimpleNnetContext (const Nnet &nnet, int32 *left_context, int32 *right_context)
 ComputeSimpleNnetContext computes the left-context and right-context of a nnet. More...
 
void PerturbParams (BaseFloat stddev, Nnet *nnet)
 Calls PerturbParams (with the given stddev) on all updatable components of the nnet. More...
 
void ComponentDotProducts (const Nnet &nnet1, const Nnet &nnet2, VectorBase< BaseFloat > *dot_prod)
 Returns dot products between two networks of the same structure (calls the DotProduct functions of the Updatable components and fill in the output vector). More...
 
std::string PrintVectorPerUpdatableComponent (const Nnet &nnet, const VectorBase< BaseFloat > &vec)
 This function is for printing, to a string, a vector with one element per updatable component of the nnet (e.g. More...
 
BaseFloat DotProduct (const Nnet &nnet1, const Nnet &nnet2)
 Returns dot product between two networks of the same structure (calls the DotProduct functions of the Updatable components and sums up the return values). More...
 
void ZeroComponentStats (Nnet *nnet)
 Zeroes the component stats in all nonlinear components in the nnet. More...
 
void SetLearningRate (BaseFloat learning_rate, Nnet *nnet)
 Sets the underlying learning rate for all the components in the nnet to this value. More...
 
void SetNnetAsGradient (Nnet *nnet)
 Sets nnet as gradient by Setting is_gradient_ to true and learning_rate_ to 1 for each UpdatableComponent in nnet. More...
 
void ScaleNnet (BaseFloat scale, Nnet *nnet)
 Scales the nnet parameters and stats by this scale. More...
 
void AddNnetComponents (const Nnet &src, const Vector< BaseFloat > &alphas, BaseFloat scale, Nnet *dest)
 Does *dest += alpha * src for updatable components (affects nnet parameters), and *dest += scale * src for other components (affects stored stats). More...
 
void AddNnet (const Nnet &src, BaseFloat alpha, Nnet *dest)
 Does *dest += alpha * src (affects nnet parameters and stored stats). More...
 
int32 NumParameters (const Nnet &src)
 Returns the total of the number of parameters in the updatable components of the nnet. More...
 
void VectorizeNnet (const Nnet &src, VectorBase< BaseFloat > *params)
 Copies the nnet parameters to *params, whose dimension must be equal to NumParameters(src). More...
 
void UnVectorizeNnet (const VectorBase< BaseFloat > &params, Nnet *dest)
 Copies the parameters from params to *dest. More...
 
int32 NumUpdatableComponents (const Nnet &dest)
 Returns the number of updatable components in the nnet. More...
 
void FreezeNaturalGradient (bool freeze, Nnet *nnet)
 Controls if natural gradient will be updated. More...
 
void ConvertRepeatedToBlockAffine (CompositeComponent *c_component)
 
void ConvertRepeatedToBlockAffine (Nnet *nnet)
 Convert all components of type RepeatedAffineComponent or NaturalGradientRepeatedAffineComponent to BlockAffineComponent in nnet. More...
 
std::string NnetInfo (const Nnet &nnet)
 This function returns various info about the neural net. More...
 
void SetDropoutProportion (BaseFloat dropout_proportion, Nnet *nnet)
 This function sets the dropout proportion in all dropout components to dropout_proportion value. More...
 
bool HasBatchnorm (const Nnet &nnet)
 Returns true if nnet has at least one component of type BatchNormComponent. More...
 
void ScaleBatchnormStats (BaseFloat batchnorm_stats_scale, Nnet *nnet)
 This function scales the batchorm stats of any batchnorm components (components of type BatchNormComponent) in 'nnet' by the scale 'batchnorm_stats_scale'. More...
 
void RecomputeStats (const std::vector< NnetExample > &egs, Nnet *nnet)
 This function zeros the stored component-level stats in the nnet using ZeroComponentStats(), then recomputes them with the supplied egs. More...
 
void SetBatchnormTestMode (bool test_mode, Nnet *nnet)
 This function affects only components of type BatchNormComponent. More...
 
void SetDropoutTestMode (bool test_mode, Nnet *nnet)
 This function affects components of child-classes of RandomComponent. More...
 
void ResetGenerators (Nnet *nnet)
 This function calls 'ResetGenerator()' on all components in 'nnet' that inherit from class RandomComponent. More...
 
void FindOrphanComponents (const Nnet &nnet, std::vector< int32 > *components)
 This function finds a list of components that are never used, and outputs the integer comopnent indexes (you can use these to index nnet.GetComponentNames() to get their names). More...
 
void FindOrphanNodes (const Nnet &nnet, std::vector< int32 > *nodes)
 This function finds a list of nodes that are never used to compute any output, and outputs the integer node indexes (you can use these to index nnet.GetNodeNames() to get their names). More...
 
void ConstrainOrthonormalInternal (BaseFloat scale, CuMatrixBase< BaseFloat > *M)
 
void ConstrainOrthonormal (Nnet *nnet)
 This function, to be called after processing every minibatch, is responsible for enforcing the orthogonality constraint for any components of type LinearComponent or inheriting from AffineComponent that have the "orthonormal_constraint" value set. More...
 
void ConsolidateMemory (Nnet *nnet)
 This just calls ConsolidateMemory() on all the components of the nnet. More...
 
void ReduceRankOfComponents (const std::string component_name_pattern, int32 rank, Nnet *nnet)
 
void ReadEditConfig (std::istream &config_file, Nnet *nnet)
 ReadEditConfig() reads a file with a similar-looking format to the config file read by Nnet::ReadConfig(), but this consists of a sequence of operations to perform on an existing network, mostly modifying components. More...
 
bool NnetIsRecurrent (const Nnet &nnet)
 Returns true if 'nnet' has some kind of recurrency. More...
 
void CollapseModel (const CollapseModelConfig &config, Nnet *nnet)
 This function modifies the neural net for efficiency, in a way that suitable to be done in test time. More...
 
bool UpdateNnetWithMaxChange (const Nnet &delta_nnet, BaseFloat max_param_change, BaseFloat max_change_scale, BaseFloat scale, Nnet *nnet, std::vector< int32 > *num_max_change_per_component_applied, int32 *num_max_change_global_applied)
 This function does the operation '*nnet += scale * delta_nnet', while respecting any max-parameter-change (max-param-change) specified in the updatable components, and also the global max-param-change specified as 'max_param_change'. More...
 
int32 GetNumNvalues (const std::vector< NnetIo > &io_vec, bool exhaustive)
 This utility function can be used to obtain the number of distinct 'n' values in a training example. More...
 
void ApplyL2Regularization (const Nnet &nnet, BaseFloat l2_regularize_scale, Nnet *delta_nnet)
 This function is used as part of the regular training workflow, prior to UpdateNnetWithMaxChange(). More...
 
BaseFloat KlDivergence (const Vector< BaseFloat > &p, const Vector< BaseFloat > &q)
 
void PrintPriorDiagnostics (const Vector< BaseFloat > &old_priors, const Vector< BaseFloat > &new_priors)
 
void SetPriors (const TransitionModel &tmodel, const Vector< double > &transition_accs, double prior_floor, AmNnetSimple *am_nnet)
 
double ComputeObjf (bool batchnorm_test_mode, bool dropout_test_mode, const std::vector< NnetExample > &egs, const Nnet &nnet, NnetComputeProb *prob_computer)
 
void UpdateNnetMovingAverage (int32 num_models, const Nnet &nnet, Nnet *moving_average_nnet)
 
void RenameOutputs (const std::string &new_name, NnetExample *eg)
 
void ScaleSupervisionWeight (BaseFloat weight, NnetExample *eg)
 
int32 GetCount (double expected_count)
 
bool ContainsSingleExample (const NnetExample &eg, int32 *min_input_t, int32 *max_input_t, int32 *min_output_t, int32 *max_output_t)
 Returns true if the "eg" contains just a single example, meaning that all the "n" values in the indexes are zero, and the example has NnetIo members named both "input" and "output". More...
 
void FilterExample (const NnetExample &eg, int32 min_input_t, int32 max_input_t, int32 min_output_t, int32 max_output_t, NnetExample *eg_out)
 This function filters the indexes (and associated feature rows) in a NnetExample, removing any index/row in an NnetIo named "input" with t < min_input_t or t > max_input_t and any index/row in an NnetIo named "output" with t < min_output_t or t > max_output_t. More...
 
bool SelectFromExample (const NnetExample &eg, std::string frame_str, int32 left_context, int32 right_context, int32 frame_shift, NnetExample *eg_out)
 This function is responsible for possibly selecting one frame from multiple supervised frames, and reducing the left and right context as specified. More...
 
static bool ProcessFile (const discriminative::SplitDiscriminativeSupervisionOptions &config, const TransitionModel &tmodel, const MatrixBase< BaseFloat > &feats, const MatrixBase< BaseFloat > *ivector_feats, int32 ivector_period, const discriminative::DiscriminativeSupervision &supervision, const std::string &utt_id, bool compress, UtteranceSplitter *utt_splitter, NnetDiscriminativeExampleWriter *example_writer)
 
void ApplyAffineTransform (MatrixBase< BaseFloat > &transform, int32 num_channels, MatrixBase< BaseFloat > *image, FillMode fill_mode)
 This function applies a geometric transformation 'transform' to the image. More...
 
void PerturbImage (const ImageAugmentationConfig &config, MatrixBase< BaseFloat > *image)
 This function randomly modifies (perturbs) the image by applying different geometric transformations according to the options in 'config'. More...
 
void PerturbImageInNnetExample (const ImageAugmentationConfig &config, NnetExample *eg)
 This function does image perturbation as directed by 'config' The example 'eg' is expected to contain a NnetIo member with the name 'input', representing an image. More...
 
static bool ProcessFile (const GeneralMatrix &feats, const MatrixBase< BaseFloat > *ivector_feats, int32 ivector_period, const MatrixBase< BaseFloat > &targets, const std::string &utt_id, bool compress, int32 num_targets, int32 length_tolerance, UtteranceSplitter *utt_splitter, NnetExampleWriter *example_writer)
 
static bool ProcessFile (const GeneralMatrix &feats, const MatrixBase< BaseFloat > *ivector_feats, int32 ivector_period, const Posterior &pdf_post, const std::string &utt_id, bool compress, int32 num_pdfs, int32 length_tolerance, UtteranceSplitter *utt_splitter, NnetExampleWriter *example_writer)
 
int32 NumOutputIndexes (const NnetExample &eg)
 
static void RunNnetComputation (const MatrixBase< BaseFloat > &features, const Nnet &nnet, CachingOptimizingCompiler *compiler, Vector< BaseFloat > *xvector)
 
static void ProcessRangeFile (const std::string &range_rxfilename, unordered_map< std::string, std::vector< ChunkInfo *> > *utt_to_chunks)
 
static void WriteExamples (const MatrixBase< BaseFloat > &feats, const std::vector< ChunkInfo *> &chunks, const std::string &utt, bool compress, int32 num_pdfs, int32 *num_egs_written, std::vector< NnetExampleWriter *> *example_writers)
 

Variables

static bool computation_checker_warned_unused_input = false
 Checks that we never use variables before they are allocated or after they are deallocated, and some other checks that can be done from the MatrixAccesses. More...
 
const int kNoTime = std::numeric_limits<int32>::min()
 

Typedef Documentation

◆ Cindex

typedef std::pair<int32, Index> Cindex

Definition at line 115 of file nnet-common.h.

◆ NnetChainExampleWriter

◆ NnetDiscriminativeExampleWriter

◆ NnetExampleWriter

◆ RandomAccessNnetChainExampleReader

◆ RandomAccessNnetDiscriminativeExampleReader

◆ RandomAccessNnetExampleReader

◆ SequentialNnetChainExampleReader

◆ SequentialNnetDiscriminativeExampleReader

◆ SequentialNnetExampleReader

Enumeration Type Documentation

◆ AccessType

enum AccessType
Enumerator
kReadAccess 
kWriteAccess 
kReadWriteAccess 

Definition at line 75 of file nnet-analyze.h.

◆ CommandType

CommandType is an enum that describes the category of the command used in the NnetComputation.

We declare it outside that class because it's so frequently used and we got tired of typing NnetComputation:: everywhere. We document the commands here. Note: for operations that naturally need to operate on entire matrices (i.e. allocation commands and input and output commands), we use the submatrix indexes of them, which turns out to be more convenient for optimization; but these submatrix indexes must refer to the whole of a matrix.

  • kAllocMatrix. Allocate a matrix (its values will be undefined). arg1 = submatrix index, which must refer to a whole matrix.
  • kDeallocMatrix: Deallocate a matrix. arg1 = submatrix index.
  • kSwapMatrix: initialize matrix with submatrix index arg1 using memory from matrix with submatrix index arg2 (using shallow swap). Both submatrices must refer to whole matrices. The expectation is that prior to the swap, arg1 was empty and arg2 was nonempty, but the execution code does not enforce this.
  • kSetConst: set all elements of submatrix index 'arg1' to the value 'alpha'.
  • kPropagate: Forward computation of neural net, see Component::Propagate()
    • arg1 is is component-index in neural net
    • arg2 is index into ComponentPrecomputedIndexes (0 if NULL; always 0 for simple Components)
    • arg3 is sub-matrix index of input
    • arg4 is sub-matrix index of output
    • arg5 is the index of the memo saved from Propagate()'s return value, or 0 if it saves no memo.
    • arg6 is 1 if we need to call StoreStats() after the Propagate, or 0 if we don't. We used to have a separate command for storing the stats, but that has been removed.
  • kBackprop: Do the back-propagation operation, see Component::Backprop()
    • arg1 is index of component in neural net
    • arg2 is index into ComponentPrecomputedIndexes (0 if NULL; always 0 for simple Components)
    • arg3 is submatrix-index of input value (input to Propagate()); 0 if unused
    • arg4 is submatrix-index of output value (output of Propagate()); 0 if unused
    • arg5 is submatrix-index of output derivative
    • arg6 is submatrix-index of input derivative; 0 if unused.
    • arg7 is the index of the memo which is generated from the corresponding Propagate() function if the flag kUsesMemo is set; 0 if unused.
  • kBackpropNoModelUpdate: as kBackprop, but does not set the 'to_update' argument to the Backprop call, even if the model is updatable, so it skips the model-update phase of backprop.
  • kMatrixCopy: Copy (alpha times contents of sub-matrix arg2) to sub-matrix arg1, currently implemented as copy then scale. Note: to implement scaling a matrix, you can use kMatrixCopy with arg1 == arg2 and it won't do any redundant copying.
  • kMatrixAdd: Add (alpha times contents of sub-matrix arg2) to sub-matrix arg1
  • kCopyRows: call CopyRows() on sub-matrix arg1 with sub-matrix arg2 and indexes[arg3] as arguments, then if alpha != 1.0, scale sub-matrix arg1 by alpha.
  • kAddRows: call AddRows() on sub-matrix arg1 with alpha, sub-matrix arg2 and indexes[arg3] as arguments.
  • kAddRowsMulti, kAddToRowsMulti, kCopyRowsMulti, kCopyToRowsMulti: Call the corresponding function in class CuMatrix (Actually the names do not have 'Multi' in them, but they are the ones that accept a vector of 'Real*'.
    • arg1 is sub-matrix index of *this matrix in operation
    • arg2 is index into "indexes_multi", of which each pair is (sub-matrix index, row index) (or (-1,-1) for NULL marker), which is turned into a vector of BaseFloat* (pointers to matrix rows) before being given as the argument to the function. In the 'Add' functions 'alpha' is provided as an argument; for the 'Copy' functions, we scale the destination by 'alpha' after the copy, if alpha != 1.0. (However, for implementation reasons, kCopyToRowsMulti does not currently support alpha != 1.0 and will crash, so we avoid generating this code).
  • kAddRowRanges: call AddRowRanges() on sub-matrix arg1, with arg2 as source sub-matrix, and indexes given indexes_ranges[arg3]. We use the "alpha" as if AddRowRanges() accepted that argument, even though it doesn't (we fake it using other calls, if alpha != 1.0).
  • kCompressMatrix: Compresses the matrix which should be referred to by submatrix-index arg1. arg2 is a number that determines the compression type (it's converted from the enum CuCompressedMatrixType; 1=int8, 2=uint8, 3=int16, 4=uint16), and alpha determines the 'range' parameter (c.f. NewCuCompressedMatrix()). arg3 will be converted to the 'truncate' argument to the class CuCompressedMatrix; it should be false (0) if you know that the input is limited to the allowed range, and true (1) if the input may exceed that range (see docs for CuCompresedMatrix).
  • kDecompressMatrix: Decompresses the matrix which is referred to by submatrix-index arg1 (it should previously have been compressed).
  • kAcceptInput: accepts a matrix of input from the user, which may be either features, or derivatives w.r.t. the output. arg1 is the submatrix index of a whole matrix that the input goes to, and arg2 is the index of the network node associated with it (e.g. the node of "input" or "ivector"), for puroses of double checking.
  • kProvideOutput: outputs a matrix to the user: either a network output, or a matrix of derivatives w.r.t. an input. arg1 is the submatrix index of the output (which we expect to be a whole matrix), arg2 is the index of the network node associated with it (e.g. the node for "output").
  • kNoOperation: does nothing, and will be removed by optimization code (sometimes useful during optimization)
  • kNoOperationPermanent: like kNoOperation, but won't be removed by optimization code. This is used to ensure that for 'trivial' computations, which just copy the input to the output, the block of commands for the forward or backward propagation is nonempty (to avoid confusing the computation code).
  • kNoOperationMarker: does nothing, but used to mark end of a block of commands (like forward commands).
  • kNoOperationLabel: does nothing, but is the destination for the kGotoLabel command.
  • kGotoLabel: jumps to the kNoOperationLabel command. arg1 must be set to the location of that command. Since there are no conditionals, the kGotoLabel command should be the last command, as remaining commands will be unreachable.
Enumerator
kAllocMatrix 
kDeallocMatrix 
kSwapMatrix 
kSetConst 
kPropagate 
kBackprop 
kBackpropNoModelUpdate 
kMatrixCopy 
kMatrixAdd 
kCopyRows 
kAddRows 
kCopyRowsMulti 
kCopyToRowsMulti 
kAddRowsMulti 
kAddToRowsMulti 
kAddRowRanges 
kCompressMatrix 
kDecompressMatrix 
kAcceptInput 
kProvideOutput 
kNoOperation 
kNoOperationPermanent 
kNoOperationMarker 
kNoOperationLabel 
kGotoLabel 

Definition at line 288 of file nnet-computation.h.

288  {
296  kGotoLabel };

◆ ComponentProperties

Enumerator
kSimpleComponent 
kUpdatableComponent 
kPropagateInPlace 
kPropagateAdds 
kReordersIndexes 
kBackpropAdds 
kBackpropNeedsInput 
kBackpropNeedsOutput 
kBackpropInPlace 
kStoresStats 
kInputContiguous 
kOutputContiguous 
kUsesMemo 
kRandomComponent 

Definition at line 37 of file nnet-component-itf.h.

37  {
38  kSimpleComponent = 0x001, // true if number of rows of input equals number of rows
39  // of output and this component doesn't care about the indexes
40  // (i.e. it maps each row of input to each row of output without
41  // regard to the index values). Will normally be true.
42  kUpdatableComponent = 0x002, // true if the component has parameters that can
43  // be updated. Components that return this flag
44  // must be dynamic_castable to type
45  // UpdatableComponent (but components of type
46  // UpdatableComponent do not have to return this
47  // flag, e.g. if this instance is not really
48  // updatable).
49  kPropagateInPlace = 0x004, // true if we can do the propagate operation in-place
50  // (input and output matrices are the same).
51  // Note: if doing backprop, you'd also need to check
52  // that the kBackpropNeedsInput property is not true.
53  kPropagateAdds = 0x008, // true if the Propagate function adds to, rather
54  // than setting, its output, for non-in-place
55  // propagation. The Component chooses whether to add
56  // or set, and the calling code has to accommodate
57  // it.
58  kReordersIndexes = 0x010, // true if the ReorderIndexes function might reorder
59  // the indexes (otherwise we can skip calling it).
60  // Must not be set for simple components.
61  kBackpropAdds = 0x020, // true if the Backprop function adds to, rather than
62  // setting, the "in_deriv" output for non-in-place
63  // backprop. The Component chooses whether to add or
64  // set, and the calling code has to accommodate it.
65  kBackpropNeedsInput = 0x040, // true if backprop operation needs access to
66  // forward-pass input.
67  kBackpropNeedsOutput = 0x080, // true if backprop operation needs access to
68  // forward-pass output (e.g. true for Sigmoid).
69  kBackpropInPlace = 0x100, // true if we can do the backprop operation in-place
70  // (input and output matrices may be the same).
71  kStoresStats = 0x200, // true if the StoreStats operation stores
72  // statistics e.g. on average node activations and
73  // derivatives of the nonlinearity, (as it does for
74  // Tanh, Sigmoid, ReLU and Softmax).
75  kInputContiguous = 0x400, // true if the component requires its input data (and
76  // input derivatives) to have Stride()== NumCols().
77  kOutputContiguous = 0x800, // true if the component requires its input data (and
78  // output derivatives) to have Stride()== NumCols().
79  kUsesMemo = 0x1000, // true if the component returns a void* pointer from its
80  // Propagate() function that needs to be passed into the
81  // corresponding Backprop function.
82  kRandomComponent = 0x2000 // true if the component has some kind of
83  // randomness, like DropoutComponent (these should
84  // inherit from class RandomComponent.
85 };

◆ FillMode

enum FillMode
Enumerator
kNearest 
kReflect 

Definition at line 31 of file nnet3-egs-augment-image.cc.

◆ NodeType

enum NodeType
Enumerator
kInput 
kDescriptor 
kComponent 
kDimRange 
kNone 

Definition at line 55 of file nnet-nnet.h.

◆ ObjectiveType

This enum is for a kind of annotation we associate with output nodes of the network; it's for the convenience of calling code so that if the objective is one of a few standard types, we can compute it directly and know how to interpret the supervision labels.

However, the core of the framework never makes use of the objective types, other than making them available to calling code which then supplies the derivatives.

  • Objective type kLinear is intended for Neural nets where the final component is a LogSoftmaxComponent, so the log-prob (negative cross-entropy) objective is just a linear function of the input.
  • Objective type kQuadratic is used to mean the objective function f(x, y) = -0.5 (x-y).(x-y), which is to be maximized, as in the kLinear case.
Enumerator
kLinear 
kQuadratic 

Definition at line 52 of file nnet-nnet.h.

Function Documentation

◆ AddNnet()

void AddNnet ( const Nnet src,
BaseFloat  alpha,
Nnet dest 
)

Does *dest += alpha * src (affects nnet parameters and stored stats).

Definition at line 340 of file nnet-utils.cc.

References Component::Add(), Nnet::GetComponent(), KALDI_ERR, and Nnet::NumComponents().

Referenced by main(), kaldi::ReadModels(), NnetDiscriminativeTrainer::Train(), and UpdateNnetMovingAverage().

340  {
341  if (src.NumComponents() != dest->NumComponents())
342  KALDI_ERR << "Trying to add incompatible nnets.";
343  for (int32 c = 0; c < src.NumComponents(); c++) {
344  const Component *src_comp = src.GetComponent(c);
345  Component *dest_comp = dest->GetComponent(c);
346  dest_comp->Add(alpha, *src_comp);
347  }
348 }
#define KALDI_ERR
Definition: kaldi-error.h:123

◆ AddNnetComponents()

void AddNnetComponents ( const Nnet src,
const Vector< BaseFloat > &  alphas,
BaseFloat  scale,
Nnet dest 
)

Does *dest += alpha * src for updatable components (affects nnet parameters), and *dest += scale * src for other components (affects stored stats).

Here, alphas is a vector of size equal to the number of updatable components

Definition at line 313 of file nnet-utils.cc.

References Component::Add(), VectorBase< Real >::Dim(), Nnet::GetComponent(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, kUpdatableComponent, Nnet::NumComponents(), and Component::Properties().

Referenced by UpdateNnetWithMaxChange().

314  {
315  if (src.NumComponents() != dest->NumComponents())
316  KALDI_ERR << "Trying to add incompatible nnets.";
317  int32 i = 0;
318  for (int32 c = 0; c < src.NumComponents(); c++) {
319  const Component *src_comp = src.GetComponent(c);
320  Component *dest_comp = dest->GetComponent(c);
321  if (src_comp->Properties() & kUpdatableComponent) {
322  // For now all updatable components inherit from class UpdatableComponent.
323  // If that changes in future, we will change this code.
324  const UpdatableComponent *src_uc =
325  dynamic_cast<const UpdatableComponent*>(src_comp);
326  UpdatableComponent *dest_uc =
327  dynamic_cast<UpdatableComponent*>(dest_comp);
328  if (src_uc == NULL || dest_uc == NULL)
329  KALDI_ERR << "Updatable component does not inherit from class "
330  "UpdatableComponent; change this code.";
331  KALDI_ASSERT(i < alphas.Dim());
332  dest_uc->Add(alphas(i++), *src_uc);
333  } else { // add stored stats
334  dest_comp->Add(scale, *src_comp);
335  }
336  }
337  KALDI_ASSERT(i == alphas.Dim());
338 }
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ AddTimeOffsetToComputationRequest()

void kaldi::nnet3::AddTimeOffsetToComputationRequest ( int32  t_offset,
ComputationRequest request 
)

Definition at line 235 of file nnet-compile-looped.cc.

References rnnlm::i, ComputationRequest::inputs, rnnlm::j, and ComputationRequest::outputs.

Referenced by ExtrapolateComputationRequest().

236  {
237  for (size_t i = 0; i < request->inputs.size(); i++) {
238  size_t size = request->inputs[i].indexes.size();
239  for (size_t j = 0; j < size; j++)
240  request->inputs[i].indexes[j].t += t_offset;
241  }
242  for (size_t i = 0; i < request->outputs.size(); i++) {
243  size_t size = request->outputs[i].indexes.size();
244  for (size_t j = 0; j < size; j++)
245  request->outputs[i].indexes[j].t += t_offset;
246  }
247 }

◆ AppendCindexes()

void AppendCindexes ( int32  node,
const std::vector< Index > &  indexes,
std::vector< Cindex > *  out 
)

Appends to 'out' the pairs (node, indexes[0]), (node, indexes[1]), ...

Definition at line 1375 of file nnet-compile.cc.

References rnnlm::i.

Referenced by Compiler::OutputDebugInfo().

1376  {
1377  size_t indexes_size = indexes.size();
1378  if (indexes_size > out->size())
1379  out->reserve(out->size() + indexes_size);
1380  for (size_t i = 0; i < indexes_size; i++)
1381  out->push_back(Cindex(node, indexes[i]));
1382 }
std::pair< int32, Index > Cindex
Definition: nnet-common.h:115

◆ ApplyAffineTransform()

void kaldi::nnet3::ApplyAffineTransform ( MatrixBase< BaseFloat > &  transform,
int32  num_channels,
MatrixBase< BaseFloat > *  image,
FillMode  fill_mode 
)

This function applies a geometric transformation 'transform' to the image.

Reference: Digital Image Processing book by Gonzalez and Woods.

Parameters
[in]transformThe 3x3 geometric transformation matrix to apply.
[in]num_channelsNumber of channels (i.e. colors) of the image
[in,out]imageThe image matrix to be modified. image->NumRows() is the width (number of x values) in the image; image->NumCols() is the height times number of channels (channel varies the fastest).

Definition at line 110 of file nnet3-egs-augment-image.cc.

References KALDI_ASSERT, kNearest, ImageAugmentationConfig::num_channels, MatrixBase< Real >::NumCols(), and MatrixBase< Real >::NumRows().

Referenced by PerturbImage().

113  {
114  int32 num_rows = image->NumRows(),
115  num_cols = image->NumCols(),
116  height = num_cols / num_channels,
117  width = num_rows;
118  KALDI_ASSERT(num_cols % num_channels == 0);
119  Matrix<BaseFloat> original_image(*image);
120  for (int32 r = 0; r < width; r++) {
121  for (int32 c = 0; c < height; c++) {
122  // (r_old, c_old) is the coordinate of the pixel in the original image
123  // while (r, c) is the coordinate in the new (transformed) image.
124  BaseFloat r_old = transform(0, 0) * r +
125  transform(0, 1) * c + transform(0, 2);
126  BaseFloat c_old = transform(1, 0) * r +
127  transform(1, 1) * c + transform(1, 2);
128  // We are going to do bilinear interpolation between 4 closest points
129  // to the point (r_old, c_old) of the original image. We have:
130  // r1 <= r_old <= r2
131  // c1 <= c_old <= c2
132  int32 r1 = static_cast<int32>(floor(r_old));
133  int32 c1 = static_cast<int32>(floor(c_old));
134  int32 r2 = r1 + 1;
135  int32 c2 = c1 + 1;
136 
137  // These weights determine how much each of the 4 points contributes
138  // to the final interpolated value:
139  BaseFloat weight_11 = (r2 - r_old) * (c2 - c_old),
140  weight_12 = (r2 - r_old) * (c_old - c1),
141  weight_21 = (r_old - r1) * (c2 - c_old),
142  weight_22 = (r_old - r1) * (c_old - c1);
143  // Handle edge conditions:
144  if (fill_mode == kNearest) {
145  if (r1 < 0) {
146  r1 = 0;
147  if (r2 < 0) r2 = 0;
148  }
149  if (r2 >= width) {
150  r2 = width - 1;
151  if (r1 >= width) r1 = width - 1;
152  }
153  if (c1 < 0) {
154  c1 = 0;
155  if (c2 < 0) c2 = 0;
156  }
157  if (c2 >= height) {
158  c2 = height - 1;
159  if (c1 >= height) c1 = height - 1;
160  }
161  } else {
162  KALDI_ASSERT(fill_mode == kReflect);
163  if (r1 < 0) {
164  r1 = - r1;
165  if (r2 < 0) r2 = - r2;
166  }
167  if (r2 >= width) {
168  r2 = 2 * width - 2 - r2;
169  if (r1 >= width) r1 = 2 * width - 2 - r1;
170  }
171  if (c1 < 0) {
172  c1 = - c1;
173  if (c2 < 0) c2 = -c2;
174  }
175  if (c2 >= height) {
176  c2 = 2 * height - 2 - c2;
177  if (c1 >= height) c1 = 2 * height - 2 - c1;
178  }
179  }
180  for (int32 ch = 0; ch < num_channels; ch++) {
181  // find the values at the 4 points
182  BaseFloat p11 = original_image(r1, num_channels * c1 + ch),
183  p12 = original_image(r1, num_channels * c2 + ch),
184  p21 = original_image(r2, num_channels * c1 + ch),
185  p22 = original_image(r2, num_channels * c2 + ch);
186  (*image)(r, num_channels * c + ch) = weight_11 * p11 + weight_12 * p12 +
187  weight_21 * p21 + weight_22 * p22;
188  }
189  }
190  }
191 }
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:64
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61

◆ ApplyL2Regularization()

void ApplyL2Regularization ( const Nnet nnet,
BaseFloat  l2_regularize_scale,
Nnet delta_nnet 
)

This function is used as part of the regular training workflow, prior to UpdateNnetWithMaxChange().

For each updatable component c in the neural net, suppose it has a l2-regularization constant alpha set at the component level (see UpdatableComponent::L2Regularization()), and a learning-rate eta, then this function does (and this is not real code):

delta_nnet->c -= 2.0 * l2_regularize_scale * alpha * eta * nnet.c

The factor of -1.0 comes from the fact that we are maximizing, and we'd add the l2 regularization term (of the form ||||_2^2, i.e. squared l2 norm) in the objective function with negative sign; the factor of 2.0 comes from the derivative of the squared parameters. The factor 'l2_regularize_scale' is provided to this function, see below for an explanation.

Note: the way we do it is a little bit approximate, due to the interaction with natural gradient. The issue is that the regular gradients are multiplied by the inverse of the approximated, smoothed and factored inverse Fisher matrix, but the l2 gradients are not. This means that what we're optimizing is not exactly the (regular objective plus the L2 term)– we could view it as optimizing (regular objective plus the l2 term times the Fisher matrix)– with the proviso that the Fisher matrix has been scaled in such a way that the amount of parameter change is not affected, so this is not an issue of affecting the overall strength of l2, just an issue of the direction-wise weighting. In effect, the l2 term will be larger, relative to the gradient contribution, in directions where the Fisher matrix is large. This is probably not ideal– but it's hard to judge without experiments. Anyway the l2 effect is small enough, and the Fisher matrix sufficiently smoothed with the identity, that I doubt this makes much of a difference.

Parameters
[in]nnetThe neural net that is being trained; expected to be different from delta_nnet
[in]l2_regularize_scaleA scale on the l2 regularization. Usually this will be equal to the number of distinct examples (e.g. the number of chunks of speech– more precisely, the number of distinct 'n' values) in the minibatch, but this is multiplied by a configuration value –l2-regularize-factor passed in from the command line. The reason for making l2 proportional to the number of elements in the minibatch is that we add the parameter gradients over the minibatch (we don't average), so multiplying the l2 factor by the number of elements in the minibatch is necessary to make the amount of l2 vs. gradient contribution stay the same when we vary the minibatch size. The –l2-regularize-factor option is provided so that the calling script can correct for the effects of parallelization via model-averaging (we'd normally set this to 1/num-parallel-jobs).
[out]delta_nnetThe neural net containing the parameter updates; this is a copy of 'nnet' that is used for purposes of momentum and applying max-change values. This is what this code adds to.

Definition at line 2151 of file nnet-utils.cc.

References Component::Add(), Nnet::GetComponent(), KALDI_ASSERT, kUpdatableComponent, UpdatableComponent::L2Regularization(), UpdatableComponent::LearningRate(), Nnet::NumComponents(), and Component::Properties().

Referenced by CollapseModelConfig::CollapseModelConfig(), NnetChainTrainer::TrainInternal(), NnetTrainer::TrainInternal(), NnetChainTrainer::TrainInternalBackstitch(), and NnetTrainer::TrainInternalBackstitch().

2153  {
2154  if (l2_regularize_scale == 0.0)
2155  return;
2156  for (int32 c = 0; c < nnet.NumComponents(); c++) {
2157  const Component *src_component_in = nnet.GetComponent(c);
2158  if (src_component_in->Properties() & kUpdatableComponent) {
2159  const UpdatableComponent *src_component =
2160  dynamic_cast<const UpdatableComponent*>(src_component_in);
2161  UpdatableComponent *dest_component =
2162  dynamic_cast<UpdatableComponent*>(delta_nnet->GetComponent(c));
2163  // The following code will segfault if they aren't both updatable, which
2164  // would be a bug in the calling code.
2165  BaseFloat lrate = dest_component->LearningRate(),
2166  l2_regularize = dest_component->L2Regularization();
2167  KALDI_ASSERT(lrate >= 0 && l2_regularize >= 0);
2168  BaseFloat scale = -2.0 * l2_regularize_scale * lrate * l2_regularize;
2169  if (scale != 0.0)
2170  dest_component->Add(scale, *src_component);
2171  }
2172  }
2173 }
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ AssertGraphEqual()

bool kaldi::nnet3::AssertGraphEqual ( const std::vector< std::vector< int32 > > &  graph1,
const std::vector< std::vector< int32 > > &  graph2 
)

Definition at line 26 of file nnet-graph-test.cc.

References rnnlm::i, and rnnlm::j.

Referenced by UnitTestComputeGraphTranspose(), UnitTestFindSccs(), and UnitTestMakeSccGraph().

27  {
28  if (graph1.size() != graph2.size()) { return false; }
29  for (int32 i = 0; i < graph1.size(); ++i) {
30  if (graph1[i].size() != graph2[i].size()) { return false; }
31  for (int32 j = 0; j < graph1[i].size(); ++j) {
32  if (graph1[i][j] != graph2[i][j]) { return false; }
33  }
34  }
35  return true;
36 }

◆ AssertVectorEqual()

bool kaldi::nnet3::AssertVectorEqual ( const std::vector< int32 > &  vec1,
const std::vector< int32 > &  vec2 
)

Definition at line 38 of file nnet-graph-test.cc.

References rnnlm::i.

Referenced by UnitTestComputeTopSortOrder(), and UnitTestComputeTopSortOrder2().

39  {
40  if (vec1.size() != vec2.size()) { return false; }
41  for (int32 i = 0; i < vec1.size(); ++i) {
42  if (vec1[i] != vec2[i]) { return false; }
43  }
44  return true;
45 }

◆ BuildTestGraph()

void kaldi::nnet3::BuildTestGraph ( std::vector< std::vector< int32 > > *  graph)

Definition at line 47 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeGraphTranspose(), UnitTestFindSccs(), and UnitTestMakeSccGraph().

47  {
48  KALDI_ASSERT(graph != NULL);
49  graph->clear();
50  graph->resize(8);
51 
52  // We create the following graph for testing.
53  // 0 --> 4
54  // 1 --> 0
55  // 2 --> 1 3
56  // 3 --> 2
57  // 4 --> 1
58  // 5 --> 1 4 6
59  // 6 --> 5
60  // 7 --> 7 3 6
61  std::vector<int32> tmp;
62  tmp.resize(1); tmp[0] = 4; (*graph)[0] = tmp;
63  tmp.resize(1); tmp[0] = 0; (*graph)[1] = tmp;
64  tmp.resize(2); tmp[0] = 1; tmp[1] = 3; (*graph)[2] = tmp;
65  tmp.resize(1); tmp[0] = 2; (*graph)[3] = tmp;
66  tmp.resize(1); tmp[0] = 1; (*graph)[4] = tmp;
67  tmp.resize(3); tmp[0] = 1; tmp[1] = 4; tmp[2] = 6; (*graph)[5] = tmp;
68  tmp.resize(1); tmp[0] = 5; (*graph)[6] = tmp;
69  tmp.resize(3); tmp[0] = 7; tmp[1] = 3; tmp[2] = 6; (*graph)[7] = tmp;
70 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ BuildTestGraphTranspose()

void kaldi::nnet3::BuildTestGraphTranspose ( std::vector< std::vector< int32 > > *  graph)

Definition at line 72 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeGraphTranspose().

72  {
73  KALDI_ASSERT(graph != NULL);
74  graph->clear();
75  graph->resize(8);
76 
77  // We create the following graph for testing.
78  // 0 --> 1
79  // 1 --> 2 4 5
80  // 2 --> 3
81  // 3 --> 2 7
82  // 4 --> 0 5
83  // 5 --> 6
84  // 6 --> 5 7
85  // 7 --> 7
86  std::vector<int32> tmp;
87  tmp.resize(1); tmp[0] = 1; (*graph)[0] = tmp;
88  tmp.resize(3); tmp[0] = 2; tmp[1] = 4; tmp[2] = 5; (*graph)[1] = tmp;
89  tmp.resize(1); tmp[0] = 3; (*graph)[2] = tmp;
90  tmp.resize(2); tmp[0] = 2; tmp[1] = 7; (*graph)[3] = tmp;
91  tmp.resize(2); tmp[0] = 0; tmp[1] = 5; (*graph)[4] = tmp;
92  tmp.resize(1); tmp[0] = 6; (*graph)[5] = tmp;
93  tmp.resize(2); tmp[0] = 5; tmp[1] = 7; (*graph)[6] = tmp;
94  tmp.resize(1); tmp[0] = 7; (*graph)[7] = tmp;
95 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ BuildTestSccGraph()

void kaldi::nnet3::BuildTestSccGraph ( std::vector< std::vector< int32 > > *  scc_graph)

Definition at line 114 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeTopSortOrder(), and UnitTestMakeSccGraph().

114  {
115  KALDI_ASSERT(scc_graph != NULL);
116  scc_graph->clear();
117  scc_graph->resize(4);
118 
119  // We create the following SCC graph for testing.
120  // 0 -->
121  // 1 --> 0
122  // 2 --> 0
123  // 3 --> 1 2
124  std::vector<int32> tmp;
125  tmp.resize(0); (*scc_graph)[0] = tmp;
126  tmp.resize(1); tmp[0] = 0; (*scc_graph)[1] = tmp;
127  tmp.resize(1); tmp[0] = 0; (*scc_graph)[2] = tmp;
128  tmp.resize(2); tmp[0] = 1; tmp[1] = 2; (*scc_graph)[3] = tmp;
129 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ BuildTestSccs()

void kaldi::nnet3::BuildTestSccs ( std::vector< std::vector< int32 > > *  sccs)

Definition at line 97 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestFindSccs(), and UnitTestMakeSccGraph().

97  {
98  KALDI_ASSERT(sccs != NULL);
99  sccs->clear();
100  sccs->resize(4);
101 
102  // We create the following SCCs for testing.
103  // 0 --> 1 4 0
104  // 1 --> 3 2
105  // 2 --> 6 5
106  // 3 --> 7
107  std::vector<int32> tmp;
108  tmp.resize(3); tmp[0] = 1; tmp[1] = 4; tmp[2] = 0; (*sccs)[0] = tmp;
109  tmp.resize(2); tmp[0] = 3; tmp[1] = 2; (*sccs)[1] = tmp;
110  tmp.resize(2); tmp[0] = 6; tmp[1] = 5; (*sccs)[2] = tmp;
111  tmp.resize(1); tmp[0] = 7; (*sccs)[3] = tmp;
112 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ BuildTestTopSortOrder()

void kaldi::nnet3::BuildTestTopSortOrder ( std::vector< int32 > *  node_to_order)

Definition at line 131 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeTopSortOrder().

131  {
132  KALDI_ASSERT(node_to_order != NULL);
133  node_to_order->clear();
134  node_to_order->resize(4);
135 
136  // The topological sorting order of the above SCC graph is as follows (from
137  // our particular algorithm):
138  // 0 --> 3
139  // 1 --> 2
140  // 2 --> 1
141  // 3 --> 0
142  (*node_to_order)[0] = 3;
143  (*node_to_order)[1] = 2;
144  (*node_to_order)[2] = 1;
145  (*node_to_order)[3] = 0;
146 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ CheckComputation()

void CheckComputation ( const Nnet nnet,
const NnetComputation computation,
bool  check_rewrite = false 
)

This is a convenience interface for class ComputationChecker.

Call it with check_rewrite = true only if the computation is pre-optimization. If the computation is an 'online' computation, this function treats it specially.

Definition at line 1145 of file nnet-analyze.cc.

References ComputationChecker::Check(), CheckComputationOptions::check_rewrite, CheckComputationOnline(), NnetComputation::commands, KALDI_ERR, kGotoLabel, and NnetComputation::Print().

Referenced by CachingOptimizingCompiler::CompileViaShortcut(), and Optimize().

1147  {
1148  try {
1149  if (!computation.commands.empty() &&
1150  computation.commands.back().command_type == kGotoLabel) {
1151  // Online computations need to be treated specially.
1152  CheckComputationOnline(nnet, computation, check_rewrite);
1153  } else {
1154  CheckComputationOptions opts;
1155  opts.check_rewrite = check_rewrite;
1156  ComputationChecker checker(opts, nnet, computation);
1157  checker.Check();
1158  }
1159  } catch (...) {
1160  computation.Print(std::cerr, nnet);
1161  KALDI_ERR << "Computation check failed for computation printed above "
1162  "(actual error message is above computation)";
1163  }
1164 }
#define KALDI_ERR
Definition: kaldi-error.h:123
static void CheckComputationOnline(const Nnet &nnet, NnetComputation computation, bool check_rewrite)

◆ CheckComputationOnline()

static void kaldi::nnet3::CheckComputationOnline ( const Nnet nnet,
NnetComputation  computation,
bool  check_rewrite 
)
static

Definition at line 1118 of file nnet-analyze.cc.

References ComputationChecker::Check(), CheckComputationOptions::check_rewrite, CheckComputationOptions::check_unused_variables, NnetComputation::commands, KALDI_ASSERT, kDeallocMatrix, kGotoLabel, kSwapMatrix, and kaldi::swap().

Referenced by CheckComputation().

1120  {
1121  int32 num_commands = computation.commands.size();
1122  KALDI_ASSERT(computation.commands[num_commands-1].command_type == kGotoLabel);
1123  for (int32 c = num_commands - 2;
1124  c >= 0 && computation.commands[c].command_type == kSwapMatrix;
1125  c--) {
1126  // this command can be interpreted as "initialize matrix referred to by
1127  // c.arg2 with the matrix referred to by c.arg2".
1128  // Because this would be interpreted by the analysis code as initializing a
1129  // matrix that has already been initialized, we turn this into a command
1130  // that just deallocates the matrix in c.arg2. [note: all these indexes
1131  // are actually submatrix indexes].
1132  computation.commands[c].command_type = kDeallocMatrix;
1133  std::swap(computation.commands[c].arg1, computation.commands[c].arg2);
1134  }
1135 
1136  CheckComputationOptions opts;
1137  opts.check_rewrite = check_rewrite;
1138  opts.check_unused_variables = false;
1139  // We can always do this check with online computations, since they do not
1140  // have the RemoveUnnecessaryAllocation() optimization applied.
1141  ComputationChecker checker(opts, nnet, computation);
1142  checker.Check();
1143 }
void swap(basic_filebuf< CharT, Traits > &x, basic_filebuf< CharT, Traits > &y)
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ CheckStringsApproxEqual()

bool kaldi::nnet3::CheckStringsApproxEqual ( const std::string &  a,
const std::string &  b,
int32  tolerance = 3 
)

Definition at line 39 of file nnet-component-test.cc.

References KALDI_WARN, and kaldi::StringsApproxEqual().

Referenced by TestNnetComponentAddScale(), TestNnetComponentIo(), TestNnetComponentUpdatable(), and TestNnetComponentVectorizeUnVectorize().

41  {
42  if (!StringsApproxEqual(a, b, tolerance)) {
43  KALDI_WARN << "Strings differ: " << a
44  << "\nvs.\n" << b;
45  return false;
46  } else {
47  return true;
48  }
49 }
bool StringsApproxEqual(const std::string &a, const std::string &b, int32 decimal_places_tolerance)
This function returns true when two text strings are approximately equal, and false when they are not...
Definition: text-utils.cc:335
#define KALDI_WARN
Definition: kaldi-error.h:126

◆ CollapseModel()

void CollapseModel ( const CollapseModelConfig config,
Nnet nnet 
)

This function modifies the neural net for efficiency, in a way that suitable to be done in test time.

For example, it tries to get rid of dropout, batchnorm and fixed-scale components, and to collapse subsequent affine components if doing so won't hurt speed.

Definition at line 2007 of file nnet-utils.cc.

References ModelCollapser::Collapse().

Referenced by CollapseModelConfig::CollapseModelConfig(), main(), and UnitTestNnetCompute().

2008  {
2009  ModelCollapser c(config, nnet);
2010  c.Collapse();
2011 }

◆ CompileLooped()

void CompileLooped ( const Nnet nnet,
const NnetOptimizeOptions optimize_opts,
const ComputationRequest request1,
const ComputationRequest request2,
const ComputationRequest request3,
NnetComputation computation 
)

CompileLooped() provides an internal interface for 'looped' computation.

It's usable for inference only (not training), meaning that backprop is not supported (for now, at least). CompileLooped() allows you to do the neural net computation for small chunks with increasing 't' values, and naturally cache the intermediate activations (rather than recomputing them every time you see new input data).

This function does both compilation and optimization, so it's like a combination of Compiler::CreateComputation() [nnet-compile.h] and Optimize() [nnet-optimize.h].

You provide 3 computation requests. request1 is the first computation request of an utterance (or other type of segment) that contains any required extra left context in the input. request2 and request3 are the second and third computation request, and must have exactly the same structure, except for a fixed time offset (change in 't' index) between them. This will be extrapolated to an infinite sequence of further requests (request4, request5, etc.). In practice the way it's done is that we extrapolate to a small finite number of requests (like 10), and then attempt to identify a common structure in the computation where, after processing, as an example, the 3nd computation request, the active variables can be identified with those present at, say, the 7th computation request, and we then cut and splice the computation together at this points, like making a tape loop, by adding a goto statement that jumps from the end of the 7th computation request to the end of the 3rd computation request. We also have to identify the variables with each other (merge variables).

That's done in the optimization code.

Definition at line 329 of file nnet-compile-looped.cc.

References CompileLoopedInternal(), Timer::Elapsed(), KALDI_ERR, KALDI_LOG, and KALDI_VLOG.

Referenced by DecodableNnetSimpleLoopedInfo::Init(), and UnitTestNnetCompileLooped().

334  {
335  int32 num_requests1 = 5, factor = 2, max_requests = 100,
336  num_requests;
337 
338  Timer timer;
339 
340  for (num_requests = num_requests1; num_requests <= max_requests;
341  num_requests *= factor) {
342  if (CompileLoopedInternal(nnet, optimize_opts,
343  request1, request2, request3,
344  num_requests, computation)) {
345  KALDI_LOG << "Spent " << timer.Elapsed()
346  << " seconds in looped compilation.";
347  return;
348  } else {
349  KALDI_VLOG(2) << "Looped compilation failed with "
350  << num_requests << " requests, trying "
351  << (num_requests * factor);
352  }
353  }
354  KALDI_ERR << "Looped compilation failed with "
355  << (num_requests/factor) << " requests, which "
356  << "we expect should be enough... something "
357  << "went wrong.";
358 }
static bool CompileLoopedInternal(const Nnet &nnet, NnetOptimizeOptions optimize_opts, const ComputationRequest &request1, const ComputationRequest &request2, const ComputationRequest &request3, int32 num_requests, NnetComputation *computation)
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_VLOG(v)
Definition: kaldi-error.h:132
#define KALDI_LOG
Definition: kaldi-error.h:129

◆ CompileLoopedInternal()

static bool kaldi::nnet3::CompileLoopedInternal ( const Nnet nnet,
NnetOptimizeOptions  optimize_opts,
const ComputationRequest request1,
const ComputationRequest request2,
const ComputationRequest request3,
int32  num_requests,
NnetComputation computation 
)
static

Definition at line 284 of file nnet-compile-looped.cc.

References NnetComputation::commands, Compiler::CreateComputation(), ExtrapolateComputationRequest(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, KALDI_LOG, kGotoLabel, MaxOutputTimeInRequest(), Optimize(), NnetOptimizeOptions::optimize_looped_computation, and ComputationRequest::Print().

Referenced by CompileLooped().

291  {
292 
293  KALDI_ASSERT(num_requests >= 3);
294  std::vector<ComputationRequest> extra_requests(num_requests - 3);
295  const ComputationRequest *prev_request = &request2;
296  const ComputationRequest *cur_request = &request3;
297  for (int32 i = 0; i < num_requests - 3; i++) {
298  if (!ExtrapolateComputationRequest(*prev_request, *cur_request,
299  &(extra_requests[i]))) {
300  KALDI_LOG << "prev_request is:";
301  prev_request->Print(std::cerr);
302  KALDI_LOG << "cur_request is:";
303  cur_request->Print(std::cerr);
304  KALDI_ERR << "Computation requests do not have the right relationship";
305  }
306  prev_request = cur_request;
307  cur_request = &(extra_requests[i]);
308  }
309 
310  std::vector<const ComputationRequest*> requests;
311  requests.push_back(&request1);
312  requests.push_back(&request2);
313  requests.push_back(&request3);
314  for (int32 i = 0; i < num_requests - 3; i++)
315  requests.push_back(&(extra_requests[i]));
316  Compiler compiler(requests, nnet);
317  CompilerOptions compiler_opts;
318  compiler.CreateComputation(compiler_opts, computation);
319  optimize_opts.optimize_looped_computation = true;
320 
321  int32 dont_really_care = MaxOutputTimeInRequest(request3);
322  Optimize(optimize_opts, nnet,
323  dont_really_care, computation);
324 
325  return computation->commands.size() != 0 &&
326  computation->commands.back().command_type == kGotoLabel;
327 }
static bool ExtrapolateComputationRequest(const ComputationRequest &request1, const ComputationRequest &request2, ComputationRequest *request3)
int32 MaxOutputTimeInRequest(const ComputationRequest &request)
#define KALDI_ERR
Definition: kaldi-error.h:123
void Optimize(const NnetOptimizeOptions &config, const Nnet &nnet, int32 max_output_time_in_request, NnetComputation *computation)
This is the top-level function for optimizing a computation.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
#define KALDI_LOG
Definition: kaldi-error.h:129

◆ ComponentDotProducts()

void ComponentDotProducts ( const Nnet nnet1,
const Nnet nnet2,
VectorBase< BaseFloat > *  dot_prod 
)

Returns dot products between two networks of the same structure (calls the DotProduct functions of the Updatable components and fill in the output vector).

Definition at line 211 of file nnet-utils.cc.

References VectorBase< Real >::Data(), VectorBase< Real >::Dim(), UpdatableComponent::DotProduct(), Nnet::GetComponent(), KALDI_ASSERT, kUpdatableComponent, Nnet::NumComponents(), and Component::Properties().

Referenced by main().

213  {
214  KALDI_ASSERT(nnet1.NumComponents() == nnet2.NumComponents());
215  int32 updatable_c = 0;
216  for (int32 c = 0; c < nnet1.NumComponents(); c++) {
217  const Component *comp1 = nnet1.GetComponent(c),
218  *comp2 = nnet2.GetComponent(c);
219  if (comp1->Properties() & kUpdatableComponent) {
220  const UpdatableComponent
221  *u_comp1 = dynamic_cast<const UpdatableComponent*>(comp1),
222  *u_comp2 = dynamic_cast<const UpdatableComponent*>(comp2);
223  KALDI_ASSERT(u_comp1 != NULL && u_comp2 != NULL);
224  dot_prod->Data()[updatable_c] = u_comp1->DotProduct(*u_comp2);
225  updatable_c++;
226  }
227  }
228  KALDI_ASSERT(updatable_c == dot_prod->Dim());
229 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeAccuracy()

void ComputeAccuracy ( const GeneralMatrix supervision,
const CuMatrixBase< BaseFloat > &  nnet_output,
BaseFloat tot_weight,
BaseFloat tot_accuracy,
VectorBase< BaseFloat > *  tot_weight_vec = NULL,
VectorBase< BaseFloat > *  tot_accuracy_vec = NULL 
)

This function computes the frame accuracy for this minibatch.

It interprets the supervision information in "supervision" as labels or soft labels; it picks the maximum element in each row and treats that as the label for purposes of computing the accuracy (in situations where you would care about the accuracy, there will normally be just one nonzero label). The hypothesized labels are computed by taking the neural net output (supplied as a CuMatrix), and finding the maximum element in each row. See also the function ComputeObjectiveFunction, declared in nnet-training.h.

Parameters
[in]supervisionThe supervision information (no elements may be negative); only the maximum in each row matters (although we expect that usually there will be just one nonzero element in each row); and the sum of each row is interpreted as a weighting factor (although we expect that this sum will usually be one).
[in]nnet_outputThe neural net output must have the same dimensions as the supervision. Only the index of the maximum value in each row matters. Ties will be broken in an unspecified way.
[out]tot_weightThe sum of the values in the supervision matrix
[out]tot_accuracyThe total accuracy, equal to the sum over all row indexes r such that the maximum column index of row r of supervision and nnet_output is the same, of the sum of the r'th row of supervision (i.e. the row's weight).
[out]tot_weight_vecIf non-NULL, we write to this location the counts per-class in the supervision matrix. This is expected to have the same dimension as the corresponding output in the network.
[out]tot_accuracy_vecIf non-NULL, we write to this location the accuracy per-class. For index j, the value is equal to the sum over all row indexes r such that the maximum column index of row r of supervision is j and nnet_output is also j, of the sum of the r'th row of supervision (i.e. the row's weight)

Definition at line 206 of file nnet-diagnostics.cc.

References CuArrayBase< T >::CopyToVec(), VectorBase< Real >::Dim(), CuMatrixBase< Real >::FindRowMaxId(), GeneralMatrix::GetFullMatrix(), GeneralMatrix::GetMatrix(), GeneralMatrix::GetSparseMatrix(), KALDI_ASSERT, KALDI_ERR, kaldi::kCompressedMatrix, kaldi::kFullMatrix, kaldi::kSparseMatrix, SparseVector< Real >::Max(), VectorBase< Real >::Max(), CuMatrixBase< Real >::NumCols(), GeneralMatrix::NumCols(), CuMatrixBase< Real >::NumRows(), GeneralMatrix::NumRows(), SparseMatrix< Real >::Row(), VectorBase< Real >::Set(), SparseVector< Real >::Sum(), VectorBase< Real >::Sum(), and GeneralMatrix::Type().

Referenced by NnetComputeProb::ProcessOutputs().

211  {
212  int32 num_rows = nnet_output.NumRows(),
213  num_cols = nnet_output.NumCols();
214  KALDI_ASSERT(supervision.NumRows() == num_rows &&
215  supervision.NumCols() == num_cols);
216 
217  if (tot_accuracy_vec || tot_weight_vec)
218  KALDI_ASSERT(tot_accuracy_vec && tot_weight_vec &&
219  tot_accuracy_vec->Dim() == num_cols &&
220  tot_weight_vec->Dim() == num_cols);
221  if (tot_accuracy_vec) tot_accuracy_vec->Set(0.0);
222  if (tot_weight_vec) tot_weight_vec->Set(0.0);
223 
224  CuArray<int32> best_index(num_rows);
225  nnet_output.FindRowMaxId(&best_index);
226  std::vector<int32> best_index_cpu;
227  // wasteful copy, but doesn't dominate.
228  best_index.CopyToVec(&best_index_cpu);
229 
230 
231  double tot_weight = 0.0,
232  tot_accuracy = 0.0;
233 
234  // note: we expect that in most cases where this code is called,
235  // supervision.Type() will be kSparseMatrix.
236  switch (supervision.Type()) {
237  case kCompressedMatrix: {
238  Matrix<BaseFloat> mat;
239  supervision.GetMatrix(&mat);
240  for (int32 r = 0; r < num_rows; r++) {
241  SubVector<BaseFloat> vec(mat, r);
242  BaseFloat row_sum = vec.Sum();
243  int32 best_index;
244  vec.Max(&best_index); // discard max value.
245  tot_weight += row_sum;
246  if (tot_weight_vec)
247  (*tot_weight_vec)(best_index) += row_sum;
248  if (best_index == best_index_cpu[r]) {
249  tot_accuracy += row_sum;
250  if (tot_accuracy_vec)
251  (*tot_accuracy_vec)(best_index) += row_sum;
252  }
253  }
254  break;
255  }
256  case kFullMatrix: {
257  const Matrix<BaseFloat> &mat = supervision.GetFullMatrix();
258  for (int32 r = 0; r < num_rows; r++) {
259  SubVector<BaseFloat> vec(mat, r);
260  BaseFloat row_sum = vec.Sum();
261  int32 best_index;
262  vec.Max(&best_index); // discard max value.
263  tot_weight += row_sum;
264  if (tot_weight_vec)
265  (*tot_weight_vec)(best_index) += row_sum;
266  if (best_index == best_index_cpu[r]) {
267  tot_accuracy += row_sum;
268  if (tot_accuracy_vec)
269  (*tot_accuracy_vec)(best_index) += row_sum;
270  }
271  }
272  break;
273  }
274  case kSparseMatrix: {
275  const SparseMatrix<BaseFloat> &smat = supervision.GetSparseMatrix();
276  for (int32 r = 0; r < num_rows; r++) {
277  const SparseVector<BaseFloat> &row = smat.Row(r);
278  BaseFloat row_sum = row.Sum();
279  int32 best_index;
280  row.Max(&best_index);
281  KALDI_ASSERT(best_index < num_cols);
282  tot_weight += row_sum;
283  if (tot_weight_vec)
284  (*tot_weight_vec)(best_index) += row_sum;
285  if (best_index == best_index_cpu[r]) {
286  tot_accuracy += row_sum;
287  if (tot_accuracy_vec)
288  (*tot_accuracy_vec)(best_index) += row_sum;
289  }
290  }
291  break;
292  }
293  default: KALDI_ERR << "Bad general-matrix type.";
294  }
295  *tot_weight_out = tot_weight;
296  *tot_accuracy_out = tot_accuracy;
297 }
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeCommandAttributes()

void ComputeCommandAttributes ( const Nnet nnet,
const NnetComputation computation,
const ComputationVariables vars,
std::vector< CommandAttributes > *  attributes 
)

Definition at line 284 of file nnet-analyze.cc.

References NnetComputation::Command::arg1, NnetComputation::Command::arg2, NnetComputation::Command::arg3, NnetComputation::Command::arg4, NnetComputation::Command::arg5, NnetComputation::Command::arg6, NnetComputation::Command::command_type, NnetComputation::commands, count, Nnet::GetComponent(), CommandAttributes::has_side_effects, rnnlm::i, NnetComputation::indexes, NnetComputation::indexes_multi, IndexesMultiToSubmatrixIndexes(), kAcceptInput, kAddRowRanges, kAddRows, kAddRowsMulti, kAddToRowsMulti, KALDI_ERR, kAllocMatrix, kBackprop, kBackpropAdds, kBackpropNoModelUpdate, kCompressMatrix, kCopyRows, kCopyRowsMulti, kCopyToRowsMulti, kDeallocMatrix, kDecompressMatrix, kGotoLabel, kMatrixAdd, kMatrixCopy, kNoOperation, kNoOperationLabel, kNoOperationMarker, kNoOperationPermanent, kPropagate, kPropagateAdds, kProvideOutput, kReadAccess, kReadWriteAccess, kSetConst, kSwapMatrix, kUpdatableComponent, kWriteAccess, CommandAttributes::matrices_read, CommandAttributes::matrices_written, Component::Properties(), ComputationVariables::RecordAccessForSubmatrix(), kaldi::SortAndUniq(), CommandAttributes::submatrices_read, CommandAttributes::submatrices_written, CommandAttributes::variables_read, and CommandAttributes::variables_written.

Referenced by NnetComputer::Init(), Analyzer::Init(), and MoveSizingCommands().

288  {
289  int32 num_commands = computation.commands.size();
290  attributes->clear();
291  attributes->resize(num_commands);
292  for (int32 command_index = 0; command_index < num_commands; command_index++) {
293  const NnetComputation::Command &c = computation.commands[command_index];
294  CommandAttributes &attr = (*attributes)[command_index];
295  switch (c.command_type) {
296  case kAllocMatrix:
297  case kDeallocMatrix:
298  case kSwapMatrix:
299  break; // the commands above leave the matrix undefined.
300  case kSetConst:
301  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
302  break;
303  case kPropagate:
304  vars.RecordAccessForSubmatrix(c.arg3, kReadAccess, &attr);
305  if (nnet.GetComponent(c.arg1)->Properties() & kPropagateAdds)
306  vars.RecordAccessForSubmatrix(c.arg4, kReadWriteAccess, &attr);
307  else
308  vars.RecordAccessForSubmatrix(c.arg4, kWriteAccess, &attr);
309  break;
310  case kBackprop:
312  vars.RecordAccessForSubmatrix(c.arg3, kReadAccess, &attr);
313  vars.RecordAccessForSubmatrix(c.arg4, kReadAccess, &attr);
314  vars.RecordAccessForSubmatrix(c.arg5, kReadAccess, &attr);
315  if (nnet.GetComponent(c.arg1)->Properties() & kBackpropAdds)
316  vars.RecordAccessForSubmatrix(c.arg6, kReadWriteAccess, &attr);
317  else
318  vars.RecordAccessForSubmatrix(c.arg6, kWriteAccess, &attr);
319  if (c.command_type == kBackprop &&
320  nnet.GetComponent(c.arg1)->Properties() & kUpdatableComponent)
321  attr.has_side_effects = true;
322  break;
323  case kMatrixCopy:
324  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
325  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
326  break;
327  case kMatrixAdd:
328  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
329  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
330  break;
331  case kAddRows:
332  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
333  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
334  break;
335  case kCopyRows: {
336  const std::vector<int32> &indexes = computation.indexes[c.arg3];
337  // if there are -1's in "indexes", then the result of the operation
338  // will depend on the initial value of the matrix, so it's
339  // a "rw" operation, not a "write" operation.
340  if (std::count(indexes.begin(), indexes.end(), -1) > 0)
341  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
342  else
343  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
344  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
345  break;
346  }
347  case kAddRowsMulti: {
348  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
349  std::vector<int32> submatrix_indexes;
350  IndexesMultiToSubmatrixIndexes(computation.indexes_multi[c.arg2],
351  &submatrix_indexes);
352  for (size_t i = 0; i < submatrix_indexes.size(); i++)
353  vars.RecordAccessForSubmatrix(submatrix_indexes[i],
354  kReadAccess, &attr);
355  break;
356  }
357  case kCopyRowsMulti: {
358  std::vector<int32> submatrix_indexes;
359  IndexesMultiToSubmatrixIndexes(computation.indexes_multi[c.arg2],
360  &submatrix_indexes);
361  // note: the CopyRows command assigns zero in cases where
362  // there is no source for some row
363  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
364  for (size_t i = 0; i < submatrix_indexes.size(); i++)
365  vars.RecordAccessForSubmatrix(submatrix_indexes[i],
366  kReadAccess, &attr);
367  break;
368  }
369  case kAddToRowsMulti:
370  case kCopyToRowsMulti: {
371  vars.RecordAccessForSubmatrix(c.arg1, kReadAccess, &attr);
372  // if the submatrixes we're writing to (in kCopyToRowsMulti) had all
373  // rows covered, it would be a pure write operation.
374  std::vector<int32> submatrix_indexes;
375  IndexesMultiToSubmatrixIndexes(computation.indexes_multi[c.arg2],
376  &submatrix_indexes);
377  for (size_t i = 0; i < submatrix_indexes.size(); i++)
378  vars.RecordAccessForSubmatrix(submatrix_indexes[i], kReadWriteAccess,
379  &attr);
380  break;
381  }
382  case kAddRowRanges: {
383  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
384  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
385  break;
386  }
387  case kCompressMatrix: {
388  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
389  break;
390  }
391  case kDecompressMatrix: {
392  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
393  break;
394  }
395  case kAcceptInput: {
396  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
397  break;
398  }
399  case kProvideOutput: {
400  vars.RecordAccessForSubmatrix(c.arg1, kReadAccess, &attr);
401  break;
402  }
403  case kNoOperation:
405  case kNoOperationMarker:
406  case kNoOperationLabel:
407  case kGotoLabel:
408  break;
409  default:
410  KALDI_ERR << "Unknown command type.";
411  }
412  SortAndUniq(&attr.variables_read);
413  SortAndUniq(&attr.variables_written);
414  SortAndUniq(&attr.submatrices_read);
415  SortAndUniq(&attr.submatrices_written);
416  SortAndUniq(&attr.matrices_read);
417  SortAndUniq(&attr.matrices_written);
418  }
419 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq&#39;s (removes duplicates) from a vector.
Definition: stl-utils.h:39
static void IndexesMultiToSubmatrixIndexes(const std::vector< std::pair< int32, int32 > > &indexes_multi, std::vector< int32 > *submatrix_indexes)
given a vector of pairs from computation.indexes_multi_indexes containing paris (submatrix-index, row-index), this function outputs to "submatrix_indexes" all (unique) submatrix indexes that appear; and it outputs to "contains_null_marker" true if the pair (-1, -1) appears anywhere in indexes_multi, and false otherwise.
const size_t count
#define KALDI_ERR
Definition: kaldi-error.h:123

◆ ComputeCommandPairs()

static void kaldi::nnet3::ComputeCommandPairs ( const std::pair< std::vector< int32 >, std::vector< int32 > > &  lists,
std::vector< std::pair< int32, int32 > > *  pairs 
)
static

Definition at line 328 of file nnet-optimize.cc.

References kaldi::CopyVectorToSet(), rnnlm::d, and KALDI_PARANOID_ASSERT.

Referenced by RemoveUnnecessaryAllocation().

330  {
331  std::vector<int32> d_list = lists.first;
332 
333  std::set<int32> a_set;
334  CopyVectorToSet(lists.second, &a_set);
335 
336  std::vector<int32>::reverse_iterator iter = d_list.rbegin(),
337  end = d_list.rend();
338 
339  // from the latest to the earliest deallocation command...
340  for (; iter != end; ++iter) {
341  int32 d = *iter;
342  std::set<int32>::iterator a_iter = a_set.upper_bound(d);
343  // a_iter is an iterator to the first element a of the set 'a_set' such
344  // that a > d, or a_set.end() if no such element exists.
345  if (a_iter == a_set.end())
346  continue; // we will output no pair for this d.
347  int32 a = *a_iter;
348  KALDI_PARANOID_ASSERT(a > d); // or code error
349  a_set.erase(a_iter); // remove this a from 'a_set' so it doesn't get used
350  // twice
351  pairs->push_back(std::pair<int32,int32>(d, a));
352  }
353 }
void CopyVectorToSet(const std::vector< A > &v, std::set< A > *s)
Copies the contents of a vector to a set.
Definition: stl-utils.h:174
#define KALDI_PARANOID_ASSERT(cond)
Definition: kaldi-error.h:178

◆ ComputeComputationGraph()

void kaldi::nnet3::ComputeComputationGraph ( const Nnet nnet,
const ComputationRequest request,
ComputationGraph graph 
)

Definition at line 1171 of file nnet-computation-graph.cc.

References kaldi::nnet3::computation_graph::AddInputToGraph(), kaldi::nnet3::computation_graph::AddOutputToGraph(), ComputationGraph::cindexes, NetworkNode::component_index, ComputationGraph::dependencies, NetworkNode::descriptor, ComputationGraph::GetCindexId(), Nnet::GetComponent(), Descriptor::GetDependencies(), Component::GetInputIndexes(), Nnet::GetNode(), rnnlm::i, ComputationGraph::is_input, KALDI_ASSERT, KALDI_ERR, kComponent, kDescriptor, kDimRange, kInput, ComputationRequest::misc_info, rnnlm::n, NetworkNode::node_index, NetworkNode::node_type, kaldi::SortAndUniq(), and NetworkNode::u.

1173  {
1174  using namespace computation_graph;
1175  // make sure graph is empty at the start.
1176  KALDI_ASSERT(graph->cindexes.empty());
1177 
1178  AddInputToGraph(request, nnet, graph);
1179  AddOutputToGraph(request, nnet, graph);
1180 
1181  // queue of cindex_ids to process.
1182  std::vector<int32> queue(graph->cindexes.size());
1183  for (int32 i = 0; i < graph->cindexes.size(); i++)
1184  queue.push_back(i);
1185 
1186  while (!queue.empty()) {
1187  int32 cindex_id = queue.back();
1188  queue.pop_back();
1189  if (static_cast<int32>(graph->dependencies.size()) <= cindex_id)
1190  graph->dependencies.resize(cindex_id + 1);
1191 
1192  if (graph->is_input[cindex_id])
1193  continue;
1194  Cindex cindex = graph->cindexes[cindex_id];
1195 
1196  // find the dependencies of this cindex.
1197  int32 n = cindex.first;
1198  const Index &index = cindex.second;
1199  const NetworkNode &node = nnet.GetNode(n);
1200 
1201  std::vector<Cindex> input_cindexes;
1202 
1203  // the following switch statement sets up "input_cindexes".
1204  switch (node.node_type) {
1205  case kDescriptor: {
1206  // desc describes how this node obtains its input from other nodes.
1207  const Descriptor &desc = node.descriptor;
1208  desc.GetDependencies(index, &input_cindexes);
1209  break;
1210  }
1211  case kComponent: {
1212  int32 c = node.u.component_index;
1213  const Component *component = nnet.GetComponent(c);
1214  std::vector<Index> input_indexes;
1215  component->GetInputIndexes(request.misc_info, index,
1216  &input_indexes);
1217  // each Component node should be preceded by a node that describes its
1218  // input, of type kDescriptor
1219  KALDI_ASSERT(nnet.GetNode(n-1).node_type ==
1220  kDescriptor);
1221 
1222  input_cindexes.resize(input_indexes.size());
1223  for (size_t i = 0; i < input_indexes.size(); i++) {
1224  input_cindexes[i].first = n - 1; // preceding node.
1225  input_cindexes[i].second = input_indexes[i];
1226  }
1227  break;
1228  }
1229  case kDimRange: {
1230  input_cindexes.resize(1);
1231  input_cindexes[0] = Cindex(node.u.node_index, index);
1232  break;
1233  }
1234  case kInput: default:
1235  // for kInput, you should have hit the "continue" statement above.
1236  KALDI_ERR << "Invalid node type";
1237  }
1238  std::vector<int32> &this_dep = graph->dependencies[cindex_id];
1239 
1240  int32 num_dependencies = input_cindexes.size();
1241  this_dep.resize(num_dependencies);
1242  for (size_t i = 0; i < num_dependencies; i++) {
1243  bool is_input = false, is_new;
1244  int32 dep_cindex_id = graph->GetCindexId(input_cindexes[i],
1245  is_input, &is_new);
1246  this_dep[i] = dep_cindex_id;
1247  if (is_new)
1248  queue.push_back(dep_cindex_id);
1249  }
1250 
1251  // remove duplicates of dependencies.
1252  SortAndUniq(&this_dep);
1253  }
1254 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq&#39;s (removes duplicates) from a vector.
Definition: stl-utils.h:39
void AddInputToGraph(const ComputationRequest &request, const Nnet &nnet, ComputationGraph *graph)
std::pair< int32, Index > Cindex
Definition: nnet-common.h:115
void AddOutputToGraph(const ComputationRequest &request, const Nnet &nnet, ComputationGraph *graph)
struct rnnlm::@11::@12 n
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeComputationPhases()

void ComputeComputationPhases ( const Nnet nnet,
const ComputationGraph computation_graph,
std::vector< std::vector< std::vector< int32 > > > *  phases_per_segment 
)

This function divides a computation into 'phases', where a 'phase' is a collection of cindexes which can (as far as the computation graph is concerned) all be computed at the same time, and depend only on cindexes previously computed in earlier phases.

So the phases are an ordering of the Cindexes in the computation, but an ordering that depends on graph-theoretic considerations only, and not practical concerns like whether the cindexes belong to the same node [for that, see the notion of steps].

Parameters
[in]nnetThe neural network this computation is for
[in]graphThe computation graph that we're computing phases for.
[out]phases_per_segmentThe phases, listed separately for each segment of the computation [there will be just one segment in the normal case, more in the online-recognition case]. Consider just one segment for now. Suppose the computation can be completed in 20 phases, then (*phases)[0].size() will be 20 at exit, and (*phases)[0][0] will be a sorted list of cindex_ids. that belong to the first phase, and so on. (Remember, a cindex_id is an index into graph->cindexes; it compactly identifies a cindex.) The sets represented by the int32's in 'phases_per_segment' will be disjoint and will cover all elements in [0 .. computation.cindexes.size() - 1].

Note: we assume you have called PruneComputationGraph() before this function. Even so, this function will be crash if the computation cannot actually be computed– there are some mal-formed computations where you can build the computation graph but not the ordering of cindexes because there are dependencies forward and backward in time that intertwine.

Definition at line 1425 of file nnet-computation-graph.cc.

References ComputationGraph::cindexes, ComputeComputationPhasesForEpoch(), kaldi::nnet3::computation_graph::ComputeDependenciesSubset(), kaldi::nnet3::computation_graph::ComputeEpochInfo(), ComputeGraphTranspose(), KALDI_ASSERT, ComputationGraph::segment_ends, and SumVectorSizes().

Referenced by Compiler::CreateComputation().

1428  {
1429  using namespace computation_graph;
1430  int32 num_cindex_ids = graph.cindexes.size();
1431 
1432  std::vector<int32> cindex_id_to_segment_and_epoch;
1433  std::vector<std::vector<std::vector<int32 > > > epochs_per_segment;
1434  std::vector<bool> epoch_is_trivial;
1435  ComputeEpochInfo(nnet, graph, &cindex_id_to_segment_and_epoch,
1436  &epochs_per_segment, &epoch_is_trivial);
1437 
1438  KALDI_ASSERT(SumVectorSizes(epochs_per_segment) == num_cindex_ids);
1439 
1440  // dependencies_subset contains just the subset of dependencies
1441  // of each cindex_id, that have the same epoch index as
1442  // cindex_id itself. This will be used to correctly order
1443  // cindexes within a certain epoch (relevant for things like
1444  // LSTMs).
1445  std::vector<std::vector<int32> > dependencies_subset;
1446  ComputeDependenciesSubset(graph, cindex_id_to_segment_and_epoch,
1447  &dependencies_subset);
1448  // destroy cindex_id_to_segment_and_epoch, it's no longer needed.
1449  { std::vector<int32> temp; temp.swap(cindex_id_to_segment_and_epoch); }
1450 
1451  // depend_on_subset is a subset of the normal "depend_on" list (i.e. a list of
1452  // all cindex_ids that depend on the current cindex_id), limited to just those
1453  // cindex_ids that have the same epoch index.
1454  std::vector<std::vector<int32> > depend_on_subset;
1455  ComputeGraphTranspose(dependencies_subset, &depend_on_subset);
1456 
1457  int32 num_epoch_indexes = epoch_is_trivial.size(),
1458  num_segments = graph.segment_ends.size();
1459 
1460  // "phase_indexes" is used inside ComputeComputationPhasesForEpoch.
1461  std::vector<int32> phase_indexes(num_cindex_ids, -1);
1462 
1463  phases_per_segment->clear();
1464  phases_per_segment->resize(num_segments);
1465 
1466  for (int32 segment = 0; segment < num_segments; segment++) {
1467  phases_per_segment->reserve(50); // minimize unnecessary copies. 50 is
1468  // very arbitrarily chosen.
1469  for (int32 epoch = 0; epoch < num_epoch_indexes; epoch++)
1471  epochs_per_segment[segment][epoch],
1472  dependencies_subset,
1473  depend_on_subset,
1474  epoch_is_trivial[epoch],
1475  &phase_indexes,
1476  &((*phases_per_segment)[segment]));
1477  }
1478 
1479 
1480  // make sure everything was computable. If the next assert fails it's likely
1481  // a bug in this function or in PruneComputataionGraph.
1482  KALDI_ASSERT(SumVectorSizes(*phases_per_segment) == num_cindex_ids);
1483 }
static void ComputeEpochInfo(const Nnet &nnet, const ComputationGraph &graph, std::vector< int32 > *cindex_id_to_segment_and_epoch, std::vector< std::vector< std::vector< int32 > > > *epochs_per_segment, std::vector< bool > *epoch_is_trivial)
This function computes certain information about "epochs" of cindex_ids.
static int32 SumVectorSizes(const std::vector< std::vector< std::vector< int32 > > > &vec)
void ComputeGraphTranspose(const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *graph_transpose)
Outputs a graph in which the order of arcs is reversed.
Definition: nnet-graph.cc:63
static void ComputeComputationPhasesForEpoch(const Nnet &nnet, const ComputationGraph &graph, const std::vector< int32 > &this_epoch, const std::vector< std::vector< int32 > > &dependencies_subset, const std::vector< std::vector< int32 > > &depend_on_subset, bool epoch_is_trivial, std::vector< int32 > *phase_indexes, std::vector< std::vector< int32 > > *phases)
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
static void ComputeDependenciesSubset(const ComputationGraph &graph, const std::vector< int32 > &cindex_id_to_segment_and_epoch, std::vector< std::vector< int32 > > *dependencies_subset)
This function outputs to dependencies_subset[c], for each cindex_id c, the subset of elements d of gr...

◆ ComputeComputationPhasesForEpoch()

static void kaldi::nnet3::ComputeComputationPhasesForEpoch ( const Nnet nnet,
const ComputationGraph graph,
const std::vector< int32 > &  this_epoch,
const std::vector< std::vector< int32 > > &  dependencies_subset,
const std::vector< std::vector< int32 > > &  depend_on_subset,
bool  epoch_is_trivial,
std::vector< int32 > *  phase_indexes,
std::vector< std::vector< int32 > > *  phases 
)
inlinestatic

Definition at line 1326 of file nnet-computation-graph.cc.

References rnnlm::d, KALDI_ASSERT, KALDI_ERR, and kaldi::SortAndUniq().

Referenced by ComputeComputationPhases().

1334  {
1335  std::vector<int32> this_phase, next_phase_candidates;
1336 
1337  if (this_epoch.empty())
1338  return;
1339 
1340  if (epoch_is_trivial) { // an optimization
1341  this_phase = this_epoch;
1342  } else {
1343  // Start out with all elements of this epoch that have no
1344  // dependencies within the same epoch (i.e. those that
1345  // can be computed first).
1346  std::vector<int32>::const_iterator iter = this_epoch.begin(),
1347  end = this_epoch.end();
1348  for (; iter != end; ++iter) {
1349  int32 cindex_id = *iter;
1350  if (dependencies_subset[cindex_id].empty())
1351  this_phase.push_back(cindex_id);
1352  }
1353  }
1354 
1355  // if the next assert fails, the graph at the level of cindex_ids is not acyclic.
1356  KALDI_ASSERT(!this_phase.empty() &&
1357  "Trying to process computation with cycles");
1358 
1359  while (!this_phase.empty()) {
1360  // The next two lines are a more efficient version of:
1361  // phases->push_back(this_phase);
1362  phases->resize(phases->size() + 1);
1363  phases->back().swap(this_phase);
1364  // The next if-statement is an optimization: if for this epoch index
1365  // there is just one node, we can skip the rest of this loop. Note: if
1366  // epoch == 0, even if there is just one node, cindex_ids from
1367  // multiple nodes may be put here because of the rule that cindex_ids which
1368  // are inputs always get epoch 0. But it's still true that they
1369  // will have no dependencies, so we can still skip the code below.
1370  if (epoch_is_trivial)
1371  return;
1372 
1373  int32 cur_phase_index = phases->size() - 1;
1374 
1375  // next_phases_candidates is a list of cindexes that we should check
1376  // whether they are computable now, because one of the things they depend
1377  // on just became computable.
1378  next_phase_candidates.clear();
1379  std::vector<int32>::const_iterator this_phase_iter = phases->back().begin(),
1380  this_phase_end = phases->back().end();
1381 
1382  for (; this_phase_iter != this_phase_end; ++this_phase_iter) {
1383  int32 c = *this_phase_iter; // c is a cindex_id with phase cur_phase_index.
1384  (*phase_indexes)[c] = cur_phase_index;
1385  std::vector<int32>::const_iterator iter = depend_on_subset[c].begin(),
1386  end = depend_on_subset[c].end();
1387  for (; iter != end; ++iter) {
1388  int32 d = *iter; // cindex_id that depends on c.
1389  next_phase_candidates.push_back(d);
1390  }
1391  }
1392  SortAndUniq(&next_phase_candidates);
1393  // note, at this point 'this_phase' will be the empty vector [see the 'swap'
1394  // above].
1395  this_phase.reserve(next_phase_candidates.size());
1396  // now check the candidates that might be in the next phase, and put any
1397  // members that we are currently able to compute into "this_phase".
1398  std::vector<int32>::const_iterator iter = next_phase_candidates.begin(),
1399  end = next_phase_candidates.end();
1400  for (; iter != end; ++iter) {
1401  int32 c = *iter;
1402  std::vector<int32>::const_iterator
1403  dep_iter = dependencies_subset[c].begin(),
1404  dep_end = dependencies_subset[c].end();
1405  for (; dep_iter != dep_end; ++dep_iter) {
1406  int32 d = *dep_iter; // d is cindex_id that c depends on.
1407  if ((*phase_indexes)[d] < 0) // we can't compute c yet because something we depend
1408  break; // on has not yet been computed.
1409  }
1410  if (dep_iter == dep_end) {
1411  // we reached the end and did not break -> all dependencies satisfied
1412  this_phase.push_back(c);
1413  }
1414  }
1415  if (!next_phase_candidates.empty() && this_phase.empty()) {
1416  // this should have been caught earlier so likely a code error rather than
1417  // a problem with user input.
1418  KALDI_ERR << "Your model has a type of recurrence that cannot be computed. "
1419  << "E.g. if x[t] depends on both x[t+1] and x[t-1]... no order "
1420  << "of computation will work.";
1421  }
1422  }
1423 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq&#39;s (removes duplicates) from a vector.
Definition: stl-utils.h:39
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeExampleComputationRequestSimple()

void ComputeExampleComputationRequestSimple ( const Nnet nnet,
ComputationRequest request,
std::vector< Matrix< BaseFloat > > *  inputs 
)

This function computes an example computation request, for testing purposes.

The "Simple" in the name means that it currently only supports neural nets that satisfy IsSimple(nnet) (defined in nnet-utils.h). If there are 2 inputs, the "input" will be first, followed by "ivector".

In order to expand the range of things you can test with this (mainly to stop crashes with statistics-pooling/statistics-extraction components), this function always generates computation-requests where at least 3 successive frames of input are requested.

Definition at line 1338 of file nnet-test-utils.cc.

References ComputeSimpleNnetContext(), Nnet::InputDim(), ComputationRequest::inputs, IsSimpleNnet(), KALDI_ASSERT, rnnlm::n, ComputationRequest::need_model_derivative, ComputationRequest::outputs, kaldi::Rand(), and ComputationRequest::store_component_stats.

Referenced by NnetGenerationOptions::NnetGenerationOptions(), UnitTestNnetAnalyze(), UnitTestNnetCompile(), UnitTestNnetCompileMulti(), UnitTestNnetCompute(), UnitTestNnetInputDerivatives(), UnitTestNnetModelDerivatives(), and UnitTestNnetOptimizeWithOptions().

1341  {
1342  KALDI_ASSERT(IsSimpleNnet(nnet));
1343 
1344  int32 left_context, right_context;
1345  ComputeSimpleNnetContext(nnet, &left_context, &right_context);
1346 
1347  int32 num_output_frames = 1 + Rand() % 10,
1348  output_start_frame = Rand() % 10,
1349  num_examples = 1 + Rand() % 4,
1350  output_end_frame = output_start_frame + num_output_frames,
1351  input_start_frame = output_start_frame - left_context - (Rand() % 3),
1352  input_end_frame = output_end_frame + right_context + (Rand() % 3),
1353  n_offset = Rand() % 2;
1354  bool need_deriv = (Rand() % 2 == 0);
1355  // make sure there are at least 3 frames of input available. this makes a
1356  // difference for our tests of statistics-pooling and statistics-extraction
1357  // component.
1358  if (input_end_frame < input_start_frame + 3)
1359  input_end_frame = input_start_frame + 3;
1360 
1361  request->inputs.clear();
1362  request->outputs.clear();
1363  inputs->clear();
1364 
1365  std::vector<Index> input_indexes, ivector_indexes, output_indexes;
1366  for (int32 n = n_offset; n < n_offset + num_examples; n++) {
1367  for (int32 t = input_start_frame; t < input_end_frame; t++)
1368  input_indexes.push_back(Index(n, t, 0));
1369  for (int32 t = output_start_frame; t < output_end_frame; t++)
1370  output_indexes.push_back(Index(n, t, 0));
1371  ivector_indexes.push_back(Index(n, 0, 0));
1372  }
1373  request->outputs.push_back(IoSpecification("output", output_indexes));
1374  if (need_deriv || (Rand() % 3 == 0))
1375  request->outputs.back().has_deriv = true;
1376  request->inputs.push_back(IoSpecification("input", input_indexes));
1377  if (need_deriv && (Rand() % 2 == 0))
1378  request->inputs.back().has_deriv = true;
1379  int32 input_dim = nnet.InputDim("input");
1380  KALDI_ASSERT(input_dim > 0);
1381  inputs->push_back(
1382  Matrix<BaseFloat>((input_end_frame - input_start_frame) * num_examples,
1383  input_dim));
1384  inputs->back().SetRandn();
1385  int32 ivector_dim = nnet.InputDim("ivector"); // may not exist.
1386  if (ivector_dim != -1) {
1387  request->inputs.push_back(IoSpecification("ivector", ivector_indexes));
1388  inputs->push_back(Matrix<BaseFloat>(num_examples, ivector_dim));
1389  inputs->back().SetRandn();
1390  if (need_deriv && (Rand() % 2 == 0))
1391  request->inputs.back().has_deriv = true;
1392  }
1393  if (Rand() % 2 == 0)
1394  request->need_model_derivative = need_deriv;
1395  if (Rand() % 2 == 0)
1396  request->store_component_stats = true;
1397 }
void ComputeSimpleNnetContext(const Nnet &nnet, int32 *left_context, int32 *right_context)
ComputeSimpleNnetContext computes the left-context and right-context of a nnet.
Definition: nnet-utils.cc:146
struct rnnlm::@11::@12 n
int Rand(struct RandomState *state)
Definition: kaldi-math.cc:44
bool IsSimpleNnet(const Nnet &nnet)
This function returns true if the nnet has the following properties: It has an output called "output"...
Definition: nnet-utils.cc:52
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeGraphTranspose()

void ComputeGraphTranspose ( const std::vector< std::vector< int32 > > &  graph,
std::vector< std::vector< int32 > > *  graph_transpose 
)

Outputs a graph in which the order of arcs is reversed.

Definition at line 63 of file nnet-graph.cc.

References rnnlm::n.

Referenced by ComputeComputationPhases(), FindOrphanNodes(), and UnitTestComputeGraphTranspose().

64  {
65  int32 size = graph.size();
66  graph_transpose->clear();
67  graph_transpose->resize(size);
68  for (int32 n = 0; n < size; n++) {
69  const std::vector<int32> &nodes = graph[n];
70  std::vector<int32>::const_iterator iter = nodes.begin(), end = nodes.end();
71  for (; iter != end; ++iter) {
72  int32 dest = *iter;
73  (*graph_transpose)[dest].push_back(n);
74  }
75  }
76 }
struct rnnlm::@11::@12 n

◆ ComputeMatrixAccesses()

void ComputeMatrixAccesses ( const Nnet nnet,
const NnetComputation computation,
const ComputationVariables variables,
const std::vector< CommandAttributes > &  command_attributes,
std::vector< MatrixAccesses > *  matrix_accesses 
)

This function organizes information in the CommandAttributes in a way that is convenient to access per matrix.

See the declaration of struct MatrixAccesses for the output format; the output "matrix_accesses" is indexed by matrix index (the same index as computation.matrices).

Definition at line 467 of file nnet-analyze.cc.

References NnetComputation::Command::arg1, NnetComputation::Command::arg2, NnetComputation::Command::command_type, NnetComputation::commands, kaldi::IsSortedAndUniq(), NnetComputation::IsWholeMatrix(), kAcceptInput, KALDI_ASSERT, KALDI_ERR, kAllocMatrix, kDeallocMatrix, kProvideOutput, kReadAccess, kReadWriteAccess, kSwapMatrix, kWriteAccess, NnetComputation::matrices, CommandAttributes::matrices_read, CommandAttributes::matrices_written, kaldi::SortAndUniq(), and NnetComputation::submatrices.

Referenced by Analyzer::Init(), MatrixAccesses::MatrixAccesses(), and MoveSizingCommands().

472  {
473  int32 num_matrices = computation.matrices.size(),
474  num_commands = command_attributes.size();
475  matrix_accesses->clear();
476  matrix_accesses->resize(num_matrices);
477  for (int32 c = 0; c < num_commands; c++) {
478  const CommandAttributes &attr = command_attributes[c];
479  KALDI_ASSERT(IsSortedAndUniq(attr.matrices_read));
480  KALDI_ASSERT(IsSortedAndUniq(attr.matrices_written));
481  std::vector<int32> all_matrices;
482  all_matrices.reserve(attr.matrices_read.size() +
483  attr.matrices_written.size());
484  all_matrices.insert(all_matrices.end(), attr.matrices_read.begin(),
485  attr.matrices_read.end());
486  all_matrices.insert(all_matrices.end(), attr.matrices_written.begin(),
487  attr.matrices_written.end());
488  SortAndUniq(&all_matrices);
489 
490  std::vector<int32>::const_iterator iter = all_matrices.begin(),
491  end = all_matrices.end();
492  for (; iter != end; ++iter) {
493  int32 matrix_index = *iter;
494  bool is_read = std::binary_search(attr.matrices_read.begin(),
495  attr.matrices_read.end(),
496  matrix_index),
497  is_written = (!is_read ? true :
498  std::binary_search(attr.matrices_written.begin(),
499  attr.matrices_written.end(),
500  matrix_index));
501  if (is_read && is_written) {
502  (*matrix_accesses)[matrix_index].accesses.push_back(
503  Access(c, kReadWriteAccess));
504  } else if (is_read) {
505  (*matrix_accesses)[matrix_index].accesses.push_back(
506  Access(c, kReadAccess));
507  } else {
508  (*matrix_accesses)[matrix_index].accesses.push_back(
509  Access(c, kWriteAccess));
510  }
511  }
512  // Now set up allocate_command, deallocate_command,
513  // is_input and is_output.
514  const NnetComputation::Command &command = computation.commands[c];
515  int32 matrix_index1, matrix_index2;
516 
517  switch (command.command_type) {
518  case kAllocMatrix:
519  if (!computation.IsWholeMatrix(command.arg1))
520  KALDI_ERR << "Command does not operate on whole matrix";
521  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
522  if ((*matrix_accesses)[matrix_index1].allocate_command != -1)
523  KALDI_ERR << "Matrix " << matrix_index1 << " initialized twice.";
524  (*matrix_accesses)[matrix_index1].allocate_command = c;
525  break;
526  case kSwapMatrix:
527  if (!computation.IsWholeMatrix(command.arg1))
528  KALDI_ERR << "Command does not operate on whole matrix";
529  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
530  KALDI_ASSERT(computation.IsWholeMatrix(command.arg2));
531  matrix_index2 = computation.submatrices[command.arg2].matrix_index;
532  if ((*matrix_accesses)[matrix_index1].allocate_command != -1)
533  KALDI_ERR << "Matrix " << matrix_index1 << " initialized twice.";
534  (*matrix_accesses)[matrix_index1].allocate_command = c;
535  if ((*matrix_accesses)[matrix_index2].deallocate_command != -1)
536  KALDI_ERR << "Matrix " << matrix_index2 << " destroyed twice.";
537  (*matrix_accesses)[matrix_index2].deallocate_command = c;
538  break;
539  case kDeallocMatrix:
540  if (!computation.IsWholeMatrix(command.arg1))
541  KALDI_ERR << "Command does not operate on whole matrix";
542  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
543  if ((*matrix_accesses)[matrix_index1].deallocate_command != -1)
544  KALDI_ERR << "Matrix " << matrix_index1 << " destroyed twice.";
545  (*matrix_accesses)[matrix_index1].deallocate_command = c;
546  break;
547  case kAcceptInput:
548  if (!computation.IsWholeMatrix(command.arg1))
549  KALDI_ERR << "Command does not operate on whole matrix";
550  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
551  (*matrix_accesses)[matrix_index1].is_input = true;
552  // If a certain matrix is accepted as input multiple times, we
553  // count the first one as allocating it (the second will just
554  // allocate it again, which is harmless).
555  if ((*matrix_accesses)[matrix_index1].allocate_command == -1)
556  (*matrix_accesses)[matrix_index1].allocate_command = c;
557  break;
558  case kProvideOutput:
559  if (!computation.IsWholeMatrix(command.arg1))
560  KALDI_ERR << "Command does not operate on whole matrix";
561  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
562  (*matrix_accesses)[matrix_index1].is_output = true;
563  break;
564  default:
565  ;
566  }
567  }
568 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq&#39;s (removes duplicates) from a vector.
Definition: stl-utils.h:39
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
bool IsSortedAndUniq(const std::vector< T > &vec)
Returns true if the vector is sorted and contains each element only once.
Definition: stl-utils.h:63

◆ ComputeMatrixToSubmatrix()

void ComputeMatrixToSubmatrix ( const NnetComputation computation,
std::vector< std::vector< int32 > > *  mat_to_submat 
)

This function computes a vector 'mat_to_submat', indexed by matrix index, such that (*mat_to_submat)[m] is a list of all the submatrix indexes that refer to matrix m.

Note, (*mat_to_submat)[0] will be the empty vector.

Definition at line 1166 of file nnet-analyze.cc.

References KALDI_ASSERT, NnetComputation::matrices, and NnetComputation::submatrices.

Referenced by VariableMergingOptimizer::VariableMergingOptimizer().

1168  {
1169  int32 num_matrices = computation.matrices.size(),
1170  num_submatrices = computation.submatrices.size();
1171  mat_to_submat->clear();
1172  mat_to_submat->resize(num_matrices);
1173  for (int32 submatrix_index = 1;
1174  submatrix_index < num_submatrices;
1175  submatrix_index++) {
1176  int32 matrix_index = computation.submatrices[submatrix_index].matrix_index;
1177  KALDI_ASSERT(matrix_index > 0 && matrix_index < num_matrices);
1178  (*mat_to_submat)[matrix_index].push_back(submatrix_index);
1179  }
1180 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeMinAndMaxTimes()

void kaldi::nnet3::ComputeMinAndMaxTimes ( const std::vector< Index > &  indexes,
int32 *  min_t,
int32 *  max_t 
)

Definition at line 31 of file nnet-derivative-test.cc.

References KALDI_ASSERT, and rnnlm::n.

Referenced by SetDerivTimesOptions().

33  {
34  KALDI_ASSERT(!indexes.empty());
35  *min_t = indexes[0].t;
36  *max_t = *min_t;
37  for (int32 n = 1; n < static_cast<int32>(indexes.size()); n++) {
38  *min_t = std::min(*min_t, indexes[n].t);
39  *max_t = std::max(*max_t, indexes[n].t);
40  }
41 }
struct rnnlm::@11::@12 n
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeNnetComputationEpochs()

void ComputeNnetComputationEpochs ( const Nnet nnet,
std::vector< int32 > *  node_to_epoch 
)

This function computes the order in which we need to compute each node in the graph, where each node-index n maps to an epoch-index t = 0, 1, ...

that says when we should compute it. Nodes that are part of a strongly connected component (SCC) will all be computed at the same time, but any two nodes that are not part of an SCC will have different epoch-index, and these epoch-indexes will be such that a node computed at a larger epoch-index may depend on a node computed at a smaller epoch-index, but not vice versa.

Internally it calls NnetToDirectedGraph, FindSccs, MakeSccGraph and ComputeTopSortOrder.

Definition at line 265 of file nnet-graph.cc.

References ComputeTopSortOrder(), FindSccs(), kaldi::GetVerboseLevel(), rnnlm::i, rnnlm::j, KALDI_ASSERT, KALDI_VLOG, MakeSccGraph(), NnetToDirectedGraph(), and PrintGraphToString().

Referenced by kaldi::nnet3::computation_graph::ComputeEpochInfo().

266  {
267  KALDI_ASSERT(node_to_epoch != NULL);
268 
269  std::vector<std::vector<int32> > graph;
270  NnetToDirectedGraph(nnet, &graph);
271  KALDI_VLOG(6) << "graph is: " << PrintGraphToString(graph);
272 
273  std::vector<std::vector<int32> > sccs;
274  FindSccs(graph, &sccs);
275 
276  std::vector<std::vector<int32> > scc_graph;
277  MakeSccGraph(graph, sccs, &scc_graph);
278  KALDI_VLOG(6) << "scc graph is: " << PrintGraphToString(scc_graph);
279 
280  std::vector<int32> scc_node_to_epoch;
281  ComputeTopSortOrder(scc_graph, &scc_node_to_epoch);
282  if (GetVerboseLevel() >= 6) {
283  std::ostringstream os;
284  for (int32 i = 0; i < scc_node_to_epoch.size(); i++)
285  os << scc_node_to_epoch[i] << ", ";
286  KALDI_VLOG(6) << "scc_node_to_epoch is: " << os.str();
287  }
288 
289  node_to_epoch->clear();
290  node_to_epoch->resize(graph.size());
291  for (int32 i = 0; i < sccs.size(); ++i) {
292  for (int32 j = 0; j < sccs[i].size(); ++j) {
293  int32 node = sccs[i][j];
294  KALDI_ASSERT(node >= 0 && node < graph.size());
295  (*node_to_epoch)[node] = scc_node_to_epoch[i];
296  }
297  }
298 }
void NnetToDirectedGraph(const Nnet &nnet, std::vector< std::vector< int32 > > *graph)
This function takes an nnet and turns it to a directed graph on nodes.
Definition: nnet-graph.cc:30
int32 GetVerboseLevel()
Definition: kaldi-error.h:57
void ComputeTopSortOrder(const std::vector< std::vector< int32 > > &graph, std::vector< int32 > *node_to_order)
Given an acyclic graph (where each std::vector<int32> is a list of destination-nodes of arcs coming f...
Definition: nnet-graph.cc:223
void FindSccs(const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *sccs)
Given a directed graph (where each std::vector<int32> is a list of destination-nodes of arcs coming f...
Definition: nnet-graph.cc:156
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
void MakeSccGraph(const std::vector< std::vector< int32 > > &graph, const std::vector< std::vector< int32 > > &sccs, std::vector< std::vector< int32 > > *scc_graph)
Given a list of sccs of a graph (e.g.
Definition: nnet-graph.cc:164
#define KALDI_VLOG(v)
Definition: kaldi-error.h:132
std::string PrintGraphToString(const std::vector< std::vector< int32 > > &graph)
Prints a graph to a string in a pretty way for human readability, e.g.
Definition: nnet-graph.cc:248

◆ ComputeObjectiveFunction()

void ComputeObjectiveFunction ( const GeneralMatrix supervision,
ObjectiveType  objective_type,
const std::string &  output_name,
bool  supply_deriv,
NnetComputer computer,
BaseFloat tot_weight,
BaseFloat tot_objf 
)

This function computes the objective function, and if supply_deriv = true, supplies its derivative to the NnetComputation object.

See also the function ComputeAccuracy(), declared in nnet-diagnostics.h.

Parameters
[in]supervisionA GeneralMatrix, typically derived from a NnetExample, containing the supervision posteriors or features.
[in]objective_typeThe objective function type: kLinear = output * supervision, or kQuadratic = -0.5 * (output - supervision)^2. kLinear is used for softmax objectives; the network contains a LogSoftmax layer which correctly normalizes its output.
[in]output_nameThe name of the output node (e.g. "output"), used to look up the output in the NnetComputer object.
[in]supply_derivIf this is true, this function will compute the derivative of the objective function and supply it to the network using the function NnetComputer::AcceptOutputDeriv
[in,out]computerThe NnetComputer object, from which we get the output using GetOutput and to which we may supply the derivatives using AcceptOutputDeriv.
[out]tot_weightThe total weight of the training examples. In the kLinear case, this is the sum of the supervision matrix; in the kQuadratic case, it is the number of rows of the supervision matrix. In order to make it possible to weight samples with quadratic objective functions, we may at some point make it possible for the supervision matrix to have an extra column containing weights. At the moment, this is not supported.
[out]tot_objfThe total objective function; divide this by the tot_weight to get the normalized objective function.

Definition at line 370 of file nnet-training.cc.

References NnetComputer::AcceptInput(), CuMatrixBase< Real >::CopyFromGeneralMat(), CuSparseMatrix< Real >::CopyToMat(), GeneralMatrix::GetFullMatrix(), GeneralMatrix::GetMatrix(), NnetComputer::GetOutput(), GeneralMatrix::GetSparseMatrix(), KALDI_ERR, kaldi::kCompressedMatrix, kaldi::kFullMatrix, kLinear, kQuadratic, kaldi::kSparseMatrix, kaldi::kTrans, kaldi::kUndefined, CuMatrixBase< Real >::NumCols(), GeneralMatrix::NumCols(), CuMatrixBase< Real >::NumRows(), GeneralMatrix::NumRows(), CuSparseMatrix< Real >::Sum(), CuMatrixBase< Real >::Sum(), CuMatrix< Real >::Swap(), kaldi::TraceMatMat(), kaldi::TraceMatSmat(), and GeneralMatrix::Type().

Referenced by NnetComputeProb::ProcessOutputs(), and NnetTrainer::ProcessOutputs().

376  {
377  const CuMatrixBase<BaseFloat> &output = computer->GetOutput(output_name);
378 
379  if (output.NumCols() != supervision.NumCols())
380  KALDI_ERR << "Nnet versus example output dimension (num-classes) "
381  << "mismatch for '" << output_name << "': " << output.NumCols()
382  << " (nnet) vs. " << supervision.NumCols() << " (egs)\n";
383 
384  switch (objective_type) {
385  case kLinear: {
386  // objective is x * y.
387  switch (supervision.Type()) {
388  case kSparseMatrix: {
389  const SparseMatrix<BaseFloat> &post = supervision.GetSparseMatrix();
390  CuSparseMatrix<BaseFloat> cu_post(post);
391  // The cross-entropy objective is computed by a simple dot product,
392  // because after the LogSoftmaxLayer, the output is already in the form
393  // of log-likelihoods that are normalized to sum to one.
394  *tot_weight = cu_post.Sum();
395  *tot_objf = TraceMatSmat(output, cu_post, kTrans);
396  if (supply_deriv) {
397  CuMatrix<BaseFloat> output_deriv(output.NumRows(), output.NumCols(),
398  kUndefined);
399  cu_post.CopyToMat(&output_deriv);
400  computer->AcceptInput(output_name, &output_deriv);
401  }
402  break;
403  }
404  case kFullMatrix: {
405  // there is a redundant matrix copy in here if we're not using a GPU
406  // but we don't anticipate this code branch being used in many cases.
407  CuMatrix<BaseFloat> cu_post(supervision.GetFullMatrix());
408  *tot_weight = cu_post.Sum();
409  *tot_objf = TraceMatMat(output, cu_post, kTrans);
410  if (supply_deriv)
411  computer->AcceptInput(output_name, &cu_post);
412  break;
413  }
414  case kCompressedMatrix: {
415  Matrix<BaseFloat> post;
416  supervision.GetMatrix(&post);
417  CuMatrix<BaseFloat> cu_post;
418  cu_post.Swap(&post);
419  *tot_weight = cu_post.Sum();
420  *tot_objf = TraceMatMat(output, cu_post, kTrans);
421  if (supply_deriv)
422  computer->AcceptInput(output_name, &cu_post);
423  break;
424  }
425  }
426  break;
427  }
428  case kQuadratic: {
429  // objective is -0.5 (x - y)^2
430  CuMatrix<BaseFloat> diff(supervision.NumRows(),
431  supervision.NumCols(),
432  kUndefined);
433  diff.CopyFromGeneralMat(supervision);
434  diff.AddMat(-1.0, output);
435  *tot_weight = diff.NumRows();
436  *tot_objf = -0.5 * TraceMatMat(diff, diff, kTrans);
437  if (supply_deriv)
438  computer->AcceptInput(output_name, &diff);
439  break;
440  }
441  default:
442  KALDI_ERR << "Objective function type " << objective_type
443  << " not handled.";
444  }
445 }
#define KALDI_ERR
Definition: kaldi-error.h:123
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
We need to declare this here as it will be a friend function.
Real TraceMatSmat(const MatrixBase< Real > &A, const SparseMatrix< Real > &B, MatrixTransposeType trans)

◆ ComputeObjf()

double kaldi::nnet3::ComputeObjf ( bool  batchnorm_test_mode,
bool  dropout_test_mode,
const std::vector< NnetExample > &  egs,
const Nnet nnet,
NnetComputeProb prob_computer 
)

Definition at line 35 of file nnet3-combine.cc.

References NnetComputeProb::Compute(), NnetComputeProb::GetTotalObjective(), KALDI_ASSERT, NnetComputeProb::Reset(), SetBatchnormTestMode(), and SetDropoutTestMode().

Referenced by main().

37  {
38  if (batchnorm_test_mode || dropout_test_mode) {
39  Nnet nnet_copy(nnet);
40  if (batchnorm_test_mode)
41  SetBatchnormTestMode(true, &nnet_copy);
42  if (dropout_test_mode)
43  SetDropoutTestMode(true, &nnet_copy);
44  NnetComputeProbOptions compute_prob_opts;
45  NnetComputeProb prob_computer_test(compute_prob_opts, nnet_copy);
46  return ComputeObjf(false, false, egs, nnet_copy, &prob_computer_test);
47  } else {
48  prob_computer->Reset();
49  std::vector<NnetExample>::const_iterator iter = egs.begin(),
50  end = egs.end();
51  for (; iter != end; ++iter)
52  prob_computer->Compute(*iter);
53  double tot_weights,
54  tot_objf = prob_computer->GetTotalObjective(&tot_weights);
55  KALDI_ASSERT(tot_weights > 0.0);
56  // inf/nan tot_objf->return -inf objective.
57  if (!(tot_objf == tot_objf && tot_objf - tot_objf == 0))
58  return -std::numeric_limits<double>::infinity();
59  // we prefer to deal with normalized objective functions.
60  return tot_objf / tot_weights;
61  }
62 }
void Compute(const NnetExample &eg)
void SetBatchnormTestMode(bool test_mode, Nnet *nnet)
This function affects only components of type BatchNormComponent.
Definition: nnet-utils.cc:555
This class is for computing cross-entropy and accuracy values in a neural network, for diagnostics.
double ComputeObjf(bool batchnorm_test_mode, bool dropout_test_mode, const std::vector< NnetExample > &egs, const Nnet &nnet, NnetComputeProb *prob_computer)
void SetDropoutTestMode(bool test_mode, Nnet *nnet)
This function affects components of child-classes of RandomComponent.
Definition: nnet-utils.cc:564
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
double GetTotalObjective(double *tot_weight) const

◆ ComputeSimpleNnetContext()

void ComputeSimpleNnetContext ( const Nnet nnet,
int32 *  left_context,
int32 *  right_context 
)

ComputeSimpleNnetContext computes the left-context and right-context of a nnet.

The nnet must satisfy IsSimpleNnet(nnet).

It does this by constructing a ComputationRequest with a certain number of inputs available, outputs can be computed.. It does the same after shifting the time index of the output to all values 0, 1, ... n-1, where n is the output of nnet.Modulus(). Then it returns the largest left context and the largest right context that it infers from any of these computation requests.

Definition at line 146 of file nnet-utils.cc.

References ComputeSimpleNnetContextForShift(), IsSimpleNnet(), KALDI_ASSERT, KALDI_ERR, and Nnet::Modulus().

Referenced by ComputeExampleComputationRequestSimple(), CreateLoopedComputationRequestSimple(), CachingOptimizingCompiler::GetSimpleNnetContext(), Nnet::Info(), DecodableNnetSimpleLoopedInfo::Init(), NnetInfo(), AmNnetSimple::SetContext(), and UnitTestNnetContext().

148  {
149  KALDI_ASSERT(IsSimpleNnet(nnet));
150  int32 modulus = nnet.Modulus();
151  // modulus >= 1 is a number such that the network ought to be
152  // invariant to time shifts (of both the input and output) that
153  // are a multiple of this number. We need to test all shifts modulo
154  // this number in case the left and right context vary at all within
155  // this range.
156 
157  std::vector<int32> left_contexts(modulus + 1);
158  std::vector<int32> right_contexts(modulus + 1);
159 
160  // window_size is a number which needs to be greater than the total context
161  // of the nnet, else we won't be able to work out the context. Large window
162  // size will make this code slow, so we start off with small window size, and
163  // if it isn't enough, we keep doubling it up to a maximum.
164  int32 window_size = 40, max_window_size = 800;
165 
166  while (window_size < max_window_size) {
167 
168  // by going "<= modulus" instead of "< modulus" we do one more computation
169  // than we really need; it becomes a sanity check.
170  int32 input_start;
171  for (input_start = 0; input_start <= modulus; input_start++) {
172  if (!ComputeSimpleNnetContextForShift(nnet, input_start, window_size,
173  &(left_contexts[input_start]),
174  &(right_contexts[input_start])))
175  break;
176  }
177  if (input_start <= modulus) {
178  // We broke from the loop over 'input_start', which means there was
179  // a failure in ComputeSimpleNnextContextForShift-- we assume at
180  // this point that it was because window_size was too small.
181  window_size *= 2;
182  continue;
183  }
184 
185  KALDI_ASSERT(left_contexts[0] == left_contexts[modulus] &&
186  "nnet does not have the properties we expect.");
187  KALDI_ASSERT(right_contexts[0] == right_contexts[modulus] &&
188  "nnet does not have the properties we expect.");
189  *left_context =
190  *std::max_element(left_contexts.begin(), left_contexts.end());
191  *right_context =
192  *std::max_element(right_contexts.begin(), right_contexts.end());
193  // Success.
194  return;
195  }
196  KALDI_ERR << "Failure in ComputeSimpleNnetContext (perhaps not a simple nnet?)";
197 }
#define KALDI_ERR
Definition: kaldi-error.h:123
static bool ComputeSimpleNnetContextForShift(const Nnet &nnet, int32 input_start, int32 window_size, int32 *left_context, int32 *right_context)
Definition: nnet-utils.cc:92
bool IsSimpleNnet(const Nnet &nnet)
This function returns true if the nnet has the following properties: It has an output called "output"...
Definition: nnet-utils.cc:52
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeSimpleNnetContextForShift()

static bool kaldi::nnet3::ComputeSimpleNnetContextForShift ( const Nnet nnet,
int32  input_start,
int32  window_size,
int32 *  left_context,
int32 *  right_context 
)
static

Definition at line 92 of file nnet-utils.cc.

References EvaluateComputationRequest(), Nnet::GetNodeIndex(), IoSpecification::indexes, ComputationRequest::inputs, KALDI_ASSERT, Nnet::Modulus(), rnnlm::n, IoSpecification::name, and ComputationRequest::outputs.

Referenced by ComputeSimpleNnetContext().

97  {
98 
99  int32 input_end = input_start + window_size;
100  IoSpecification input;
101  input.name = "input";
102  IoSpecification output;
103  output.name = "output";
104  IoSpecification ivector; // we might or might not use this.
105  ivector.name = "ivector";
106 
107  int32 n = rand() % 10;
108  // in the IoSpecification for now we we will request all the same indexes at
109  // output that we requested at input.
110  for (int32 t = input_start; t < input_end; t++) {
111  input.indexes.push_back(Index(n, t));
112  output.indexes.push_back(Index(n, t));
113  }
114 
115  // most networks will just require the ivector at time t = 0,
116  // but this might not always be the case, and some might use rounding
117  // descriptors with the iVector which might require it at an earlier
118  // frame than the regular input, so we provide the iVector in as wide a range
119  // as it might possibly be needed.
120  for (int32 t = input_start - nnet.Modulus(); t < input_end; t++) {
121  ivector.indexes.push_back(Index(n, t));
122  }
123 
124  ComputationRequest request;
125  request.inputs.push_back(input);
126  request.outputs.push_back(output);
127  if (nnet.GetNodeIndex("ivector") != -1)
128  request.inputs.push_back(ivector);
129  std::vector<std::vector<bool> > computable;
130  EvaluateComputationRequest(nnet, request, &computable);
131 
132  KALDI_ASSERT(computable.size() == 1);
133  std::vector<bool> &output_ok = computable[0];
134  std::vector<bool>::iterator iter =
135  std::find(output_ok.begin(), output_ok.end(), true);
136  int32 first_ok = iter - output_ok.begin();
137  int32 first_not_ok = std::find(iter, output_ok.end(), false) -
138  output_ok.begin();
139  if (first_ok == window_size || first_not_ok <= first_ok)
140  return false;
141  *left_context = first_ok;
142  *right_context = window_size - first_not_ok;
143  return true;
144 }
void EvaluateComputationRequest(const Nnet &nnet, const ComputationRequest &request, std::vector< std::vector< bool > > *is_computable)
Given an nnet and a computation request, this function works out which requested outputs in the compu...
Definition: nnet-utils.cc:71
struct rnnlm::@11::@12 n
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeSubmatIndexHistogram()

void kaldi::nnet3::ComputeSubmatIndexHistogram ( const std::vector< std::vector< std::pair< int32, int32 > > >  sorted_submat_lists,
unordered_map< int32, std::vector< int32 > > *  submat_histogram 
)

Definition at line 113 of file nnet-compile-utils.cc.

References rnnlm::i, rnnlm::j, and KALDI_ASSERT.

Referenced by SplitLocations().

123  {
124  KALDI_ASSERT(sorted_submat_lists.size() > 0);
125  // computing the submat_histogram
126  // counting the occurrences of each element in the current submat_list;
127  // each new occurrence of the same element, in this list, is counted
128  // as a seperate symbol for frequency counts
129  for (int32 i = 0; i < sorted_submat_lists.size(); i++) {
130  int j = 0;
131  unordered_map<int32, std::vector<int32> >::iterator histogram_iterator
132  = submat_histogram->end();
133  int32 repetition_count = 0;
134  while (j < sorted_submat_lists[i].size()) {
135  if ((histogram_iterator == submat_histogram->end()) ||
136  (histogram_iterator->first != sorted_submat_lists[i][j].first)) {
137  histogram_iterator =
138  submat_histogram->find(sorted_submat_lists[i][j].first);
139  repetition_count = 0;
140  // if a histogram entry was not found for this submat_index, add one
141  if (histogram_iterator == submat_histogram->end()) {
142  (*submat_histogram)[sorted_submat_lists[i][j].first];
143  histogram_iterator = submat_histogram->find(
144  sorted_submat_lists[i][j].first);
145  }
146  }
147 
148  if (repetition_count >= (histogram_iterator->second).size()) {
149  // this is the first time the submat_index repeated this many times
150  // so add an entry for this in the count vector
151  (histogram_iterator->second).push_back(1);
152  } else {
153  (histogram_iterator->second)[repetition_count]++;
154  }
155  repetition_count++;
156  j++;
157  }
158  }
159 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeTopSortOrder()

void ComputeTopSortOrder ( const std::vector< std::vector< int32 > > &  graph,
std::vector< int32 > *  node_to_order 
)

Given an acyclic graph (where each std::vector<int32> is a list of destination-nodes of arcs coming from the current node), compute a topological ordering of the graph nodes.

The output format is that node_to_order[n] contains an integer t = 0, 1, ... which is the order of node n in a topological sorting. node_to_order should contain some permutation of the numbers 0 ... graph.size() - 1. This function should crash if the graph contains cycles.

Definition at line 223 of file nnet-graph.cc.

References ComputeTopSortOrderRecursive(), rnnlm::i, and KALDI_ASSERT.

Referenced by ComputeNnetComputationEpochs(), UnitTestComputeTopSortOrder(), and UnitTestComputeTopSortOrder2().

224  {
225  // Internally we use DFS, but we only put the node to <node_to_order> when all
226  // its parents have been visited.
227  KALDI_ASSERT(node_to_order != NULL);
228  node_to_order->resize(graph.size());
229 
230  std::vector<bool> cycle_detector(graph.size(), false);
231  std::vector<bool> is_visited(graph.size(), false);
232 
233  std::vector<int32> reversed_orders;
234  for(int32 i = 0; i < graph.size(); ++i) {
235  if (!is_visited[i]) {
236  ComputeTopSortOrderRecursive(i, graph, &cycle_detector,
237  &is_visited, &reversed_orders);
238  }
239  }
240 
241  KALDI_ASSERT(node_to_order->size() == reversed_orders.size());
242  for (int32 i = 0; i < reversed_orders.size(); ++i) {
243  KALDI_ASSERT(reversed_orders[i] >= 0 && reversed_orders[i] < graph.size());
244  (*node_to_order)[reversed_orders[i]] = graph.size() - i - 1;
245  }
246 }
void ComputeTopSortOrderRecursive(int32 node, const std::vector< std::vector< int32 > > &graph, std::vector< bool > *cycle_detector, std::vector< bool > *is_visited, std::vector< int32 > *reversed_orders)
Definition: nnet-graph.cc:196
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeTopSortOrderRecursive()

void kaldi::nnet3::ComputeTopSortOrderRecursive ( int32  node,
const std::vector< std::vector< int32 > > &  graph,
std::vector< bool > *  cycle_detector,
std::vector< bool > *  is_visited,
std::vector< int32 > *  reversed_orders 
)

Definition at line 196 of file nnet-graph.cc.

References rnnlm::i, KALDI_ASSERT, and KALDI_ERR.

Referenced by ComputeTopSortOrder().

200  {
201  KALDI_ASSERT(node >= 0 && node < graph.size());
202  KALDI_ASSERT(cycle_detector != NULL);
203  KALDI_ASSERT(is_visited != NULL);
204  KALDI_ASSERT(reversed_orders != NULL);
205  if ((*cycle_detector)[node]) {
206  KALDI_ERR << "Cycle detected when computing the topological sorting order";
207  }
208 
209  if (!(*is_visited)[node]) {
210  (*cycle_detector)[node] = true;
211  for (int32 i = 0; i < graph[node].size(); ++i) {
212  ComputeTopSortOrderRecursive(graph[node][i], graph,
213  cycle_detector, is_visited, reversed_orders);
214  }
215  (*cycle_detector)[node] = false;
216  (*is_visited)[node] = true;
217  // At this point we have added all the children to <reversed_orders>, so we
218  // can add the current now.
219  reversed_orders->push_back(node);
220  }
221 }
void ComputeTopSortOrderRecursive(int32 node, const std::vector< std::vector< int32 > > &graph, std::vector< bool > *cycle_detector, std::vector< bool > *is_visited, std::vector< int32 > *reversed_orders)
Definition: nnet-graph.cc:196
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ComputeVariableAccesses()

void ComputeVariableAccesses ( const ComputationVariables variables,
const std::vector< CommandAttributes > &  command_attributes,
std::vector< std::vector< Access > > *  variable_accesses 
)

After the command-level attributes have been computed, this function organizes them per variable (see class ComputationVariables for how a variable is defined; it is part of a matrix).

Parameters
[in]variablesThe definition of variables for this computation
[in]command_attributesA vector of attributes, one per command, as obtained from ComputeCommandAttributes().
[out]variable_accessesThe output will have a size equal to the number of variables, and each element will be a vector of accesses, sorted by command index; each command will only be listed once in this vector.

Definition at line 421 of file nnet-analyze.cc.

References kaldi::IsSortedAndUniq(), KALDI_ASSERT, kReadAccess, kReadWriteAccess, kWriteAccess, ComputationVariables::NumVariables(), kaldi::SortAndUniq(), CommandAttributes::variables_read, and CommandAttributes::variables_written.

Referenced by Analyzer::Init(), MoveSizingCommands(), and Access::operator<().

424  {
425  int32 num_variables = variables.NumVariables(),
426  num_commands = command_attributes.size();
427  variable_accesses->clear();
428  variable_accesses->resize(num_variables);
429  for (int32 c = 0; c < num_commands; c++) {
430  const CommandAttributes &attr = command_attributes[c];
431  KALDI_ASSERT(IsSortedAndUniq(attr.variables_read));
432  KALDI_ASSERT(IsSortedAndUniq(attr.variables_written));
433  std::vector<int32> all_variables;
434  all_variables.reserve(attr.variables_read.size() +
435  attr.variables_written.size());
436  all_variables.insert(all_variables.end(), attr.variables_read.begin(),
437  attr.variables_read.end());
438  all_variables.insert(all_variables.end(), attr.variables_written.begin(),
439  attr.variables_written.end());
440  SortAndUniq(&all_variables);
441 
442  std::vector<int32>::const_iterator iter = all_variables.begin(),
443  end = all_variables.end();
444  for (; iter != end; ++iter) {
445  int32 variable_index = *iter;
446  bool is_read = std::binary_search(attr.variables_read.begin(),
447  attr.variables_read.end(),
448  variable_index),
449  is_written = (!is_read ? true :
450  std::binary_search(attr.variables_written.begin(),
451  attr.variables_written.end(),
452  variable_index));
453  if (is_read && is_written) {
454  (*variable_accesses)[variable_index].push_back(
455  Access(c, kReadWriteAccess));
456  } else if (is_read) {
457  (*variable_accesses)[variable_index].push_back(
458  Access(c, kReadAccess));
459  } else {
460  (*variable_accesses)[variable_index].push_back(
461  Access(c, kWriteAccess));
462  }
463  }
464  }
465 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq&#39;s (removes duplicates) from a vector.
Definition: stl-utils.h:39
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
bool IsSortedAndUniq(const std::vector< T > &vec)
Returns true if the vector is sorted and contains each element only once.
Definition: stl-utils.h:63

◆ ConsolidateIoOperations()

void ConsolidateIoOperations ( const Nnet nnet,
NnetComputation computation 
)

This optimization puts the input operations (kAcceptInput) and output operations (kProvideOutput) at the very beginning or end of segments of computation, respectively.

This is actually necessary for computations to be run easily, because if these commands were interspersed with the regular commands, you'd have to call computer.Run() between the individual AcceptInput() and GetOutput() function calls.

Definition at line 869 of file nnet-optimize.cc.

References NnetComputation::commands, kAcceptInput, KALDI_ASSERT, kNoOperationMarker, kProvideOutput, and SplitComputationIntoSegments().

Referenced by Compiler::CreateComputation(), and Optimize().

870  {
871  // These segments, represented as (start-index, end-index),
872  // are segments of the computation separated by kNoOperationMarker.
873  std::vector<std::pair<int32, int32> > segments;
874  SplitComputationIntoSegments(*computation, &segments);
875 
876  int32 num_commands = computation->commands.size();
877  std::vector<NnetComputation::Command> reordered_commands(num_commands);
878  // put kNoOperationMarker between all segments in the reordered commands.
879  for (size_t s = 0; s + 1 < segments.size(); s++)
880  reordered_commands[segments[s].second].command_type = kNoOperationMarker;
881 
882  // for each segment we'll divide the commands up into those that must appear
883  // at the left of the segment (kAcceptInput for inputs and output-derivs), those
884  // that must appear in the middle (most commands), those that must appear
885  // on the right (kProvideOutput for output nodes and input derivatives).
886  std::vector<int32> left_commands, middle_commands, right_commands;
887 
888  for (size_t s = 0; s < segments.size(); s++) {
889  int32 segment_start = segments[s].first,
890  segment_end = segments[s].second;
891  left_commands.clear();
892  middle_commands.clear();
893  right_commands.clear();
894  for (int32 c = segment_start; c < segment_end; c++) {
895  if (computation->commands[c].command_type == kProvideOutput) {
896  right_commands.push_back(c);
897  } else if (computation->commands[c].command_type == kAcceptInput) {
898  left_commands.push_back(c);
899  } else {
900  middle_commands.push_back(c);
901  }
902  }
903  std::vector<int32>::const_iterator iter = left_commands.begin(),
904  end = left_commands.end();
905  int32 c = segment_start;
906  for (; iter != end; ++iter, ++c)
907  reordered_commands[c] = computation->commands[*iter];
908  iter = middle_commands.begin();
909  end = middle_commands.end();
910  for (; iter != end; ++iter, ++c)
911  reordered_commands[c] = computation->commands[*iter];
912  iter = right_commands.begin();
913  end = right_commands.end();
914  for (; iter != end; ++iter, ++c)
915  reordered_commands[c] = computation->commands[*iter];
916  KALDI_ASSERT(c == segment_end);
917  }
918  computation->commands.swap(reordered_commands);
919 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
static void SplitComputationIntoSegments(const NnetComputation &computation, std::vector< std::pair< int32, int32 > > *segments)
Split the computation up into segments bounded by kNoOperationMarker.

◆ ConsolidateMemory()

void ConsolidateMemory ( Nnet nnet)

This just calls ConsolidateMemory() on all the components of the nnet.

This is called by the training code after processing the first minibatch. On some components this will do nothing; on some components it will reallocate certain quantities that have been allocated during training (mostly the contents of NaturalGradientOnline objects, and stats for NonlinearComponents) so that they can be put into low memory. This will tend to minimize memory fragmentation. Read comments in ../cudamatrix/cu-allocator.h for more explanation.

Definition at line 1061 of file nnet-utils.cc.

References Component::ConsolidateMemory(), Nnet::GetComponent(), kaldi::GetVerboseLevel(), KALDI_VLOG, and Nnet::NumComponents().

Referenced by CollapseModelConfig::CollapseModelConfig(), NonlinearComponent::OutputDim(), NnetChainTrainer::Train(), and NnetTrainer::Train().

1061  {
1062 #if HAVE_CUDA == 1
1063  if (CuDevice::Instantiate().Enabled()) {
1064  bool print_memory_info = (GetVerboseLevel() >= 1);
1065  if (print_memory_info) {
1066  KALDI_VLOG(1) << "Consolidating memory; will print memory usage before "
1067  "and after consolidating:";
1068  g_cuda_allocator.PrintMemoryUsage();
1069  }
1070  for (int32 c = 0; c < nnet->NumComponents(); c++) {
1071  Component *comp = nnet->GetComponent(c);
1072  comp->ConsolidateMemory();
1073  }
1074  if (print_memory_info) {
1075  g_cuda_allocator.PrintMemoryUsage();
1076  }
1077  }
1078 #endif
1079 }
int32 GetVerboseLevel()
Definition: kaldi-error.h:57
#define KALDI_VLOG(v)
Definition: kaldi-error.h:132

◆ ConsolidateModelUpdate()

void ConsolidateModelUpdate ( const Nnet nnet,
NnetComputation computation 
)

This optimization consolidates the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices.

This consolidates the model-update parts of the backprop into larger operations (applicable mostly to recurrent setups)– internally it uses class ModelUpdateConsolidator.

This is useful for recurrent networks. The resulting computation separates the backprop for data-derivatives from the model-update part of backprop.

Will fail if called a second time.

Definition at line 1551 of file nnet-optimize-utils.cc.

References ModelUpdateConsolidator::ConsolidateModelUpdate(), and NnetComputation::need_model_derivative.

Referenced by Optimize().

1552  {
1553  // This following if-statement is an optimization: if the computation
1554  // request(s) had need_model_derivative == false, there would be nothing to
1555  // optimize, so don't bother trying.
1556  if (!computation->need_model_derivative)
1557  return;
1558  ModelUpdateConsolidator consolidator(nnet, computation);
1559  consolidator.ConsolidateModelUpdate();
1560 }

◆ ConstrainOrthonormal()

void ConstrainOrthonormal ( Nnet nnet)

This function, to be called after processing every minibatch, is responsible for enforcing the orthogonality constraint for any components of type LinearComponent or inheriting from AffineComponent that have the "orthonormal_constraint" value set.

This function, to be called after processing every minibatch, is responsible for enforcing the orthogonality constraint for any components of type LinearComponent or inheriting from AffineComponent that have the "orthonormal-constraint" value set to a nonzero value.

Technically what we are doing is constraining the parameter matrix M to be a "semi-orthogonal" matrix times a constant alpha. That is: if num-rows > num-cols, this amounts to asserting that M M^T == alpha^2 I; otherwise, that M^T M == alpha^2 I.

If, for a particular component, orthonormal-constraint > 0.0, then that value becomes the "alpha" mentioned above. If orthonormal-constraint == 0.0, then nothing is done. If orthonormal-constraint < 0.0, then it's like letting alpha "float", i.e. we try to make M closer to (any constant alpha) times a semi-orthogonal matrix.

In order to make it efficient on GPU, it doesn't make it completely orthonormal, it just makes it closer to being orthonormal (times the 'orthonormal_constraint' value). Over multiple iterations this rapidly makes it almost exactly orthonormal.

See http://www.danielpovey.com/files/2018_interspeech_tdnnf.pdf

Definition at line 1022 of file nnet-utils.cc.

References ConstrainOrthonormalInternal(), CuMatrixBase< Real >::CopyFromMat(), Nnet::GetComponent(), kaldi::kTrans, AffineComponent::LinearParams(), TdnnComponent::LinearParams(), CuMatrixBase< Real >::NumCols(), Nnet::NumComponents(), CuMatrixBase< Real >::NumRows(), AffineComponent::OrthonormalConstraint(), TdnnComponent::OrthonormalConstraint(), LinearComponent::OrthonormalConstraint(), LinearComponent::Params(), and kaldi::RandInt().

Referenced by CollapseModelConfig::CollapseModelConfig(), NnetChainTrainer::TrainInternal(), NnetTrainer::TrainInternal(), NnetChainTrainer::TrainInternalBackstitch(), and NnetTrainer::TrainInternalBackstitch().

1022  {
1023 
1024  for (int32 c = 0; c < nnet->NumComponents(); c++) {
1025  Component *component = nnet->GetComponent(c);
1026  CuMatrixBase<BaseFloat> *params = NULL;
1027  BaseFloat orthonormal_constraint = 0.0;
1028 
1029  LinearComponent *lc = dynamic_cast<LinearComponent*>(component);
1030  if (lc != NULL && lc->OrthonormalConstraint() != 0.0) {
1031  orthonormal_constraint = lc->OrthonormalConstraint();
1032  params = &(lc->Params());
1033  }
1034  AffineComponent *ac = dynamic_cast<AffineComponent*>(component);
1035  if (ac != NULL && ac->OrthonormalConstraint() != 0.0) {
1036  orthonormal_constraint = ac->OrthonormalConstraint();
1037  params = &(ac->LinearParams());
1038  }
1039  TdnnComponent *tc = dynamic_cast<TdnnComponent*>(component);
1040  if (tc != NULL && tc->OrthonormalConstraint() != 0.0) {
1041  orthonormal_constraint = tc->OrthonormalConstraint();
1042  params = &(tc->LinearParams());
1043  }
1044  if (orthonormal_constraint == 0.0 || RandInt(0, 3) != 0) {
1045  // For efficiency, only do this every 4 or so minibatches-- it won't have
1046  // time stray far from the constraint in between.
1047  continue;
1048  }
1049 
1050  int32 rows = params->NumRows(), cols = params->NumCols();
1051  if (rows <= cols) {
1052  ConstrainOrthonormalInternal(orthonormal_constraint, params);
1053  } else {
1054  CuMatrix<BaseFloat> params_trans(*params, kTrans);
1055  ConstrainOrthonormalInternal(orthonormal_constraint, &params_trans);
1056  params->CopyFromMat(params_trans, kTrans);
1057  }
1058  }
1059 }
void ConstrainOrthonormalInternal(BaseFloat scale, CuMatrixBase< BaseFloat > *M)
Definition: nnet-utils.cc:896
float BaseFloat
Definition: kaldi-types.h:29
int32 RandInt(int32 min_val, int32 max_val, struct RandomState *state)
Definition: kaldi-math.cc:94

◆ ConstrainOrthonormalInternal()

void kaldi::nnet3::ConstrainOrthonormalInternal ( BaseFloat  scale,
CuMatrixBase< BaseFloat > *  M 
)

Definition at line 896 of file nnet-utils.cc.

References CuMatrixBase< Real >::AddMat(), CuMatrixBase< Real >::AddMatMat(), CuMatrixBase< Real >::AddToDiag(), CuMatrixBase< Real >::CopyLowerToUpper(), CuMatrixBase< Real >::FrobeniusNorm(), kaldi::GetVerboseLevel(), KALDI_ASSERT, KALDI_VLOG, kaldi::kNoTrans, kaldi::kTrans, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrixBase< Real >::SymAddMat2(), CuMatrixBase< Real >::Trace(), and kaldi::TraceMatMat().

Referenced by ConstrainOrthonormal().

896  {
897  KALDI_ASSERT(scale != 0.0);
898 
899  // We'd like to enforce the rows of M to be orthonormal.
900  // define P = M M^T. If P is unit then M has orthonormal rows.
901  // We actually want P to equal scale^2 * I, so that M's rows are
902  // orthogonal with 2-norms equal to 'scale'.
903  // We (notionally) add to the objective function, the value
904  // -alpha times the sum of squared elements of Q = (P - scale^2 * I).
905  int32 rows = M->NumRows(), cols = M->NumCols();
906  CuMatrix<BaseFloat> M_update(rows, cols);
907  CuMatrix<BaseFloat> P(rows, rows);
908  P.SymAddMat2(1.0, *M, kNoTrans, 0.0);
909  P.CopyLowerToUpper();
910 
911  // The 'update_speed' is a constant that determines how fast we approach a
912  // matrix with the desired properties (larger -> faster). Larger values will
913  // update faster but will be more prone to instability. 0.125 (1/8) is the
914  // value that gives us the fastest possible convergence when we are already
915  // close to be a semi-orthogonal matrix (in fact, it will lead to quadratic
916  // convergence).
917  // See http://www.danielpovey.com/files/2018_interspeech_tdnnf.pdf
918  // for more details.
919  BaseFloat update_speed = 0.125;
920  bool floating_scale = (scale < 0.0);
921 
922 
923  if (floating_scale) {
924  // This (letting the scale "float") is described in Sec. 2.3 of
925  // http://www.danielpovey.com/files/2018_interspeech_tdnnf.pdf,
926  // where 'scale' here is written 'alpha' in the paper.
927  //
928  // We pick the scale that will give us an update to M that is
929  // orthogonal to M (viewed as a vector): i.e., if we're doing
930  // an update M := M + X, then we want to have tr(M X^T) == 0.
931  // The following formula is what gives us that.
932  // With P = M M^T, our update formula is doing to be:
933  // M := M + (-4 * alpha * (P - scale^2 I) * M).
934  // (The math below explains this update formula; for now, it's
935  // best to view it as an established fact).
936  // So X (the change in M) is -4 * alpha * (P - scale^2 I) * M,
937  // where alpha == update_speed / scale^2.
938  // We want tr(M X^T) == 0. First, forget the -4*alpha, because
939  // we don't care about constant factors. So we want:
940  // tr(M * M^T * (P - scale^2 I)) == 0.
941  // Since M M^T == P, that means:
942  // tr(P^2 - scale^2 P) == 0,
943  // or scale^2 = tr(P^2) / tr(P).
944  // Note: P is symmetric so it doesn't matter whether we use tr(P P) or
945  // tr(P^T P); we use tr(P^T P) because I believe it's faster to compute.
946 
947  BaseFloat trace_P = P.Trace(), trace_P_P = TraceMatMat(P, P, kTrans);
948 
949  scale = std::sqrt(trace_P_P / trace_P);
950 
951  // The following is a tweak to avoid divergence when the eigenvalues aren't
952  // close to being the same. trace_P is the sum of eigenvalues of P, and
953  // trace_P_P is the sum-square of eigenvalues of P. Treat trace_P as a sum
954  // of positive values, and trace_P_P as their sumsq. Then mean = trace_P /
955  // dim, and trace_P_P cannot be less than dim * (trace_P / dim)^2,
956  // i.e. trace_P_P >= trace_P^2 / dim. If ratio = trace_P_P * dim /
957  // trace_P^2, then ratio >= 1.0, and the excess above 1.0 is a measure of
958  // how far we are from convergence. If we're far from convergence, we make
959  // the learning rate slower to reduce the risk of divergence, since the
960  // update may not be stable for starting points far from equilibrium.
961  BaseFloat ratio = (trace_P_P * P.NumRows() / (trace_P * trace_P));
962  KALDI_ASSERT(ratio > 0.999);
963  if (ratio > 1.02) {
964  update_speed *= 0.5; // Slow down the update speed to reduce the risk of divergence.
965  if (ratio > 1.1) update_speed *= 0.5; // Slow it down even more.
966  }
967  }
968 
969  P.AddToDiag(-1.0 * scale * scale);
970 
971  // We may want to un-comment the following code block later on if we have a
972  // problem with instability in setups with a non-floating orthonormal
973  // constraint.
974  /*
975  if (!floating_scale) {
976  // This is analogous to the stuff with 'ratio' above, but when we don't have
977  // a floating scale. It reduces the chances of divergence when we have
978  // a bad initialization.
979  BaseFloat error = P.FrobeniusNorm(),
980  error_proportion = error * error / P.NumRows();
981  // 'error_proportion' is the sumsq of elements in (P - I) divided by the
982  // sumsq of elements of I. It should be much less than one (i.e. close to
983  // zero) if the error is small.
984  if (error_proportion > 0.02) {
985  update_speed *= 0.5;
986  if (error_proportion > 0.1)
987  update_speed *= 0.5;
988  }
989  }
990  */
991 
992  if (GetVerboseLevel() >= 1) {
993  BaseFloat error = P.FrobeniusNorm();
994  KALDI_VLOG(2) << "Error in orthogonality is " << error;
995  }
996 
997  // see Sec. 2.2 of http://www.danielpovey.com/files/2018_interspeech_tdnnf.pdf
998  // for explanation of the 1/(scale*scale) factor, but there is a difference in
999  // notation; 'scale' here corresponds to 'alpha' in the paper, and
1000  // 'update_speed' corresponds to 'nu' in the paper.
1001  BaseFloat alpha = update_speed / (scale * scale);
1002 
1003  // At this point, the matrix P contains what, in the math, would be Q =
1004  // P-scale^2*I. The derivative of the objective function w.r.t. an element q(i,j)
1005  // of Q is now equal to -2*alpha*q(i,j), i.e. we could write q_deriv(i,j)
1006  // = -2*alpha*q(i,j) This is also the derivative of the objective function
1007  // w.r.t. p(i,j): i.e. p_deriv(i,j) = -2*alpha*q(i,j).
1008  // Suppose we have define this matrix as 'P_deriv'.
1009  // The derivative of the objective w.r.t M equals
1010  // 2 * P_deriv * M, which equals -4*alpha*(P-scale^2*I)*M.
1011  // (Currently the matrix P contains what, in the math, is P-scale^2*I).
1012  M_update.AddMatMat(-4.0 * alpha, P, kNoTrans, *M, kNoTrans, 0.0);
1013  M->AddMat(1.0, M_update);
1014 }
int32 GetVerboseLevel()
Definition: kaldi-error.h:57
float BaseFloat
Definition: kaldi-types.h:29
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
We need to declare this here as it will be a friend function.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
#define KALDI_VLOG(v)
Definition: kaldi-error.h:132

◆ ContainsSingleExample()

bool kaldi::nnet3::ContainsSingleExample ( const NnetExample eg,
int32 *  min_input_t,
int32 *  max_input_t,
int32 *  min_output_t,
int32 *  max_output_t 
)

Returns true if the "eg" contains just a single example, meaning that all the "n" values in the indexes are zero, and the example has NnetIo members named both "input" and "output".

Also computes the minimum and maximum "t" values in the "input" and "output" NnetIo members.

Definition at line 82 of file nnet3-copy-egs.cc.

References rnnlm::i, NnetIo::indexes, NnetExample::io, KALDI_ASSERT, KALDI_WARN, and NnetIo::name.

Referenced by SelectFromExample().

86  {
87  bool done_input = false, done_output = false;
88  int32 num_indexes = eg.io.size();
89  for (int32 i = 0; i < num_indexes; i++) {
90  const NnetIo &io = eg.io[i];
91  std::vector<Index>::const_iterator iter = io.indexes.begin(),
92  end = io.indexes.end();
93  // Should not have an empty input/output type.
94  KALDI_ASSERT(!io.indexes.empty());
95  if (io.name == "input" || io.name == "output") {
96  int32 min_t = iter->t, max_t = iter->t;
97  for (; iter != end; ++iter) {
98  int32 this_t = iter->t;
99  min_t = std::min(min_t, this_t);
100  max_t = std::max(max_t, this_t);
101  if (iter->n != 0) {
102  KALDI_WARN << "Example does not contain just a single example; "
103  << "too late to do frame selection or reduce context.";
104  return false;
105  }
106  }
107  if (io.name == "input") {
108  done_input = true;
109  *min_input_t = min_t;
110  *max_input_t = max_t;
111  } else {
112  KALDI_ASSERT(io.name == "output");
113  done_output = true;
114  *min_output_t = min_t;
115  *max_output_t = max_t;
116  }
117  } else {
118  for (; iter != end; ++iter) {
119  if (iter->n != 0) {
120  KALDI_WARN << "Example does not contain just a single example; "
121  << "too late to do frame selection or reduce context.";
122  return false;
123  }
124  }
125  }
126  }
127  if (!done_input) {
128  KALDI_WARN << "Example does not have any input named 'input'";
129  return false;
130  }
131  if (!done_output) {
132  KALDI_WARN << "Example does not have any output named 'output'";
133  return false;
134  }
135  return true;
136 }
std::vector< Index > indexes
"indexes" is a vector the same length as features.NumRows(), explaining the meaning of each row of th...
Definition: nnet-example.h:42
#define KALDI_WARN
Definition: kaldi-error.h:126
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165
std::string name
the name of the input in the neural net; in simple setups it will just be "input".
Definition: nnet-example.h:36
std::vector< NnetIo > io
"io" contains the input and output.
Definition: nnet-example.h:116

◆ ConvertAdditionToAssignment()

void ConvertAdditionToAssignment ( const Nnet nnet,
NnetComputation computation 
)

This converts addition operations (things with Add in their names) to copy operations (things with Copy in their names).

This is slightly more efficient, and it may later allow us to remove unnecessary zeroing.

Definition at line 430 of file nnet-optimize.cc.

References NnetComputation::Command::alpha, Analyzer::command_attributes, NnetComputation::Command::command_type, NnetComputation::commands, ComputationAnalysis::FirstNontrivialAccess(), Analyzer::Init(), kAddRows, kAddRowsMulti, kAddToRowsMulti, KALDI_ASSERT, KALDI_ERR, kCopyRows, kCopyRowsMulti, kCopyToRowsMulti, kMatrixAdd, and kMatrixCopy.

Referenced by Optimize().

431  {
432  Analyzer analyzer;
433  analyzer.Init(nnet, *computation);
434  ComputationAnalysis analysis(*computation, analyzer);
435  int32 num_commands = computation->commands.size();
436  for (int32 command = 0; command < num_commands; command++) {
437  NnetComputation::Command &c = computation->commands[command];
438  switch (c.command_type) {
439  case kMatrixAdd: case kAddRows: case kAddRowsMulti:
440  case kAddToRowsMulti: {
441  const std::vector<int32> &submatrices_written =
442  analyzer.command_attributes[command].submatrices_written;
443  KALDI_ASSERT(!submatrices_written.empty());
444  std::vector<int32>::const_iterator iter = submatrices_written.begin(),
445  end = submatrices_written.end();
446  bool can_convert = true;
447  for (; iter != end; ++iter) {
448  int32 submatrix_written = *iter;
449  int32 first_access_command = analysis.FirstNontrivialAccess(
450  submatrix_written);
451  // first_access_command is first command other than zeroing and
452  // allocation that accesses this submatrix. It can be assumed to be a
453  // write command, since it makes no sense to read a variable before
454  // it's written to. If it's before this command then we need to add
455  // rather than copy; we can't do the conversion to a copy command.
456  if (first_access_command != command) {
457  can_convert = false;
458  break;
459  }
460  }
461  if (can_convert) { // convert to a copy command.
462  switch (c.command_type) {
463  case kMatrixAdd: c.command_type = kMatrixCopy;
464  break;
465  case kAddRows: c.command_type = kCopyRows;
466  break;
467  case kAddRowsMulti: c.command_type = kCopyRowsMulti;
468  break;
469  // note: kCopyToRowsMulti does not currently support alpha != 1.0.
470  case kAddToRowsMulti: if (c.alpha == 1.0) c.command_type = kCopyToRowsMulti;
471  break;
472  default: KALDI_ERR << "Unexpected command type.";
473  }
474  }
475  break;
476  }
477  default:
478  break;
479  }
480  }
481 }
#define KALDI_ERR
Definition: kaldi-error.h:123
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ConvertNumNValues()

static void kaldi::nnet3::ConvertNumNValues ( int32  n_stride,
int32  old_N,
int32  new_N,
const std::vector< Index > &  indexes_in,
std::vector< Index > *  indexes_out 
)
static

Definition at line 3106 of file nnet-optimize-utils.cc.

References KALDI_ASSERT, Index::n, and rnnlm::n.

Referenced by ComputationExpander::ExpandIndexes(), and IoSpecificationIsDecomposable().

3108  {
3109  int32 size_in = indexes_in.size();
3110  KALDI_ASSERT(size_in > 0 && indexes_in[size_in - 1].n == old_N - 1);
3111  int32 block_size_in = n_stride * old_N,
3112  block_size_out = n_stride * new_N;
3113 
3114  indexes_out->resize((size_in / old_N) * new_N);
3115  for (int32 i_in = 0; i_in < size_in; i_in++) {
3116  if (indexes_in[i_in].n != 0)
3117  continue;
3118  Index index(indexes_in[i_in]);
3119  int32 block_index = i_in / block_size_in,
3120  offset_within_block = i_in % block_size_in;
3121 
3122 
3123  int32 i_out = block_index * block_size_out +
3124  offset_within_block;
3125  for (int32 n = 0; n < new_N; n++, i_out += n_stride) {
3126  index.n = n;
3127  (*indexes_out)[i_out] = index;
3128  }
3129  }
3130 }
struct rnnlm::@11::@12 n
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ConvertRepeatedToBlockAffine() [1/2]

void kaldi::nnet3::ConvertRepeatedToBlockAffine ( CompositeComponent c_component)

Definition at line 438 of file nnet-utils.cc.

References CompositeComponent::GetComponent(), rnnlm::i, KALDI_ASSERT, CompositeComponent::NumComponents(), CompositeComponent::SetComponent(), and Component::Type().

Referenced by ConvertRepeatedToBlockAffine(), main(), UnitTestConvertRepeatedToBlockAffine(), and UnitTestConvertRepeatedToBlockAffineComposite().

438  {
439  for(int32 i = 0; i < c_component->NumComponents(); i++) {
440  const Component *c = c_component->GetComponent(i);
441  KALDI_ASSERT(c->Type() != "CompositeComponent" &&
442  "Nesting CompositeComponent within CompositeComponent is not allowed.\n"
443  "(We may change this as more complicated components are introduced.)");
444 
445  if(c->Type() == "RepeatedAffineComponent" ||
446  c->Type() == "NaturalGradientRepeatedAffineComponent") {
447  // N.B.: NaturalGradientRepeatedAffineComponent is a subclass of
448  // RepeatedAffineComponent.
449  const RepeatedAffineComponent *rac =
450  dynamic_cast<const RepeatedAffineComponent*>(c);
451  KALDI_ASSERT(rac != NULL);
452  BlockAffineComponent *bac = new BlockAffineComponent(*rac);
453  // following call deletes rac
454  c_component->SetComponent(i, bac);
455  }
456  }
457 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ConvertRepeatedToBlockAffine() [2/2]

void ConvertRepeatedToBlockAffine ( Nnet nnet)

Convert all components of type RepeatedAffineComponent or NaturalGradientRepeatedAffineComponent to BlockAffineComponent in nnet.

Definition at line 459 of file nnet-utils.cc.

References ConvertRepeatedToBlockAffine(), Nnet::GetComponent(), rnnlm::i, KALDI_ASSERT, Nnet::NumComponents(), Nnet::SetComponent(), and Component::Type().

459  {
460  for(int32 i = 0; i < nnet->NumComponents(); i++) {
461  const Component *const_c = nnet->GetComponent(i);
462  if(const_c->Type() == "RepeatedAffineComponent" ||
463  const_c->Type() == "NaturalGradientRepeatedAffineComponent") {
464  // N.B.: NaturalGradientRepeatedAffineComponent is a subclass of
465  // RepeatedAffineComponent.
466  const RepeatedAffineComponent *rac =
467  dynamic_cast<const RepeatedAffineComponent*>(const_c);
468  KALDI_ASSERT(rac != NULL);
469  BlockAffineComponent *bac = new BlockAffineComponent(*rac);
470  // following call deletes rac
471  nnet->SetComponent(i, bac);
472  } else if (const_c->Type() == "CompositeComponent") {
473  // We must modify the composite component, so we use the
474  // non-const GetComponent() call here.
475  Component *c = nnet->GetComponent(i);
476  CompositeComponent *cc = dynamic_cast<CompositeComponent*>(c);
477  KALDI_ASSERT(cc != NULL);
479  }
480  }
481 }
void ConvertRepeatedToBlockAffine(Nnet *nnet)
Convert all components of type RepeatedAffineComponent or NaturalGradientRepeatedAffineComponent to B...
Definition: nnet-utils.cc:459
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ ConvertToIndexes()

bool ConvertToIndexes ( const std::vector< std::pair< int32, int32 > > &  location_vector,
int32 *  first_value,
std::vector< int32 > *  second_values 
)

If it is the case for some i >= 0 that all the .first elements of "location_vector" are either i or -1, then output i to first_value and the .second elements into "second_values", and return true.

Otherwise return false and the outputs are don't-cares.

Definition at line 377 of file nnet-compile-utils.cc.

Referenced by Compiler::CompileBackwardFromSubmatLocations(), Compiler::CompileForwardFromSubmatLocations(), SplitLocationsBackward(), UnitTestSplitLocations(), and UnitTestSplitLocationsBackward().

380  {
381  *first_value = -1;
382  second_values->clear();
383  second_values->reserve(location_vector.size());
384  std::vector<std::pair<int32, int32> >::const_iterator iter;
385  for (iter = location_vector.begin(); iter < location_vector.end(); ++iter) {
386  if (iter->first != -1) {
387  if (*first_value == -1)
388  *first_value = iter->first;
389  if (iter->first != *first_value)
390  return false;
391  second_values->push_back(iter->second);
392  } else {
393  second_values->push_back(-1);
394  }
395  }
396  return true;
397 }

◆ CopyPairVector() [1/2]

static void kaldi::nnet3::CopyPairVector ( const CuArray< Int32Pair > &  in,
std::vector< std::pair< int32, int32 > > *  out 
)
static

Definition at line 31 of file nnet-general-component.cc.

References CuArrayBase< T >::CopyToVec().

Referenced by StatisticsExtractionComponentPrecomputedIndexes::Read(), StatisticsPoolingComponentPrecomputedIndexes::Read(), StatisticsExtractionComponentPrecomputedIndexes::Write(), and StatisticsPoolingComponentPrecomputedIndexes::Write().

32  {
33  in.CopyToVec(reinterpret_cast<std::vector<Int32Pair>*>(out));
34 }
void CopyToVec(std::vector< T > *dst) const
This function resizes *dst if needed.
Definition: cu-array-inl.h:173

◆ CopyPairVector() [2/2]

static void kaldi::nnet3::CopyPairVector ( const std::vector< std::pair< int32, int32 > > &  in,
CuArray< Int32Pair > *  out 
)
static

Definition at line 36 of file nnet-general-component.cc.

References CuArray< T >::CopyFromVec().

37  {
38  const std::vector<Int32Pair> *in_cast =
39  reinterpret_cast<const std::vector<Int32Pair>*>(&in);
40  out->CopyFromVec(*in_cast);
41 }
void CopyFromVec(const std::vector< T > &src)
This function resizes if needed.
Definition: cu-array-inl.h:119

◆ CreateComputationRequestInternal()

static void kaldi::nnet3::CreateComputationRequestInternal ( int32  begin_input_t,
int32  end_input_t,
int32  begin_output_t,
int32  end_output_t,
int32  num_sequences,
int32  frame_subsampling_factor,
const std::set< int32 > &  ivector_times,
ComputationRequest request 
)
static

Definition at line 113 of file nnet-compile-looped.cc.

References ComputationRequest::inputs, rnnlm::n, and ComputationRequest::outputs.

Referenced by CreateLoopedComputationRequest().

119  {
120  request->inputs.reserve(2);
121  request->inputs.clear();
122  request->inputs.resize(1 + (ivector_times.empty() ? 0 : 1));
123  request->inputs[0].name = "input";
124  request->inputs[0].has_deriv = false;
125  request->outputs.clear();
126  request->outputs.resize(1);
127  request->outputs[0].name = "output";
128  request->outputs[0].has_deriv = false;
129  if (!ivector_times.empty()) {
130  request->inputs[1].name = "ivector";
131  request->inputs[1].has_deriv = false;
132  }
133 
134  // in the computation request the 'n' indexes (the sequence/utterance indexes)
135  // have the larger stride than 't', although this is opposite to the way it's
136  // done inside the computation. This is for user convenience where it may be
137  // easier to deal with submatrixes per sequence.
138  for (int32 n = 0; n < num_sequences; n++) {
139  int32 x = 0;
140  for (int32 t = begin_input_t; t < end_input_t; t++) {
141  request->inputs[0].indexes.push_back(Index(n, t, x));
142  }
143  for (int32 t = begin_output_t;
144  t < end_output_t;
145  t += frame_subsampling_factor)
146  request->outputs[0].indexes.push_back(Index(n, t, x));
147  }
148  if (!ivector_times.empty()) {
149  request->inputs.resize(2);
150  request->inputs[1].name = "ivector";
151  request->inputs[1].has_deriv = false;
152  for (int32 n = 0; n < num_sequences; n++) {
153  // note: std::sets store things in sorted order.
154  for (std::set<int32>::const_iterator iter = ivector_times.begin();
155  iter != ivector_times.end(); ++iter) {
156  int32 t = *iter, x = 0;
157  request->inputs[1].indexes.push_back(Index(n, t, x));
158  }
159  }
160  }
161 }
struct rnnlm::@11::@12 n

◆ CreateLoopedComputationRequest()

void CreateLoopedComputationRequest ( const Nnet nnet,
int32  chunk_size,
int32  frame_subsampling_factor,
int32  ivector_period,
int32  left_context_begin,
int32  right_context,
int32  num_sequences,
ComputationRequest request1,
ComputationRequest request2,
ComputationRequest request3 
)

This function creates computation request suitable for giving to ComputeLooped().

It's intended for use with a 'simple' nnet (one satisfying IsSimpleNnet()), and this basically means that the inputs must be named "input" and possibly "ivector", and that there is an output named "output", and that those are the ones you care about (it won't generate any other outputs or use any other inputs).

If you want to use looped computation for different types of neural net, you should use the deeper interface, CompileLooped().

Parameters
[in]nnetThe neural net this computation request is to be used with. This is used to check whether the neural net accepts iVectors, and to work out the left-context and right-context required by the network.
[in]chunk_sizeThe number of frames of output that will be generated for each chunk (note: this is the shift in the t-index, which will not equal the number of output frames if frame_subsampling_factor != 1). Note: it is required that chunk_size be a multiple of ivector_period, frame_subsampling_factor, and nnet.Modulus(). You should use GetChunkSize() to compute the chunk size, giving it an advisory/ minimum chunksize, to make sure it satisfies these properties.
[in]frame_subsampling_factorThis will normally be 1, but may be more than 1 (e.g. 3) in chain systems; it determines the frame-skipping on the output, so we evaluate the output with 't' at multiples of this value.
[in]ivector_periodThe period with which iVectors are to be supplied to the network (if you're using iVectors). Not necessarily the same as the period with which the ivectors are extracted or stored on disk (–online-ivector-period). You will normally set this to the chunk size. It must divide the chunk size (if you're using iVectors) Note: you should call ModifyNnetIvectorPeriod on 'nnet' before calling this function; otherwise the neural net will most likely not actually be able to consume the iVector with this frequency.
[in]left_context_beginThis should be the left-context of the network plus any additional left-context (provided via the option –extra-left-context-begin) that should be supplied to the network on top of the minimum that the network requires. We call this left_context_begin because this only relates to the start of the utterance (t=0).
[in]right_contextThis should be the right-context of the network, plus any additional right-context ("extra-right-context") that should be supplied to the network on top of the minimum that the network requires (currently extra-right-context != 0 is is not supported at the command-line level).
[in]num_sequencesThe number of separate 'n' values to put in the computation; normally this will be just 1, but it can be increased to allow simultaneous operation on multiple streams of input.
[out]request1The first of the 3 requests that this function generates, that the user should then supply to CompileLooped(). Note: this will tend to be the largest computation request in terms of input, because we have to provide enough left and right context that it can evaluate the first chunk. Note: as elsewhere, the job of duplicating first and last frames enough to provide the required left/right context to the network, is left to the caller (at runtime, not during compilation).
[out]request2The second of the 3 requests that this function generates. Caution: none of the inputs and outputs should overlap.
[out]request3The third of the 3 requests that this function generates. It will be the same as request2, except for a time offset.

Definition at line 164 of file nnet-compile-looped.cc.

References CreateComputationRequestInternal(), Nnet::InputDim(), KALDI_ASSERT, Mod(), and Nnet::Modulus().

Referenced by CreateLoopedComputationRequestSimple(), and DecodableNnetSimpleLoopedInfo::Init().

173  {
174  bool has_ivector = (nnet.InputDim("ivector") > 0);
175  KALDI_ASSERT(chunk_size % frame_subsampling_factor == 0 &&
176  chunk_size % nnet.Modulus() == 0 &&
177  chunk_size % ivector_period == 0);
178  KALDI_ASSERT(left_context_begin >= 0 && right_context >= 0);
179  // note, 'end' is one past the last one.
180  int32 chunk1_input_begin_t = - left_context_begin,
181  chunk1_input_end_t = chunk_size + right_context,
182  chunk2_input_begin_t = chunk1_input_end_t,
183  chunk2_input_end_t = chunk2_input_begin_t + chunk_size,
184  chunk3_input_begin_t = chunk2_input_end_t,
185  chunk3_input_end_t = chunk3_input_begin_t + chunk_size;
186 
187 
188  // work out the times at which i-vectors are required.
189  std::set<int32> ivector_times1, ivector_times2, ivector_times3;
190  if (has_ivector) {
191  for (int32 t = chunk1_input_begin_t; t < chunk1_input_end_t; t++) {
192  int32 ivector_t = t - Mod(t, ivector_period);
193  ivector_times1.insert(ivector_t);
194  }
195  for (int32 t = chunk2_input_begin_t; t < chunk2_input_end_t; t++) {
196  int32 ivector_t = t - Mod(t, ivector_period);
197  if (ivector_times2.count(ivector_t) == 0 &&
198  ivector_times1.count(ivector_t) == 0)
199  ivector_times2.insert(ivector_t);
200  }
201  for (int32 t = chunk3_input_begin_t; t < chunk3_input_end_t; t++) {
202  int32 ivector_t = t - Mod(t, ivector_period);
203  if (ivector_times3.count(ivector_t) == 0 &&
204  ivector_times2.count(ivector_t) == 0 &&
205  ivector_times1.count(ivector_t) == 0)
206  ivector_times3.insert(ivector_t);
207  }
208  }
209 
211  chunk1_input_begin_t, chunk1_input_end_t,
212  0, chunk_size,
213  num_sequences, frame_subsampling_factor,
214  ivector_times1,
215  request1);
216 
218  chunk2_input_begin_t, chunk2_input_end_t,
219  chunk_size, chunk_size * 2,
220  num_sequences, frame_subsampling_factor,
221  ivector_times2,
222  request2);
223 
225  chunk3_input_begin_t, chunk3_input_end_t,
226  chunk_size * 2, chunk_size * 3,
227  num_sequences, frame_subsampling_factor,
228  ivector_times3,
229  request3);
230 
231 }
static void CreateComputationRequestInternal(int32 begin_input_t, int32 end_input_t, int32 begin_output_t, int32 end_output_t, int32 num_sequences, int32 frame_subsampling_factor, const std::set< int32 > &ivector_times, ComputationRequest *request)
I Mod(I m, I n)
Mod(m, n), defined for integers m and n where n > 0, returns the modulus m % n, defined as the intege...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ CreateLoopedComputationRequestSimple()

void CreateLoopedComputationRequestSimple ( const Nnet nnet,
int32  chunk_size,
int32  frame_subsampling_factor,
int32  ivector_period,
int32  extra_left_context_begin,
int32  extra_right_context,
int32  num_sequences,
ComputationRequest request1,
ComputationRequest request2,
ComputationRequest request3 
)

This function is deprecated.

It has the same interface as CreateLoopedComputationRequest(), except that the left and right context are specified in a different way (as just the 'extra' part). It is deprecated because this function has to work out the left and right context of the network, which turns out to be quite slow if it's done after you call ModifyNnetIvectorPeriod().

Definition at line 361 of file nnet-compile-looped.cc.

References ComputeSimpleNnetContext(), and CreateLoopedComputationRequest().

Referenced by UnitTestNnetCompileLooped().

370  {
371  int32 left_context, right_context;
372  ComputeSimpleNnetContext(nnet, &left_context, &right_context);
373 
374  CreateLoopedComputationRequest(nnet, chunk_size, frame_subsampling_factor,
375  ivector_period,
376  extra_left_context_begin + left_context,
377  extra_right_context + right_context,
378  num_sequences, request1, request2, request3);
379 }
void CreateLoopedComputationRequest(const Nnet &nnet, int32 chunk_size, int32 frame_subsampling_factor, int32 ivector_period, int32 left_context_begin, int32 right_context, int32 num_sequences, ComputationRequest *request1, ComputationRequest *request2, ComputationRequest *request3)
This function creates computation request suitable for giving to ComputeLooped(). ...
void ComputeSimpleNnetContext(const Nnet &nnet, int32 *left_context, int32 *right_context)
ComputeSimpleNnetContext computes the left-context and right-context of a nnet.
Definition: nnet-utils.cc:146

◆ DescriptorTokenize()

bool DescriptorTokenize ( const std::string &  input,
std::vector< std::string > *  tokens 
)

This function tokenizes input when parsing Descriptor configuration values.

A token in this context is not the same as a generic Kaldi token, e.g. as defined in IsToken() in util/text_utils.h, which just means a non-empty whitespace-free string. Here a token is more like a programming-language token, and currently the following are allowed as tokens: "(" ")" ","

  • A nonempty string beginning with A-Za-z_, and containing only -_A-Za-z0-9.
  • An integer, optionally beginning with - or + and then a nonempty sequence of 0-9.

This function should return false and print an informative error with local context if it can't tokenize the input.

Definition at line 377 of file nnet-parse.cc.

References kaldi::ConvertStringToReal(), ErrorContext(), IsValidName(), KALDI_ASSERT, and KALDI_WARN.

Referenced by NormalizeTextDescriptor(), Nnet::ProcessComponentNodeConfigLine(), Nnet::ProcessOutputNodeConfigLine(), Nnet::RemoveSomeNodes(), ModelCollapser::ReplaceNodeInDescriptor(), UnitTestDescriptorIo(), UnitTestDescriptorTokenize(), and UnitTestGeneralDescriptor().

378  {
379  KALDI_ASSERT(tokens != NULL);
380  size_t start = input.find_first_not_of(" \t"), size = input.size();
381  tokens->clear();
382  while (start < size) {
383  KALDI_ASSERT(!isspace(input[start]));
384  if (input[start] == '(' || input[start] == ')' || input[start] == ',') {
385  tokens->push_back(std::string(input, start, 1));
386  start = input.find_first_not_of(" \t", start + 1);
387  } else {
388  size_t found = input.find_first_of(" \t(),", start);
389  KALDI_ASSERT(found != start);
390  if (found == std::string::npos) {
391  std::string str(input, start, input.size() - start);
392  BaseFloat tmp;
393  if (!IsValidName(str) && !ConvertStringToReal(str, &tmp)) {
394  KALDI_WARN << "Could not tokenize line " << ErrorContext(std::string(input, start));
395  return false;
396  }
397  tokens->push_back(str);
398  break;
399  } else {
400  if (input[found] == '(' || input[found] == ')' || input[found] == ',') {
401  std::string str(input, start, found - start);
402  BaseFloat tmp;
403  if (!IsValidName(str) && !ConvertStringToReal(str, &tmp)) {
404  KALDI_WARN << "Could not tokenize line " << ErrorContext(std::string(input, start));
405  return false;
406  }
407  tokens->push_back(str);
408  start = found;
409  } else {
410  std::string str(input, start, found - start);
411  BaseFloat tmp;
412  if (!IsValidName(str) && !ConvertStringToReal(str, &tmp)) {
413  KALDI_WARN << "Could not tokenize line " << ErrorContext(std::string(input, start));
414  return false;
415  }
416  tokens->push_back(str);
417  start = input.find_first_not_of(" \t", found);
418  }
419  }
420  }
421  }
422  return true;
423 }
float BaseFloat
Definition: kaldi-types.h:29
std::string ErrorContext(const std::string &str)
Definition: nnet-parse.cc:461
bool ConvertStringToReal(const std::string &str, T *out)
ConvertStringToReal converts a string into either float or double and returns false if there was any ...
Definition: text-utils.cc:238
#define KALDI_WARN
Definition: kaldi-error.h:126
bool IsValidName(const std::string &name)
Returns true if &#39;name&#39; would be a valid name for a component or node in a Nnet.
Definition: nnet-parse.cc:425
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ DotProduct()

BaseFloat DotProduct ( const Nnet nnet1,
const Nnet nnet2 
)

Returns dot product between two networks of the same structure (calls the DotProduct functions of the Updatable components and sums up the return values).

Definition at line 250 of file nnet-utils.cc.

References UpdatableComponent::DotProduct(), Nnet::GetComponent(), KALDI_ASSERT, kUpdatableComponent, Nnet::NumComponents(), and Component::Properties().

Referenced by AffineComponent::BackpropNeedsOutput(), BlockAffineComponent::BackpropNeedsOutput(), Convolutional1dComponent::BackpropNeedsOutput(), ScaleAndOffsetComponent::Copy(), ConstantComponent::IsComputable(), LinearComponent::LinearComponent(), AffineComponent::Properties(), BlockAffineComponent::Properties(), RepeatedAffineComponent::Properties(), PerElementScaleComponent::Properties(), PerElementOffsetComponent::Properties(), ConstantFunctionComponent::Properties(), ConvolutionComponent::Properties(), LstmNonlinearityComponent::Properties(), NnetDiscriminativeTrainer::Train(), CompositeComponent::Type(), UnitTestNnetModelDerivatives(), and UpdatableComponent::~UpdatableComponent().

251  {
252  KALDI_ASSERT(nnet1.NumComponents() == nnet2.NumComponents());
253  BaseFloat ans = 0.0;
254  for (int32 c = 0; c < nnet1.NumComponents(); c++) {
255  const Component *comp1 = nnet1.GetComponent(c),
256  *comp2 = nnet2.GetComponent(c);
257  if (comp1->Properties() & kUpdatableComponent) {
258  const UpdatableComponent
259  *u_comp1 = dynamic_cast<const UpdatableComponent*>(comp1),
260  *u_comp2 = dynamic_cast<const UpdatableComponent*>(comp2);
261  KALDI_ASSERT(u_comp1 != NULL && u_comp2 != NULL);
262  ans += u_comp1->DotProduct(*u_comp2);
263  }
264  }
265  return ans;
266 }
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:165

◆ EnsureContiguousProperty()

void EnsureContiguousProperty ( const std::vector< int32 > &  indexes,
std::vector< std::vector< int32 > > *  indexes_out 
)

This function takes a vector of indexes and splits it up into as separate vectors of the same size, as needed to ensure that the 'contiguous property' holds.

This is done via padding with -1's. An example will clarify this. Suppose the input is: [ -1 1 1 1 2 2 1 1 ] which lacks the contiguous property because 1's appear in 2 different places, it would split it up as [ -1 1 1 1 2 2 -1 -1 ] [ -1 -1 -1 -1 -1 -1 1 1 ] If 'indexes' is empty or only contains -1's, 'indexes_out' will be empty.

Definition at line 401 of file nnet-compile-utils.cc.

References rnnlm::i.

Referenced by SplitLocationsBackward(), and UnitTestEnsureContiguousProperty().

403  {
404  indexes_out->clear();
405  indexes_out->reserve(3);
406  if (indexes.empty()) return;
407  int32 max_value = *std::max_element(indexes.begin(), indexes.end());
408  if (max_value == -1) return;
409  std::vector<int32> num_segments_seen(max_value + 1, 0);
410  int32 dim = indexes.size(), num_output_vectors = 0;
411  for (int32 i = 0; i < dim;) {
412  // note, we increment i within the loop.
413  if (indexes[i] == -1) {
414  i++;
415  continue;
416  }
417  int32 value = indexes[i], start_index = i;
418  for (; i < dim && indexes[i] == value; i++);
419  int32 end_index = i; // one past the end.
420  // the input 'indexes' contains a sequence of possibly-repeated instances of
421  // the value 'value', starting at index 'start_index', with 'end_index' as
422  // one past the end.
423  int32 this_num_segments_seen = num_segments_seen[value]++;
424  if (this_num_segments_seen >= num_output_vectors) { // we have nowhere to
425  // put it.
426  indexes_out->resize(++num_output_vectors);
427  indexes_out->back().resize(dim, -1); // fill newly added vector with -1's.
428  }
429  std::vector<int32> &this_out_vec((*indexes_out)[this_num_segments_seen]);
430  std::vector<int32>::iterator iter = this_out_vec.begin() + start_index,
431  end = this_out_vec.begin() + end_index;
432  // Fill the appropriate range of the output vector with 'value'
433  for (; iter != end; ++iter) *iter = value;
434  }
435 }

◆ ErrorContext() [1/2]

std::string ErrorContext ( std::istream &  is)

Return a string used in error messages.

Here, "is" will be from an istringstream derived from a single line or part of a line. If "is" is at EOF or in error state, this should just say "end of line", else if the contents of "is" before EOF is <20 characters it should return it all, else it should return the first 20 characters followed by "...".

Definition at line 451 of file nnet-parse.cc.

Referenced by DescriptorTokenize().

451  {
452  if (!is.good()) return "end of line";
453  char buf[21];
454  is.read(buf, 21);
455  if (is) {
456  return (std::string(buf, 20) + "...");
457  }
458  return std::string(buf, is.gcount());
459 }

◆ ErrorContext() [2/2]

std::string ErrorContext ( const std::string &  str)

Definition at line 461 of file nnet-parse.cc.

461  {
462  if (str.size() == 0) return "end of line";
463  if (str.size() <= 20) return str;
464  return std::string(str, 0, 20) + "...";
465 }

◆ EvaluateComputationRequest()

void EvaluateComputationRequest ( const Nnet nnet,
const ComputationRequest request,
std::vector< std::vector< bool > > *  is_computable 
)

Given an nnet and a computation request, this function works out which requested outputs in the computation request are computable; it outputs this information as a vector "is_computable" indexed by the same indexes as request.outputs.

It does this by executing some of the early stages of compilation.

Definition at line 71 of file nnet-utils.cc.

References ComputationGraphBuilder::Compute(), ComputationGraphBuilder::GetComputableInfo(), Nnet::GetNodeNames(), kaldi::GetVerboseLevel(),