All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
kaldi::nnet3 Namespace Reference

Namespaces

 attention
 
 computation_graph
 
 time_height_convolution
 

Classes

struct  Access
 
class  AffineComponent
 
class  AmNnetSimple
 
struct  Analyzer
 This struct exists to set up various pieces of analysis; it helps avoid the repetition of code where we compute all these things in sequence. More...
 
class  BackpropTruncationComponent
 
class  BackpropTruncationComponentPrecomputedIndexes
 
class  BatchNormComponent
 
class  BinarySumDescriptor
 BinarySumDescriptor can represent either A + B, or (A if defined, else B). More...
 
class  BlockAffineComponent
 This class implements an affine transform using a block diagonal matrix e.g., one whose weight matrix is all zeros except for blocks on the diagonal. More...
 
class  CachingOptimizingCompiler
 This class enables you to do the compilation and optimization in one call, and also ensures that if the ComputationRequest is identical to the previous one, the compilation process is not repeated. More...
 
struct  CachingOptimizingCompilerOptions
 
class  ChainExampleMerger
 This class is responsible for arranging examples in groups that have the same strucure (i.e. More...
 
struct  ChainObjectiveInfo
 
class  ChainTrainerMemoryHolder
 
struct  CheckComputationOptions
 
struct  ChunkInfo
 
struct  ChunkTimeInfo
 struct ChunkTimeInfo is used by class UtteranceSplitter to output information about how we split an utterance into chunks. More...
 
struct  CindexHasher
 
class  CindexSet
 
struct  CindexVectorHasher
 
class  ClipGradientComponent
 
struct  CollapseModelConfig
 Config class for the CollapseModel function. More...
 
struct  CommandAttributes
 
struct  CommandPairComparator
 
struct  ComparePair
 
class  Compiler
 This class creates an initial version of the NnetComputation, without any optimization or sharing of matrices. More...
 
struct  CompilerOptions
 
class  Component
 Abstract base-class for neural-net components. More...
 
class  ComponentPrecomputedIndexes
 
class  CompositeComponent
 CompositeComponent is a component representing a sequence of [simple] components. More...
 
class  ComputationAnalysis
 This class performs various kinds of specific analysis on top of what class Analyzer gives you immediately. More...
 
class  ComputationCache
 Class ComputationCache is used inside class CachingOptimizingCompiler to cache previously computed computations. More...
 
class  ComputationChecker
 
class  ComputationExpander
 
struct  ComputationGraph
 The first step in compilation is to turn the ComputationSpecification into a ComputationGraph, where for each Cindex we have a list of other Cindexes that it depends on. More...
 
class  ComputationGraphBuilder
 An abstract representation of a set of Cindexes. More...
 
class  ComputationLoopedOptimizer
 
class  ComputationRenumberer
 
struct  ComputationRequest
 
struct  ComputationRequestHasher
 
struct  ComputationRequestPtrEqual
 
class  ComputationStepsComputer
 This class arranges the cindex_ids of the computation into a sequence of lists called "steps", which will correspond roughly to the commands in the compiled computation. More...
 
class  ComputationVariables
 This class relates the matrices and sub-matrices in the computation to imaginary "variables", such that we can think of the operations as operating on sets of individual variables, and we can then do analysis that lets us do optimization. More...
 
class  ConfigLine
 This class is responsible for parsing input like hi-there xx=yyy a=b c empty= f-oo=Append(bar, sss) ba_z=123 bing='a b c' baz="a b c d='a b' e" and giving you access to the fields, in this case. More...
 
class  ConstantComponent
 
class  ConstantFunctionComponent
 
class  ConstantSumDescriptor
 This is an alternative base-case of SumDescriptor (an alternative to SimpleSumDescriptor) which represents a constant term, e.g. More...
 
class  ConvolutionComponent
 WARNING, this component is deprecated in favor of TimeHeightConvolutionComponent, and will be deleted. More...
 
class  DecodableAmNnetLoopedOnline
 
class  DecodableAmNnetSimple
 
class  DecodableAmNnetSimpleLooped
 
class  DecodableAmNnetSimpleParallel
 
class  DecodableNnetLoopedOnline
 
class  DecodableNnetLoopedOnlineBase
 
class  DecodableNnetSimple
 
class  DecodableNnetSimpleLooped
 
class  DecodableNnetSimpleLoopedInfo
 When you instantiate class DecodableNnetSimpleLooped, you should give it a const reference to this class, that has been previously initialized. More...
 
class  DerivativeTimeLimiter
 
class  Descriptor
 
class  DiscriminativeExampleMerger
 This class is responsible for arranging examples in groups that have the same strucure (i.e. More...
 
struct  DiscriminativeObjectiveFunctionInfo
 
class  DistributeComponent
 This Component takes a larger input-dim than output-dim, where the input-dim must be a multiple of the output-dim, and distributes different blocks of the input dimension to different 'x' values. More...
 
class  DistributeComponentPrecomputedIndexes
 
class  DropoutComponent
 
class  DropoutMaskComponent
 
class  ElementwiseProductComponent
 
struct  ExampleGenerationConfig
 
class  ExampleMerger
 This class is responsible for arranging examples in groups that have the same strucure (i.e. More...
 
class  ExampleMergingConfig
 
class  ExampleMergingStats
 This class is responsible for storing, and displaying in log messages, statistics about how examples of different sizes (c.f. More...
 
struct  FirstElementComparator
 
struct  FirstElementIsEqualComparator
 
class  FixedAffineComponent
 FixedAffineComponent is an affine transform that is supplied at network initialization time and is not trainable. More...
 
class  FixedBiasComponent
 FixedBiasComponent applies a fixed per-element bias; it's similar to the AddShift component in the nnet1 setup (and only needed for nnet1 model conversion. More...
 
class  FixedScaleComponent
 FixedScaleComponent applies a fixed per-element scale; it's similar to the Rescale component in the nnet1 setup (and only needed for nnet1 model conversion). More...
 
class  ForwardingDescriptor
 A ForwardingDescriptor describes how we copy data from another NetworkNode, or from multiple other NetworkNodes, possibly with a scalar weight. More...
 
struct  GeneralDescriptor
 This class is only used when parsing Descriptors. More...
 
class  GeneralDropoutComponent
 GeneralDropoutComponent implements dropout, including a continuous variant where the thing we multiply is not just zero or one, but may be a continuous value. More...
 
class  GeneralDropoutComponentPrecomputedIndexes
 
struct  ImageAugmentationConfig
 
struct  Index
 struct Index is intended to represent the various indexes by which we number the rows of the matrices that the Components process: mainly 'n', the index of the member of the minibatch, 't', used for the frame index in speech recognition, and 'x', which is a catch-all extra index which we might use in convolutional setups or for other reasons. More...
 
struct  IndexHasher
 
struct  IndexLessNxt
 
class  IndexSet
 An abstract representation of a set of Indexes. More...
 
struct  IndexVectorHasher
 
struct  IoSpecification
 
struct  IoSpecificationHasher
 
class  LinearComponent
 
class  LogSoftmaxComponent
 
class  LstmNonlinearityComponent
 
struct  MatrixAccesses
 
class  MatrixExtender
 
class  MaxpoolingComponent
 
class  MemoryCompressionOptimizer
 This class is used in the function OptimizeMemoryCompression(), once we determine that there is some potential to do memory compression for this computation. More...
 
struct  MiscComputationInfo
 
class  ModelCollapser
 
class  ModelUpdateConsolidator
 This class is responsible for consolidating the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices. More...
 
class  NaturalGradientAffineComponent
 
class  NaturalGradientPerElementScaleComponent
 NaturalGradientPerElementScaleComponent is like PerElementScaleComponent but it uses a natural gradient update for the per-element scales. More...
 
class  NaturalGradientRepeatedAffineComponent
 
struct  NetworkNode
 NetworkNode is used to represent, three types of thing: either an input of the network (which pretty much just states the dimension of the input vector); a Component (e.g. More...
 
class  Nnet
 
class  NnetChainComputeProb
 This class is for computing objective-function values in a nnet3+chain setup, for diagnostics. More...
 
struct  NnetChainExample
 NnetChainExample is like NnetExample, but specialized for lattice-free (chain) training. More...
 
struct  NnetChainExampleStructureCompare
 This comparator object compares just the structural aspects of the NnetChainExample without looking at the value of the features. More...
 
struct  NnetChainExampleStructureHasher
 This hashing object hashes just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetChainSupervision
 
class  NnetChainTrainer
 This class is for single-threaded training of neural nets using the 'chain' model. More...
 
struct  NnetChainTrainingOptions
 
struct  NnetComputation
 
struct  NnetComputeOptions
 
class  NnetComputeProb
 This class is for computing cross-entropy and accuracy values in a neural network, for diagnostics. More...
 
struct  NnetComputeProbOptions
 
class  NnetComputer
 class NnetComputer is responsible for executing the computation described in the "computation" object. More...
 
class  NnetComputerFromEg
 
class  NnetDiscriminativeComputeObjf
 This class is for computing objective-function values in a nnet3 discriminative training, for diagnostics. More...
 
struct  NnetDiscriminativeExample
 NnetDiscriminativeExample is like NnetExample, but specialized for sequence training. More...
 
struct  NnetDiscriminativeExampleStructureCompare
 This comparator object compares just the structural aspects of the NnetDiscriminativeExample without looking at the value of the features. More...
 
struct  NnetDiscriminativeExampleStructureHasher
 This hashing object hashes just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetDiscriminativeOptions
 
struct  NnetDiscriminativeSupervision
 
class  NnetDiscriminativeTrainer
 This class is for single-threaded discriminative training of neural nets. More...
 
struct  NnetExample
 NnetExample is the input data and corresponding label (or labels) for one or more frames of input, used for standard cross-entropy training of neural nets (and possibly for other objective functions). More...
 
struct  NnetExampleStructureCompare
 This comparator object compares just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetExampleStructureHasher
 This hashing object hashes just the structural aspects of the NnetExample without looking at the value of the features. More...
 
struct  NnetGenerationOptions
 
struct  NnetIo
 
struct  NnetIoStructureCompare
 This comparison object compares just the structural aspects of the NnetIo object (name, indexes, feature dimension) without looking at the value of features. More...
 
struct  NnetIoStructureHasher
 This hashing object hashes just the structural aspects of the NnetIo object (name, indexes, feature dimension) without looking at the value of features. More...
 
class  NnetLdaStatsAccumulator
 
struct  NnetOptimizeOptions
 
struct  NnetSimpleComputationOptions
 
struct  NnetSimpleLoopedComputationOptions
 
class  NnetTrainer
 This class is for single-threaded training of neural nets using standard objective functions such as cross-entropy (implemented with logsoftmax nonlinearity and a linear objective function) and quadratic loss. More...
 
struct  NnetTrainerOptions
 
class  NonlinearComponent
 
class  NoOpComponent
 NoOpComponent just duplicates its input. More...
 
class  NormalizeComponent
 
struct  ObjectiveFunctionInfo
 
class  OffsetForwardingDescriptor
 Offsets in 't' and 'x' values of other ForwardingDescriptors. More...
 
class  OnlineNaturalGradient
 Keywords for search: natural gradient, naturalgradient, NG-SGD. More...
 
class  OnlineNaturalGradientSimple
 
class  OptionalSumDescriptor
 This is the case of class SumDescriptor, in which we contain just one term, and that term is optional (an IfDefined() expression). More...
 
struct  PairIsEqualComparator
 
struct  PerDimObjectiveInfo
 
class  PerElementOffsetComponent
 
class  PerElementScaleComponent
 PerElementScaleComponent scales each dimension of its input with a separate trainable scale; it's like a linear component with a diagonal matrix. More...
 
class  PermuteComponent
 PermuteComponent changes the order of the columns (i.e. More...
 
class  PnormComponent
 
class  RandomComponent
 
class  RectifiedLinearComponent
 
class  RepeatedAffineComponent
 
class  ReplaceIndexForwardingDescriptor
 This ForwardingDescriptor modifies the indexes (n, t, x) by replacing one of them (normally t) with a constant value and keeping the rest. More...
 
class  RestrictedAttentionComponent
 RestrictedAttentionComponent implements an attention model with restricted temporal context. More...
 
class  RoundingForwardingDescriptor
 For use in clockwork RNNs and the like, this forwarding-descriptor rounds the time-index t down to the the closest t' <= t that is an exact multiple of t_modulus_. More...
 
class  RowOpsSplitter
 
class  ScaleAndOffsetComponent
 
class  SigmoidComponent
 
class  SimpleForwardingDescriptor
 SimpleForwardingDescriptor is the base-case of ForwardingDescriptor, consisting of a source node in the graph with a given scalar weight (which will in the normal case be 1.0). More...
 
struct  SimpleObjectiveInfo
 
class  SimpleSumDescriptor
 This is the normal base-case of SumDescriptor which just wraps a ForwardingDescriptor. More...
 
class  SoftmaxComponent
 
class  StatisticsExtractionComponent
 
class  StatisticsExtractionComponentPrecomputedIndexes
 
class  StatisticsPoolingComponent
 
class  StatisticsPoolingComponentPrecomputedIndexes
 
class  SumBlockComponent
 SumBlockComponent sums over blocks of its input: for instance, if you create one with the config "input-dim=400 output-dim=100", its output will be the sum over the 4 100-dimensional blocks of the input. More...
 
class  SumDescriptor
 This is an abstract base-class. More...
 
class  SumGroupComponent
 SumGroupComponent is used to sum up groups of posteriors. More...
 
class  SvdApplier
 
class  SwitchingForwardingDescriptor
 Chooses from different inputs based on the the time index modulo (the number of ForwardingDescriptors given as inputs). More...
 
class  TanhComponent
 
struct  TarjanNode
 
class  TimeHeightConvolutionComponent
 TimeHeightConvolutionComponent implements 2-dimensional convolution where one of the dimensions of convolution (which traditionally would be called the width axis) is identified with time (i.e. More...
 
class  UpdatableComponent
 Class UpdatableComponent is a Component which has trainable parameters; it extends the interface of Component. More...
 
class  UtteranceSplitter
 
class  VariableMergingOptimizer
 This class is responsible for merging matrices, although you probably want to access it via the the function VariableMergingOptimization(). More...
 

Typedefs

typedef TableWriter
< KaldiObjectHolder
< NnetChainExample > > 
NnetChainExampleWriter
 
typedef SequentialTableReader
< KaldiObjectHolder
< NnetChainExample > > 
SequentialNnetChainExampleReader
 
typedef
RandomAccessTableReader
< KaldiObjectHolder
< NnetChainExample > > 
RandomAccessNnetChainExampleReader
 
typedef std::pair< int32, IndexCindex
 
typedef TableWriter
< KaldiObjectHolder
< NnetDiscriminativeExample > > 
NnetDiscriminativeExampleWriter
 
typedef SequentialTableReader
< KaldiObjectHolder
< NnetDiscriminativeExample > > 
SequentialNnetDiscriminativeExampleReader
 
typedef
RandomAccessTableReader
< KaldiObjectHolder
< NnetDiscriminativeExample > > 
RandomAccessNnetDiscriminativeExampleReader
 
typedef TableWriter
< KaldiObjectHolder
< NnetExample > > 
NnetExampleWriter
 
typedef SequentialTableReader
< KaldiObjectHolder
< NnetExample > > 
SequentialNnetExampleReader
 
typedef
RandomAccessTableReader
< KaldiObjectHolder
< NnetExample > > 
RandomAccessNnetExampleReader
 

Enumerations

enum  AccessType { kReadAccess, kWriteAccess, kReadWriteAccess }
 
enum  ComponentProperties {
  kSimpleComponent = 0x001, kUpdatableComponent = 0x002, kPropagateInPlace = 0x004, kPropagateAdds = 0x008,
  kReordersIndexes = 0x010, kBackpropAdds = 0x020, kBackpropNeedsInput = 0x040, kBackpropNeedsOutput = 0x080,
  kBackpropInPlace = 0x100, kStoresStats = 0x200, kInputContiguous = 0x400, kOutputContiguous = 0x800,
  kUsesMemo = 0x1000, kRandomComponent = 0x2000
}
 
enum  CommandType {
  kAllocMatrix, kDeallocMatrix, kSwapMatrix, kSetConst,
  kPropagate, kBackprop, kBackpropNoModelUpdate, kMatrixCopy,
  kMatrixAdd, kCopyRows, kAddRows, kCopyRowsMulti,
  kCopyToRowsMulti, kAddRowsMulti, kAddToRowsMulti, kAddRowRanges,
  kCompressMatrix, kDecompressMatrix, kAcceptInput, kProvideOutput,
  kNoOperation, kNoOperationPermanent, kNoOperationMarker, kNoOperationLabel,
  kGotoLabel
}
 CommandType is an enum that describes the category of the command used in the NnetComputation. More...
 
enum  ObjectiveType { kLinear, kQuadratic }
 This enum is for a kind of annotation we associate with output nodes of the network; it's for the convenience of calling code so that if the objective is one of a few standard types, we can compute it directly and know how to interpret the supervision labels. More...
 
enum  NodeType {
  kInput, kDescriptor, kComponent, kDimRange,
  kNone
}
 
enum  FillMode { kNearest, kReflect }
 

Functions

void UnitTestPreconditionDirectionsOnline ()
 
std::string PrintCommand (int32 num_commands, int32 command)
 
void UnitTestNnetAnalyze ()
 
static void IndexesMultiToSubmatrixIndexes (const std::vector< std::pair< int32, int32 > > &indexes_multi, std::vector< int32 > *submatrix_indexes)
 given a vector of pairs from computation.indexes_multi_indexes containing paris (submatrix-index, row-index), this function outputs to "submatrix_indexes" all (unique) submatrix indexes that appear; and it outputs to "contains_null_marker" true if the pair (-1, -1) appears anywhere in indexes_multi, and false otherwise. More...
 
void ComputeCommandAttributes (const Nnet &nnet, const NnetComputation &computation, const ComputationVariables &vars, std::vector< CommandAttributes > *attributes)
 
void ComputeVariableAccesses (const ComputationVariables &variables, const std::vector< CommandAttributes > &command_attributes, std::vector< std::vector< Access > > *variable_accesses)
 After the command-level attributes have been computed, this function organizes them per variable (see class ComputationVariables for how a variable is defined; it is part of a matrix). More...
 
void ComputeMatrixAccesses (const Nnet &nnet, const NnetComputation &computation, const ComputationVariables &variables, const std::vector< CommandAttributes > &command_attributes, std::vector< MatrixAccesses > *matrix_accesses)
 This function organizes information in the CommandAttributes in a way that is convenient to access per matrix. More...
 
static void CheckComputationOnline (const Nnet &nnet, NnetComputation computation, bool check_rewrite)
 
void CheckComputation (const Nnet &nnet, const NnetComputation &computation, bool check_rewrite=false)
 This is a convenience interface for class ComputationChecker. More...
 
void ComputeMatrixToSubmatrix (const NnetComputation &computation, std::vector< std::vector< int32 > > *mat_to_submat)
 This function computes a vector 'mat_to_submat', indexed by matrix index, such that (*mat_to_submat)[m] is a list of all the submatrix indexes that refer to matrix m. More...
 
void PrintMatrixAccesses (std::ostream &os, const std::vector< MatrixAccesses > &matrix_accesses)
 This function is to be used in debugging; it produces human-readable output. More...
 
void PrintCommandAttributes (std::ostream &os, const std::vector< CommandAttributes > &attributes)
 This function is to be used in debugging; it produces human-readable output. More...
 
void GetCommandsOfType (const NnetComputation &computation, CommandType t, std::vector< int32 > *command_indexes)
 This utility function works out from a computation, the command-indexes of the commands of the given type. More...
 
int64 GetMaxMemoryUse (const NnetComputation &computation)
 
int32 MaxMemoryUsage (const NnetComputation &computation)
 Returns the total memory, in bytes, used by the computation (just the temporary memory, not counting the memory used by the nnet itself). More...
 
static bool HasXentOutputs (const Nnet &nnet)
 
void RecomputeStats (const std::vector< NnetChainExample > &egs, const chain::ChainTrainingOptions &chain_config, const fst::StdVectorFst &den_fst, Nnet *nnet)
 This function zeros the stored component-level stats in the nnet using ZeroComponentStats(), then recomputes them with the supplied egs. More...
 
static void MergeSupervision (const std::vector< const NnetChainSupervision * > &inputs, NnetChainSupervision *output)
 
void MergeChainExamples (bool compress, std::vector< NnetChainExample > *input, NnetChainExample *output)
 This function merges a list of NnetChainExample objects into a single one– intended to be used when forming minibatches for neural net training. More...
 
void GetChainComputationRequest (const Nnet &nnet, const NnetChainExample &eg, bool need_model_derivative, bool store_component_stats, bool use_xent_regularization, bool use_xent_derivative, ComputationRequest *computation_request)
 This function takes a NnetChainExample and produces a ComputationRequest. More...
 
void ShiftChainExampleTimes (int32 frame_shift, const std::vector< std::string > &exclude_names, NnetChainExample *eg)
 Shifts the time-index t of everything in the input of "eg" by adding "t_offset" to all "t" values– but excluding those with names listed in "exclude_names", e.g. More...
 
int32 GetNnetChainExampleSize (const NnetChainExample &a)
 
int32 GetChainNnetExampleSize (const NnetChainExample &a)
 This function returns the 'size' of a chain example as defined for purposes of merging egs, which is defined as the largest number of Indexes in any of the inputs or outputs of the example. More...
 
void UnitTestIndexIo ()
 
void UnitTestCindexIo ()
 
static void WriteIndexVectorElementBinary (std::ostream &os, const std::vector< Index > &vec, int32 i)
 
static void ReadIndexVectorElementBinary (std::istream &is, int32 i, std::vector< Index > *vec)
 
void WriteIndexVector (std::ostream &os, bool binary, const std::vector< Index > &vec)
 
void ReadIndexVector (std::istream &is, bool binary, std::vector< Index > *vec)
 
static void WriteCindexVectorElementBinary (std::ostream &os, const std::vector< Cindex > &vec, int32 i)
 
static void ReadCindexVectorElementBinary (std::istream &is, int32 i, std::vector< Cindex > *vec)
 
void WriteCindexVector (std::ostream &os, bool binary, const std::vector< Cindex > &vec)
 
void ReadCindexVector (std::istream &is, bool binary, std::vector< Cindex > *vec)
 
std::ostream & operator<< (std::ostream &ostream, const Index &index)
 
std::ostream & operator<< (std::ostream &ostream, const Cindex &cindex)
 
void PrintCindex (std::ostream &os, const Cindex &cindex, const std::vector< std::string > &node_names)
 
void PrintIndexes (std::ostream &ostream, const std::vector< Index > &indexes)
 this will only be used for pretty-printing. More...
 
void PrintCindexes (std::ostream &ostream, const std::vector< Cindex > &cindexes, const std::vector< std::string > &node_names)
 this will only be used for pretty-printing. More...
 
void PrintIntegerVector (std::ostream &os, const std::vector< int32 > &ints)
 
void AppendCindexes (int32 node, const std::vector< Index > &indexes, std::vector< Cindex > *out)
 Appends to 'out' the pairs (node, indexes[0]), (node, indexes[1]), ... More...
 
void ModifyNnetIvectorPeriod (int32 ivector_period, Nnet *nnet)
 This function modifies the descriptors in the neural network to change the periodicity with which it expects to read an iVector at its input. More...
 
int32 GetChunkSize (const Nnet &nnet, int32 frame_subsampling_factor, int32 advised_chunk_size)
 
template<class I >
Mod (I m, I n)
 Mod(m, n), defined for integers m and n where n > 0, returns the modulus m % n, defined as the integer 0 <= i < n such that i and m are congruent modulo n; for instance, Mod(13, 10) = 3. More...
 
static void CreateComputationRequestInternal (int32 begin_input_t, int32 end_input_t, int32 begin_output_t, int32 end_output_t, int32 num_sequences, int32 frame_subsampling_factor, const std::set< int32 > &ivector_times, ComputationRequest *request)
 
void CreateLoopedComputationRequest (const Nnet &nnet, int32 chunk_size, int32 frame_subsampling_factor, int32 ivector_period, int32 left_context_begin, int32 right_context, int32 num_sequences, ComputationRequest *request1, ComputationRequest *request2, ComputationRequest *request3)
 This function creates computation request suitable for giving to ComputeLooped(). More...
 
void AddTimeOffsetToComputationRequest (int32 t_offset, ComputationRequest *request)
 
static bool ExtrapolateComputationRequest (const ComputationRequest &request1, const ComputationRequest &request2, ComputationRequest *request3)
 
static bool CompileLoopedInternal (const Nnet &nnet, NnetOptimizeOptions optimize_opts, const ComputationRequest &request1, const ComputationRequest &request2, const ComputationRequest &request3, int32 num_requests, NnetComputation *computation)
 
void CompileLooped (const Nnet &nnet, const NnetOptimizeOptions &optimize_opts, const ComputationRequest &request1, const ComputationRequest &request2, const ComputationRequest &request3, NnetComputation *computation)
 CompileLooped() provides an internal interface for 'looped' computation. More...
 
void CreateLoopedComputationRequestSimple (const Nnet &nnet, int32 chunk_size, int32 frame_subsampling_factor, int32 ivector_period, int32 extra_left_context_begin, int32 extra_right_context, int32 num_sequences, ComputationRequest *request1, ComputationRequest *request2, ComputationRequest *request3)
 This function is deprecated. More...
 
void UnitTestNnetCompile ()
 
void UnitTestNnetCompileMulti ()
 
void UnitTestNnetCompileLooped ()
 
void PrintVectorVectorPair (std::vector< std::vector< std::pair< int32, int32 > > > vec_vec_pair)
 
void UnitTestSplitLocationsBackward (bool verbose)
 
void UnitTestHasContiguousProperty ()
 
void UnitTestEnsureContiguousProperty ()
 
void UnitTestSplitLocations (bool verbose)
 
bool SecondElementComparator (const std::pair< int32, int32 > &first_pair, const std::pair< int32, int32 > &second_pair)
 
void SortSubmatLists (const std::vector< std::vector< std::pair< int32, int32 > > > submat_lists, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, int32 *max_submat_list_size)
 
void ComputeSubmatIndexHistogram (const std::vector< std::vector< std::pair< int32, int32 > > > sorted_submat_lists, unordered_map< int32, std::vector< int32 > > *submat_histogram)
 
void FindSubmatIndexInSubmatLists (int32 submat_index, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::vector< std::pair< int32, int32 > >::iterator > *output_iterator_list, int32 *max_remaining_submat_list_size)
 
void ExtractGivenPairsFromSubmatLists (std::vector< std::vector< std::pair< int32, int32 > >::iterator > &input_iterator_list, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::pair< int32, int32 > > *list_of_pairs)
 
static void ExtractLastPairFromSubmatLists (std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::pair< int32, int32 > > *list_of_pairs)
 
static void SplitLocationsUsingSubmatHistogram (int32 max_submat_list_size, std::vector< std::vector< std::pair< int32, int32 > > > *sorted_submat_lists, std::vector< std::pair< int32, int32 > > *submat_histogram_vector, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 
void SplitLocations (const std::vector< std::vector< std::pair< int32, int32 > > > &submat_lists, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 The input to this function is a vector of lists of pairs, and this function splits it up into a list of vectors of pairs. More...
 
bool ConvertToIndexes (const std::vector< std::pair< int32, int32 > > &location_vector, int32 *first_value, std::vector< int32 > *second_values)
 If it is the case for some i >= 0 that all the .first elements of "location_vector" are either i or -1, then output i to first_value and the .second elements into "second_values", and return true. More...
 
void EnsureContiguousProperty (const std::vector< int32 > &indexes, std::vector< std::vector< int32 > > *indexes_out)
 This function takes a vector of indexes and splits it up into as separate vectors of the same size, as needed to ensure that the 'contiguous property' holds. More...
 
void SplitPairList (std::vector< std::pair< int32, int32 > > &list, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 This function splits a vector of pairs into a list of vectors of pairs. More...
 
void SplitLocationsBackward (const std::vector< std::vector< std::pair< int32, int32 > > > &submat_lists, std::vector< std::vector< std::pair< int32, int32 > > > *split_lists)
 This function has the same interface as SplitLocations(); however, it ensures certain additional properties of the output "split_lists", which are necessary because of the way it is used in backprop code. More...
 
bool HasContiguousProperty (const std::vector< int32 > &indexes, std::vector< std::pair< int32, int32 > > *reverse_indexes)
 This function returns true if for each integer i != -1, all the indexes j at which indexes[j] == i are consecutive with no gaps (more formally: if j1 < j2 < j3 and indexes[j1] != -1 and indexes[j1] == indexes[j3], then indexes[j1] == indexes[j2]). More...
 
void GetNxList (const std::vector< Index > &indexes, std::vector< std::pair< int32, int32 > > *pairs)
 This function outputs a unique, lexicographically sorted list of the pairs of (n, x) values that are encountered in the provided list of Indexes. More...
 
void GetTList (const std::vector< Index > &indexes, std::vector< int32 > *t_values)
 This function outputs a sorted, unique list of the 't' values that are encountered in the provided list of Indexes If 't' values equal to kNoTime are encountered, they are ignored and are not output. More...
 
static void ResetSeed (int32 rand_seed, const Component &c)
 
bool CheckStringsApproxEqual (const std::string &a, const std::string &b, int32 tolerance=3)
 
void TestNnetComponentIo (Component *c)
 
void TestNnetComponentCopy (Component *c)
 
void TestNnetComponentAddScale (Component *c)
 
void TestNnetComponentVectorizeUnVectorize (Component *c)
 
void TestNnetComponentUpdatable (Component *c)
 
void TestSimpleComponentPropagateProperties (const Component &c)
 
bool TestSimpleComponentDataDerivative (const Component &c, BaseFloat perturb_delta)
 
bool TestSimpleComponentModelDerivative (const Component &c, BaseFloat perturb_delta, bool test_derivative)
 
void UnitTestNnetComponent ()
 
std::ostream & operator<< (std::ostream &os, const ComputationGraphBuilder::ComputableInfo &info)
 This is to be used in logging only. More...
 
void ComputeComputationGraph (const Nnet &nnet, const ComputationRequest &request, ComputationGraph *graph)
 
static int32 SumVectorSizes (const std::vector< std::vector< int32 > > &vec)
 
static int32 SumVectorSizes (const std::vector< std::vector< std::vector< int32 > > > &vec)
 
static void ComputeComputationPhasesForEpoch (const Nnet &nnet, const ComputationGraph &graph, const std::vector< int32 > &this_epoch, const std::vector< std::vector< int32 > > &dependencies_subset, const std::vector< std::vector< int32 > > &depend_on_subset, bool epoch_is_trivial, std::vector< int32 > *phase_indexes, std::vector< std::vector< int32 > > *phases)
 
void ComputeComputationPhases (const Nnet &nnet, const ComputationGraph &computation_graph, std::vector< std::vector< std::vector< int32 > > > *phases_per_segment)
 This function divides a computation into 'phases', where a 'phase' is a collection of cindexes which can (as far as the computation graph is concerned) all be computed at the same time, and depend only on cindexes previously computed in earlier phases. More...
 
static void GetIndexesStrings (const Nnet &nnet, const NnetComputation &computation, std::vector< std::string > *indexes_strings)
 
static void GetIndexesMultiStrings (const Nnet &nnet, const NnetComputation &computation, std::vector< std::string > *indexes_multi_strings)
 
static void PrintCommand (std::ostream &os_out, const Nnet &nnet, const NnetComputation &computation, int32 command_index, const std::vector< std::string > &submatrix_strings, const std::vector< std::string > &indexes_strings, const std::vector< std::string > &indexes_multi_strings)
 
static void PrintComputationPreamble (std::ostream &os, const NnetComputation &c, const Nnet &nnet, const std::vector< std::string > &submatrix_strings, const std::vector< std::string > &indexes_strings, const std::vector< std::string > &indexes_multi_strings)
 
std::ostream & operator<< (std::ostream &os, NnetComputation &computation)
 
void UnitTestNnetComputationIo (NnetComputation *computation)
 
void UnitTestComputationRequestIo (ComputationRequest *request)
 
void TestNnetDecodable (Nnet *nnet)
 
void UnitTestNnetCompute ()
 
void ComputeMinAndMaxTimes (const std::vector< Index > &indexes, int32 *min_t, int32 *max_t)
 
void SetDerivTimesOptions (const ComputationRequest &request, NnetOptimizeOptions *opt_config)
 
void UnitTestNnetModelDerivatives ()
 
void UnitTestNnetInputDerivatives ()
 
ForwardingDescriptorGenRandForwardingDescriptor (int32 num_nodes)
 
SumDescriptorGenRandSumDescriptor (int32 num_nodes)
 
void GenRandDescriptor (int32 num_nodes, Descriptor *desc)
 
void UnitTestDescriptorIo ()
 
void UnitTestGeneralDescriptor ()
 
std::string NormalizeTextDescriptor (const std::vector< std::string > &node_names, const std::string &desc_str)
 
void UnitTestGeneralDescriptorSpecial ()
 
static std::string ParsingContext (const std::string *token_ptr)
 
static void ExpectToken (const std::string &token, const std::string &what_we_are_parsing, const std::string **next_token)
 
static int32 ReadIntegerToken (const std::string &what_we_are_parsing, const std::string **next_token)
 
void ComputeAccuracy (const GeneralMatrix &supervision, const CuMatrixBase< BaseFloat > &nnet_output, BaseFloat *tot_weight, BaseFloat *tot_accuracy, VectorBase< BaseFloat > *tot_weight_vec=NULL, VectorBase< BaseFloat > *tot_accuracy_vec=NULL)
 This function computes the frame accuracy for this minibatch. More...
 
void MergeSupervision (const std::vector< const NnetDiscriminativeSupervision * > &inputs, NnetDiscriminativeSupervision *output)
 
void MergeDiscriminativeExamples (bool compress, std::vector< NnetDiscriminativeExample > *input, NnetDiscriminativeExample *output)
 
void GetDiscriminativeComputationRequest (const Nnet &nnet, const NnetDiscriminativeExample &eg, bool need_model_derivative, bool store_component_stats, bool use_xent_regularization, bool use_xent_derivative, ComputationRequest *computation_request)
 This function takes a NnetDiscriminativeExample and produces a ComputationRequest. More...
 
void ShiftDiscriminativeExampleTimes (int32 frame_shift, const std::vector< std::string > &exclude_names, NnetDiscriminativeExample *eg)
 Shifts the time-index t of everything in the input of "eg" by adding "t_offset" to all "t" values– but excluding those with names listed in "exclude_names", e.g. More...
 
int32 GetNnetDiscriminativeExampleSize (const NnetDiscriminativeExample &a)
 
void MergeDiscriminativeExamples (std::vector< NnetDiscriminativeExample > *input, bool compress, NnetDiscriminativeExample *output)
 Appends the given vector of examples (which must be non-empty) into a single output example. More...
 
int32 GetDiscriminativeNnetExampleSize (const NnetDiscriminativeExample &a)
 This function returns the 'size' of a discriminative example as defined for purposes of merging egs, which is defined as the largest number of Indexes in any of the inputs or outputs of the example. More...
 
void UnitTestNnetExample ()
 
void UnitTestNnetMergeExamples ()
 
static void GetIoNames (const std::vector< NnetExample > &src, std::vector< std::string > *names_vec)
 
static void GetIoSizes (const std::vector< NnetExample > &src, const std::vector< std::string > &names, std::vector< int32 > *sizes)
 
static void MergeIo (const std::vector< NnetExample > &src, const std::vector< std::string > &names, const std::vector< int32 > &sizes, bool compress, NnetExample *merged_eg)
 
void MergeExamples (const std::vector< NnetExample > &src, bool compress, NnetExample *dest)
 Merge a set of input examples into a single example (typically the size of "src" will be the minibatch size). More...
 
void ShiftExampleTimes (int32 t_offset, const std::vector< std::string > &exclude_names, NnetExample *eg)
 Shifts the time-index t of everything in the "eg" by adding "t_offset" to all "t" values. More...
 
void GetComputationRequest (const Nnet &nnet, const NnetExample &eg, bool need_model_derivative, bool store_component_stats, ComputationRequest *computation_request)
 This function takes a NnetExample (which should already have been frame-selected, if desired, and merged into a minibatch) and produces a ComputationRequest. More...
 
void WriteVectorAsChar (std::ostream &os, bool binary, const VectorBase< BaseFloat > &vec)
 
void ReadVectorAsChar (std::istream &is, bool binary, Vector< BaseFloat > *vec)
 
void RoundUpNumFrames (int32 frame_subsampling_factor, int32 *num_frames, int32 *num_frames_overlap)
 
int32 GetNnetExampleSize (const NnetExample &a)
 This function returns the 'size' of a nnet-example as defined for purposes of merging egs, which is defined as the largest number of Indexes in any of the inputs or outputs of the example. More...
 
static void CopyPairVector (const CuArray< Int32Pair > &in, std::vector< std::pair< int32, int32 > > *out)
 
static void CopyPairVector (const std::vector< std::pair< int32, int32 > > &in, CuArray< Int32Pair > *out)
 
bool AssertGraphEqual (const std::vector< std::vector< int32 > > &graph1, const std::vector< std::vector< int32 > > &graph2)
 
bool AssertVectorEqual (const std::vector< int32 > &vec1, const std::vector< int32 > &vec2)
 
void BuildTestGraph (std::vector< std::vector< int32 > > *graph)
 
void BuildTestGraphTranspose (std::vector< std::vector< int32 > > *graph)
 
void BuildTestSccs (std::vector< std::vector< int32 > > *sccs)
 
void BuildTestSccGraph (std::vector< std::vector< int32 > > *scc_graph)
 
void BuildTestTopSortOrder (std::vector< int32 > *node_to_order)
 
void UnitTestComputeGraphTranspose ()
 
void UnitTestFindSccs ()
 
void UnitTestMakeSccGraph ()
 
void UnitTestComputeTopSortOrder ()
 
void UnitTestComputeTopSortOrder2 ()
 
void NnetToDirectedGraph (const Nnet &nnet, std::vector< std::vector< int32 > > *graph)
 This function takes an nnet and turns it to a directed graph on nodes. More...
 
void ComputeGraphTranspose (const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *graph_transpose)
 Outputs a graph in which the order of arcs is reversed. More...
 
void TarjanSccRecursive (int32 node, const std::vector< std::vector< int32 > > &graph, int32 *global_index, std::vector< TarjanNode > *tarjan_nodes, std::vector< int32 > *tarjan_stack, std::vector< std::vector< int32 > > *sccs)
 
void FindSccsTarjan (const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *sccs)
 
void FindSccs (const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *sccs)
 Given a directed graph (where each std::vector<int32> is a list of destination-nodes of arcs coming from the current node), partition it into strongly connected components (i.e. More...
 
void MakeSccGraph (const std::vector< std::vector< int32 > > &graph, const std::vector< std::vector< int32 > > &sccs, std::vector< std::vector< int32 > > *scc_graph)
 Given a list of sccs of a graph (e.g. More...
 
void ComputeTopSortOrderRecursive (int32 node, const std::vector< std::vector< int32 > > &graph, std::vector< bool > *cycle_detector, std::vector< bool > *is_visited, std::vector< int32 > *reversed_orders)
 
void ComputeTopSortOrder (const std::vector< std::vector< int32 > > &graph, std::vector< int32 > *node_to_order)
 Given an acyclic graph (where each std::vector<int32> is a list of destination-nodes of arcs coming from the current node), compute a topological ordering of the graph nodes. More...
 
std::string PrintGraphToString (const std::vector< std::vector< int32 > > &graph)
 Prints a graph to a string in a pretty way for human readability, e.g. More...
 
void ComputeNnetComputationEpochs (const Nnet &nnet, std::vector< int32 > *node_to_epoch)
 This function computes the order in which we need to compute each node in the graph, where each node-index n maps to an epoch-index t = 0, 1, ... More...
 
bool GraphHasCycles (const std::vector< std::vector< int32 > > &graph)
 This function returns 'true' if the graph represented in 'graph' contains cycles (including cycles where a single node has an arc to itself). More...
 
void UnitTestNnetIo ()
 
static bool UnitTestNnetOptimizeWithOptions (int32 srand_seed, NnetOptimizeOptions opt_config, CachingOptimizingCompilerOptions compiler_config)
 
static void UnitTestNnetOptimizeInternal (int32 srand_seed)
 
static void UnitTestNnetOptimize ()
 
void IdentifySubmatrixArgs (NnetComputation::Command *command, std::vector< int32 * > *submatrix_args)
 This function outputs to "submatrix_args" the addresses of a subset of arguments arg1 through arg6 in "command", that correspond to the indexes of submatrices. More...
 
void IdentifySubmatrixArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *submatrix_args)
 This function outputs to "submatrix_args" the addresses of the args (arguments arg1 through arg6) in the vector "commands", that correspond to the indexes of submatrices. More...
 
void IdentifyMatrixArgsInComputation (NnetComputation *computation, std::vector< int32 * > *matrix_args)
 
void IdentifyIndexesMultiArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *indexes_multi_args)
 Identifies in the vector of commands, arguments that correspond to indexes into the computation's indexes_multi array, and outputs a list of pointers to those arguments to 'indexes_multi_args'. More...
 
void IdentifyIndexesRangesArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *indexes_ranges_args)
 Identifies in the vector of commands, arguments that correspond to indexes into the computation's 'indexes_ranges' array, and outputs a list of pointers to those arguments to 'indexes_ranges_args'. More...
 
void IdentifyIndexesArgs (std::vector< NnetComputation::Command > *commands, std::vector< int32 * > *indexes_args)
 Identifies in the vector of commands, arguments that correspond to indexes into the computation's 'indexes' array, and outputs a list of pointers to those arguments to 'indexes_args'. More...
 
void IdentifySubmatrixArgsInComputation (NnetComputation *computation, std::vector< int32 * > *submatrix_args)
 This function outputs to "submatrix_args" the addresses of integers in 'computation' that correspond to submatrices. More...
 
void RenumberComputation (NnetComputation *computation)
 This function detects submatrices and matrices that are never used (e.g. More...
 
void RemoveNoOps (NnetComputation *computation)
 Removes commands of type kNoOperation in the computation. More...
 
static
NnetComputation::SubMatrixInfo 
GetSubMatrixOfSubMatrix (const NnetComputation &computation, int32 submat_a, int32 submat_b)
 This static function returns a SubMatrixInfo corresponding to replacing the matrix-index in a's "matrix_index" with, essentially, sub-matrix b. More...
 
void ExtendMatrices (NnetComputation *computation)
 This is not really an optimization in itself but it can make things easier for class VariableMergingOptimizer (usually called by its wrapper VariableMergingOptimization()). More...
 
void ConsolidateModelUpdate (const Nnet &nnet, NnetComputation *computation)
 This optimization consolidates the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices. More...
 
void LimitDerivativeTimes (const Nnet &nnet, int32 min_deriv_time, int32 max_deriv_time, NnetComputation *computation)
 
static bool IndexesHaveSpecialStructure (const std::vector< int32 > &indexes, int32 *first_nonnegative_pos, int32 *first_nonnegative_value, int32 *num_nonnegative_indexes)
 
bool ReplaceRowWithMatrixOps (NnetComputation *computation)
 This function detects cases where commands of type kCopyRows, kAddRows or kAddToRows can be converted to commands of type kMatrixCopy or kMatrixAdd, and converts them (this may involve adding submatrices). More...
 
static void FindNumLeadingAndTrailingNegatives (const std::vector< int32 > &vec, int32 *num_leading_negatives, int32 *num_trailing_negatives)
 
static bool SnipSingleRowOp (NnetComputation *computation, int32 command_index)
 
static void FindNumLeadingAndTrailingNegatives (const std::vector< std::pair< int32, int32 > > &vec, int32 *num_leading_negatives, int32 *num_trailing_negatives)
 
static bool SnipMultiRowOp (NnetComputation *computation, int32 command_index)
 
static void FindNumLeadingAndTrailingIdenticals (const std::vector< std::pair< int32, int32 > > &vec, int32 *num_leading_identicals, int32 *num_trailing_identicals)
 
static bool SnipRangesRowOp (NnetComputation *computation, int32 command_index)
 
bool SnipRowOps (NnetComputation *computation)
 This function detects cases where commands of type kCopyRows, kAddRows, kAddRowsMulti, kAddToRowsMulti, kCopyRowsMulti, kCopyToRowsMulti or kAddRowRanges use indexes that start or end with -1's or equivalents, and replace them with similar commands that act on a sub-matrix of the matrices they are currently acting on. More...
 
bool SplitRowOps (NnetComputation *computation)
 This function detects cases where commands of type kAddRowsMulti, kAddToRowsMulti, kCopyRowsMulti, kCopyToRowsMulti use indexes that correspond to at most two submatrices, in two distinct ranges without gaps filled by -1's, and could be converted to at most two commands of type kMatrixAdd, kMatrixCopy, kAddRows or kCopyRows. More...
 
static int32 FindNStride (const std::vector< Index > &indexes, bool full_check)
 
static int32 FindNStride (const std::vector< Cindex > &cindexes, bool full_check)
 
static void ConvertNumNValues (int32 n_stride, int32 old_N, int32 new_N, const std::vector< Index > &indexes_in, std::vector< Index > *indexes_out)
 
void ExpandComputation (const Nnet &nnet, const MiscComputationInfo &misc_info, const NnetComputation &computation, bool need_debug_info, int32 num_n_values, NnetComputation *expanded_computation)
 This function is used in 'shortcut' compilation to expand a computation that has been compiled for exactly 2 'n' values, to one that is suitable for some num_n_values > 2. More...
 
static bool IoSpecificationIsDecomposable (const IoSpecification &io_spec, IoSpecification *mini_io_spec, int32 *num_n_values_out)
 
bool RequestIsDecomposable (const ComputationRequest &request, ComputationRequest *mini_request, int32 *num_n_values)
 This function, used in 'shortcut' compilation where we first compile a smaller computation with the same structure but only 2 distinct 'n' values, works out whether a computation is 'decomposable'; if so, it returns true and outputs the 'mini_request' with the same structure, and the number of 'n' values. More...
 
void OptimizeLoopedComputation (const Nnet &nnet, NnetComputation *computation)
 This function tries to optimize computation 'computation' for an 'looped' computation. More...
 
void FixGotoLabel (NnetComputation *computation)
 This function ensures that the arg1 of a final command of type kGotoLabel is the same as the command with type kNoOperationLabel. More...
 
bool MatrixIsUnused (const Analyzer &analyzer, const NnetComputation &computation, int32 m)
 This function returns true if matrix 1 <= m < computation->matrices.size() is unused, defined as: it is not an input or an output, and is not accessed other than via commands of type kAllocMatrix, kDeallocMatrix, and kSetConst. More...
 
void RemoveCommandsForUnusedMatrix (const Analyzer &analyzer, int32 m, NnetComputation *computation)
 This function removes from 'computation' the commands accessing matrix 'm', which is assumed to be unused according to the MatrixIsUnused() command above. More...
 
void InsertCommands (std::vector< std::pair< int32, NnetComputation::Command > > *commands, NnetComputation *computation)
 Inserts commands into the computation at the requested places. More...
 
void OptimizeMemoryCompression (const Nnet &nnet, int32 memory_compression_level, NnetComputation *computation)
 Performs optimization to reduce memory usage where possible, making use of the kCompressMatrix and kDecompressMatrix commands. More...
 
int32 MaxOutputTimeInRequest (const ComputationRequest &request)
 
void MoveSizingCommands (const Nnet &nnet, NnetComputation *computation)
 This optimization moves commands that allocate and zero matrices to as late as possible, and moves commands that deallocate matrices to as early as possible. More...
 
void RemoveUnnecessaryZeroing (const Nnet &nnet, NnetComputation *computation)
 This optimization function removes, where possible, commands of type type kSetConst. More...
 
static void ComputeCommandPairs (const std::pair< std::vector< int32 >, std::vector< int32 > > &lists, std::vector< std::pair< int32, int32 > > *pairs)
 
void RemoveUnnecessaryAllocation (const Nnet &nnet, NnetComputation *computation)
 This optimization detects cases where we deallocate a matrix, and then later allocate another matrix of the same size; and replaces them with commands of type kAllocFromOther or kAllocFromOtherZeroed. More...
 
void VariableMergingOptimization (const NnetOptimizeOptions &config, const Nnet &nnet, NnetComputation *computation)
 This wraps class VariableMergingOptimizer in a simplified interface. More...
 
void ConvertAdditionToAssignment (const Nnet &nnet, NnetComputation *computation)
 This converts addition operations (things with Add in their names) to copy operations (things with Copy in their names). More...
 
void Optimize (const NnetOptimizeOptions &config, const Nnet &nnet, int32 max_output_time_in_request, NnetComputation *computation)
 This is the top-level function for optimizing a computation. More...
 
static void SplitComputationIntoSegments (const NnetComputation &computation, std::vector< std::pair< int32, int32 > > *segments)
 Split the computation up into segments bounded by kNoOperationMarker. More...
 
void ConsolidateIoOperations (const Nnet &nnet, NnetComputation *computation)
 This optimization puts the input operations (kAcceptInput) and output operations (kProvideOutput) at the very beginning or end of segments of computation, respectively. More...
 
void LimitDerivativeTimes (const Nnet &nnet, const ComputationRequest &request, const NnetOptimizeOptions &opts, NnetComputation *computation)
 This optimization, which has no effect unless you set –min-deriv-time or –max-deriv-time, modifies the backprop operations for efficiency based on the assumption that derivatives for any Cindex with t < min_deriv_time or t > max_deriv_time are zero. More...
 
void UnitTestConfigLineParse ()
 
void UnitTestReadConfig ()
 
void UnitTestDescriptorTokenize ()
 
void UnitTestSummarizeVector ()
 
void UnitTestNameMatchesPattern ()
 
void ExpectOneOrTwoTokens (std::istream &is, bool binary, const std::string &token1, const std::string &token2)
 This function is like ExpectToken but for two tokens, and it will either accept token1 and then token2, or just token2. More...
 
bool ParseFromString (const std::string &name, std::string *string, int32 *param)
 Function used in Init routines. More...
 
bool ParseFromString (const std::string &name, std::string *string, bool *param)
 This version of ParseFromString is for parameters of type bool, which can appear as any string beginning with f, F, t or T. More...
 
bool ParseFromString (const std::string &name, std::string *string, BaseFloat *param)
 This version of ParseFromString is for parameters of type BaseFloat. More...
 
bool ParseFromString (const std::string &name, std::string *string, std::string *param)
 This version of ParseFromString is for parsing strings. More...
 
bool ParseFromString (const std::string &name, std::string *string, std::vector< int32 > *param)
 This version of ParseFromString handles colon-separated or comma-separated lists of integers. More...
 
bool DescriptorTokenize (const std::string &input, std::vector< std::string > *tokens)
 This function tokenizes input when parsing Descriptor configuration values. More...
 
bool IsValidName (const std::string &name)
 Returns true if 'name' would be a valid name for a component or node in a Nnet. More...
 
void ReadConfigLines (std::istream &is, std::vector< std::string > *lines)
 This function reads in a config file and *appends* its contents to a vector of lines; it is responsible for removing comments (anything after '#') and stripping out any lines that contain only whitespace after comment removal. More...
 
std::string ErrorContext (std::istream &is)
 Return a string used in error messages. More...
 
std::string ErrorContext (const std::string &str)
 
static void PrintFloatSuccinctly (std::ostream &os, BaseFloat f)
 
std::string SummarizeVector (const VectorBase< BaseFloat > &vec)
 Returns a string that summarizes a vector fairly succintly, for printing stats in info lines. More...
 
std::string SummarizeVector (const VectorBase< double > &vec)
 
std::string SummarizeVector (const CuVectorBase< BaseFloat > &cu_vec)
 
void PrintParameterStats (std::ostringstream &os, const std::string &name, const CuVectorBase< BaseFloat > &params, bool include_mean=false)
 Print to 'os' some information about the mean and standard deviation of some parameters, used in Info() functions in nnet-simple-component.cc. More...
 
void PrintParameterStats (std::ostringstream &os, const std::string &name, const CuMatrix< BaseFloat > &params, bool include_mean=false, bool include_row_norms=false, bool include_column_norms=false, bool include_singular_values=false)
 Print to 'os' some information about the mean and standard deviation of some parameters, used in Info() functions in nnet-simple-component.cc. More...
 
void ParseConfigLines (const std::vector< std::string > &lines, std::vector< ConfigLine > *config_lines)
 This function converts config-lines from a simple sequence of strings as output by ReadConfigLines(), into a sequence of first-tokens and name-value pairs. More...
 
bool NameMatchesPattern (const char *name, const char *pattern)
 
int32 YzxVectorIndex (int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
 
int32 ZyxVectorIndex (int32 x, int32 y, int32 z, int32 input_x_dim, int32 input_y_dim, int32 input_z_dim)
 
void RearrangeIndexes (const std::vector< std::vector< int32 > > &in, std::vector< std::vector< int32 > > *out)
 
void GenerateConfigSequenceSimplest (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceSimpleContext (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceSimple (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceStatistics (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceRnn (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceRnnClockwork (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceLstm (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceLstmWithTruncation (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceLstmType2 (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceCnn (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceCnnNew (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceRestrictedAttention (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceDistribute (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 
void GenerateConfigSequenceCompositeBlock (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 Generate a config string with a composite component composed only of block affine, repeated affine, and natural gradient repeated affine components. More...
 
void GenerateConfigSequence (const NnetGenerationOptions &opts, std::vector< std::string > *configs)
 Generates a sequence of at least one config files, output as strings, where the first in the sequence is the initial nnet, and the remaining ones may do things like add layers. More...
 
void ComputeExampleComputationRequestSimple (const Nnet &nnet, ComputationRequest *request, std::vector< Matrix< BaseFloat > > *inputs)
 This function computes an example computation request, for testing purposes. More...
 
static void GenerateRandomComponentConfig (std::string *component_type, std::string *config)
 
ComponentGenerateRandomSimpleComponent ()
 Generates random simple component for testing. More...
 
bool NnetParametersAreIdentical (const Nnet &nnet1, const Nnet &nnet2, BaseFloat threshold)
 Used for testing that the updatable parameters in two networks are the same. More...
 
void GenerateSimpleNnetTrainingExample (int32 num_supervised_frames, int32 left_context, int32 right_context, int32 input_dim, int32 output_dim, int32 ivector_dim, NnetExample *example)
 Low-level function that generates an nnet training example. More...
 
bool ExampleApproxEqual (const NnetExample &eg1, const NnetExample &eg2, BaseFloat delta)
 Returns true if the examples are approximately equal (only intended to be used in testing). More...
 
void ComputeObjectiveFunction (const GeneralMatrix &supervision, ObjectiveType objective_type, const std::string &output_name, bool supply_deriv, NnetComputer *computer, BaseFloat *tot_weight, BaseFloat *tot_objf)
 This function computes the objective function, and if supply_deriv = true, supplies its derivative to the NnetComputation object. More...
 
void UnitTestNnetContext ()
 
void UnitTestConvertRepeatedToBlockAffine ()
 
void UnitTestConvertRepeatedToBlockAffineComposite ()
 
int32 NumOutputNodes (const Nnet &nnet)
 returns the number of output nodes of this nnet. More...
 
int32 NumInputNodes (const Nnet &nnet)
 returns the number of input nodes of this nnet. More...
 
bool IsSimpleNnet (const Nnet &nnet)
 This function returns true if the nnet has the following properties: It has an output called "output" (other outputs are allowed but may be ignored). More...
 
void EvaluateComputationRequest (const Nnet &nnet, const ComputationRequest &request, std::vector< std::vector< bool > > *is_computable)
 Given an nnet and a computation request, this function works out which requested outputs in the computation request are computable; it outputs this information as a vector "is_computable" indexed by the same indexes as request.outputs. More...
 
static void ComputeSimpleNnetContextForShift (const Nnet &nnet, int32 input_start, int32 window_size, int32 *left_context, int32 *right_context)
 
void ComputeSimpleNnetContext (const Nnet &nnet, int32 *left_context, int32 *right_context)
 ComputeSimpleNnetContext computes the left-context and right-context of a nnet. More...
 
void PerturbParams (BaseFloat stddev, Nnet *nnet)
 Calls PerturbParams (with the given stddev) on all updatable components of the nnet. More...
 
void ComponentDotProducts (const Nnet &nnet1, const Nnet &nnet2, VectorBase< BaseFloat > *dot_prod)
 Returns dot products between two networks of the same structure (calls the DotProduct functions of the Updatable components and fill in the output vector). More...
 
std::string PrintVectorPerUpdatableComponent (const Nnet &nnet, const VectorBase< BaseFloat > &vec)
 This function is for printing, to a string, a vector with one element per updatable component of the nnet (e.g. More...
 
BaseFloat DotProduct (const Nnet &nnet1, const Nnet &nnet2)
 Returns dot product between two networks of the same structure (calls the DotProduct functions of the Updatable components and sums up the return values). More...
 
void ZeroComponentStats (Nnet *nnet)
 Zeroes the component stats in all nonlinear components in the nnet. More...
 
void SetLearningRate (BaseFloat learning_rate, Nnet *nnet)
 Sets the underlying learning rate for all the components in the nnet to this value. More...
 
void SetNnetAsGradient (Nnet *nnet)
 Sets nnet as gradient by Setting is_gradient_ to true and learning_rate_ to 1 for each UpdatableComponent in nnet. More...
 
void ScaleNnet (BaseFloat scale, Nnet *nnet)
 Scales the nnet parameters and stats by this scale. More...
 
void AddNnetComponents (const Nnet &src, const Vector< BaseFloat > &alphas, BaseFloat scale, Nnet *dest)
 Does *dest += alpha * src for updatable components (affects nnet parameters), and *dest += scale * src for other components (affects stored stats). More...
 
void AddNnet (const Nnet &src, BaseFloat alpha, Nnet *dest)
 Does *dest += alpha * src (affects nnet parameters and stored stats). More...
 
int32 NumParameters (const Nnet &src)
 Returns the total of the number of parameters in the updatable components of the nnet. More...
 
void VectorizeNnet (const Nnet &src, VectorBase< BaseFloat > *params)
 Copies the nnet parameters to *params, whose dimension must be equal to NumParameters(src). More...
 
void UnVectorizeNnet (const VectorBase< BaseFloat > &params, Nnet *dest)
 Copies the parameters from params to *dest. More...
 
int32 NumUpdatableComponents (const Nnet &dest)
 Returns the number of updatable components in the nnet. More...
 
void FreezeNaturalGradient (bool freeze, Nnet *nnet)
 Controls if natural gradient will be updated. More...
 
void ConvertRepeatedToBlockAffine (CompositeComponent *c_component)
 
void ConvertRepeatedToBlockAffine (Nnet *nnet)
 Convert all components of type RepeatedAffineComponent or NaturalGradientRepeatedAffineComponent to BlockAffineComponent in nnet. More...
 
std::string NnetInfo (const Nnet &nnet)
 This function returns various info about the neural net. More...
 
void SetDropoutProportion (BaseFloat dropout_proportion, Nnet *nnet)
 This function sets the dropout proportion in all dropout components to dropout_proportion value. More...
 
bool HasBatchnorm (const Nnet &nnet)
 Returns true if nnet has at least one component of type BatchNormComponent. More...
 
void ScaleBatchnormStats (BaseFloat batchnorm_stats_scale, Nnet *nnet)
 This function scales the batchorm stats of any batchnorm components (components of type BatchNormComponent) in 'nnet' by the scale 'batchnorm_stats_scale'. More...
 
void RecomputeStats (const std::vector< NnetExample > &egs, Nnet *nnet)
 This function zeros the stored component-level stats in the nnet using ZeroComponentStats(), then recomputes them with the supplied egs. More...
 
void SetBatchnormTestMode (bool test_mode, Nnet *nnet)
 This function affects only components of type BatchNormComponent. More...
 
void SetDropoutTestMode (bool test_mode, Nnet *nnet)
 This function affects components of child-classes of RandomComponent. More...
 
void ResetGenerators (Nnet *nnet)
 This function calls 'ResetGenerator()' on all components in 'nnet' that inherit from class RandomComponent. More...
 
void FindOrphanComponents (const Nnet &nnet, std::vector< int32 > *components)
 This function finds a list of components that are never used, and outputs the integer comopnent indexes (you can use these to index nnet.GetComponentNames() to get their names). More...
 
void FindOrphanNodes (const Nnet &nnet, std::vector< int32 > *nodes)
 This function finds a list of nodes that are never used to compute any output, and outputs the integer node indexes (you can use these to index nnet.GetNodeNames() to get their names). More...
 
void ConstrainOrthonormalInternal (BaseFloat scale, CuMatrixBase< BaseFloat > *M)
 
void ConstrainOrthonormal (Nnet *nnet)
 This function, to be called after processing every minibatch, is responsible for enforcing the orthogonality constraint for any components of type LinearComponent or inheriting from AffineComponent that have the "orthonormal_constraint" value set. More...
 
void ReduceRankOfComponents (const std::string component_name_pattern, int32 rank, Nnet *nnet)
 
void ReadEditConfig (std::istream &config_file, Nnet *nnet)
 ReadEditConfig() reads a file with a similar-looking format to the config file read by Nnet::ReadConfig(), but this consists of a sequence of operations to perform on an existing network, mostly modifying components. More...
 
bool NnetIsRecurrent (const Nnet &nnet)
 Returns true if 'nnet' has some kind of recurrency. More...
 
void CollapseModel (const CollapseModelConfig &config, Nnet *nnet)
 This function modifies the neural net for efficiency, in a way that suitable to be done in test time. More...
 
bool UpdateNnetWithMaxChange (const Nnet &delta_nnet, BaseFloat max_param_change, BaseFloat max_change_scale, BaseFloat scale, Nnet *nnet, std::vector< int32 > *num_max_change_per_component_applied, int32 *num_max_change_global_applied)
 This function does the operation '*nnet += scale * delta_nnet', while respecting any max-parameter-change (max-param-change) specified in the updatable components, and also the global max-param-change specified as 'max_param_change'. More...
 
int32 GetNumNvalues (const std::vector< NnetIo > &io_vec, bool exhaustive)
 This utility function can be used to obtain the number of distinct 'n' values in a training example. More...
 
void ApplyL2Regularization (const Nnet &nnet, BaseFloat l2_regularize_scale, Nnet *delta_nnet)
 This function is used as part of the regular training workflow, prior to UpdateNnetWithMaxChange(). More...
 
BaseFloat KlDivergence (const Vector< BaseFloat > &p, const Vector< BaseFloat > &q)
 
void PrintPriorDiagnostics (const Vector< BaseFloat > &old_priors, const Vector< BaseFloat > &new_priors)
 
void SetPriors (const TransitionModel &tmodel, const Vector< double > &transition_accs, double prior_floor, AmNnetSimple *am_nnet)
 
double ComputeObjf (bool batchnorm_test_mode, bool dropout_test_mode, const std::vector< NnetExample > &egs, const Nnet &nnet, NnetComputeProb *prob_computer)
 
void UpdateNnetMovingAverage (int32 num_models, const Nnet &nnet, Nnet *moving_average_nnet)
 
void RenameOutputs (const std::string &new_name, NnetExample *eg)
 
void ScaleSupervisionWeight (BaseFloat weight, NnetExample *eg)
 
int32 GetCount (double expected_count)
 
bool ContainsSingleExample (const NnetExample &eg, int32 *min_input_t, int32 *max_input_t, int32 *min_output_t, int32 *max_output_t)
 Returns true if the "eg" contains just a single example, meaning that all the "n" values in the indexes are zero, and the example has NnetIo members named both "input" and "output". More...
 
void FilterExample (const NnetExample &eg, int32 min_input_t, int32 max_input_t, int32 min_output_t, int32 max_output_t, NnetExample *eg_out)
 This function filters the indexes (and associated feature rows) in a NnetExample, removing any index/row in an NnetIo named "input" with t < min_input_t or t > max_input_t and any index/row in an NnetIo named "output" with t < min_output_t or t > max_output_t. More...
 
bool SelectFromExample (const NnetExample &eg, std::string frame_str, int32 left_context, int32 right_context, int32 frame_shift, NnetExample *eg_out)
 This function is responsible for possibly selecting one frame from multiple supervised frames, and reducing the left and right context as specified. More...
 
static bool ProcessFile (const discriminative::SplitDiscriminativeSupervisionOptions &config, const TransitionModel &tmodel, const MatrixBase< BaseFloat > &feats, const MatrixBase< BaseFloat > *ivector_feats, int32 ivector_period, const discriminative::DiscriminativeSupervision &supervision, const std::string &utt_id, bool compress, UtteranceSplitter *utt_splitter, NnetDiscriminativeExampleWriter *example_writer)
 
void ApplyAffineTransform (MatrixBase< BaseFloat > &transform, int32 num_channels, MatrixBase< BaseFloat > *image, FillMode fill_mode)
 This function applies a geometric transformation 'transform' to the image. More...
 
void PerturbImage (const ImageAugmentationConfig &config, MatrixBase< BaseFloat > *image)
 This function randomly modifies (perturbs) the image by applying different geometric transformations according to the options in 'config'. More...
 
void PerturbImageInNnetExample (const ImageAugmentationConfig &config, NnetExample *eg)
 This function does image perturbation as directed by 'config' The example 'eg' is expected to contain a NnetIo member with the name 'input', representing an image. More...
 
static bool ProcessFile (const GeneralMatrix &feats, const MatrixBase< BaseFloat > *ivector_feats, int32 ivector_period, const MatrixBase< BaseFloat > &targets, const std::string &utt_id, bool compress, int32 num_targets, int32 length_tolerance, UtteranceSplitter *utt_splitter, NnetExampleWriter *example_writer)
 
static bool ProcessFile (const GeneralMatrix &feats, const MatrixBase< BaseFloat > *ivector_feats, int32 ivector_period, const Posterior &pdf_post, const std::string &utt_id, bool compress, int32 num_pdfs, int32 length_tolerance, UtteranceSplitter *utt_splitter, NnetExampleWriter *example_writer)
 
int32 NumOutputIndexes (const NnetExample &eg)
 
static void RunNnetComputation (const MatrixBase< BaseFloat > &features, const Nnet &nnet, CachingOptimizingCompiler *compiler, Vector< BaseFloat > *xvector)
 
static void ProcessRangeFile (const std::string &range_rxfilename, unordered_map< std::string, std::vector< ChunkInfo * > > *utt_to_chunks)
 
static void WriteExamples (const MatrixBase< BaseFloat > &feats, const std::vector< ChunkInfo * > &chunks, const std::string &utt, bool compress, int32 num_pdfs, int32 *num_egs_written, std::vector< NnetExampleWriter * > *example_writers)
 

Variables

static bool computation_checker_warned_unused_input = false
 Checks that we never use variables before they are allocated or after they are deallocated, and some other checks that can be done from the MatrixAccesses. More...
 
const int kNoTime = std::numeric_limits<int32>::min()
 

Typedef Documentation

typedef std::pair<int32, Index> Cindex

Definition at line 115 of file nnet-common.h.

Enumeration Type Documentation

enum AccessType
Enumerator
kReadAccess 
kWriteAccess 
kReadWriteAccess 

Definition at line 75 of file nnet-analyze.h.

CommandType is an enum that describes the category of the command used in the NnetComputation.

We declare it outside that class because it's so frequently used and we got tired of typing NnetComputation:: everywhere. We document the commands here. Note: for operations that naturally need to operate on entire matrices (i.e. allocation commands and input and output commands), we use the submatrix indexes of them, which turns out to be more convenient for optimization; but these submatrix indexes must refer to the whole of a matrix.

  • kAllocMatrix. Allocate a matrix (its values will be undefined). arg1 = submatrix index, which must refer to a whole matrix.
  • kDeallocMatrix: Deallocate a matrix. arg1 = submatrix index.
  • kSwapMatrix: initialize matrix with submatrix index arg1 using memory from matrix with submatrix index arg2 (using shallow swap). Both submatrices must refer to whole matrices. The expectation is that prior to the swap, arg1 was empty and arg2 was nonempty, but the execution code does not enforce this.
  • kSetConst: set all elements of submatrix index 'arg1' to the value 'alpha'.
  • kPropagate: Forward computation of neural net, see Component::Propagate()
    • arg1 is is component-index in neural net
    • arg2 is index into ComponentPrecomputedIndexes (0 if NULL; always 0 for simple Components)
    • arg3 is sub-matrix index of input
    • arg4 is sub-matrix index of output
    • arg5 is the index of the memo saved from Propagate()'s return value, or 0 if it saves no memo.
    • arg6 is 1 if we need to call StoreStats() after the Propagate, or 0 if we don't. We used to have a separate command for storing the stats, but that has been removed.
  • kBackprop: Do the back-propagation operation, see Component::Backprop()
    • arg1 is index of component in neural net
    • arg2 is index into ComponentPrecomputedIndexes (0 if NULL; always 0 for simple Components)
    • arg3 is submatrix-index of input value (input to Propagate()); 0 if unused
    • arg4 is submatrix-index of output value (output of Propagate()); 0 if unused
    • arg5 is submatrix-index of output derivative
    • arg6 is submatrix-index of input derivative; 0 if unused.
    • arg7 is the index of the memo which is generated from the corresponding Propagate() function if the flag kUsesMemo is set; 0 if unused.
  • kBackpropNoModelUpdate: as kBackprop, but does not set the 'to_update' argument to the Backprop call, even if the model is updatable, so it skips the model-update phase of backprop.
  • kMatrixCopy: Copy (alpha times contents of sub-matrix arg2) to sub-matrix arg1, currently implemented as copy then scale. Note: to implement scaling a matrix, you can use kMatrixCopy with arg1 == arg2 and it won't do any redundant copying.
  • kMatrixAdd: Add (alpha times contents of sub-matrix arg2) to sub-matrix arg1
  • kCopyRows: call CopyRows() on sub-matrix arg1 with sub-matrix arg2 and indexes[arg3] as arguments, then if alpha != 1.0, scale sub-matrix arg1 by alpha.
  • kAddRows: call AddRows() on sub-matrix arg1 with alpha, sub-matrix arg2 and indexes[arg3] as arguments.
  • kAddRowsMulti, kAddToRowsMulti, kCopyRowsMulti, kCopyToRowsMulti: Call the corresponding function in class CuMatrix (Actually the names do not have 'Multi' in them, but they are the ones that accept a vector of 'Real*'.
    • arg1 is sub-matrix index of *this matrix in operation
    • arg2 is index into "indexes_multi", of which each pair is (sub-matrix index, row index) (or (-1,-1) for NULL marker), which is turned into a vector of BaseFloat* (pointers to matrix rows) before being given as the argument to the function. In the 'Add' functions 'alpha' is provided as an argument; for the 'Copy' functions, we scale the destination by 'alpha' after the copy, if alpha != 1.0. (However, for implementation reasons, kCopyToRowsMulti does not currently support alpha != 1.0 and will crash, so we avoid generating this code).
  • kAddRowRanges: call AddRowRanges() on sub-matrix arg1, with arg2 as source sub-matrix, and indexes given indexes_ranges[arg3]. We use the "alpha" as if AddRowRanges() accepted that argument, even though it doesn't (we fake it using other calls, if alpha != 1.0).
  • kCompressMatrix: Compresses the matrix which should be referred to by submatrix-index arg1. arg2 is a number that determines the compression type (it's converted from the enum CuCompressedMatrixType; 1=int8, 2=uint8, 3=int16, 4=uint16), and alpha determines the 'range' parameter (c.f. NewCuCompressedMatrix()). arg3 will be converted to the 'truncate' argument to the class CuCompressedMatrix; it should be false (0) if you know that the input is limited to the allowed range, and true (1) if the input may exceed that range (see docs for CuCompresedMatrix).
  • kDecompressMatrix: Decompresses the matrix which is referred to by submatrix-index arg1 (it should previously have been compressed).
  • kAcceptInput: accepts a matrix of input from the user, which may be either features, or derivatives w.r.t. the output. arg1 is the submatrix index of a whole matrix that the input goes to, and arg2 is the index of the network node associated with it (e.g. the node of "input" or "ivector"), for puroses of double checking.
  • kProvideOutput: outputs a matrix to the user: either a network output, or a matrix of derivatives w.r.t. an input. arg1 is the submatrix index of the output (which we expect to be a whole matrix), arg2 is the index of the network node associated with it (e.g. the node for "output").
  • kNoOperation: does nothing, and will be removed by optimization code (sometimes useful during optimization)
  • kNoOperationPermanent: like kNoOperation, but won't be removed by optimization code. This is used to ensure that for 'trivial' computations, which just copy the input to the output, the block of commands for the forward or backward propagation is nonempty (to avoid confusing the computation code).
  • kNoOperationMarker: does nothing, but used to mark end of a block of commands (like forward commands).
  • kNoOperationLabel: does nothing, but is the destination for the kGotoLabel command.
  • kGotoLabel: jumps to the kNoOperationLabel command. arg1 must be set to the location of that command. Since there are no conditionals, the kGotoLabel command should be the last command, as remaining commands will be unreachable.
Enumerator
kAllocMatrix 
kDeallocMatrix 
kSwapMatrix 
kSetConst 
kPropagate 
kBackprop 
kBackpropNoModelUpdate 
kMatrixCopy 
kMatrixAdd 
kCopyRows 
kAddRows 
kCopyRowsMulti 
kCopyToRowsMulti 
kAddRowsMulti 
kAddToRowsMulti 
kAddRowRanges 
kCompressMatrix 
kDecompressMatrix 
kAcceptInput 
kProvideOutput 
kNoOperation 
kNoOperationPermanent 
kNoOperationMarker 
kNoOperationLabel 
kGotoLabel 

Definition at line 288 of file nnet-computation.h.

288  {
296  kGotoLabel };
Enumerator
kSimpleComponent 
kUpdatableComponent 
kPropagateInPlace 
kPropagateAdds 
kReordersIndexes 
kBackpropAdds 
kBackpropNeedsInput 
kBackpropNeedsOutput 
kBackpropInPlace 
kStoresStats 
kInputContiguous 
kOutputContiguous 
kUsesMemo 
kRandomComponent 

Definition at line 37 of file nnet-component-itf.h.

37  {
38  kSimpleComponent = 0x001, // true if number of rows of input equals number of rows
39  // of output and this component doesn't care about the indexes
40  // (i.e. it maps each row of input to each row of output without
41  // regard to the index values). Will normally be true.
42  kUpdatableComponent = 0x002, // true if the component has parameters that can
43  // be updated. Components that return this flag
44  // must be dynamic_castable to type
45  // UpdatableComponent (but components of type
46  // UpdatableComponent do not have to return this
47  // flag, e.g. if this instance is not really
48  // updatable).
49  kPropagateInPlace = 0x004, // true if we can do the propagate operation in-place
50  // (input and output matrices are the same).
51  // Note: if doing backprop, you'd also need to check
52  // that the kBackpropNeedsInput property is not true.
53  kPropagateAdds = 0x008, // true if the Propagate function adds to, rather
54  // than setting, its output, for non-in-place
55  // propagation. The Component chooses whether to add
56  // or set, and the calling code has to accommodate
57  // it.
58  kReordersIndexes = 0x010, // true if the ReorderIndexes function might reorder
59  // the indexes (otherwise we can skip calling it).
60  // Must not be set for simple components.
61  kBackpropAdds = 0x020, // true if the Backprop function adds to, rather than
62  // setting, the "in_deriv" output for non-in-place
63  // backprop. The Component chooses whether to add or
64  // set, and the calling code has to accommodate it.
65  kBackpropNeedsInput = 0x040, // true if backprop operation needs access to
66  // forward-pass input.
67  kBackpropNeedsOutput = 0x080, // true if backprop operation needs access to
68  // forward-pass output (e.g. true for Sigmoid).
69  kBackpropInPlace = 0x100, // true if we can do the backprop operation in-place
70  // (input and output matrices may be the same).
71  kStoresStats = 0x200, // true if the StoreStats operation stores
72  // statistics e.g. on average node activations and
73  // derivatives of the nonlinearity, (as it does for
74  // Tanh, Sigmoid, ReLU and Softmax).
75  kInputContiguous = 0x400, // true if the component requires its input data (and
76  // input derivatives) to have Stride()== NumCols().
77  kOutputContiguous = 0x800, // true if the component requires its input data (and
78  // output derivatives) to have Stride()== NumCols().
79  kUsesMemo = 0x1000, // true if the component returns a void* pointer from its
80  // Propagate() function that needs to be passed into the
81  // corresponding Backprop function.
82  kRandomComponent = 0x2000 // true if the component has some kind of
83  // randomness, like DropoutComponent (these should
84  // inherit from class RandomComponent.
85 };
enum FillMode
Enumerator
kNearest 
kReflect 

Definition at line 31 of file nnet3-egs-augment-image.cc.

enum NodeType
Enumerator
kInput 
kDescriptor 
kComponent 
kDimRange 
kNone 

Definition at line 55 of file nnet-nnet.h.

This enum is for a kind of annotation we associate with output nodes of the network; it's for the convenience of calling code so that if the objective is one of a few standard types, we can compute it directly and know how to interpret the supervision labels.

However, the core of the framework never makes use of the objective types, other than making them available to calling code which then supplies the derivatives.

  • Objective type kLinear is intended for Neural nets where the final component is a LogSoftmaxComponent, so the log-prob (negative cross-entropy) objective is just a linear function of the input.
  • Objective type kQuadratic is used to mean the objective function f(x, y) = -0.5 (x-y).(x-y), which is to be maximized, as in the kLinear case.
Enumerator
kLinear 
kQuadratic 

Definition at line 52 of file nnet-nnet.h.

Function Documentation

void AddNnet ( const Nnet &  src,
BaseFloat  alpha,
Nnet *  dest 
)

Does *dest += alpha * src (affects nnet parameters and stored stats).

Definition at line 318 of file nnet-utils.cc.

References Component::Add(), Nnet::GetComponent(), KALDI_ERR, and Nnet::NumComponents().

Referenced by main(), NnetDiscriminativeTrainer::Train(), and UpdateNnetMovingAverage().

318  {
319  if (src.NumComponents() != dest->NumComponents())
320  KALDI_ERR << "Trying to add incompatible nnets.";
321  for (int32 c = 0; c < src.NumComponents(); c++) {
322  const Component *src_comp = src.GetComponent(c);
323  Component *dest_comp = dest->GetComponent(c);
324  dest_comp->Add(alpha, *src_comp);
325  }
326 }
#define KALDI_ERR
Definition: kaldi-error.h:127
void AddNnetComponents ( const Nnet &  src,
const Vector< BaseFloat > &  alphas,
BaseFloat  scale,
Nnet *  dest 
)

Does *dest += alpha * src for updatable components (affects nnet parameters), and *dest += scale * src for other components (affects stored stats).

Here, alphas is a vector of size equal to the number of updatable components

Definition at line 291 of file nnet-utils.cc.

References Component::Add(), VectorBase< Real >::Dim(), Nnet::GetComponent(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, kUpdatableComponent, Nnet::NumComponents(), and Component::Properties().

Referenced by UpdateNnetWithMaxChange().

292  {
293  if (src.NumComponents() != dest->NumComponents())
294  KALDI_ERR << "Trying to add incompatible nnets.";
295  int32 i = 0;
296  for (int32 c = 0; c < src.NumComponents(); c++) {
297  const Component *src_comp = src.GetComponent(c);
298  Component *dest_comp = dest->GetComponent(c);
299  if (src_comp->Properties() & kUpdatableComponent) {
300  // For now all updatable components inherit from class UpdatableComponent.
301  // If that changes in future, we will change this code.
302  const UpdatableComponent *src_uc =
303  dynamic_cast<const UpdatableComponent*>(src_comp);
304  UpdatableComponent *dest_uc =
305  dynamic_cast<UpdatableComponent*>(dest_comp);
306  if (src_uc == NULL || dest_uc == NULL)
307  KALDI_ERR << "Updatable component does not inherit from class "
308  "UpdatableComponent; change this code.";
309  KALDI_ASSERT(i < alphas.Dim());
310  dest_uc->Add(alphas(i++), *src_uc);
311  } else { // add stored stats
312  dest_comp->Add(scale, *src_comp);
313  }
314  }
315  KALDI_ASSERT(i == alphas.Dim());
316 }
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:63
void kaldi::nnet3::AddTimeOffsetToComputationRequest ( int32  t_offset,
ComputationRequest *  request 
)

Definition at line 224 of file nnet-compile-looped.cc.

References rnnlm::i, ComputationRequest::inputs, rnnlm::j, and ComputationRequest::outputs.

Referenced by ExtrapolateComputationRequest().

225  {
226  for (size_t i = 0; i < request->inputs.size(); i++) {
227  size_t size = request->inputs[i].indexes.size();
228  for (size_t j = 0; j < size; j++)
229  request->inputs[i].indexes[j].t += t_offset;
230  }
231  for (size_t i = 0; i < request->outputs.size(); i++) {
232  size_t size = request->outputs[i].indexes.size();
233  for (size_t j = 0; j < size; j++)
234  request->outputs[i].indexes[j].t += t_offset;
235  }
236 }
void AppendCindexes ( int32  node,
const std::vector< Index > &  indexes,
std::vector< Cindex > *  out 
)

Appends to 'out' the pairs (node, indexes[0]), (node, indexes[1]), ...

Definition at line 1375 of file nnet-compile.cc.

References rnnlm::i.

Referenced by Compiler::OutputDebugInfo().

1376  {
1377  size_t indexes_size = indexes.size();
1378  if (indexes_size > out->size())
1379  out->reserve(out->size() + indexes_size);
1380  for (size_t i = 0; i < indexes_size; i++)
1381  out->push_back(Cindex(node, indexes[i]));
1382 }
std::pair< int32, Index > Cindex
Definition: nnet-common.h:115
void kaldi::nnet3::ApplyAffineTransform ( MatrixBase< BaseFloat > &  transform,
int32  num_channels,
MatrixBase< BaseFloat > *  image,
FillMode  fill_mode 
)

This function applies a geometric transformation 'transform' to the image.

Reference: Digital Image Processing book by Gonzalez and Woods.

Parameters
[in]transformThe 3x3 geometric transformation matrix to apply.
[in]num_channelsNumber of channels (i.e. colors) of the image
[in,out]imageThe image matrix to be modified. image->NumRows() is the width (number of x values) in the image; image->NumCols() is the height times number of channels (channel varies the fastest).

Definition at line 110 of file nnet3-egs-augment-image.cc.

References KALDI_ASSERT, kNearest, kReflect, MatrixBase< Real >::NumCols(), and MatrixBase< Real >::NumRows().

Referenced by PerturbImage().

113  {
114  int32 num_rows = image->NumRows(),
115  num_cols = image->NumCols(),
116  height = num_cols / num_channels,
117  width = num_rows;
118  KALDI_ASSERT(num_cols % num_channels == 0);
119  Matrix<BaseFloat> original_image(*image);
120  for (int32 r = 0; r < width; r++) {
121  for (int32 c = 0; c < height; c++) {
122  // (r_old, c_old) is the coordinate of the pixel in the original image
123  // while (r, c) is the coordinate in the new (transformed) image.
124  BaseFloat r_old = transform(0, 0) * r +
125  transform(0, 1) * c + transform(0, 2);
126  BaseFloat c_old = transform(1, 0) * r +
127  transform(1, 1) * c + transform(1, 2);
128  // We are going to do bilinear interpolation between 4 closest points
129  // to the point (r_old, c_old) of the original image. We have:
130  // r1 <= r_old <= r2
131  // c1 <= c_old <= c2
132  int32 r1 = static_cast<int32>(floor(r_old));
133  int32 c1 = static_cast<int32>(floor(c_old));
134  int32 r2 = r1 + 1;
135  int32 c2 = c1 + 1;
136 
137  // These weights determine how much each of the 4 points contributes
138  // to the final interpolated value:
139  BaseFloat weight_11 = (r2 - r_old) * (c2 - c_old),
140  weight_12 = (r2 - r_old) * (c_old - c1),
141  weight_21 = (r_old - r1) * (c2 - c_old),
142  weight_22 = (r_old - r1) * (c_old - c1);
143  // Handle edge conditions:
144  if (fill_mode == kNearest) {
145  if (r1 < 0) {
146  r1 = 0;
147  if (r2 < 0) r2 = 0;
148  }
149  if (r2 >= width) {
150  r2 = width - 1;
151  if (r1 >= width) r1 = width - 1;
152  }
153  if (c1 < 0) {
154  c1 = 0;
155  if (c2 < 0) c2 = 0;
156  }
157  if (c2 >= height) {
158  c2 = height - 1;
159  if (c1 >= height) c1 = height - 1;
160  }
161  } else {
162  KALDI_ASSERT(fill_mode == kReflect);
163  if (r1 < 0) {
164  r1 = - r1;
165  if (r2 < 0) r2 = - r2;
166  }
167  if (r2 >= width) {
168  r2 = 2 * width - 2 - r2;
169  if (r1 >= width) r1 = 2 * width - 2 - r1;
170  }
171  if (c1 < 0) {
172  c1 = - c1;
173  if (c2 < 0) c2 = -c2;
174  }
175  if (c2 >= height) {
176  c2 = 2 * height - 2 - c2;
177  if (c1 >= height) c1 = 2 * height - 2 - c1;
178  }
179  }
180  for (int32 ch = 0; ch < num_channels; ch++) {
181  // find the values at the 4 points
182  BaseFloat p11 = original_image(r1, num_channels * c1 + ch),
183  p12 = original_image(r1, num_channels * c2 + ch),
184  p21 = original_image(r2, num_channels * c1 + ch),
185  p22 = original_image(r2, num_channels * c2 + ch);
186  (*image)(r, num_channels * c + ch) = weight_11 * p11 + weight_12 * p12 +
187  weight_21 * p21 + weight_22 * p22;
188  }
189  }
190  }
191 }
float BaseFloat
Definition: kaldi-types.h:29
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:64
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ApplyL2Regularization ( const Nnet &  nnet,
BaseFloat  l2_regularize_scale,
Nnet *  delta_nnet 
)

This function is used as part of the regular training workflow, prior to UpdateNnetWithMaxChange().

For each updatable component c in the neural net, suppose it has a l2-regularization constant alpha set at the component level (see UpdatableComponent::L2Regularization()), and a learning-rate eta, then this function does (and this is not real code):

delta_nnet->c -= 2.0 * l2_regularize_scale * alpha * eta * nnet.c

The factor of -1.0 comes from the fact that we are maximizing, and we'd add the l2 regularization term (of the form ||||_2^2, i.e. squared l2 norm) in the objective function with negative sign; the factor of 2.0 comes from the derivative of the squared parameters. The factor 'l2_regularize_scale' is provided to this function, see below for an explanation.

Note: the way we do it is a little bit approximate, due to the interaction with natural gradient. The issue is that the regular gradients are multiplied by the inverse of the approximated, smoothed and factored inverse Fisher matrix, but the l2 gradients are not. This means that what we're optimizing is not exactly the (regular objective plus the L2 term)– we could view it as optimizing (regular objective plus the l2 term times the Fisher matrix)– with the proviso that the Fisher matrix has been scaled in such a way that the amount of parameter change is not affected, so this is not an issue of affecting the overall strength of l2, just an issue of the direction-wise weighting. In effect, the l2 term will be larger, relative to the gradient contribution, in directions where the Fisher matrix is large. This is probably not ideal– but it's hard to judge without experiments. Anyway the l2 effect is small enough, and the Fisher matrix sufficiently smoothed with the identity, that I doubt this makes much of a difference.

Parameters
[in]nnetThe neural net that is being trained; expected to be different from delta_nnet
[in]l2_regularize_scaleA scale on the l2 regularization. Usually this will be equal to the number of distinct examples (e.g. the number of chunks of speech– more precisely, the number of distinct 'n' values) in the minibatch, but this is multiplied by a configuration value –l2-regularize-factor passed in from the command line. The reason for making l2 proportional to the number of elements in the minibatch is that we add the parameter gradients over the minibatch (we don't average), so multiplying the l2 factor by the number of elements in the minibatch is necessary to make the amount of l2 vs. gradient contribution stay the same when we vary the minibatch size. The –l2-regularize-factor option is provided so that the calling script can correct for the effects of parallelization via model-averaging (we'd normally set this to 1/num-parallel-jobs).
[out]delta_nnetThe neural net containing the parameter updates; this is a copy of 'nnet' that is used for purposes of momentum and applying max-change values. This is what this code adds to.

Definition at line 2030 of file nnet-utils.cc.

References Component::Add(), Nnet::GetComponent(), KALDI_ASSERT, kUpdatableComponent, UpdatableComponent::L2Regularization(), UpdatableComponent::LearningRate(), Nnet::NumComponents(), and Component::Properties().

Referenced by NnetChainTrainer::TrainInternal(), NnetTrainer::TrainInternal(), NnetChainTrainer::TrainInternalBackstitch(), and NnetTrainer::TrainInternalBackstitch().

2032  {
2033  if (l2_regularize_scale == 0.0)
2034  return;
2035  for (int32 c = 0; c < nnet.NumComponents(); c++) {
2036  const Component *src_component_in = nnet.GetComponent(c);
2037  if (src_component_in->Properties() & kUpdatableComponent) {
2038  const UpdatableComponent *src_component =
2039  dynamic_cast<const UpdatableComponent*>(src_component_in);
2040  UpdatableComponent *dest_component =
2041  dynamic_cast<UpdatableComponent*>(delta_nnet->GetComponent(c));
2042  // The following code will segfault if they aren't both updatable, which
2043  // would be a bug in the calling code.
2044  BaseFloat lrate = dest_component->LearningRate(),
2045  l2_regularize = dest_component->L2Regularization();
2046  KALDI_ASSERT(lrate >= 0 && l2_regularize >= 0);
2047  BaseFloat scale = -2.0 * l2_regularize_scale * lrate * l2_regularize;
2048  if (scale != 0.0)
2049  dest_component->Add(scale, *src_component);
2050  }
2051  }
2052 }
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool kaldi::nnet3::AssertGraphEqual ( const std::vector< std::vector< int32 > > &  graph1,
const std::vector< std::vector< int32 > > &  graph2 
)

Definition at line 26 of file nnet-graph-test.cc.

References rnnlm::i, and rnnlm::j.

Referenced by UnitTestComputeGraphTranspose(), UnitTestFindSccs(), and UnitTestMakeSccGraph().

27  {
28  if (graph1.size() != graph2.size()) { return false; }
29  for (int32 i = 0; i < graph1.size(); ++i) {
30  if (graph1[i].size() != graph2[i].size()) { return false; }
31  for (int32 j = 0; j < graph1[i].size(); ++j) {
32  if (graph1[i][j] != graph2[i][j]) { return false; }
33  }
34  }
35  return true;
36 }
bool kaldi::nnet3::AssertVectorEqual ( const std::vector< int32 > &  vec1,
const std::vector< int32 > &  vec2 
)

Definition at line 38 of file nnet-graph-test.cc.

References rnnlm::i.

Referenced by UnitTestComputeTopSortOrder(), and UnitTestComputeTopSortOrder2().

39  {
40  if (vec1.size() != vec2.size()) { return false; }
41  for (int32 i = 0; i < vec1.size(); ++i) {
42  if (vec1[i] != vec2[i]) { return false; }
43  }
44  return true;
45 }
void kaldi::nnet3::BuildTestGraph ( std::vector< std::vector< int32 > > *  graph)

Definition at line 47 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeGraphTranspose(), UnitTestFindSccs(), and UnitTestMakeSccGraph().

47  {
48  KALDI_ASSERT(graph != NULL);
49  graph->clear();
50  graph->resize(8);
51 
52  // We create the following graph for testing.
53  // 0 --> 4
54  // 1 --> 0
55  // 2 --> 1 3
56  // 3 --> 2
57  // 4 --> 1
58  // 5 --> 1 4 6
59  // 6 --> 5
60  // 7 --> 7 3 6
61  std::vector<int32> tmp;
62  tmp.resize(1); tmp[0] = 4; (*graph)[0] = tmp;
63  tmp.resize(1); tmp[0] = 0; (*graph)[1] = tmp;
64  tmp.resize(2); tmp[0] = 1; tmp[1] = 3; (*graph)[2] = tmp;
65  tmp.resize(1); tmp[0] = 2; (*graph)[3] = tmp;
66  tmp.resize(1); tmp[0] = 1; (*graph)[4] = tmp;
67  tmp.resize(3); tmp[0] = 1; tmp[1] = 4; tmp[2] = 6; (*graph)[5] = tmp;
68  tmp.resize(1); tmp[0] = 5; (*graph)[6] = tmp;
69  tmp.resize(3); tmp[0] = 7; tmp[1] = 3; tmp[2] = 6; (*graph)[7] = tmp;
70 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::BuildTestGraphTranspose ( std::vector< std::vector< int32 > > *  graph)

Definition at line 72 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeGraphTranspose().

72  {
73  KALDI_ASSERT(graph != NULL);
74  graph->clear();
75  graph->resize(8);
76 
77  // We create the following graph for testing.
78  // 0 --> 1
79  // 1 --> 2 4 5
80  // 2 --> 3
81  // 3 --> 2 7
82  // 4 --> 0 5
83  // 5 --> 6
84  // 6 --> 5 7
85  // 7 --> 7
86  std::vector<int32> tmp;
87  tmp.resize(1); tmp[0] = 1; (*graph)[0] = tmp;
88  tmp.resize(3); tmp[0] = 2; tmp[1] = 4; tmp[2] = 5; (*graph)[1] = tmp;
89  tmp.resize(1); tmp[0] = 3; (*graph)[2] = tmp;
90  tmp.resize(2); tmp[0] = 2; tmp[1] = 7; (*graph)[3] = tmp;
91  tmp.resize(2); tmp[0] = 0; tmp[1] = 5; (*graph)[4] = tmp;
92  tmp.resize(1); tmp[0] = 6; (*graph)[5] = tmp;
93  tmp.resize(2); tmp[0] = 5; tmp[1] = 7; (*graph)[6] = tmp;
94  tmp.resize(1); tmp[0] = 7; (*graph)[7] = tmp;
95 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::BuildTestSccGraph ( std::vector< std::vector< int32 > > *  scc_graph)

Definition at line 114 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeTopSortOrder(), and UnitTestMakeSccGraph().

114  {
115  KALDI_ASSERT(scc_graph != NULL);
116  scc_graph->clear();
117  scc_graph->resize(4);
118 
119  // We create the following SCC graph for testing.
120  // 0 -->
121  // 1 --> 0
122  // 2 --> 0
123  // 3 --> 1 2
124  std::vector<int32> tmp;
125  tmp.resize(0); (*scc_graph)[0] = tmp;
126  tmp.resize(1); tmp[0] = 0; (*scc_graph)[1] = tmp;
127  tmp.resize(1); tmp[0] = 0; (*scc_graph)[2] = tmp;
128  tmp.resize(2); tmp[0] = 1; tmp[1] = 2; (*scc_graph)[3] = tmp;
129 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::BuildTestSccs ( std::vector< std::vector< int32 > > *  sccs)

Definition at line 97 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestFindSccs(), and UnitTestMakeSccGraph().

97  {
98  KALDI_ASSERT(sccs != NULL);
99  sccs->clear();
100  sccs->resize(4);
101 
102  // We create the following SCCs for testing.
103  // 0 --> 1 4 0
104  // 1 --> 3 2
105  // 2 --> 6 5
106  // 3 --> 7
107  std::vector<int32> tmp;
108  tmp.resize(3); tmp[0] = 1; tmp[1] = 4; tmp[2] = 0; (*sccs)[0] = tmp;
109  tmp.resize(2); tmp[0] = 3; tmp[1] = 2; (*sccs)[1] = tmp;
110  tmp.resize(2); tmp[0] = 6; tmp[1] = 5; (*sccs)[2] = tmp;
111  tmp.resize(1); tmp[0] = 7; (*sccs)[3] = tmp;
112 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::BuildTestTopSortOrder ( std::vector< int32 > *  node_to_order)

Definition at line 131 of file nnet-graph-test.cc.

References KALDI_ASSERT.

Referenced by UnitTestComputeTopSortOrder().

131  {
132  KALDI_ASSERT(node_to_order != NULL);
133  node_to_order->clear();
134  node_to_order->resize(4);
135 
136  // The topological sorting order of the above SCC graph is as follows (from
137  // our particular algorithm):
138  // 0 --> 3
139  // 1 --> 2
140  // 2 --> 1
141  // 3 --> 0
142  (*node_to_order)[0] = 3;
143  (*node_to_order)[1] = 2;
144  (*node_to_order)[2] = 1;
145  (*node_to_order)[3] = 0;
146 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void CheckComputation ( const Nnet &  nnet,
const NnetComputation &  computation,
bool  check_rewrite = false 
)

This is a convenience interface for class ComputationChecker.

Call it with check_rewrite = true only if the computation is pre-optimization. If the computation is an 'online' computation, this function treats it specially.

Definition at line 1145 of file nnet-analyze.cc.

References ComputationChecker::Check(), CheckComputationOptions::check_rewrite, CheckComputationOnline(), NnetComputation::commands, KALDI_ERR, kGotoLabel, and NnetComputation::Print().

Referenced by CachingOptimizingCompiler::CompileViaShortcut(), and Optimize().

1147  {
1148  try {
1149  if (!computation.commands.empty() &&
1150  computation.commands.back().command_type == kGotoLabel) {
1151  // Online computations need to be treated specially.
1152  CheckComputationOnline(nnet, computation, check_rewrite);
1153  } else {
1154  CheckComputationOptions opts;
1155  opts.check_rewrite = check_rewrite;
1156  ComputationChecker checker(opts, nnet, computation);
1157  checker.Check();
1158  }
1159  } catch (...) {
1160  computation.Print(std::cerr, nnet);
1161  KALDI_ERR << "Computation check failed for computation printed above "
1162  "(actual error message is above computation)";
1163  }
1164 }
#define KALDI_ERR
Definition: kaldi-error.h:127
static void CheckComputationOnline(const Nnet &nnet, NnetComputation computation, bool check_rewrite)
static void kaldi::nnet3::CheckComputationOnline ( const Nnet &  nnet,
NnetComputation  computation,
bool  check_rewrite 
)
static

Definition at line 1118 of file nnet-analyze.cc.

References ComputationChecker::Check(), CheckComputationOptions::check_rewrite, CheckComputationOptions::check_unused_variables, NnetComputation::commands, KALDI_ASSERT, kDeallocMatrix, kGotoLabel, kSwapMatrix, and kaldi::swap().

Referenced by CheckComputation().

1120  {
1121  int32 num_commands = computation.commands.size();
1122  KALDI_ASSERT(computation.commands[num_commands-1].command_type == kGotoLabel);
1123  for (int32 c = num_commands - 2;
1124  c >= 0 && computation.commands[c].command_type == kSwapMatrix;
1125  c--) {
1126  // this command can be interpreted as "initialize matrix referred to by
1127  // c.arg2 with the matrix referred to by c.arg2".
1128  // Because this would be interpreted by the analysis code as initializing a
1129  // matrix that has already been initialized, we turn this into a command
1130  // that just deallocates the matrix in c.arg2. [note: all these indexes
1131  // are actually submatrix indexes].
1132  computation.commands[c].command_type = kDeallocMatrix;
1133  std::swap(computation.commands[c].arg1, computation.commands[c].arg2);
1134  }
1135 
1136  CheckComputationOptions opts;
1137  opts.check_rewrite = check_rewrite;
1138  opts.check_unused_variables = false;
1139  // We can always do this check with online computations, since they do not
1140  // have the RemoveUnnecessaryAllocation() optimization applied.
1141  ComputationChecker checker(opts, nnet, computation);
1142  checker.Check();
1143 }
void swap(basic_filebuf< CharT, Traits > &x, basic_filebuf< CharT, Traits > &y)
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool kaldi::nnet3::CheckStringsApproxEqual ( const std::string &  a,
const std::string &  b,
int32  tolerance = 3 
)

Definition at line 39 of file nnet-component-test.cc.

References KALDI_WARN, and kaldi::StringsApproxEqual().

Referenced by TestNnetComponentAddScale(), TestNnetComponentIo(), TestNnetComponentUpdatable(), and TestNnetComponentVectorizeUnVectorize().

41  {
42  if (!StringsApproxEqual(a, b, tolerance)) {
43  KALDI_WARN << "Strings differ: " << a
44  << "\nvs.\n" << b;
45  return false;
46  } else {
47  return true;
48  }
49 }
bool StringsApproxEqual(const std::string &a, const std::string &b, int32 decimal_places_tolerance)
This function returns true when two text strings are approximately equal, and false when they are not...
Definition: text-utils.cc:335
#define KALDI_WARN
Definition: kaldi-error.h:130
void CollapseModel ( const CollapseModelConfig &  config,
Nnet *  nnet 
)

This function modifies the neural net for efficiency, in a way that suitable to be done in test time.

For example, it tries to get rid of dropout, batchnorm and fixed-scale components, and to collapse subsequent affine components if doing so won't hurt speed.

Definition at line 1886 of file nnet-utils.cc.

References ModelCollapser::Collapse().

Referenced by main(), and UnitTestNnetCompute().

1887  {
1888  ModelCollapser c(config, nnet);
1889  c.Collapse();
1890 }
void CompileLooped ( const Nnet &  nnet,
const NnetOptimizeOptions &  optimize_opts,
const ComputationRequest &  request1,
const ComputationRequest &  request2,
const ComputationRequest &  request3,
NnetComputation *  computation 
)

CompileLooped() provides an internal interface for 'looped' computation.

It's usable for inference only (not training), meaning that backprop is not supported (for now, at least). CompileLooped() allows you to do the neural net computation for small chunks with increasing 't' values, and naturally cache the intermediate activations (rather than recomputing them every time you see new input data).

This function does both compilation and optimization, so it's like a combination of Compiler::CreateComputation() [nnet-compile.h] and Optimize() [nnet-optimize.h].

You provide 3 computation requests. request1 is the first computation request of an utterance (or other type of segment) that contains any required extra left context in the input. request2 and request3 are the second and third computation request, and must have exactly the same structure, except for a fixed time offset (change in 't' index) between them. This will be extrapolated to an infinite sequence of further requests (request4, request5, etc.). In practice the way it's done is that we extrapolate to a small finite number of requests (like 10), and then attempt to identify a common structure in the computation where, after processing, as an example, the 3nd computation request, the active variables can be identified with those present at, say, the 7th computation request, and we then cut and splice the computation together at this points, like making a tape loop, by adding a goto statement that jumps from the end of the 7th computation request to the end of the 3rd computation request. We also have to identify the variables with each other (merge variables).

That's done in the optimization code.

Definition at line 318 of file nnet-compile-looped.cc.

References CompileLoopedInternal(), Timer::Elapsed(), KALDI_ERR, KALDI_LOG, and KALDI_VLOG.

Referenced by DecodableNnetSimpleLoopedInfo::Init(), and UnitTestNnetCompileLooped().

323  {
324  int32 num_requests1 = 5, factor = 2, max_requests = 100,
325  num_requests;
326 
327  Timer timer;
328 
329  for (num_requests = num_requests1; num_requests <= max_requests;
330  num_requests *= factor) {
331  if (CompileLoopedInternal(nnet, optimize_opts,
332  request1, request2, request3,
333  num_requests, computation)) {
334  KALDI_LOG << "Spent " << timer.Elapsed()
335  << " seconds in looped compilation.";
336  return;
337  } else {
338  KALDI_VLOG(2) << "Looped compilation failed with "
339  << num_requests << " requests, trying "
340  << (num_requests * factor);
341  }
342  }
343  KALDI_ERR << "Looped compilation failed with "
344  << (num_requests/factor) << " requests, which "
345  << "we expect should be enough... something "
346  << "went wrong.";
347 }
static bool CompileLoopedInternal(const Nnet &nnet, NnetOptimizeOptions optimize_opts, const ComputationRequest &request1, const ComputationRequest &request2, const ComputationRequest &request3, int32 num_requests, NnetComputation *computation)
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_VLOG(v)
Definition: kaldi-error.h:136
#define KALDI_LOG
Definition: kaldi-error.h:133
static bool kaldi::nnet3::CompileLoopedInternal ( const Nnet &  nnet,
NnetOptimizeOptions  optimize_opts,
const ComputationRequest &  request1,
const ComputationRequest &  request2,
const ComputationRequest &  request3,
int32  num_requests,
NnetComputation *  computation 
)
static

Definition at line 273 of file nnet-compile-looped.cc.

References NnetComputation::commands, Compiler::CreateComputation(), ExtrapolateComputationRequest(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, KALDI_LOG, kGotoLabel, MaxOutputTimeInRequest(), Optimize(), NnetOptimizeOptions::optimize_looped_computation, and ComputationRequest::Print().

Referenced by CompileLooped().

280  {
281 
282  KALDI_ASSERT(num_requests >= 3);
283  std::vector<ComputationRequest> extra_requests(num_requests - 3);
284  const ComputationRequest *prev_request = &request2;
285  const ComputationRequest *cur_request = &request3;
286  for (int32 i = 0; i < num_requests - 3; i++) {
287  if (!ExtrapolateComputationRequest(*prev_request, *cur_request,
288  &(extra_requests[i]))) {
289  KALDI_LOG << "prev_request is:";
290  prev_request->Print(std::cerr);
291  KALDI_LOG << "cur_request is:";
292  cur_request->Print(std::cerr);
293  KALDI_ERR << "Computation requests do not have the right relationship";
294  }
295  prev_request = cur_request;
296  cur_request = &(extra_requests[i]);
297  }
298 
299  std::vector<const ComputationRequest*> requests;
300  requests.push_back(&request1);
301  requests.push_back(&request2);
302  requests.push_back(&request3);
303  for (int32 i = 0; i < num_requests - 3; i++)
304  requests.push_back(&(extra_requests[i]));
305  Compiler compiler(requests, nnet);
306  CompilerOptions compiler_opts;
307  compiler.CreateComputation(compiler_opts, computation);
308  optimize_opts.optimize_looped_computation = true;
309 
310  int32 dont_really_care = MaxOutputTimeInRequest(request3);
311  Optimize(optimize_opts, nnet,
312  dont_really_care, computation);
313 
314  return computation->commands.size() != 0 &&
315  computation->commands.back().command_type == kGotoLabel;
316 }
static bool ExtrapolateComputationRequest(const ComputationRequest &request1, const ComputationRequest &request2, ComputationRequest *request3)
int32 MaxOutputTimeInRequest(const ComputationRequest &request)
#define KALDI_ERR
Definition: kaldi-error.h:127
void Optimize(const NnetOptimizeOptions &config, const Nnet &nnet, int32 max_output_time_in_request, NnetComputation *computation)
This is the top-level function for optimizing a computation.
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
#define KALDI_LOG
Definition: kaldi-error.h:133
void ComponentDotProducts ( const Nnet &  nnet1,
const Nnet &  nnet2,
VectorBase< BaseFloat > *  dot_prod 
)

Returns dot products between two networks of the same structure (calls the DotProduct functions of the Updatable components and fill in the output vector).

Definition at line 189 of file nnet-utils.cc.

References VectorBase< Real >::Data(), VectorBase< Real >::Dim(), UpdatableComponent::DotProduct(), Nnet::GetComponent(), KALDI_ASSERT, kUpdatableComponent, Nnet::NumComponents(), and Component::Properties().

Referenced by main().

191  {
192  KALDI_ASSERT(nnet1.NumComponents() == nnet2.NumComponents());
193  int32 updatable_c = 0;
194  for (int32 c = 0; c < nnet1.NumComponents(); c++) {
195  const Component *comp1 = nnet1.GetComponent(c),
196  *comp2 = nnet2.GetComponent(c);
197  if (comp1->Properties() & kUpdatableComponent) {
198  const UpdatableComponent
199  *u_comp1 = dynamic_cast<const UpdatableComponent*>(comp1),
200  *u_comp2 = dynamic_cast<const UpdatableComponent*>(comp2);
201  KALDI_ASSERT(u_comp1 != NULL && u_comp2 != NULL);
202  dot_prod->Data()[updatable_c] = u_comp1->DotProduct(*u_comp2);
203  updatable_c++;
204  }
205  }
206  KALDI_ASSERT(updatable_c == dot_prod->Dim());
207 }
Real * Data()
Returns a pointer to the start of the vector's data.
Definition: kaldi-vector.h:69
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:63
void ComputeAccuracy ( const GeneralMatrix &  supervision,
const CuMatrixBase< BaseFloat > &  nnet_output,
BaseFloat *  tot_weight,
BaseFloat *  tot_accuracy,
VectorBase< BaseFloat > *  tot_weight_vec = NULL,
VectorBase< BaseFloat > *  tot_accuracy_vec = NULL 
)

This function computes the frame accuracy for this minibatch.

It interprets the supervision information in "supervision" as labels or soft labels; it picks the maximum element in each row and treats that as the label for purposes of computing the accuracy (in situations where you would care about the accuracy, there will normally be just one nonzero label). The hypothesized labels are computed by taking the neural net output (supplied as a CuMatrix), and finding the maximum element in each row. See also the function ComputeObjectiveFunction, declared in nnet-training.h.

Parameters
[in]supervisionThe supervision information (no elements may be negative); only the maximum in each row matters (although we expect that usually there will be just one nonzero element in each row); and the sum of each row is interpreted as a weighting factor (although we expect that this sum will usually be one).
[in]nnet_outputThe neural net output must have the same dimensions as the supervision. Only the index of the maximum value in each row matters. Ties will be broken in an unspecified way.
[out]tot_weightThe sum of the values in the supervision matrix
[out]tot_accuracyThe total accuracy, equal to the sum over all row indexes r such that the maximum column index of row r of supervision and nnet_output is the same, of the sum of the r'th row of supervision (i.e. the row's weight).
[out]tot_weight_vecIf non-NULL, we write to this location the counts per-class in the supervision matrix. This is expected to have the same dimension as the corresponding output in the network.
[out]tot_accuracy_vecIf non-NULL, we write to this location the accuracy per-class. For index j, the value is equal to the sum over all row indexes r such that the maximum column index of row r of supervision is j and nnet_output is also j, of the sum of the r'th row of supervision (i.e. the row's weight)

Definition at line 206 of file nnet-diagnostics.cc.

References CuArrayBase< T >::CopyToVec(), VectorBase< Real >::Dim(), CuMatrixBase< Real >::FindRowMaxId(), GeneralMatrix::GetFullMatrix(), GeneralMatrix::GetMatrix(), GeneralMatrix::GetSparseMatrix(), KALDI_ASSERT, KALDI_ERR, kaldi::kCompressedMatrix, kaldi::kFullMatrix, kaldi::kSparseMatrix, SparseVector< Real >::Max(), VectorBase< Real >::Max(), CuMatrixBase< Real >::NumCols(), GeneralMatrix::NumCols(), CuMatrixBase< Real >::NumRows(), GeneralMatrix::NumRows(), SparseMatrix< Real >::Row(), VectorBase< Real >::Set(), SparseVector< Real >::Sum(), VectorBase< Real >::Sum(), and GeneralMatrix::Type().

Referenced by NnetComputeProb::ProcessOutputs().

211  {
212  int32 num_rows = nnet_output.NumRows(),
213  num_cols = nnet_output.NumCols();
214  KALDI_ASSERT(supervision.NumRows() == num_rows &&
215  supervision.NumCols() == num_cols);
216 
217  if (tot_accuracy_vec || tot_weight_vec)
218  KALDI_ASSERT(tot_accuracy_vec && tot_weight_vec &&
219  tot_accuracy_vec->Dim() == num_cols &&
220  tot_weight_vec->Dim() == num_cols);
221  if (tot_accuracy_vec) tot_accuracy_vec->Set(0.0);
222  if (tot_weight_vec) tot_weight_vec->Set(0.0);
223 
224  CuArray<int32> best_index(num_rows);
225  nnet_output.FindRowMaxId(&best_index);
226  std::vector<int32> best_index_cpu;
227  // wasteful copy, but doesn't dominate.
228  best_index.CopyToVec(&best_index_cpu);
229 
230 
231  double tot_weight = 0.0,
232  tot_accuracy = 0.0;
233 
234  // note: we expect that in most cases where this code is called,
235  // supervision.Type() will be kSparseMatrix.
236  switch (supervision.Type()) {
237  case kCompressedMatrix: {
238  Matrix<BaseFloat> mat;
239  supervision.GetMatrix(&mat);
240  for (int32 r = 0; r < num_rows; r++) {
241  SubVector<BaseFloat> vec(mat, r);
242  BaseFloat row_sum = vec.Sum();
243  int32 best_index;
244  vec.Max(&best_index); // discard max value.
245  tot_weight += row_sum;
246  if (tot_weight_vec)
247  (*tot_weight_vec)(best_index) += row_sum;
248  if (best_index == best_index_cpu[r]) {
249  tot_accuracy += row_sum;
250  if (tot_accuracy_vec)
251  (*tot_accuracy_vec)(best_index) += row_sum;
252  }
253  }
254  break;
255  }
256  case kFullMatrix: {
257  const Matrix<BaseFloat> &mat = supervision.GetFullMatrix();
258  for (int32 r = 0; r < num_rows; r++) {
259  SubVector<BaseFloat> vec(mat, r);
260  BaseFloat row_sum = vec.Sum();
261  int32 best_index;
262  vec.Max(&best_index); // discard max value.
263  tot_weight += row_sum;
264  if (tot_weight_vec)
265  (*tot_weight_vec)(best_index) += row_sum;
266  if (best_index == best_index_cpu[r]) {
267  tot_accuracy += row_sum;
268  if (tot_accuracy_vec)
269  (*tot_accuracy_vec)(best_index) += row_sum;
270  }
271  }
272  break;
273  }
274  case kSparseMatrix: {
275  const SparseMatrix<BaseFloat> &smat = supervision.GetSparseMatrix();
276  for (int32 r = 0; r < num_rows; r++) {
277  const SparseVector<BaseFloat> &row = smat.Row(r);
278  BaseFloat row_sum = row.Sum();
279  int32 best_index;
280  row.Max(&best_index);
281  KALDI_ASSERT(best_index < num_cols);
282  tot_weight += row_sum;
283  if (tot_weight_vec)
284  (*tot_weight_vec)(best_index) += row_sum;
285  if (best_index == best_index_cpu[r]) {
286  tot_accuracy += row_sum;
287  if (tot_accuracy_vec)
288  (*tot_accuracy_vec)(best_index) += row_sum;
289  }
290  }
291  break;
292  }
293  default: KALDI_ERR << "Bad general-matrix type.";
294  }
295  *tot_weight_out = tot_weight;
296  *tot_accuracy_out = tot_accuracy;
297 }
void FindRowMaxId(CuArray< int32 > *id) const
Find the id of the maximal element for each row (resizes the 'id' array to the appropriate size)...
Definition: cu-matrix.cc:1813
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
float BaseFloat
Definition: kaldi-types.h:29
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Set(Real f)
Set all members of a vector to a specified value.
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:63
void ComputeCommandAttributes ( const Nnet &  nnet,
const NnetComputation &  computation,
const ComputationVariables &  vars,
std::vector< CommandAttributes > *  attributes 
)

Definition at line 284 of file nnet-analyze.cc.

References NnetComputation::Command::arg1, NnetComputation::Command::arg2, NnetComputation::Command::arg3, NnetComputation::Command::arg4, NnetComputation::Command::arg5, NnetComputation::Command::arg6, NnetComputation::Command::command_type, NnetComputation::commands, count, Nnet::GetComponent(), CommandAttributes::has_side_effects, rnnlm::i, NnetComputation::indexes, NnetComputation::indexes_multi, IndexesMultiToSubmatrixIndexes(), kAcceptInput, kAddRowRanges, kAddRows, kAddRowsMulti, kAddToRowsMulti, KALDI_ERR, kAllocMatrix, kBackprop, kBackpropAdds, kBackpropNoModelUpdate, kCompressMatrix, kCopyRows, kCopyRowsMulti, kCopyToRowsMulti, kDeallocMatrix, kDecompressMatrix, kGotoLabel, kMatrixAdd, kMatrixCopy, kNoOperation, kNoOperationLabel, kNoOperationMarker, kNoOperationPermanent, kPropagate, kPropagateAdds, kProvideOutput, kReadAccess, kReadWriteAccess, kSetConst, kSwapMatrix, kUpdatableComponent, kWriteAccess, CommandAttributes::matrices_read, CommandAttributes::matrices_written, Component::Properties(), ComputationVariables::RecordAccessForSubmatrix(), kaldi::SortAndUniq(), CommandAttributes::submatrices_read, CommandAttributes::submatrices_written, CommandAttributes::variables_read, and CommandAttributes::variables_written.

Referenced by NnetComputer::Init(), Analyzer::Init(), and MoveSizingCommands().

288  {
289  int32 num_commands = computation.commands.size();
290  attributes->clear();
291  attributes->resize(num_commands);
292  for (int32 command_index = 0; command_index < num_commands; command_index++) {
293  const NnetComputation::Command &c = computation.commands[command_index];
294  CommandAttributes &attr = (*attributes)[command_index];
295  switch (c.command_type) {
296  case kAllocMatrix:
297  case kDeallocMatrix:
298  case kSwapMatrix:
299  break; // the commands above leave the matrix undefined.
300  case kSetConst:
301  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
302  break;
303  case kPropagate:
304  vars.RecordAccessForSubmatrix(c.arg3, kReadAccess, &attr);
305  if (nnet.GetComponent(c.arg1)->Properties() & kPropagateAdds)
306  vars.RecordAccessForSubmatrix(c.arg4, kReadWriteAccess, &attr);
307  else
308  vars.RecordAccessForSubmatrix(c.arg4, kWriteAccess, &attr);
309  break;
310  case kBackprop:
312  vars.RecordAccessForSubmatrix(c.arg3, kReadAccess, &attr);
313  vars.RecordAccessForSubmatrix(c.arg4, kReadAccess, &attr);
314  vars.RecordAccessForSubmatrix(c.arg5, kReadAccess, &attr);
315  if (nnet.GetComponent(c.arg1)->Properties() & kBackpropAdds)
316  vars.RecordAccessForSubmatrix(c.arg6, kReadWriteAccess, &attr);
317  else
318  vars.RecordAccessForSubmatrix(c.arg6, kWriteAccess, &attr);
319  if (c.command_type == kBackprop &&
320  nnet.GetComponent(c.arg1)->Properties() & kUpdatableComponent)
321  attr.has_side_effects = true;
322  break;
323  case kMatrixCopy:
324  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
325  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
326  break;
327  case kMatrixAdd:
328  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
329  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
330  break;
331  case kAddRows:
332  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
333  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
334  break;
335  case kCopyRows: {
336  const std::vector<int32> &indexes = computation.indexes[c.arg3];
337  // if there are -1's in "indexes", then the result of the operation
338  // will depend on the initial value of the matrix, so it's
339  // a "rw" operation, not a "write" operation.
340  if (std::count(indexes.begin(), indexes.end(), -1) > 0)
341  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
342  else
343  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
344  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
345  break;
346  }
347  case kAddRowsMulti: {
348  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
349  std::vector<int32> submatrix_indexes;
350  IndexesMultiToSubmatrixIndexes(computation.indexes_multi[c.arg2],
351  &submatrix_indexes);
352  for (size_t i = 0; i < submatrix_indexes.size(); i++)
353  vars.RecordAccessForSubmatrix(submatrix_indexes[i],
354  kReadAccess, &attr);
355  break;
356  }
357  case kCopyRowsMulti: {
358  std::vector<int32> submatrix_indexes;
359  IndexesMultiToSubmatrixIndexes(computation.indexes_multi[c.arg2],
360  &submatrix_indexes);
361  // note: the CopyRows command assigns zero in cases where
362  // there is no source for some row
363  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
364  for (size_t i = 0; i < submatrix_indexes.size(); i++)
365  vars.RecordAccessForSubmatrix(submatrix_indexes[i],
366  kReadAccess, &attr);
367  break;
368  }
369  case kAddToRowsMulti:
370  case kCopyToRowsMulti: {
371  vars.RecordAccessForSubmatrix(c.arg1, kReadAccess, &attr);
372  // if the submatrixes we're writing to (in kCopyToRowsMulti) had all
373  // rows covered, it would be a pure write operation.
374  std::vector<int32> submatrix_indexes;
375  IndexesMultiToSubmatrixIndexes(computation.indexes_multi[c.arg2],
376  &submatrix_indexes);
377  for (size_t i = 0; i < submatrix_indexes.size(); i++)
378  vars.RecordAccessForSubmatrix(submatrix_indexes[i], kReadWriteAccess,
379  &attr);
380  break;
381  }
382  case kAddRowRanges: {
383  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
384  vars.RecordAccessForSubmatrix(c.arg2, kReadAccess, &attr);
385  break;
386  }
387  case kCompressMatrix: {
388  vars.RecordAccessForSubmatrix(c.arg1, kReadWriteAccess, &attr);
389  break;
390  }
391  case kDecompressMatrix: {
392  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
393  break;
394  }
395  case kAcceptInput: {
396  vars.RecordAccessForSubmatrix(c.arg1, kWriteAccess, &attr);
397  break;
398  }
399  case kProvideOutput: {
400  vars.RecordAccessForSubmatrix(c.arg1, kReadAccess, &attr);
401  break;
402  }
403  case kNoOperation:
405  case kNoOperationMarker:
406  case kNoOperationLabel:
407  case kGotoLabel:
408  break;
409  default:
410  KALDI_ERR << "Unknown command type.";
411  }
412  SortAndUniq(&attr.variables_read);
413  SortAndUniq(&attr.variables_written);
414  SortAndUniq(&attr.submatrices_read);
415  SortAndUniq(&attr.submatrices_written);
416  SortAndUniq(&attr.matrices_read);
417  SortAndUniq(&attr.matrices_written);
418  }
419 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq's (removes duplicates) from a vector.
Definition: stl-utils.h:39
static void IndexesMultiToSubmatrixIndexes(const std::vector< std::pair< int32, int32 > > &indexes_multi, std::vector< int32 > *submatrix_indexes)
given a vector of pairs from computation.indexes_multi_indexes containing paris (submatrix-index, row-index), this function outputs to "submatrix_indexes" all (unique) submatrix indexes that appear; and it outputs to "contains_null_marker" true if the pair (-1, -1) appears anywhere in indexes_multi, and false otherwise.
const size_t count
#define KALDI_ERR
Definition: kaldi-error.h:127
static void kaldi::nnet3::ComputeCommandPairs ( const std::pair< std::vector< int32 >, std::vector< int32 > > &  lists,
std::vector< std::pair< int32, int32 > > *  pairs 
)
static

Definition at line 327 of file nnet-optimize.cc.

References kaldi::CopyVectorToSet(), rnnlm::d, and KALDI_PARANOID_ASSERT.

Referenced by RemoveUnnecessaryAllocation().

329  {
330  std::vector<int32> d_list = lists.first;
331 
332  std::set<int32> a_set;
333  CopyVectorToSet(lists.second, &a_set);
334 
335  std::vector<int32>::reverse_iterator iter = d_list.rbegin(),
336  end = d_list.rend();
337 
338  // from the latest to the earliest deallocation command...
339  for (; iter != end; ++iter) {
340  int32 d = *iter;
341  std::set<int32>::iterator a_iter = a_set.upper_bound(d);
342  // a_iter is an iterator to the first element a of the set 'a_set' such
343  // that a > d, or a_set.end() if no such element exists.
344  if (a_iter == a_set.end())
345  continue; // we will output no pair for this d.
346  int32 a = *a_iter;
347  KALDI_PARANOID_ASSERT(a > d); // or code error
348  a_set.erase(a_iter); // remove this a from 'a_set' so it doesn't get used
349  // twice
350  pairs->push_back(std::pair<int32,int32>(d, a));
351  }
352 }
void CopyVectorToSet(const std::vector< A > &v, std::set< A > *s)
Copies the contents of a vector to a set.
Definition: stl-utils.h:174
#define KALDI_PARANOID_ASSERT(cond)
Definition: kaldi-error.h:182
void kaldi::nnet3::ComputeComputationGraph ( const Nnet &  nnet,
const ComputationRequest &  request,
ComputationGraph *  graph 
)

Definition at line 1171 of file nnet-computation-graph.cc.

References kaldi::nnet3::computation_graph::AddInputToGraph(), kaldi::nnet3::computation_graph::AddOutputToGraph(), ComputationGraph::cindexes, NetworkNode::component_index, ComputationGraph::dependencies, NetworkNode::descriptor, ComputationGraph::GetCindexId(), Nnet::GetComponent(), Descriptor::GetDependencies(), Component::GetInputIndexes(), Nnet::GetNode(), rnnlm::i, ComputationGraph::is_input, KALDI_ASSERT, KALDI_ERR, kComponent, kDescriptor, kDimRange, kInput, ComputationRequest::misc_info, rnnlm::n, NetworkNode::node_index, NetworkNode::node_type, kaldi::SortAndUniq(), and NetworkNode::u.

1173  {
1174  using namespace computation_graph;
1175  // make sure graph is empty at the start.
1176  KALDI_ASSERT(graph->cindexes.empty());
1177 
1178  AddInputToGraph(request, nnet, graph);
1179  AddOutputToGraph(request, nnet, graph);
1180 
1181  // queue of cindex_ids to process.
1182  std::vector<int32> queue(graph->cindexes.size());
1183  for (int32 i = 0; i < graph->cindexes.size(); i++)
1184  queue.push_back(i);
1185 
1186  while (!queue.empty()) {
1187  int32 cindex_id = queue.back();
1188  queue.pop_back();
1189  if (static_cast<int32>(graph->dependencies.size()) <= cindex_id)
1190  graph->dependencies.resize(cindex_id + 1);
1191 
1192  if (graph->is_input[cindex_id])
1193  continue;
1194  Cindex cindex = graph->cindexes[cindex_id];
1195 
1196  // find the dependencies of this cindex.
1197  int32 n = cindex.first;
1198  const Index &index = cindex.second;
1199  const NetworkNode &node = nnet.GetNode(n);
1200 
1201  std::vector<Cindex> input_cindexes;
1202 
1203  // the following switch statement sets up "input_cindexes".
1204  switch (node.node_type) {
1205  case kDescriptor: {
1206  // desc describes how this node obtains its input from other nodes.
1207  const Descriptor &desc = node.descriptor;
1208  desc.GetDependencies(index, &input_cindexes);
1209  break;
1210  }
1211  case kComponent: {
1212  int32 c = node.u.component_index;
1213  const Component *component = nnet.GetComponent(c);
1214  std::vector<Index> input_indexes;
1215  component->GetInputIndexes(request.misc_info, index,
1216  &input_indexes);
1217  // each Component node should be preceded by a node that describes its
1218  // input, of type kDescriptor
1219  KALDI_ASSERT(nnet.GetNode(n-1).node_type ==
1220  kDescriptor);
1221 
1222  input_cindexes.resize(input_indexes.size());
1223  for (size_t i = 0; i < input_indexes.size(); i++) {
1224  input_cindexes[i].first = n - 1; // preceding node.
1225  input_cindexes[i].second = input_indexes[i];
1226  }
1227  break;
1228  }
1229  case kDimRange: {
1230  input_cindexes.resize(1);
1231  input_cindexes[0] = Cindex(node.u.node_index, index);
1232  break;
1233  }
1234  case kInput: default:
1235  // for kInput, you should have hit the "continue" statement above.
1236  KALDI_ERR << "Invalid node type";
1237  }
1238  std::vector<int32> &this_dep = graph->dependencies[cindex_id];
1239 
1240  int32 num_dependencies = input_cindexes.size();
1241  this_dep.resize(num_dependencies);
1242  for (size_t i = 0; i < num_dependencies; i++) {
1243  bool is_input = false, is_new;
1244  int32 dep_cindex_id = graph->GetCindexId(input_cindexes[i],
1245  is_input, &is_new);
1246  this_dep[i] = dep_cindex_id;
1247  if (is_new)
1248  queue.push_back(dep_cindex_id);
1249  }
1250 
1251  // remove duplicates of dependencies.
1252  SortAndUniq(&this_dep);
1253  }
1254 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq's (removes duplicates) from a vector.
Definition: stl-utils.h:39
void AddInputToGraph(const ComputationRequest &request, const Nnet &nnet, ComputationGraph *graph)
std::pair< int32, Index > Cindex
Definition: nnet-common.h:115
void AddOutputToGraph(const ComputationRequest &request, const Nnet &nnet, ComputationGraph *graph)
struct rnnlm::@11::@12 n
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ComputeComputationPhases ( const Nnet &  nnet,
const ComputationGraph &  computation_graph,
std::vector< std::vector< std::vector< int32 > > > *  phases_per_segment 
)

This function divides a computation into 'phases', where a 'phase' is a collection of cindexes which can (as far as the computation graph is concerned) all be computed at the same time, and depend only on cindexes previously computed in earlier phases.

So the phases are an ordering of the Cindexes in the computation, but an ordering that depends on graph-theoretic considerations only, and not practical concerns like whether the cindexes belong to the same node [for that, see the notion of steps].

Parameters
[in]nnetThe neural network this computation is for
[in]graphThe computation graph that we're computing phases for.
[out]phases_per_segmentThe phases, listed separately for each segment of the computation [there will be just one segment in the normal case, more in the online-recognition case]. Consider just one segment for now. Suppose the computation can be completed in 20 phases, then (*phases)[0].size() will be 20 at exit, and (*phases)[0][0] will be a sorted list of cindex_ids. that belong to the first phase, and so on. (Remember, a cindex_id is an index into graph->cindexes; it compactly identifies a cindex.) The sets represented by the int32's in 'phases_per_segment' will be disjoint and will cover all elements in [0 .. computation.cindexes.size() - 1].

Note: we assume you have called PruneComputationGraph() before this function. Even so, this function will be crash if the computation cannot actually be computed– there are some mal-formed computations where you can build the computation graph but not the ordering of cindexes because there are dependencies forward and backward in time that intertwine.

Definition at line 1425 of file nnet-computation-graph.cc.

References ComputationGraph::cindexes, ComputeComputationPhasesForEpoch(), kaldi::nnet3::computation_graph::ComputeDependenciesSubset(), kaldi::nnet3::computation_graph::ComputeEpochInfo(), ComputeGraphTranspose(), KALDI_ASSERT, ComputationGraph::segment_ends, and SumVectorSizes().

Referenced by Compiler::CreateComputation().

1428  {
1429  using namespace computation_graph;
1430  int32 num_cindex_ids = graph.cindexes.size();
1431 
1432  std::vector<int32> cindex_id_to_segment_and_epoch;
1433  std::vector<std::vector<std::vector<int32 > > > epochs_per_segment;
1434  std::vector<bool> epoch_is_trivial;
1435  ComputeEpochInfo(nnet, graph, &cindex_id_to_segment_and_epoch,
1436  &epochs_per_segment, &epoch_is_trivial);
1437 
1438  KALDI_ASSERT(SumVectorSizes(epochs_per_segment) == num_cindex_ids);
1439 
1440  // dependencies_subset contains just the subset of dependencies
1441  // of each cindex_id, that have the same epoch index as
1442  // cindex_id itself. This will be used to correctly order
1443  // cindexes within a certain epoch (relevant for things like
1444  // LSTMs).
1445  std::vector<std::vector<int32> > dependencies_subset;
1446  ComputeDependenciesSubset(graph, cindex_id_to_segment_and_epoch,
1447  &dependencies_subset);
1448  // destroy cindex_id_to_segment_and_epoch, it's no longer needed.
1449  { std::vector<int32> temp; temp.swap(cindex_id_to_segment_and_epoch); }
1450 
1451  // depend_on_subset is a subset of the normal "depend_on" list (i.e. a list of
1452  // all cindex_ids that depend on the current cindex_id), limited to just those
1453  // cindex_ids that have the same epoch index.
1454  std::vector<std::vector<int32> > depend_on_subset;
1455  ComputeGraphTranspose(dependencies_subset, &depend_on_subset);
1456 
1457  int32 num_epoch_indexes = epoch_is_trivial.size(),
1458  num_segments = graph.segment_ends.size();
1459 
1460  // "phase_indexes" is used inside ComputeComputationPhasesForEpoch.
1461  std::vector<int32> phase_indexes(num_cindex_ids, -1);
1462 
1463  phases_per_segment->clear();
1464  phases_per_segment->resize(num_segments);
1465 
1466  for (int32 segment = 0; segment < num_segments; segment++) {
1467  phases_per_segment->reserve(50); // minimize unnecessary copies. 50 is
1468  // very arbitrarily chosen.
1469  for (int32 epoch = 0; epoch < num_epoch_indexes; epoch++)
1471  epochs_per_segment[segment][epoch],
1472  dependencies_subset,
1473  depend_on_subset,
1474  epoch_is_trivial[epoch],
1475  &phase_indexes,
1476  &((*phases_per_segment)[segment]));
1477  }
1478 
1479 
1480  // make sure everything was computable. If the next assert fails it's likely
1481  // a bug in this function or in PruneComputataionGraph.
1482  KALDI_ASSERT(SumVectorSizes(*phases_per_segment) == num_cindex_ids);
1483 }
static void ComputeEpochInfo(const Nnet &nnet, const ComputationGraph &graph, std::vector< int32 > *cindex_id_to_segment_and_epoch, std::vector< std::vector< std::vector< int32 > > > *epochs_per_segment, std::vector< bool > *epoch_is_trivial)
This function computes certain information about "epochs" of cindex_ids.
static int32 SumVectorSizes(const std::vector< std::vector< std::vector< int32 > > > &vec)
void ComputeGraphTranspose(const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *graph_transpose)
Outputs a graph in which the order of arcs is reversed.
Definition: nnet-graph.cc:63
static void ComputeComputationPhasesForEpoch(const Nnet &nnet, const ComputationGraph &graph, const std::vector< int32 > &this_epoch, const std::vector< std::vector< int32 > > &dependencies_subset, const std::vector< std::vector< int32 > > &depend_on_subset, bool epoch_is_trivial, std::vector< int32 > *phase_indexes, std::vector< std::vector< int32 > > *phases)
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
static void ComputeDependenciesSubset(const ComputationGraph &graph, const std::vector< int32 > &cindex_id_to_segment_and_epoch, std::vector< std::vector< int32 > > *dependencies_subset)
This function outputs to dependencies_subset[c], for each cindex_id c, the subset of elements d of gr...
static void kaldi::nnet3::ComputeComputationPhasesForEpoch ( const Nnet &  nnet,
const ComputationGraph &  graph,
const std::vector< int32 > &  this_epoch,
const std::vector< std::vector< int32 > > &  dependencies_subset,
const std::vector< std::vector< int32 > > &  depend_on_subset,
bool  epoch_is_trivial,
std::vector< int32 > *  phase_indexes,
std::vector< std::vector< int32 > > *  phases 
)
inlinestatic

Definition at line 1326 of file nnet-computation-graph.cc.

References rnnlm::d, KALDI_ASSERT, KALDI_ERR, and kaldi::SortAndUniq().

Referenced by ComputeComputationPhases().

1334  {
1335  std::vector<int32> this_phase, next_phase_candidates;
1336 
1337  if (this_epoch.empty())
1338  return;
1339 
1340  if (epoch_is_trivial) { // an optimization
1341  this_phase = this_epoch;
1342  } else {
1343  // Start out with all elements of this epoch that have no
1344  // dependencies within the same epoch (i.e. those that
1345  // can be computed first).
1346  std::vector<int32>::const_iterator iter = this_epoch.begin(),
1347  end = this_epoch.end();
1348  for (; iter != end; ++iter) {
1349  int32 cindex_id = *iter;
1350  if (dependencies_subset[cindex_id].empty())
1351  this_phase.push_back(cindex_id);
1352  }
1353  }
1354 
1355  // if the next assert fails, the graph at the level of cindex_ids is not acyclic.
1356  KALDI_ASSERT(!this_phase.empty() &&
1357  "Trying to process computation with cycles");
1358 
1359  while (!this_phase.empty()) {
1360  // The next two lines are a more efficient version of:
1361  // phases->push_back(this_phase);
1362  phases->resize(phases->size() + 1);
1363  phases->back().swap(this_phase);
1364  // The next if-statement is an optimization: if for this epoch index
1365  // there is just one node, we can skip the rest of this loop. Note: if
1366  // epoch == 0, even if there is just one node, cindex_ids from
1367  // multiple nodes may be put here because of the rule that cindex_ids which
1368  // are inputs always get epoch 0. But it's still true that they
1369  // will have no dependencies, so we can still skip the code below.
1370  if (epoch_is_trivial)
1371  return;
1372 
1373  int32 cur_phase_index = phases->size() - 1;
1374 
1375  // next_phases_candidates is a list of cindexes that we should check
1376  // whether they are computable now, because one of the things they depend
1377  // on just became computable.
1378  next_phase_candidates.clear();
1379  std::vector<int32>::const_iterator this_phase_iter = phases->back().begin(),
1380  this_phase_end = phases->back().end();
1381 
1382  for (; this_phase_iter != this_phase_end; ++this_phase_iter) {
1383  int32 c = *this_phase_iter; // c is a cindex_id with phase cur_phase_index.
1384  (*phase_indexes)[c] = cur_phase_index;
1385  std::vector<int32>::const_iterator iter = depend_on_subset[c].begin(),
1386  end = depend_on_subset[c].end();
1387  for (; iter != end; ++iter) {
1388  int32 d = *iter; // cindex_id that depends on c.
1389  next_phase_candidates.push_back(d);
1390  }
1391  }
1392  SortAndUniq(&next_phase_candidates);
1393  // note, at this point 'this_phase' will be the empty vector [see the 'swap'
1394  // above].
1395  this_phase.reserve(next_phase_candidates.size());
1396  // now check the candidates that might be in the next phase, and put any
1397  // members that we are currently able to compute into "this_phase".
1398  std::vector<int32>::const_iterator iter = next_phase_candidates.begin(),
1399  end = next_phase_candidates.end();
1400  for (; iter != end; ++iter) {
1401  int32 c = *iter;
1402  std::vector<int32>::const_iterator
1403  dep_iter = dependencies_subset[c].begin(),
1404  dep_end = dependencies_subset[c].end();
1405  for (; dep_iter != dep_end; ++dep_iter) {
1406  int32 d = *dep_iter; // d is cindex_id that c depends on.
1407  if ((*phase_indexes)[d] < 0) // we can't compute c yet because something we depend
1408  break; // on has not yet been computed.
1409  }
1410  if (dep_iter == dep_end) {
1411  // we reached the end and did not break -> all dependencies satisfied
1412  this_phase.push_back(c);
1413  }
1414  }
1415  if (!next_phase_candidates.empty() && this_phase.empty()) {
1416  // this should have been caught earlier so likely a code error rather than
1417  // a problem with user input.
1418  KALDI_ERR << "Your model has a type of recurrence that cannot be computed. "
1419  << "E.g. if x[t] depends on both x[t+1] and x[t-1]... no order "
1420  << "of computation will work.";
1421  }
1422  }
1423 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq's (removes duplicates) from a vector.
Definition: stl-utils.h:39
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ComputeExampleComputationRequestSimple ( const Nnet &  nnet,
ComputationRequest *  request,
std::vector< Matrix< BaseFloat > > *  inputs 
)

This function computes an example computation request, for testing purposes.

The "Simple" in the name means that it currently only supports neural nets that satisfy IsSimple(nnet) (defined in nnet-utils.h). If there are 2 inputs, the "input" will be first, followed by "ivector".

In order to expand the range of things you can test with this (mainly to stop crashes with statistics-pooling/statistics-extraction components), this function always generates computation-requests where at least 3 successive frames of input are requested.

Definition at line 1321 of file nnet-test-utils.cc.

References ComputeSimpleNnetContext(), Nnet::InputDim(), ComputationRequest::inputs, IsSimpleNnet(), KALDI_ASSERT, rnnlm::n, ComputationRequest::need_model_derivative, ComputationRequest::outputs, kaldi::Rand(), and ComputationRequest::store_component_stats.

Referenced by UnitTestNnetAnalyze(), UnitTestNnetCompile(), UnitTestNnetCompileMulti(), UnitTestNnetCompute(), UnitTestNnetInputDerivatives(), UnitTestNnetModelDerivatives(), and UnitTestNnetOptimizeWithOptions().

1324  {
1325  KALDI_ASSERT(IsSimpleNnet(nnet));
1326 
1327  int32 left_context, right_context;
1328  ComputeSimpleNnetContext(nnet, &left_context, &right_context);
1329 
1330  int32 num_output_frames = 1 + Rand() % 10,
1331  output_start_frame = Rand() % 10,
1332  num_examples = 1 + Rand() % 10,
1333  output_end_frame = output_start_frame + num_output_frames,
1334  input_start_frame = output_start_frame - left_context - (Rand() % 3),
1335  input_end_frame = output_end_frame + right_context + (Rand() % 3),
1336  n_offset = Rand() % 2;
1337  bool need_deriv = (Rand() % 2 == 0);
1338  // make sure there are at least 3 frames of input available. this makes a
1339  // difference for our tests of statistics-pooling and statistics-extraction
1340  // component.
1341  if (input_end_frame < input_start_frame + 3)
1342  input_end_frame = input_start_frame + 3;
1343 
1344  request->inputs.clear();
1345  request->outputs.clear();
1346  inputs->clear();
1347 
1348  std::vector<Index> input_indexes, ivector_indexes, output_indexes;
1349  for (int32 n = n_offset; n < n_offset + num_examples; n++) {
1350  for (int32 t = input_start_frame; t < input_end_frame; t++)
1351  input_indexes.push_back(Index(n, t, 0));
1352  for (int32 t = output_start_frame; t < output_end_frame; t++)
1353  output_indexes.push_back(Index(n, t, 0));
1354  ivector_indexes.push_back(Index(n, 0, 0));
1355  }
1356  request->outputs.push_back(IoSpecification("output", output_indexes));
1357  if (need_deriv || (Rand() % 3 == 0))
1358  request->outputs.back().has_deriv = true;
1359  request->inputs.push_back(IoSpecification("input", input_indexes));
1360  if (need_deriv && (Rand() % 2 == 0))
1361  request->inputs.back().has_deriv = true;
1362  int32 input_dim = nnet.InputDim("input");
1363  KALDI_ASSERT(input_dim > 0);
1364  inputs->push_back(
1365  Matrix<BaseFloat>((input_end_frame - input_start_frame) * num_examples,
1366  input_dim));
1367  inputs->back().SetRandn();
1368  int32 ivector_dim = nnet.InputDim("ivector"); // may not exist.
1369  if (ivector_dim != -1) {
1370  request->inputs.push_back(IoSpecification("ivector", ivector_indexes));
1371  inputs->push_back(Matrix<BaseFloat>(num_examples, ivector_dim));
1372  inputs->back().SetRandn();
1373  if (need_deriv && (Rand() % 2 == 0))
1374  request->inputs.back().has_deriv = true;
1375  }
1376  if (Rand() % 2 == 0)
1377  request->need_model_derivative = need_deriv;
1378  if (Rand() % 2 == 0)
1379  request->store_component_stats = true;
1380 }
void ComputeSimpleNnetContext(const Nnet &nnet, int32 *left_context, int32 *right_context)
ComputeSimpleNnetContext computes the left-context and right-context of a nnet.
Definition: nnet-utils.cc:143
struct rnnlm::@11::@12 n
int Rand(struct RandomState *state)
Definition: kaldi-math.cc:44
bool IsSimpleNnet(const Nnet &nnet)
This function returns true if the nnet has the following properties: It has an output called "output"...
Definition: nnet-utils.cc:52
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ComputeGraphTranspose ( const std::vector< std::vector< int32 > > &  graph,
std::vector< std::vector< int32 > > *  graph_transpose 
)

Outputs a graph in which the order of arcs is reversed.

Definition at line 63 of file nnet-graph.cc.

References rnnlm::n.

Referenced by ComputeComputationPhases(), FindOrphanNodes(), and UnitTestComputeGraphTranspose().

64  {
65  int32 size = graph.size();
66  graph_transpose->clear();
67  graph_transpose->resize(size);
68  for (int32 n = 0; n < size; n++) {
69  const std::vector<int32> &nodes = graph[n];
70  std::vector<int32>::const_iterator iter = nodes.begin(), end = nodes.end();
71  for (; iter != end; ++iter) {
72  int32 dest = *iter;
73  (*graph_transpose)[dest].push_back(n);
74  }
75  }
76 }
struct rnnlm::@11::@12 n
void ComputeMatrixAccesses ( const Nnet &  nnet,
const NnetComputation &  computation,
const ComputationVariables &  variables,
const std::vector< CommandAttributes > &  command_attributes,
std::vector< MatrixAccesses > *  matrix_accesses 
)

This function organizes information in the CommandAttributes in a way that is convenient to access per matrix.

See the declaration of struct MatrixAccesses for the output format; the output "matrix_accesses" is indexed by matrix index (the same index as computation.matrices).

Definition at line 467 of file nnet-analyze.cc.

References NnetComputation::Command::arg1, NnetComputation::Command::arg2, NnetComputation::Command::command_type, NnetComputation::commands, kaldi::IsSortedAndUniq(), NnetComputation::IsWholeMatrix(), kAcceptInput, KALDI_ASSERT, KALDI_ERR, kAllocMatrix, kDeallocMatrix, kProvideOutput, kReadAccess, kReadWriteAccess, kSwapMatrix, kWriteAccess, NnetComputation::matrices, CommandAttributes::matrices_read, CommandAttributes::matrices_written, kaldi::SortAndUniq(), and NnetComputation::submatrices.

Referenced by Analyzer::Init(), and MoveSizingCommands().

472  {
473  int32 num_matrices = computation.matrices.size(),
474  num_commands = command_attributes.size();
475  matrix_accesses->clear();
476  matrix_accesses->resize(num_matrices);
477  for (int32 c = 0; c < num_commands; c++) {
478  const CommandAttributes &attr = command_attributes[c];
479  KALDI_ASSERT(IsSortedAndUniq(attr.matrices_read));
480  KALDI_ASSERT(IsSortedAndUniq(attr.matrices_written));
481  std::vector<int32> all_matrices;
482  all_matrices.reserve(attr.matrices_read.size() +
483  attr.matrices_written.size());
484  all_matrices.insert(all_matrices.end(), attr.matrices_read.begin(),
485  attr.matrices_read.end());
486  all_matrices.insert(all_matrices.end(), attr.matrices_written.begin(),
487  attr.matrices_written.end());
488  SortAndUniq(&all_matrices);
489 
490  std::vector<int32>::const_iterator iter = all_matrices.begin(),
491  end = all_matrices.end();
492  for (; iter != end; ++iter) {
493  int32 matrix_index = *iter;
494  bool is_read = std::binary_search(attr.matrices_read.begin(),
495  attr.matrices_read.end(),
496  matrix_index),
497  is_written = (!is_read ? true :
498  std::binary_search(attr.matrices_written.begin(),
499  attr.matrices_written.end(),
500  matrix_index));
501  if (is_read && is_written) {
502  (*matrix_accesses)[matrix_index].accesses.push_back(
503  Access(c, kReadWriteAccess));
504  } else if (is_read) {
505  (*matrix_accesses)[matrix_index].accesses.push_back(
506  Access(c, kReadAccess));
507  } else {
508  (*matrix_accesses)[matrix_index].accesses.push_back(
509  Access(c, kWriteAccess));
510  }
511  }
512  // Now set up allocate_command, deallocate_command,
513  // is_input and is_output.
514  const NnetComputation::Command &command = computation.commands[c];
515  int32 matrix_index1, matrix_index2;
516 
517  switch (command.command_type) {
518  case kAllocMatrix:
519  if (!computation.IsWholeMatrix(command.arg1))
520  KALDI_ERR << "Command does not operate on whole matrix";
521  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
522  if ((*matrix_accesses)[matrix_index1].allocate_command != -1)
523  KALDI_ERR << "Matrix " << matrix_index1 << " initialized twice.";
524  (*matrix_accesses)[matrix_index1].allocate_command = c;
525  break;
526  case kSwapMatrix:
527  if (!computation.IsWholeMatrix(command.arg1))
528  KALDI_ERR << "Command does not operate on whole matrix";
529  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
530  KALDI_ASSERT(computation.IsWholeMatrix(command.arg2));
531  matrix_index2 = computation.submatrices[command.arg2].matrix_index;
532  if ((*matrix_accesses)[matrix_index1].allocate_command != -1)
533  KALDI_ERR << "Matrix " << matrix_index1 << " initialized twice.";
534  (*matrix_accesses)[matrix_index1].allocate_command = c;
535  if ((*matrix_accesses)[matrix_index2].deallocate_command != -1)
536  KALDI_ERR << "Matrix " << matrix_index2 << " destroyed twice.";
537  (*matrix_accesses)[matrix_index2].deallocate_command = c;
538  break;
539  case kDeallocMatrix:
540  if (!computation.IsWholeMatrix(command.arg1))
541  KALDI_ERR << "Command does not operate on whole matrix";
542  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
543  if ((*matrix_accesses)[matrix_index1].deallocate_command != -1)
544  KALDI_ERR << "Matrix " << matrix_index1 << " destroyed twice.";
545  (*matrix_accesses)[matrix_index1].deallocate_command = c;
546  break;
547  case kAcceptInput:
548  if (!computation.IsWholeMatrix(command.arg1))
549  KALDI_ERR << "Command does not operate on whole matrix";
550  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
551  (*matrix_accesses)[matrix_index1].is_input = true;
552  // If a certain matrix is accepted as input multiple times, we
553  // count the first one as allocating it (the second will just
554  // allocate it again, which is harmless).
555  if ((*matrix_accesses)[matrix_index1].allocate_command == -1)
556  (*matrix_accesses)[matrix_index1].allocate_command = c;
557  break;
558  case kProvideOutput:
559  if (!computation.IsWholeMatrix(command.arg1))
560  KALDI_ERR << "Command does not operate on whole matrix";
561  matrix_index1 = computation.submatrices[command.arg1].matrix_index;
562  (*matrix_accesses)[matrix_index1].is_output = true;
563  break;
564  default:
565  ;
566  }
567  }
568 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq's (removes duplicates) from a vector.
Definition: stl-utils.h:39
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool IsSortedAndUniq(const std::vector< T > &vec)
Returns true if the vector is sorted and contains each element only once.
Definition: stl-utils.h:63
void ComputeMatrixToSubmatrix ( const NnetComputation &  computation,
std::vector< std::vector< int32 > > *  mat_to_submat 
)

This function computes a vector 'mat_to_submat', indexed by matrix index, such that (*mat_to_submat)[m] is a list of all the submatrix indexes that refer to matrix m.

Note, (*mat_to_submat)[0] will be the empty vector.

Definition at line 1166 of file nnet-analyze.cc.

References KALDI_ASSERT, NnetComputation::matrices, and NnetComputation::submatrices.

Referenced by VariableMergingOptimizer::VariableMergingOptimizer().

1168  {
1169  int32 num_matrices = computation.matrices.size(),
1170  num_submatrices = computation.submatrices.size();
1171  mat_to_submat->clear();
1172  mat_to_submat->resize(num_matrices);
1173  for (int32 submatrix_index = 1;
1174  submatrix_index < num_submatrices;
1175  submatrix_index++) {
1176  int32 matrix_index = computation.submatrices[submatrix_index].matrix_index;
1177  KALDI_ASSERT(matrix_index > 0 && matrix_index < num_matrices);
1178  (*mat_to_submat)[matrix_index].push_back(submatrix_index);
1179  }
1180 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::ComputeMinAndMaxTimes ( const std::vector< Index > &  indexes,
int32 *  min_t,
int32 *  max_t 
)

Definition at line 31 of file nnet-derivative-test.cc.

References KALDI_ASSERT, and rnnlm::n.

Referenced by SetDerivTimesOptions().

33  {
34  KALDI_ASSERT(!indexes.empty());
35  *min_t = indexes[0].t;
36  *max_t = *min_t;
37  for (int32 n = 1; n < static_cast<int32>(indexes.size()); n++) {
38  *min_t = std::min(*min_t, indexes[n].t);
39  *max_t = std::max(*max_t, indexes[n].t);
40  }
41 }
struct rnnlm::@11::@12 n
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ComputeNnetComputationEpochs ( const Nnet &  nnet,
std::vector< int32 > *  node_to_epoch 
)

This function computes the order in which we need to compute each node in the graph, where each node-index n maps to an epoch-index t = 0, 1, ...

that says when we should compute it. Nodes that are part of a strongly connected component (SCC) will all be computed at the same time, but any two nodes that are not part of an SCC will have different epoch-index, and these epoch-indexes will be such that a node computed at a larger epoch-index may depend on a node computed at a smaller epoch-index, but not vice versa.

Internally it calls NnetToDirectedGraph, FindSccs, MakeSccGraph and ComputeTopSortOrder.

Definition at line 265 of file nnet-graph.cc.

References ComputeTopSortOrder(), FindSccs(), kaldi::GetVerboseLevel(), rnnlm::i, rnnlm::j, KALDI_ASSERT, KALDI_VLOG, MakeSccGraph(), NnetToDirectedGraph(), and PrintGraphToString().

Referenced by kaldi::nnet3::computation_graph::ComputeEpochInfo().

266  {
267  KALDI_ASSERT(node_to_epoch != NULL);
268 
269  std::vector<std::vector<int32> > graph;
270  NnetToDirectedGraph(nnet, &graph);
271  KALDI_VLOG(6) << "graph is: " << PrintGraphToString(graph);
272 
273  std::vector<std::vector<int32> > sccs;
274  FindSccs(graph, &sccs);
275 
276  std::vector<std::vector<int32> > scc_graph;
277  MakeSccGraph(graph, sccs, &scc_graph);
278  KALDI_VLOG(6) << "scc graph is: " << PrintGraphToString(scc_graph);
279 
280  std::vector<int32> scc_node_to_epoch;
281  ComputeTopSortOrder(scc_graph, &scc_node_to_epoch);
282  if (GetVerboseLevel() >= 6) {
283  std::ostringstream os;
284  for (int32 i = 0; i < scc_node_to_epoch.size(); i++)
285  os << scc_node_to_epoch[i] << ", ";
286  KALDI_VLOG(6) << "scc_node_to_epoch is: " << os.str();
287  }
288 
289  node_to_epoch->clear();
290  node_to_epoch->resize(graph.size());
291  for (int32 i = 0; i < sccs.size(); ++i) {
292  for (int32 j = 0; j < sccs[i].size(); ++j) {
293  int32 node = sccs[i][j];
294  KALDI_ASSERT(node >= 0 && node < graph.size());
295  (*node_to_epoch)[node] = scc_node_to_epoch[i];
296  }
297  }
298 }
void NnetToDirectedGraph(const Nnet &nnet, std::vector< std::vector< int32 > > *graph)
This function takes an nnet and turns it to a directed graph on nodes.
Definition: nnet-graph.cc:30
int32 GetVerboseLevel()
Definition: kaldi-error.h:69
void ComputeTopSortOrder(const std::vector< std::vector< int32 > > &graph, std::vector< int32 > *node_to_order)
Given an acyclic graph (where each std::vector is a list of destination-nodes of arcs coming f...
Definition: nnet-graph.cc:223
void FindSccs(const std::vector< std::vector< int32 > > &graph, std::vector< std::vector< int32 > > *sccs)
Given a directed graph (where each std::vector is a list of destination-nodes of arcs coming f...
Definition: nnet-graph.cc:156
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void MakeSccGraph(const std::vector< std::vector< int32 > > &graph, const std::vector< std::vector< int32 > > &sccs, std::vector< std::vector< int32 > > *scc_graph)
Given a list of sccs of a graph (e.g.
Definition: nnet-graph.cc:164
#define KALDI_VLOG(v)
Definition: kaldi-error.h:136
std::string PrintGraphToString(const std::vector< std::vector< int32 > > &graph)
Prints a graph to a string in a pretty way for human readability, e.g.
Definition: nnet-graph.cc:248
void ComputeObjectiveFunction ( const GeneralMatrix &  supervision,
ObjectiveType  objective_type,
const std::string &  output_name,
bool  supply_deriv,
NnetComputer *  computer,
BaseFloat *  tot_weight,
BaseFloat *  tot_objf 
)

This function computes the objective function, and if supply_deriv = true, supplies its derivative to the NnetComputation object.

See also the function ComputeAccuracy(), declared in nnet-diagnostics.h.

Parameters
[in]supervisionA GeneralMatrix, typically derived from a NnetExample, containing the supervision posteriors or features.
[in]objective_typeThe objective function type: kLinear = output * supervision, or kQuadratic = -0.5 * (output - supervision)^2. kLinear is used for softmax objectives; the network contains a LogSoftmax layer which correctly normalizes its output.
[in]output_nameThe name of the output node (e.g. "output"), used to look up the output in the NnetComputer object.
[in]supply_derivIf this is true, this function will compute the derivative of the objective function and supply it to the network using the function NnetComputer::AcceptOutputDeriv
[in,out]computerThe NnetComputer object, from which we get the output using GetOutput and to which we may supply the derivatives using AcceptOutputDeriv.
[out]tot_weightThe total weight of the training examples. In the kLinear case, this is the sum of the supervision matrix; in the kQuadratic case, it is the number of rows of the supervision matrix. In order to make it possible to weight samples with quadratic objective functions, we may at some point make it possible for the supervision matrix to have an extra column containing weights. At the moment, this is not supported.
[out]tot_objfThe total objective function; divide this by the tot_weight to get the normalized objective function.

Definition at line 368 of file nnet-training.cc.

References NnetComputer::AcceptInput(), CuMatrixBase< Real >::CopyFromGeneralMat(), CuSparseMatrix< Real >::CopyToMat(), GeneralMatrix::GetFullMatrix(), GeneralMatrix::GetMatrix(), NnetComputer::GetOutput(), GeneralMatrix::GetSparseMatrix(), KALDI_ERR, kaldi::kCompressedMatrix, kaldi::kFullMatrix, kLinear, kQuadratic, kaldi::kSparseMatrix, kaldi::kTrans, kaldi::kUndefined, CuMatrixBase< Real >::NumCols(), GeneralMatrix::NumCols(), CuMatrixBase< Real >::NumRows(), GeneralMatrix::NumRows(), CuSparseMatrix< Real >::Sum(), CuMatrixBase< Real >::Sum(), CuMatrix< Real >::Swap(), kaldi::TraceMatMat(), kaldi::TraceMatSmat(), and GeneralMatrix::Type().

Referenced by NnetComputeProb::ProcessOutputs(), and NnetTrainer::ProcessOutputs().

374  {
375  const CuMatrixBase<BaseFloat> &output = computer->GetOutput(output_name);
376 
377  if (output.NumCols() != supervision.NumCols())
378  KALDI_ERR << "Nnet versus example output dimension (num-classes) "
379  << "mismatch for '" << output_name << "': " << output.NumCols()
380  << " (nnet) vs. " << supervision.NumCols() << " (egs)\n";
381 
382  switch (objective_type) {
383  case kLinear: {
384  // objective is x * y.
385  switch (supervision.Type()) {
386  case kSparseMatrix: {
387  const SparseMatrix<BaseFloat> &post = supervision.GetSparseMatrix();
388  CuSparseMatrix<BaseFloat> cu_post(post);
389  // The cross-entropy objective is computed by a simple dot product,
390  // because after the LogSoftmaxLayer, the output is already in the form
391  // of log-likelihoods that are normalized to sum to one.
392  *tot_weight = cu_post.Sum();
393  *tot_objf = TraceMatSmat(output, cu_post, kTrans);
394  if (supply_deriv) {
395  CuMatrix<BaseFloat> output_deriv(output.NumRows(), output.NumCols(),
396  kUndefined);
397  cu_post.CopyToMat(&output_deriv);
398  computer->AcceptInput(output_name, &output_deriv);
399  }
400  break;
401  }
402  case kFullMatrix: {
403  // there is a redundant matrix copy in here if we're not using a GPU
404  // but we don't anticipate this code branch being used in many cases.
405  CuMatrix<BaseFloat> cu_post(supervision.GetFullMatrix());
406  *tot_weight = cu_post.Sum();
407  *tot_objf = TraceMatMat(output, cu_post, kTrans);
408  if (supply_deriv)
409  computer->AcceptInput(output_name, &cu_post);
410  break;
411  }
412  case kCompressedMatrix: {
413  Matrix<BaseFloat> post;
414  supervision.GetMatrix(&post);
415  CuMatrix<BaseFloat> cu_post;
416  cu_post.Swap(&post);
417  *tot_weight = cu_post.Sum();
418  *tot_objf = TraceMatMat(output, cu_post, kTrans);
419  if (supply_deriv)
420  computer->AcceptInput(output_name, &cu_post);
421  break;
422  }
423  }
424  break;
425  }
426  case kQuadratic: {
427  // objective is -0.5 (x - y)^2
428  CuMatrix<BaseFloat> diff(supervision.NumRows(),
429  supervision.NumCols(),
430  kUndefined);
431  diff.CopyFromGeneralMat(supervision);
432  diff.AddMat(-1.0, output);
433  *tot_weight = diff.NumRows();
434  *tot_objf = -0.5 * TraceMatMat(diff, diff, kTrans);
435  if (supply_deriv)
436  computer->AcceptInput(output_name, &diff);
437  break;
438  }
439  default:
440  KALDI_ERR << "Objective function type " << objective_type
441  << " not handled.";
442  }
443 }
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
#define KALDI_ERR
Definition: kaldi-error.h:127
Real TraceMatSmat(const MatrixBase< Real > &A, const SparseMatrix< Real > &B, MatrixTransposeType trans)
double kaldi::nnet3::ComputeObjf ( bool  batchnorm_test_mode,
bool  dropout_test_mode,
const std::vector< NnetExample > &  egs,
const Nnet nnet,
NnetComputeProb prob_computer 
)

Definition at line 35 of file nnet3-combine.cc.

References NnetComputeProb::Compute(), NnetComputeProb::GetTotalObjective(), KALDI_ASSERT, NnetComputeProb::Reset(), SetBatchnormTestMode(), and SetDropoutTestMode().

Referenced by main().

37  {
38  if (batchnorm_test_mode || dropout_test_mode) {
39  Nnet nnet_copy(nnet);
40  if (batchnorm_test_mode)
41  SetBatchnormTestMode(true, &nnet_copy);
42  if (dropout_test_mode)
43  SetDropoutTestMode(true, &nnet_copy);
44  NnetComputeProbOptions compute_prob_opts;
45  NnetComputeProb prob_computer_test(compute_prob_opts, nnet_copy);
46  return ComputeObjf(false, false, egs, nnet_copy, &prob_computer_test);
47  } else {
48  prob_computer->Reset();
49  std::vector<NnetExample>::const_iterator iter = egs.begin(),
50  end = egs.end();
51  for (; iter != end; ++iter)
52  prob_computer->Compute(*iter);
53  double tot_weights,
54  tot_objf = prob_computer->GetTotalObjective(&tot_weights);
55  KALDI_ASSERT(tot_weights > 0.0);
56  // inf/nan tot_objf->return -inf objective.
57  if (!(tot_objf == tot_objf && tot_objf - tot_objf == 0))
58  return -std::numeric_limits<double>::infinity();
59  // we prefer to deal with normalized objective functions.
60  return tot_objf / tot_weights;
61  }
62 }
void Compute(const NnetExample &eg)
void SetBatchnormTestMode(bool test_mode, Nnet *nnet)
This function affects only components of type BatchNormComponent.
Definition: nnet-utils.cc:533
This class is for computing cross-entropy and accuracy values in a neural network, for diagnostics.
double ComputeObjf(bool batchnorm_test_mode, bool dropout_test_mode, const std::vector< NnetExample > &egs, const Nnet &nnet, NnetComputeProb *prob_computer)
double GetTotalObjective(double *tot_weight) const
void SetDropoutTestMode(bool test_mode, Nnet *nnet)
This function affects components of child-classes of RandomComponent.
Definition: nnet-utils.cc:542
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ComputeSimpleNnetContext ( const Nnet &  nnet,
int32 *  left_context,
int32 *  right_context 
)

ComputeSimpleNnetContext computes the left-context and right-context of a nnet.

The nnet must satisfy IsSimpleNnet(nnet).

It does this by constructing a ComputationRequest with a certain number of inputs available, outputs can be computed.. It does the same after shifting the time index of the output to all values 0, 1, ... n-1, where n is the output of nnet.Modulus(). Then it returns the largest left context and the largest right context that it infers from any of these computation requests.

Definition at line 143 of file nnet-utils.cc.

References ComputeSimpleNnetContextForShift(), IsSimpleNnet(), KALDI_ASSERT, and Nnet::Modulus().

Referenced by ComputeExampleComputationRequestSimple(), CreateLoopedComputationRequestSimple(), DecodableNnetSimple::DecodableNnetSimple(), Nnet::Info(), DecodableNnetSimpleLoopedInfo::Init(), NnetInfo(), AmNnetSimple::SetContext(), and UnitTestNnetContext().

145  {
146  KALDI_ASSERT(IsSimpleNnet(nnet));
147  int32 modulus = nnet.Modulus();
148  // modulus >= 1 is a number such that the network ought to be
149  // invariant to time shifts (of both the input and output) that
150  // are a multiple of this number. We need to test all shifts modulo
151  // this number in case the left and right context vary at all within
152  // this range.
153 
154  std::vector<int32> left_contexts(modulus + 1);
155  std::vector<int32> right_contexts(modulus + 1);
156 
157  // This will crash if the total context (left + right) is greater
158  // than window_size.
159  int32 window_size = 200;
160 
161  // by going "<= modulus" instead of "< modulus" we do one more computation
162  // than we really need; it becomes a sanity check.
163  for (int32 input_start = 0; input_start <= modulus; input_start++)
164  ComputeSimpleNnetContextForShift(nnet, input_start, window_size,
165  &(left_contexts[input_start]),
166  &(right_contexts[input_start]));
167  KALDI_ASSERT(left_contexts[0] == left_contexts[modulus] &&
168  "nnet does not have the properties we expect.");
169  KALDI_ASSERT(right_contexts[0] == right_contexts[modulus] &&
170  "nnet does not have the properties we expect.");
171  *left_context =
172  *std::max_element(left_contexts.begin(), left_contexts.end());
173  *right_context =
174  *std::max_element(right_contexts.begin(), right_contexts.end());
175 }
bool IsSimpleNnet(const Nnet &nnet)
This function returns true if the nnet has the following properties: It has an output called "output"...
Definition: nnet-utils.cc:52
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
static void ComputeSimpleNnetContextForShift(const Nnet &nnet, int32 input_start, int32 window_size, int32 *left_context, int32 *right_context)
Definition: nnet-utils.cc:89
static void kaldi::nnet3::ComputeSimpleNnetContextForShift ( const Nnet &  nnet,
int32  input_start,
int32  window_size,
int32 *  left_context,
int32 *  right_context 
)
static

Definition at line 89 of file nnet-utils.cc.

References EvaluateComputationRequest(), Nnet::GetNodeIndex(), IoSpecification::indexes, ComputationRequest::inputs, KALDI_ASSERT, KALDI_ERR, Nnet::Modulus(), rnnlm::n, IoSpecification::name, and ComputationRequest::outputs.

Referenced by ComputeSimpleNnetContext().

94  {
95 
96  int32 input_end = input_start + window_size;
97  IoSpecification input;
98  input.name = "input";
99  IoSpecification output;
100  output.name = "output";
101  IoSpecification ivector; // we might or might not use this.
102  ivector.name = "ivector";
103 
104  int32 n = rand() % 10;
105  // in the IoSpecification for now we we will request all the same indexes at
106  // output that we requested at input.
107  for (int32 t = input_start; t < input_end; t++) {
108  input.indexes.push_back(Index(n, t));
109  output.indexes.push_back(Index(n, t));
110  }
111 
112  // most networks will just require the ivector at time t = 0,
113  // but this might not always be the case, and some might use rounding
114  // descriptors with the iVector which might require it at an earlier
115  // frame than the regular input, so we provide the iVector in as wide a range
116  // as it might possibly be needed.
117  for (int32 t = input_start - nnet.Modulus(); t < input_end; t++) {
118  ivector.indexes.push_back(Index(n, t));
119  }
120 
121 
122  ComputationRequest request;
123  request.inputs.push_back(input);
124  request.outputs.push_back(output);
125  if (nnet.GetNodeIndex("ivector") != -1)
126  request.inputs.push_back(ivector);
127  std::vector<std::vector<bool> > computable;
128  EvaluateComputationRequest(nnet, request, &computable);
129 
130  KALDI_ASSERT(computable.size() == 1);
131  std::vector<bool> &output_ok = computable[0];
132  std::vector<bool>::iterator iter =
133  std::find(output_ok.begin(), output_ok.end(), true);
134  int32 first_ok = iter - output_ok.begin();
135  int32 first_not_ok = std::find(iter, output_ok.end(), false) -
136  output_ok.begin();
137  if (first_ok == window_size || first_not_ok <= first_ok)
138  KALDI_ERR << "No outputs were computable (perhaps not a simple nnet?)";
139  *left_context = first_ok;
140  *right_context = window_size - first_not_ok;
141 }
void EvaluateComputationRequest(const Nnet &nnet, const ComputationRequest &request, std::vector< std::vector< bool > > *is_computable)
Given an nnet and a computation request, this function works out which requested outputs in the compu...
Definition: nnet-utils.cc:71
struct rnnlm::@11::@12 n
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::ComputeSubmatIndexHistogram ( const std::vector< std::vector< std::pair< int32, int32 > > >  sorted_submat_lists,
unordered_map< int32, std::vector< int32 > > *  submat_histogram 
)

Definition at line 113 of file nnet-compile-utils.cc.

References rnnlm::i, rnnlm::j, and KALDI_ASSERT.

Referenced by SplitLocations().

123  {
124  KALDI_ASSERT(sorted_submat_lists.size() > 0);
125  // computing the submat_histogram
126  // counting the occurrences of each element in the current submat_list;
127  // each new occurrence of the same element, in this list, is counted
128  // as a seperate symbol for frequency counts
129  for (int32 i = 0; i < sorted_submat_lists.size(); i++) {
130  int j = 0;
131  unordered_map<int32, std::vector<int32> >::iterator histogram_iterator
132  = submat_histogram->end();
133  int32 repetition_count = 0;
134  while (j < sorted_submat_lists[i].size()) {
135  if ((histogram_iterator == submat_histogram->end()) ||
136  (histogram_iterator->first != sorted_submat_lists[i][j].first)) {
137  histogram_iterator =
138  submat_histogram->find(sorted_submat_lists[i][j].first);
139  repetition_count = 0;
140  // if a histogram entry was not found for this submat_index, add one
141  if (histogram_iterator == submat_histogram->end()) {
142  (*submat_histogram)[sorted_submat_lists[i][j].first];
143  histogram_iterator = submat_histogram->find(
144  sorted_submat_lists[i][j].first);
145  }
146  }
147 
148  if (repetition_count >= (histogram_iterator->second).size()) {
149  // this is the first time the submat_index repeated this many times
150  // so add an entry for this in the count vector
151  (histogram_iterator->second).push_back(1);
152  } else {
153  (histogram_iterator->second)[repetition_count]++;
154  }
155  repetition_count++;
156  j++;
157  }
158  }
159 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ComputeTopSortOrder ( const std::vector< std::vector< int32 > > &  graph,
std::vector< int32 > *  node_to_order 
)

Given an acyclic graph (where each std::vector<int32> is a list of destination-nodes of arcs coming from the current node), compute a topological ordering of the graph nodes.

The output format is that node_to_order[n] contains an integer t = 0, 1, ... which is the order of node n in a topological sorting. node_to_order should contain some permutation of the numbers 0 ... graph.size() - 1. This function should crash if the graph contains cycles.

Definition at line 223 of file nnet-graph.cc.

References ComputeTopSortOrderRecursive(), rnnlm::i, and KALDI_ASSERT.

Referenced by ComputeNnetComputationEpochs(), UnitTestComputeTopSortOrder(), and UnitTestComputeTopSortOrder2().

224  {
225  // Internally we use DFS, but we only put the node to <node_to_order> when all
226  // its parents have been visited.
227  KALDI_ASSERT(node_to_order != NULL);
228  node_to_order->resize(graph.size());
229 
230  std::vector<bool> cycle_detector(graph.size(), false);
231  std::vector<bool> is_visited(graph.size(), false);
232 
233  std::vector<int32> reversed_orders;
234  for(int32 i = 0; i < graph.size(); ++i) {
235  if (!is_visited[i]) {
236  ComputeTopSortOrderRecursive(i, graph, &cycle_detector,
237  &is_visited, &reversed_orders);
238  }
239  }
240 
241  KALDI_ASSERT(node_to_order->size() == reversed_orders.size());
242  for (int32 i = 0; i < reversed_orders.size(); ++i) {
243  KALDI_ASSERT(reversed_orders[i] >= 0 && reversed_orders[i] < graph.size());
244  (*node_to_order)[reversed_orders[i]] = graph.size() - i - 1;
245  }
246 }
void ComputeTopSortOrderRecursive(int32 node, const std::vector< std::vector< int32 > > &graph, std::vector< bool > *cycle_detector, std::vector< bool > *is_visited, std::vector< int32 > *reversed_orders)
Definition: nnet-graph.cc:196
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::ComputeTopSortOrderRecursive ( int32  node,
const std::vector< std::vector< int32 > > &  graph,
std::vector< bool > *  cycle_detector,
std::vector< bool > *  is_visited,
std::vector< int32 > *  reversed_orders 
)

Definition at line 196 of file nnet-graph.cc.

References rnnlm::i, KALDI_ASSERT, and KALDI_ERR.

Referenced by ComputeTopSortOrder().

200  {
201  KALDI_ASSERT(node >= 0 && node < graph.size());
202  KALDI_ASSERT(cycle_detector != NULL);
203  KALDI_ASSERT(is_visited != NULL);
204  KALDI_ASSERT(reversed_orders != NULL);
205  if ((*cycle_detector)[node]) {
206  KALDI_ERR << "Cycle detected when computing the topological sorting order";
207  }
208 
209  if (!(*is_visited)[node]) {
210  (*cycle_detector)[node] = true;
211  for (int32 i = 0; i < graph[node].size(); ++i) {
212  ComputeTopSortOrderRecursive(graph[node][i], graph,
213  cycle_detector, is_visited, reversed_orders);
214  }
215  (*cycle_detector)[node] = false;
216  (*is_visited)[node] = true;
217  // At this point we have added all the children to <reversed_orders>, so we
218  // can add the current now.
219  reversed_orders->push_back(node);
220  }
221 }
void ComputeTopSortOrderRecursive(int32 node, const std::vector< std::vector< int32 > > &graph, std::vector< bool > *cycle_detector, std::vector< bool > *is_visited, std::vector< int32 > *reversed_orders)
Definition: nnet-graph.cc:196
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ComputeVariableAccesses ( const ComputationVariables &  variables,
const std::vector< CommandAttributes > &  command_attributes,
std::vector< std::vector< Access > > *  variable_accesses 
)

After the command-level attributes have been computed, this function organizes them per variable (see class ComputationVariables for how a variable is defined; it is part of a matrix).

Parameters
[in]variablesThe definition of variables for this computation
[in]command_attributesA vector of attributes, one per command, as obtained from ComputeCommandAttributes().
[out]variable_accessesThe output will have a size equal to the number of variables, and each element will be a vector of accesses, sorted by command index; each command will only be listed once in this vector.

Definition at line 421 of file nnet-analyze.cc.

References kaldi::IsSortedAndUniq(), KALDI_ASSERT, kReadAccess, kReadWriteAccess, kWriteAccess, ComputationVariables::NumVariables(), kaldi::SortAndUniq(), CommandAttributes::variables_read, and CommandAttributes::variables_written.

Referenced by Analyzer::Init(), and MoveSizingCommands().

424  {
425  int32 num_variables = variables.NumVariables(),
426  num_commands = command_attributes.size();
427  variable_accesses->clear();
428  variable_accesses->resize(num_variables);
429  for (int32 c = 0; c < num_commands; c++) {
430  const CommandAttributes &attr = command_attributes[c];
431  KALDI_ASSERT(IsSortedAndUniq(attr.variables_read));
432  KALDI_ASSERT(IsSortedAndUniq(attr.variables_written));
433  std::vector<int32> all_variables;
434  all_variables.reserve(attr.variables_read.size() +
435  attr.variables_written.size());
436  all_variables.insert(all_variables.end(), attr.variables_read.begin(),
437  attr.variables_read.end());
438  all_variables.insert(all_variables.end(), attr.variables_written.begin(),
439  attr.variables_written.end());
440  SortAndUniq(&all_variables);
441 
442  std::vector<int32>::const_iterator iter = all_variables.begin(),
443  end = all_variables.end();
444  for (; iter != end; ++iter) {
445  int32 variable_index = *iter;
446  bool is_read = std::binary_search(attr.variables_read.begin(),
447  attr.variables_read.end(),
448  variable_index),
449  is_written = (!is_read ? true :
450  std::binary_search(attr.variables_written.begin(),
451  attr.variables_written.end(),
452  variable_index));
453  if (is_read && is_written) {
454  (*variable_accesses)[variable_index].push_back(
455  Access(c, kReadWriteAccess));
456  } else if (is_read) {
457  (*variable_accesses)[variable_index].push_back(
458  Access(c, kReadAccess));
459  } else {
460  (*variable_accesses)[variable_index].push_back(
461  Access(c, kWriteAccess));
462  }
463  }
464  }
465 }
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq's (removes duplicates) from a vector.
Definition: stl-utils.h:39
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool IsSortedAndUniq(const std::vector< T > &vec)
Returns true if the vector is sorted and contains each element only once.
Definition: stl-utils.h:63
void ConsolidateIoOperations ( const Nnet &  nnet,
NnetComputation *  computation 
)

This optimization puts the input operations (kAcceptInput) and output operations (kProvideOutput) at the very beginning or end of segments of computation, respectively.

This is actually necessary for computations to be run easily, because if these commands were interspersed with the regular commands, you'd have to call computer.Run() between the individual AcceptInput() and GetOutput() function calls.

Definition at line 857 of file nnet-optimize.cc.

References NnetComputation::commands, kAcceptInput, KALDI_ASSERT, kNoOperationMarker, kProvideOutput, and SplitComputationIntoSegments().

Referenced by Compiler::CreateComputation(), and Optimize().

858  {
859  // These segments, represented as (start-index, end-index),
860  // are segments of the computation separated by kNoOperationMarker.
861  std::vector<std::pair<int32, int32> > segments;
862  SplitComputationIntoSegments(*computation, &segments);
863 
864  int32 num_commands = computation->commands.size();
865  std::vector<NnetComputation::Command> reordered_commands(num_commands);
866  // put kNoOperationMarker between all segments in the reordered commands.
867  for (size_t s = 0; s + 1 < segments.size(); s++)
868  reordered_commands[segments[s].second].command_type = kNoOperationMarker;
869 
870  // for each segment we'll divide the commands up into those that must appear
871  // at the left of the segment (kAcceptInput for inputs and output-derivs), those
872  // that must appear in the middle (most commands), those that must appear
873  // on the right (kProvideOutput for output nodes and input derivatives).
874  std::vector<int32> left_commands, middle_commands, right_commands;
875 
876  for (size_t s = 0; s < segments.size(); s++) {
877  int32 segment_start = segments[s].first,
878  segment_end = segments[s].second;
879  left_commands.clear();
880  middle_commands.clear();
881  right_commands.clear();
882  for (int32 c = segment_start; c < segment_end; c++) {
883  if (computation->commands[c].command_type == kProvideOutput) {
884  right_commands.push_back(c);
885  } else if (computation->commands[c].command_type == kAcceptInput) {
886  left_commands.push_back(c);
887  } else {
888  middle_commands.push_back(c);
889  }
890  }
891  std::vector<int32>::const_iterator iter = left_commands.begin(),
892  end = left_commands.end();
893  int32 c = segment_start;
894  for (; iter != end; ++iter, ++c)
895  reordered_commands[c] = computation->commands[*iter];
896  iter = middle_commands.begin();
897  end = middle_commands.end();
898  for (; iter != end; ++iter, ++c)
899  reordered_commands[c] = computation->commands[*iter];
900  iter = right_commands.begin();
901  end = right_commands.end();
902  for (; iter != end; ++iter, ++c)
903  reordered_commands[c] = computation->commands[*iter];
904  KALDI_ASSERT(c == segment_end);
905  }
906  computation->commands.swap(reordered_commands);
907 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
static void SplitComputationIntoSegments(const NnetComputation &computation, std::vector< std::pair< int32, int32 > > *segments)
Split the computation up into segments bounded by kNoOperationMarker.
void ConsolidateModelUpdate ( const Nnet &  nnet,
NnetComputation *  computation 
)

This optimization consolidates the model-update part of backprop commands, for components in (e.g.) recurrent networks that need to have many separate backprop commands, into more efficient single commands operating on consolidated data in larger matrices.

This consolidates the model-update parts of the backprop into larger operations (applicable mostly to recurrent setups)– internally it uses class ModelUpdateConsolidator.

This is useful for recurrent networks. The resulting computation separates the backprop for data-derivatives from the model-update part of backprop.

Will fail if called a second time.

Definition at line 1554 of file nnet-optimize-utils.cc.

References ModelUpdateConsolidator::ConsolidateModelUpdate(), and NnetComputation::need_model_derivative.

Referenced by Optimize().

1555  {
1556  // This following if-statement is an optimization: if the computation
1557  // request(s) had need_model_derivative == false, there would be nothing to
1558  // optimize, so don't bother trying.
1559  if (!computation->need_model_derivative)
1560  return;
1561  ModelUpdateConsolidator consolidator(nnet, computation);
1562  consolidator.ConsolidateModelUpdate();
1563 }
void ConstrainOrthonormal ( Nnet *  nnet)

This function, to be called after processing every minibatch, is responsible for enforcing the orthogonality constraint for any components of type LinearComponent or inheriting from AffineComponent that have the "orthonormal_constraint" value set.

This function, to be called after processing every minibatch, is responsible for enforcing the orthogonality constraint for any components of type LinearComponent or inheriting from AffineComponent that have the "orthonormal-constraint" value set to a nonzero value.

Technically what we are doing is constraining the parameter matrix M to be a "semi-orthogonal" matrix times a constant alpha. That is: if num-rows > num-cols, this amounts to asserting that M M^T == alpha^2 I; otherwise, that M^T M == alpha^2 I.

If, for a particular component, orthonormal-constraint > 0.0, then that value becomes the "alpha" mentioned above. If orthonormal-constraint == 0.0, then nothing is done. If orthonormal-constraint < 0.0, then it's like letting alpha "float", i.e. we try to make M closer to (any constant alpha) times a semi-orthogonal matrix.

In order to make it efficient on GPU, it doesn't make it completely orthonormal, it just makes it closer to being orthonormal (times the 'orthonormal_constraint' value). Over multiple iterations this rapidly makes it almost exactly orthonormal.

Definition at line 977 of file nnet-utils.cc.

References ConstrainOrthonormalInternal(), CuMatrixBase< Real >::CopyFromMat(), Nnet::GetComponent(), kaldi::kTrans, AffineComponent::LinearParams(), CuMatrixBase< Real >::NumCols(), Nnet::NumComponents(), CuMatrixBase< Real >::NumRows(), AffineComponent::OrthonormalConstraint(), LinearComponent::OrthonormalConstraint(), LinearComponent::Params(), and kaldi::RandInt().

Referenced by NnetChainTrainer::TrainInternal(), NnetTrainer::TrainInternal(), NnetChainTrainer::TrainInternalBackstitch(), and NnetTrainer::TrainInternalBackstitch().

977  {
978 
979  for (int32 c = 0; c < nnet->NumComponents(); c++) {
980  Component *component = nnet->GetComponent(c);
981  LinearComponent *lc = dynamic_cast<LinearComponent*>(component);
982  if (lc != NULL && lc->OrthonormalConstraint() != 0.0) {
983  if (RandInt(0, 3) != 0)
984  continue; // For efficiency, only do this every 4 minibatches-- it won't
985  // stray far.
986  BaseFloat scale = lc->OrthonormalConstraint();
987 
988  CuMatrixBase<BaseFloat> &params = lc->Params();
989  int32 rows = params.NumRows(), cols = params.NumCols();
990  if (rows <= cols) {
991  ConstrainOrthonormalInternal(scale, &params);
992  } else {
993  CuMatrix<BaseFloat> params_trans(params, kTrans);
994  ConstrainOrthonormalInternal(scale, &params_trans);
995  params.CopyFromMat(params_trans, kTrans);
996  }
997  }
998 
999  AffineComponent *ac = dynamic_cast<AffineComponent*>(component);
1000  if (ac != NULL && ac->OrthonormalConstraint() != 0.0) {
1001  if (RandInt(0, 3) != 0)
1002  continue; // For efficiency, only do this every 4 minibatches-- it won't
1003  // stray far.
1004  BaseFloat scale = ac->OrthonormalConstraint();
1005  CuMatrixBase<BaseFloat> &params = ac->LinearParams();
1006  int32 rows = params.NumRows(), cols = params.NumCols();
1007  if (rows <= cols) {
1008  ConstrainOrthonormalInternal(scale, &params);
1009  } else {
1010  CuMatrix<BaseFloat> params_trans(params, kTrans);
1011  ConstrainOrthonormalInternal(scale, &params_trans);
1012  params.CopyFromMat(params_trans, kTrans);
1013  }
1014  }
1015  }
1016 }
void ConstrainOrthonormalInternal(BaseFloat scale, CuMatrixBase< BaseFloat > *M)
Definition: nnet-utils.cc:874
float BaseFloat
Definition: kaldi-types.h:29
int32 RandInt(int32 min_val, int32 max_val, struct RandomState *state)
Definition: kaldi-math.cc:94
void kaldi::nnet3::ConstrainOrthonormalInternal ( BaseFloat  scale,
CuMatrixBase< BaseFloat > *  M 
)

Definition at line 874 of file nnet-utils.cc.

References CuMatrixBase< Real >::AddMat(), CuMatrixBase< Real >::AddMatMat(), CuMatrixBase< Real >::AddToDiag(), CuMatrixBase< Real >::CopyLowerToUpper(), CuMatrixBase< Real >::FrobeniusNorm(), kaldi::GetVerboseLevel(), KALDI_ASSERT, KALDI_VLOG, kaldi::kNoTrans, kaldi::kTrans, CuMatrixBase< Real >::NumCols(), CuMatrixBase< Real >::NumRows(), CuMatrixBase< Real >::SymAddMat2(), CuMatrixBase< Real >::Trace(), and kaldi::TraceMatMat().

Referenced by ConstrainOrthonormal().

874  {
875  KALDI_ASSERT(scale != 0.0);
876 
877  // We'd like to enforce the rows of M to be orthonormal.
878  // define P = M M^T. If P is unit then M has orthonormal rows.
879  // We actually want P to equal scale^2 * I, so that M's rows are
880  // orthogonal with 2-norms equal to 'scale'.
881  // We (notionally) add to the objective function, the value
882  // -alpha times the sum of squared elements of Q = (P - scale^2 * I).
883  int32 rows = M->NumRows(), cols = M->NumCols();
884  CuMatrix<BaseFloat> M_update(rows, cols);
885  CuMatrix<BaseFloat> P(rows, rows);
886  P.SymAddMat2(1.0, *M, kNoTrans, 0.0);
887  P.CopyLowerToUpper();
888 
889  // The 'update_speed' is a constant that determines how fast we approach a
890  // matrix with the desired properties (larger -> faster). Larger values will
891  // update faster but will be more prone to instability. 0.125 (1/8) is the
892  // value that gives us the fastest possible convergence when we are already
893  // close to be a semi-orthogonal matrix (in fact, it will lead to quadratic
894  // convergence).
895  // See http://www.danielpovey.com/files/2018_interspeech_tdnnf.pdf
896  // for more details.
897  BaseFloat update_speed = 0.125;
898 
899  if (scale < 0.0) {
900  // If scale < 0.0 then it's like letting the scale "float",
901  // as in Sec. 2.3 of
902  // http://www.danielpovey.com/files/2018_interspeech_tdnnf.pdf,
903  // where 'scale' here is written 'alpha' in the paper.
904  //
905  // We pick the scale that will give us an update to M that is
906  // orthogonal to M (viewed as a vector): i.e., if we're doing
907  // an update M := M + X, then we want to have tr(M X^T) == 0.
908  // The following formula is what gives us that.
909  // With P = M M^T, our update formula is doing to be:
910  // M := M + (-4 * alpha * (P - scale^2 I) * M).
911  // (The math below explains this update formula; for now, it's
912  // best to view it as an established fact).
913  // So X (the change in M) is -4 * alpha * (P - scale^2 I) * M,
914  // where alpha == update_speed / scale^2.
915  // We want tr(M X^T) == 0. First, forget the -4*alpha, because
916  // we don't care about constant factors. So we want:
917  // tr(M * M^T * (P - scale^2 I)) == 0.
918  // Since M M^T == P, that means:
919  // tr(P^2 - scale^2 P) == 0,
920  // or scale^2 = tr(P^2) / tr(P).
921  // Note: P is symmetric so it doesn't matter whether we use tr(P P) or
922  // tr(P^T P); we use tr(P^T P) because I believe it's faster to compute.
923 
924  BaseFloat trace_P = P.Trace(), trace_P_P = TraceMatMat(P, P, kTrans);
925 
926  scale = std::sqrt(trace_P_P / trace_P);
927 
928  // The following is a tweak to avoid divergence when the eigenvalues aren't
929  // close to being the same. trace_P is the sum of eigenvalues of P, and
930  // trace_P_P is the sum-square of eigenvalues of P. Treat trace_P as a sum
931  // of positive values, and trace_P_P as their sumsq. Then mean = trace_P /
932  // dim, and trace_P_P cannot be less than dim * (trace_P / dim)^2,
933  // i.e. trace_P_P >= trace_P^2 / dim. If ratio = trace_P_P * dim /
934  // trace_P^2, then ratio >= 1.0, and the excess above 1.0 is a measure of
935  // how far we are from convergence. If we're far from convergence, we make
936  // the learning rate slower to reduce the risk of divergence, since the
937  // update may not be stable for starting points far from equilibrium.
938  BaseFloat ratio = (trace_P_P * P.NumRows() / (trace_P * trace_P));
939  KALDI_ASSERT(ratio > 0.999);
940  if (ratio > 1.02) {
941  update_speed *= 0.5; // Slow down the update speed to reduce the risk of divergence.
942  }
943  }
944 
945  // see Sec. 2.2 of http://www.danielpovey.com/files/2018_interspeech_tdnnf.pdf
946  // for explanation of the 1/(scale*scale) factor, but there is a difference in
947  // notation; 'scale' here corresponds to 'alpha' in the paper, and
948  // 'update_speed' corresponds to 'nu' in the paper.
949  BaseFloat alpha = update_speed / (scale * scale);
950 
951  P.AddToDiag(-1.0 * scale * scale);
952 
953  if (GetVerboseLevel() >= 1) {
954  BaseFloat error = P.FrobeniusNorm();
955  KALDI_VLOG(2) << "Error in orthogonality is " << error;
956  }
957 
958  // At this point, the matrix P contains what, in the math, would be Q =
959  // P-scale^2*I. The derivative of the objective function w.r.t. an element q(i,j)
960  // of Q is now equal to -2*alpha*q(i,j), i.e. we could write q_deriv(i,j)
961  // = -2*alpha*q(i,j) This is also the derivative of the objective function
962  // w.r.t. p(i,j): i.e. p_deriv(i,j) = -2*alpha*q(i,j).
963  // Suppose we have define this matrix as 'P_deriv'.
964  // The derivative of the objective w.r.t M equals
965  // 2 * P_deriv * M, which equals -4*alpha*(P-scale^2*I)*M.
966  // (Currently the matrix P contains what, in the math, is P-scale^2*I).
967  M_update.AddMatMat(-4.0 * alpha, P, kNoTrans, *M, kNoTrans, 0.0);
968  M->AddMat(1.0, M_update);
969 }
int32 GetVerboseLevel()
Definition: kaldi-error.h:69
Real TraceMatMat(const MatrixBase< Real > &A, const MatrixBase< Real > &B, MatrixTransposeType trans)
MatrixIndexT NumCols() const
Definition: cu-matrix.h:215
float BaseFloat
Definition: kaldi-types.h:29
MatrixIndexT NumRows() const
Dimensions.
Definition: cu-matrix.h:214
void AddMat(Real alpha, const CuMatrixBase< Real > &A, MatrixTransposeType trans=kNoTrans)
*this += alpha * A
Definition: cu-matrix.cc:941
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
#define KALDI_VLOG(v)
Definition: kaldi-error.h:136
bool kaldi::nnet3::ContainsSingleExample ( const NnetExample eg,
int32 *  min_input_t,
int32 *  max_input_t,
int32 *  min_output_t,
int32 *  max_output_t 
)

Returns true if the "eg" contains just a single example, meaning that all the "n" values in the indexes are zero, and the example has NnetIo members named both "input" and "output".

Also computes the minimum and maximum "t" values in the "input" and "output" NnetIo members.

Definition at line 82 of file nnet3-copy-egs.cc.

References rnnlm::i, NnetIo::indexes, NnetExample::io, KALDI_ASSERT, KALDI_WARN, and NnetIo::name.

Referenced by SelectFromExample().

86  {
87  bool done_input = false, done_output = false;
88  int32 num_indexes = eg.io.size();
89  for (int32 i = 0; i < num_indexes; i++) {
90  const NnetIo &io = eg.io[i];
91  std::vector<Index>::const_iterator iter = io.indexes.begin(),
92  end = io.indexes.end();
93  // Should not have an empty input/output type.
94  KALDI_ASSERT(!io.indexes.empty());
95  if (io.name == "input" || io.name == "output") {
96  int32 min_t = iter->t, max_t = iter->t;
97  for (; iter != end; ++iter) {
98  int32 this_t = iter->t;
99  min_t = std::min(min_t, this_t);
100  max_t = std::max(max_t, this_t);
101  if (iter->n != 0) {
102  KALDI_WARN << "Example does not contain just a single example; "
103  << "too late to do frame selection or reduce context.";
104  return false;
105  }
106  }
107  if (io.name == "input") {
108  done_input = true;
109  *min_input_t = min_t;
110  *max_input_t = max_t;
111  } else {
112  KALDI_ASSERT(io.name == "output");
113  done_output = true;
114  *min_output_t = min_t;
115  *max_output_t = max_t;
116  }
117  } else {
118  for (; iter != end; ++iter) {
119  if (iter->n != 0) {
120  KALDI_WARN << "Example does not contain just a single example; "
121  << "too late to do frame selection or reduce context.";
122  return false;
123  }
124  }
125  }
126  }
127  if (!done_input) {
128  KALDI_WARN << "Example does not have any input named 'input'";
129  return false;
130  }
131  if (!done_output) {
132  KALDI_WARN << "Example does not have any output named 'output'";
133  return false;
134  }
135  return true;
136 }
std::vector< Index > indexes
"indexes" is a vector the same length as features.NumRows(), explaining the meaning of each row of th...
Definition: nnet-example.h:42
#define KALDI_WARN
Definition: kaldi-error.h:130
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
std::string name
the name of the input in the neural net; in simple setups it will just be "input".
Definition: nnet-example.h:36
std::vector< NnetIo > io
"io" contains the input and output.
Definition: nnet-example.h:116
void ConvertAdditionToAssignment ( const Nnet &  nnet,
NnetComputation *  computation 
)

This converts addition operations (things with Add in their names) to copy operations (things with Copy in their names).

This is slightly more efficient, and it may later allow us to remove unnecessary zeroing.

Definition at line 429 of file nnet-optimize.cc.

References NnetComputation::Command::alpha, Analyzer::command_attributes, NnetComputation::Command::command_type, NnetComputation::commands, ComputationAnalysis::FirstNontrivialAccess(), Analyzer::Init(), kAddRows, kAddRowsMulti, kAddToRowsMulti, KALDI_ASSERT, KALDI_ERR, kCopyRows, kCopyRowsMulti, kCopyToRowsMulti, kMatrixAdd, and kMatrixCopy.

Referenced by Optimize().

430  {
431  Analyzer analyzer;
432  analyzer.Init(nnet, *computation);
433  ComputationAnalysis analysis(*computation, analyzer);
434  int32 num_commands = computation->commands.size();
435  for (int32 command = 0; command < num_commands; command++) {
436  NnetComputation::Command &c = computation->commands[command];
437  switch (c.command_type) {
438  case kMatrixAdd: case kAddRows: case kAddRowsMulti:
439  case kAddToRowsMulti: {
440  const std::vector<int32> &submatrices_written =
441  analyzer.command_attributes[command].submatrices_written;
442  KALDI_ASSERT(!submatrices_written.empty());
443  std::vector<int32>::const_iterator iter = submatrices_written.begin(),
444  end = submatrices_written.end();
445  bool can_convert = true;
446  for (; iter != end; ++iter) {
447  int32 submatrix_written = *iter;
448  int32 first_access_command = analysis.FirstNontrivialAccess(
449  submatrix_written);
450  // first_access_command is first command other than zeroing and
451  // allocation that accesses this submatrix. It can be assumed to be a
452  // write command, since it makes no sense to read a variable before
453  // it's written to. If it's before this command then we need to add
454  // rather than copy; we can't do the conversion to a copy command.
455  if (first_access_command != command) {
456  can_convert = false;
457  break;
458  }
459  }
460  if (can_convert) { // convert to a copy command.
461  switch (c.command_type) {
462  case kMatrixAdd: c.command_type = kMatrixCopy;
463  break;
464  case kAddRows: c.command_type = kCopyRows;
465  break;
466  case kAddRowsMulti: c.command_type = kCopyRowsMulti;
467  break;
468  // note: kCopyToRowsMulti does not currently support alpha != 1.0.
469  case kAddToRowsMulti: if (c.alpha == 1.0) c.command_type = kCopyToRowsMulti;
470  break;
471  default: KALDI_ERR << "Unexpected command type.";
472  }
473  }
474  break;
475  }
476  default:
477  break;
478  }
479  }
480 }
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
static void kaldi::nnet3::ConvertNumNValues ( int32  n_stride,
int32  old_N,
int32  new_N,
const std::vector< Index > &  indexes_in,
std::vector< Index > *  indexes_out 
)
static

Definition at line 3109 of file nnet-optimize-utils.cc.

References KALDI_ASSERT, Index::n, and rnnlm::n.

Referenced by ComputationExpander::ExpandIndexes(), and IoSpecificationIsDecomposable().

3111  {
3112  int32 size_in = indexes_in.size();
3113  KALDI_ASSERT(size_in > 0 && indexes_in[size_in - 1].n == old_N - 1);
3114  int32 block_size_in = n_stride * old_N,
3115  block_size_out = n_stride * new_N;
3116 
3117  indexes_out->resize((size_in / old_N) * new_N);
3118  for (int32 i_in = 0; i_in < size_in; i_in++) {
3119  if (indexes_in[i_in].n != 0)
3120  continue;
3121  Index index(indexes_in[i_in]);
3122  int32 block_index = i_in / block_size_in,
3123  offset_within_block = i_in % block_size_in;
3124 
3125 
3126  int32 i_out = block_index * block_size_out +
3127  offset_within_block;
3128  for (int32 n = 0; n < new_N; n++, i_out += n_stride) {
3129  index.n = n;
3130  (*indexes_out)[i_out] = index;
3131  }
3132  }
3133 }
struct rnnlm::@11::@12 n
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void kaldi::nnet3::ConvertRepeatedToBlockAffine ( CompositeComponent *  c_component)

Definition at line 416 of file nnet-utils.cc.

References CompositeComponent::GetComponent(), rnnlm::i, KALDI_ASSERT, CompositeComponent::NumComponents(), CompositeComponent::SetComponent(), and Component::Type().

Referenced by ConvertRepeatedToBlockAffine(), main(), UnitTestConvertRepeatedToBlockAffine(), and UnitTestConvertRepeatedToBlockAffineComposite().

416  {
417  for(int32 i = 0; i < c_component->NumComponents(); i++) {
418  const Component *c = c_component->GetComponent(i);
419  KALDI_ASSERT(c->Type() != "CompositeComponent" &&
420  "Nesting CompositeComponent within CompositeComponent is not allowed.\n"
421  "(We may change this as more complicated components are introduced.)");
422 
423  if(c->Type() == "RepeatedAffineComponent" ||
424  c->Type() == "NaturalGradientRepeatedAffineComponent") {
425  // N.B.: NaturalGradientRepeatedAffineComponent is a subclass of
426  // RepeatedAffineComponent.
427  const RepeatedAffineComponent *rac =
428  dynamic_cast<const RepeatedAffineComponent*>(c);
429  KALDI_ASSERT(rac != NULL);
430  BlockAffineComponent *bac = new BlockAffineComponent(*rac);
431  // following call deletes rac
432  c_component->SetComponent(i, bac);
433  }
434  }
435 }
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void ConvertRepeatedToBlockAffine ( Nnet *  nnet)

Convert all components of type RepeatedAffineComponent or NaturalGradientRepeatedAffineComponent to BlockAffineComponent in nnet.

Definition at line 437 of file nnet-utils.cc.

References ConvertRepeatedToBlockAffine(), Nnet::GetComponent(), rnnlm::i, KALDI_ASSERT, Nnet::NumComponents(), Nnet::SetComponent(), and Component::Type().

437  {
438  for(int32 i = 0; i < nnet->NumComponents(); i++) {
439  const Component *const_c = nnet->GetComponent(i);
440  if(const_c->Type() == "RepeatedAffineComponent" ||
441  const_c->Type() == "NaturalGradientRepeatedAffineComponent") {
442  // N.B.: NaturalGradientRepeatedAffineComponent is a subclass of
443  // RepeatedAffineComponent.
444  const RepeatedAffineComponent *rac =
445  dynamic_cast<const RepeatedAffineComponent*>(const_c);
446  KALDI_ASSERT(rac != NULL);
447  BlockAffineComponent *bac = new BlockAffineComponent(*rac);
448  // following call deletes rac
449  nnet->SetComponent(i, bac);
450  } else if (const_c->Type() == "CompositeComponent") {
451  // We must modify the composite component, so we use the
452  // non-const GetComponent() call here.
453  Component *c = nnet->GetComponent(i);
454  CompositeComponent *cc = dynamic_cast<CompositeComponent*>(c);
455  KALDI_ASSERT(cc != NULL);
457  }
458  }
459 }
void ConvertRepeatedToBlockAffine(Nnet *nnet)
Convert all components of type RepeatedAffineComponent or NaturalGradientRepeatedAffineComponent to B...
Definition: nnet-utils.cc:437
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool ConvertToIndexes ( const std::vector< std::pair< int32, int32 > > &  location_vector,
int32 *  first_value,
std::vector< int32 > *  second_values 
)

If it is the case for some i >= 0 that all the .first elements of "location_vector" are either i or -1, then output i to first_value and the .second elements into "second_values", and return true.

Otherwise return false and the outputs are don't-cares.

Definition at line 377 of file nnet-compile-utils.cc.

Referenced by Compiler::CompileBackwardFromSubmatLocations(), Compiler::CompileForwardFromSubmatLocations(), SplitLocationsBackward(), UnitTestSplitLocations(), and UnitTestSplitLocationsBackward().

380  {
381  *first_value = -1;
382  second_values->clear();
383  second_values->reserve(location_vector.size());
384  std::vector<std::pair<int32, int32> >::const_iterator iter;
385  for (iter = location_vector.begin(); iter < location_vector.end(); ++iter) {
386  if (iter->first != -1) {
387  if (*first_value == -1)
388  *first_value = iter->first;
389  if (iter->first != *first_value)
390  return false;
391  second_values->push_back(iter->second);
392  } else {
393  second_values->push_back(-1);
394  }
395  }
396  return true;
397 }
static void kaldi::nnet3::CopyPairVector ( const CuArray< Int32Pair > &  in,
std::vector< std::pair< int32, int32 > > *  out 
)
static

Definition at line 31 of file nnet-general-component.cc.

References CuArrayBase< T >::CopyToVec().

Referenced by StatisticsExtractionComponentPrecomputedIndexes::Read(), StatisticsPoolingComponentPrecomputedIndexes::Read(), StatisticsExtractionComponentPrecomputedIndexes::Write(), and StatisticsPoolingComponentPrecomputedIndexes::Write().

32  {
33  in.CopyToVec(reinterpret_cast<std::vector<Int32Pair>*>(out));
34 }
void CopyToVec(std::vector< T > *dst) const
This function resizes *dst if needed.
Definition: cu-array-inl.h:173
static void kaldi::nnet3::CopyPairVector ( const std::vector< std::pair< int32, int32 > > &  in,
CuArray< Int32Pair > *  out 
)
static

Definition at line 36 of file nnet-general-component.cc.

References CuArray< T >::CopyFromVec().

37  {
38  const std::vector<Int32Pair> *in_cast =
39  reinterpret_cast<const std::vector<Int32Pair>*>(&in);
40  out->CopyFromVec(*in_cast);
41 }
void CopyFromVec(const std::vector< T > &src)
This function resizes if needed.
Definition: cu-array-inl.h:119
static void kaldi::nnet3::CreateComputationRequestInternal ( int32  begin_input_t,
int32  end_input_t,
int32  begin_output_t,
int32  end_output_t,
int32  num_sequences,
int32  frame_subsampling_factor,
const std::set< int32 > &  ivector_times,
ComputationRequest *  request 
)
static

Definition at line 102 of file nnet-compile-looped.cc.

References ComputationRequest::inputs, rnnlm::n, and ComputationRequest::outputs.

Referenced by CreateLoopedComputationRequest().

108  {
109  request->inputs.reserve(2);
110  request->inputs.clear();
111  request->inputs.resize(1 + (ivector_times.empty() ? 0 : 1));
112  request->inputs[0].name = "input";
113  request->inputs[0].has_deriv = false;
114  request->outputs.clear();
115  request->outputs.resize(1);
116  request->outputs[0].name = "output";
117  request->outputs[0].has_deriv = false;
118  if (!ivector_times.empty()) {
119  request->inputs[1].name = "ivector";
120  request->inputs[1].has_deriv = false;
121  }
122 
123  // in the computation request the 'n' indexes (the sequence/utterance indexes)
124  // have the larger stride than 't', although this is opposite to the way it's
125  // done inside the computation. This is for user convenience where it may be
126  // easier to deal with submatrixes per sequence.
127  for (int32 n = 0; n < num_sequences; n++) {
128  int32 x = 0;
129  for (int32 t = begin_input_t; t < end_input_t; t++) {
130  request->inputs[0].indexes.push_back(Index(n, t, x));
131  }
132  for (int32 t = begin_output_t;
133  t < end_output_t;
134  t += frame_subsampling_factor)
135  request->outputs[0].indexes.push_back(Index(n, t, x));
136  }
137  if (!ivector_times.empty()) {
138  request->inputs.resize(2);
139  request->inputs[1].name = "ivector";
140  request->inputs[1].has_deriv = false;
141  for (int32 n = 0; n < num_sequences; n++) {
142  // note: std::sets store things in sorted order.
143  for (std::set<int32>::const_iterator iter = ivector_times.begin();
144  iter != ivector_times.end(); ++iter) {
145  int32 t = *iter, x = 0;
146  request->inputs[1].indexes.push_back(Index(n, t, x));
147  }
148  }
149  }
150 }
struct rnnlm::@11::@12 n
void CreateLoopedComputationRequest ( const Nnet &  nnet,
int32  chunk_size,
int32  frame_subsampling_factor,
int32  ivector_period,
int32  left_context_begin,
int32  right_context,
int32  num_sequences,
ComputationRequest *  request1,
ComputationRequest *  request2,
ComputationRequest *  request3 
)

This function creates computation request suitable for giving to ComputeLooped().

It's intended for use with a 'simple' nnet (one satisfying IsSimpleNnet()), and this basically means that the inputs must be named "input" and possibly "ivector", and that there is an output named "output", and that those are the ones you care about (it won't generate any other outputs or use any other inputs).

If you want to use looped computation for different types of neural net, you should use the deeper interface, CompileLooped().

Parameters
[in]nnetThe neural net this computation request is to be used with. This is used to check whether the neural net accepts iVectors, and to work out the left-context and right-context required by the network.
[in]chunk_sizeThe number of frames of output that will be generated for each chunk (note: this is the shift in the t-index, which will not equal the number of output frames if frame_subsampling_factor != 1). Note: it is required that chunk_size be a multiple of ivector_period, frame_subsampling_factor, and nnet.Modulus(). You should use GetChunkSize() to compute the chunk size, giving it an advisory/ minimum chunksize, to make sure it satisfies these properties.
[in]frame_subsampling_factorThis will normally be 1, but may be more than 1 (e.g. 3) in chain systems; it determines the frame-skipping on the output, so we evaluate the output with 't' at multiples of this value.
[in]ivector_periodThe period with which iVectors are to be supplied to the network (if you're using iVectors). Not necessarily the same as the period with which the ivectors are extracted or stored on disk (–online-ivector-period). You will normally set this to the chunk size. It must divide the chunk size (if you're using iVectors) Note: you should call ModifyNnetIvectorPeriod on 'nnet' before calling this function; otherwise the neural net will most likely not actually be able to consume the iVector with this frequency.
[in]left_context_beginThis should be the left-context of the network plus any additional left-context (provided via the option –extra-left-context-begin) that should be supplied to the network on top of the minimum that the network requires. We call this left_context_begin because this only relates to the start of the utterance (t=0).
[in]right_contextThis should be the right-context of the network, plus any additional right-context ("extra-right-context") that should be supplied to the network on top of the minimum that the network requires (currently extra-right-context != 0 is is not supported at the command-line level).
[in]num_sequencesThe number of separate 'n' values to put in the computation; normally this will be just 1, but it can be increased to allow simultaneous operation on multiple streams of input.
[out]request1The first of the 3 requests that this function generates, that the user should then supply to CompileLooped(). Note: this will tend to be the largest computation request in terms of input, because we have to provide enough left and right context that it can evaluate the first chunk. Note: as elsewhere, the job of duplicating first and last frames enough to provide the required left/right context to the network, is left to the caller (at runtime, not during compilation).
[out]request2The second of the 3 requests that this function generates. Caution: none of the inputs and outputs should overlap.
[out]request3The third of the 3 requests that this function generates. It will be the same as request2, except for a time offset.

Definition at line 153 of file nnet-compile-looped.cc.

References CreateComputationRequestInternal(), Nnet::InputDim(), KALDI_ASSERT, Mod(), and Nnet::Modulus().

Referenced by CreateLoopedComputationRequestSimple(), and DecodableNnetSimpleLoopedInfo::Init().

162  {
163  bool has_ivector = (nnet.InputDim("ivector") > 0);
164  KALDI_ASSERT(chunk_size % frame_subsampling_factor == 0 &&
165  chunk_size % nnet.Modulus() == 0 &&
166  chunk_size % ivector_period == 0);
167  KALDI_ASSERT(left_context_begin >= 0 && right_context >= 0);
168  // note, 'end' is one past the last one.
169  int32 chunk1_input_begin_t = - left_context_begin,
170  chunk1_input_end_t = chunk_size + right_context,
171  chunk2_input_begin_t = chunk1_input_end_t,
172  chunk2_input_end_t = chunk2_input_begin_t + chunk_size,
173  chunk3_input_begin_t = chunk2_input_end_t,
174  chunk3_input_end_t = chunk3_input_begin_t + chunk_size;
175 
176 
177  // work out the times at which i-vectors are required.
178  std::set<int32> ivector_times1, ivector_times2, ivector_times3;
179  if (has_ivector) {
180  for (int32 t = chunk1_input_begin_t; t < chunk1_input_end_t; t++) {
181  int32 ivector_t = t - Mod(t, ivector_period);
182  ivector_times1.insert(ivector_t);
183  }
184  for (int32 t = chunk2_input_begin_t; t < chunk2_input_end_t; t++) {
185  int32 ivector_t = t - Mod(t, ivector_period);
186  if (ivector_times2.count(ivector_t) == 0 &&
187  ivector_times1.count(ivector_t) == 0)
188  ivector_times2.insert(ivector_t);
189  }
190  for (int32 t = chunk3_input_begin_t; t < chunk3_input_end_t; t++) {
191  int32 ivector_t = t - Mod(t, ivector_period);
192  if (ivector_times3.count(ivector_t) == 0 &&
193  ivector_times2.count(ivector_t) == 0 &&
194  ivector_times1.count(ivector_t) == 0)
195  ivector_times3.insert(ivector_t);
196  }
197  }
198 
200  chunk1_input_begin_t, chunk1_input_end_t,
201  0, chunk_size,
202  num_sequences, frame_subsampling_factor,
203  ivector_times1,
204  request1);
205 
207  chunk2_input_begin_t, chunk2_input_end_t,
208  chunk_size, chunk_size * 2,
209  num_sequences, frame_subsampling_factor,
210  ivector_times2,
211  request2);
212 
214  chunk3_input_begin_t, chunk3_input_end_t,
215  chunk_size * 2, chunk_size * 3,
216  num_sequences, frame_subsampling_factor,
217  ivector_times3,
218  request3);
219 
220 }
static void CreateComputationRequestInternal(int32 begin_input_t, int32 end_input_t, int32 begin_output_t, int32 end_output_t, int32 num_sequences, int32 frame_subsampling_factor, const std::set< int32 > &ivector_times, ComputationRequest *request)
I Mod(I m, I n)
Mod(m, n), defined for integers m and n where n > 0, returns the modulus m % n, defined as the intege...
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void CreateLoopedComputationRequestSimple ( const Nnet &  nnet,
int32  chunk_size,
int32  frame_subsampling_factor,
int32  ivector_period,
int32  extra_left_context_begin,
int32  extra_right_context,
int32  num_sequences,
ComputationRequest *  request1,
ComputationRequest *  request2,
ComputationRequest *  request3 
)

This function is deprecated.

It has the same interface as CreateLoopedComputationRequest(), except that the left and right context are specified in a different way (as just the 'extra' part). It is deprecated because this function has to work out the left and right context of the network, which turns out to be quite slow if it's done after you call ModifyNnetIvectorPeriod().

Definition at line 350 of file nnet-compile-looped.cc.

References ComputeSimpleNnetContext(), and CreateLoopedComputationRequest().

Referenced by UnitTestNnetCompileLooped().

359  {
360  int32 left_context, right_context;
361  ComputeSimpleNnetContext(nnet, &left_context, &right_context);
362 
363  CreateLoopedComputationRequest(nnet, chunk_size, frame_subsampling_factor,
364  ivector_period,
365  extra_left_context_begin + left_context,
366  extra_right_context + right_context,
367  num_sequences, request1, request2, request3);
368 }
void CreateLoopedComputationRequest(const Nnet &nnet, int32 chunk_size, int32 frame_subsampling_factor, int32 ivector_period, int32 left_context_begin, int32 right_context, int32 num_sequences, ComputationRequest *request1, ComputationRequest *request2, ComputationRequest *request3)
This function creates computation request suitable for giving to ComputeLooped(). ...
void ComputeSimpleNnetContext(const Nnet &nnet, int32 *left_context, int32 *right_context)
ComputeSimpleNnetContext computes the left-context and right-context of a nnet.
Definition: nnet-utils.cc:143
bool DescriptorTokenize ( const std::string &  input,
std::vector< std::string > *  tokens 
)

This function tokenizes input when parsing Descriptor configuration values.

A token in this context is not the same as a generic Kaldi token, e.g. as defined in IsToken() in util/text_utils.h, which just means a non-empty whitespace-free string. Here a token is more like a programming-language token, and currently the following are allowed as tokens: "(" ")" ","

  • A nonempty string beginning with A-Za-z_, and containing only -_A-Za-z0-9.
  • An integer, optionally beginning with - or + and then a nonempty sequence of 0-9.

This function should return false and print an informative error with local context if it can't tokenize the input.

Definition at line 377 of file nnet-parse.cc.

References kaldi::ConvertStringToReal(), ErrorContext(), IsValidName(), KALDI_ASSERT, and KALDI_WARN.

Referenced by NormalizeTextDescriptor(), Nnet::ProcessComponentNodeConfigLine(), Nnet::ProcessOutputNodeConfigLine(), Nnet::RemoveSomeNodes(), ModelCollapser::ReplaceNodeInDescriptor(), UnitTestDescriptorIo(), UnitTestDescriptorTokenize(), and UnitTestGeneralDescriptor().

378  {
379  KALDI_ASSERT(tokens != NULL);
380  size_t start = input.find_first_not_of(" \t"), size = input.size();
381  tokens->clear();
382  while (start < size) {
383  KALDI_ASSERT(!isspace(input[start]));
384  if (input[start] == '(' || input[start] == ')' || input[start] == ',') {
385  tokens->push_back(std::string(input, start, 1));
386  start = input.find_first_not_of(" \t", start + 1);
387  } else {
388  size_t found = input.find_first_of(" \t(),", start);
389  KALDI_ASSERT(found != start);
390  if (found == std::string::npos) {
391  std::string str(input, start, input.size() - start);
392  BaseFloat tmp;
393  if (!IsValidName(str) && !ConvertStringToReal(str, &tmp)) {
394  KALDI_WARN << "Could not tokenize line " << ErrorContext(std::string(input, start));
395  return false;
396  }
397  tokens->push_back(str);
398  break;
399  } else {
400  if (input[found] == '(' || input[found] == ')' || input[found] == ',') {
401  std::string str(input, start, found - start);
402  BaseFloat tmp;
403  if (!IsValidName(str) && !ConvertStringToReal(str, &tmp)) {
404  KALDI_WARN << "Could not tokenize line " << ErrorContext(std::string(input, start));
405  return false;
406  }
407  tokens->push_back(str);
408  start = found;
409  } else {
410  std::string str(input, start, found - start);
411  BaseFloat tmp;
412  if (!IsValidName(str) && !ConvertStringToReal(str, &tmp)) {
413  KALDI_WARN << "Could not tokenize line " << ErrorContext(std::string(input, start));
414  return false;
415  }
416  tokens->push_back(str);
417  start = input.find_first_not_of(" \t", found);
418  }
419  }
420  }
421  }
422  return true;
423 }
float BaseFloat
Definition: kaldi-types.h:29
std::string ErrorContext(const std::string &str)
Definition: nnet-parse.cc:461
bool ConvertStringToReal(const std::string &str, T *out)
ConvertStringToReal converts a string into either float or double and returns false if there was any ...
Definition: text-utils.cc:238
#define KALDI_WARN
Definition: kaldi-error.h:130
bool IsValidName(const std::string &name)
Returns true if 'name' would be a valid name for a component or node in a Nnet.
Definition: nnet-parse.cc:425
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
BaseFloat DotProduct ( const Nnet &  nnet1,
const Nnet &  nnet2 
)

Returns dot product between two networks of the same structure (calls the DotProduct functions of the Updatable components and sums up the return values).

Definition at line 228 of file nnet-utils.cc.

References UpdatableComponent::DotProduct(), Nnet::GetComponent(), KALDI_ASSERT, kUpdatableComponent, Nnet::NumComponents(), and Component::Properties().

Referenced by NnetDiscriminativeTrainer::Train(), and UnitTestNnetModelDerivatives().

229  {
230  KALDI_ASSERT(nnet1.NumComponents() == nnet2.NumComponents());
231  BaseFloat ans = 0.0;
232  for (int32 c = 0; c < nnet1.NumComponents(); c++) {
233  const Component *comp1 = nnet1.GetComponent(c),
234  *comp2 = nnet2.GetComponent(c);
235  if (comp1->Properties() & kUpdatableComponent) {
236  const UpdatableComponent
237  *u_comp1 = dynamic_cast<const UpdatableComponent*>(comp1),
238  *u_comp2 = dynamic_cast<const UpdatableComponent*>(comp2);
239  KALDI_ASSERT(u_comp1 != NULL && u_comp2 != NULL);
240  ans += u_comp1->DotProduct(*u_comp2);
241  }
242  }
243  return ans;
244 }
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void EnsureContiguousProperty ( const std::vector< int32 > &  indexes,
std::vector< std::vector< int32 > > *  indexes_out 
)

This function takes a vector of indexes and splits it up into as separate vectors of the same size, as needed to ensure that the 'contiguous property' holds.

This is done via padding with -1's. An example will clarify this. Suppose the input is: [ -1 1 1 1 2 2 1 1 ] which lacks the contiguous property because 1's appear in 2 different places, it would split it up as [ -1 1 1 1 2 2 -1 -1 ] [ -1 -1 -1 -1 -1 -1 1 1 ] If 'indexes' is empty or only contains -1's, 'indexes_out' will be empty.

Definition at line 401 of file nnet-compile-utils.cc.

References rnnlm::i.

Referenced by SplitLocationsBackward(), and UnitTestEnsureContiguousProperty().

403  {
404  indexes_out->clear();
405  indexes_out->reserve(3);
406  if (indexes.empty()) return;
407  int32 max_value = *std::max_element(indexes.begin(), indexes.end());
408  if (max_value == -1) return;
409  std::vector<int32> num_segments_seen(max_value + 1, 0);
410  int32 dim = indexes.size(), num_output_vectors = 0;
411  for (int32 i = 0; i < dim;) {
412  // note, we increment i within the loop.
413  if (indexes[i] == -1) {
414  i++;
415  continue;
416  }
417  int32 value = indexes[i], start_index = i;
418  for (; i < dim && indexes[i] == value; i++);
419  int32 end_index = i; // one past the end.
420  // the input 'indexes' contains a sequence of possibly-repeated instances of
421  // the value 'value', starting at index 'start_index', with 'end_index' as
422  // one past the end.
423  int32 this_num_segments_seen = num_segments_seen[value]++;
424  if (this_num_segments_seen >= num_output_vectors) { // we have nowhere to
425  // put it.
426  indexes_out->resize(++num_output_vectors);
427  indexes_out->back().resize(dim, -1); // fill newly added vector with -1's.
428  }
429  std::vector<int32> &this_out_vec((*indexes_out)[this_num_segments_seen]);
430  std::vector<int32>::iterator iter = this_out_vec.begin() + start_index,
431  end = this_out_vec.begin() + end_index;
432  // Fill the appropriate range of the output vector with 'value'
433  for (; iter != end; ++iter) *iter = value;
434  }
435 }
std::string ErrorContext ( std::istream &  is)

Return a string used in error messages.

Here, "is" will be from an istringstream derived from a single line or part of a line. If "is" is at EOF or in error state, this should just say "end of line", else if the contents of "is" before EOF is <20 characters it should return it all, else it should return the first 20 characters followed by "...".

Definition at line 451 of file nnet-parse.cc.

Referenced by DescriptorTokenize().

451  {
452  if (!is.good()) return "end of line";
453  char buf[21];
454  is.read(buf, 21);
455  if (is) {
456  return (std::string(buf, 20) + "...");
457  }
458  return std::string(buf, is.gcount());
459 }
std::string ErrorContext ( const std::string &  str)

Definition at line 461 of file nnet-parse.cc.

461  {
462  if (str.size() == 0) return "end of line";
463  if (str.size() <= 20) return str;
464  return std::string(str, 0, 20) + "...";
465 }
void EvaluateComputationRequest ( const Nnet &  nnet,
const ComputationRequest &  request,
std::vector< std::vector< bool > > *  is_computable 
)

Given an nnet and a computation request, this function works out which requested outputs in the computation request are computable; it outputs this information as a vector "is_computable" indexed by the same indexes as request.outputs.

It does this by executing some of the early stages of compilation.

Definition at line 71 of file nnet-utils.cc.

References ComputationGraphBuilder::Compute(), ComputationGraphBuilder::GetComputableInfo(), Nnet::GetNodeNames(), kaldi::GetVerboseLevel(), KALDI_VLOG, and ComputationGraph::Print().

Referenced by ComputeSimpleNnetContextForShift().

74  {
75  ComputationGraph graph;
76  ComputationGraphBuilder builder(nnet, &graph);
77  builder.Compute(request);
78  builder.GetComputableInfo(is_computable);
79  if (GetVerboseLevel() >= 4) {
80  std::ostringstream graph_pretty;
81  graph.Print(graph_pretty, nnet.GetNodeNames());
82  KALDI_VLOG(4) << "Graph is " << graph_pretty.str();
83  }
84 }
int32 GetVerboseLevel()
Definition: kaldi-error.h:69
#define KALDI_VLOG(v)
Definition: kaldi-error.h:136
bool ExampleApproxEqual ( const NnetExample &  eg1,
const NnetExample &  eg2,
BaseFloat  delta 
)

Returns true if the examples are approximately equal (only intended to be used in testing).

Definition at line 1836 of file nnet-test-utils.cc.

References kaldi::ApproxEqual(), NnetIo::features, GeneralMatrix::GetMatrix(), rnnlm::i, NnetIo::indexes, NnetExample::io, and NnetIo::name.

Referenced by UnitTestNnetExample().

1838  {
1839  if (eg1.io.size() != eg2.io.size())
1840  return false;
1841  for (size_t i = 0; i < eg1.io.size(); i++) {
1842  NnetIo io1 = eg1.io[i], io2 = eg2.io[i];
1843  if (io1.name != io2.name || io1.indexes != io2.indexes)
1844  return false;
1845  Matrix<BaseFloat> feat1, feat2;
1846  io1.features.GetMatrix(&feat1);
1847  io2.features.GetMatrix(&feat2);
1848  if (!ApproxEqual(feat1, feat2, delta))
1849  return false;
1850  }
1851  return true;
1852 }
static bool ApproxEqual(float a, float b, float relative_tolerance=0.001)
return abs(a - b) <= relative_tolerance * (abs(a)+abs(b)).
Definition: kaldi-math.h:262
void ExpandComputation ( const Nnet &  nnet,
const MiscComputationInfo &  misc_info,
const NnetComputation &  computation,
bool  need_debug_info,
int32  num_n_values,
NnetComputation *  expanded_computation 
)

This function is used in 'shortcut' compilation to expand a computation that has been compiled for exactly 2 'n' values, to one that is suitable for some num_n_values > 2.

Parameters
[in]nnetThe neural network for which this computation is being built.
[in]misc_infoThe same MiscComputationInfo object that was present in the ComputationRequests that were originally used to generate the computation (required to generated the PrecomputedIndexes)
[in]computationThe computation that was compiled for exactly 2 'n' values (n=0 and n=1)
[in]need_debug_infoTrue if we want to retain the 'debug_info' in the output 'expanded_computation'. In any case, the 'debug_info' is required in the input computation.
[in]num_n_valuesThe number of 'n' values we want in the output computation
[out]expanded_computationThe expanded computation.

Definition at line 3807 of file nnet-optimize-utils.cc.

References ComputationExpander::Expand().

Referenced by CachingOptimizingCompiler::CompileViaShortcut().

3812  {
3813  ComputationExpander expander(nnet, misc_info, computation,
3814  need_debug_info, num_n_values,
3815  expanded_computation);
3816  expander.Expand();
3817 }
void ExpectOneOrTwoTokens ( std::istream &  is,
bool  binary,
const std::string &  token1,
const std::string &  token2 
)

This function is like ExpectToken but for two tokens, and it will either accept token1 and then token2, or just token2.

This is useful in Read functions where the first token may already have been consumed.

Definition at line 224 of file nnet-parse.cc.

References ExpectToken(), KALDI_ASSERT, KALDI_ERR, and kaldi::ReadToken().

Referenced by PnormComponent::Read(), DistributeComponent::Read(), DistributeComponentPrecomputedIndexes::Read(), RestrictedAttentionComponent::Read(), ElementwiseProductComponent::Read(), RestrictedAttentionComponent::PrecomputedIndexes::Read(), BatchNormComponent::Read(), StatisticsExtractionComponent::Read(), ConvolutionModel::Read(), TimeHeightConvolutionComponent::PrecomputedIndexes::Read(), StatisticsExtractionComponentPrecomputedIndexes::Read(), ConvolutionComputation::Read(), StatisticsPoolingComponent::Read(), StatisticsPoolingComponentPrecomputedIndexes::Read(), BackpropTruncationComponent::Read(), BackpropTruncationComponentPrecomputedIndexes::Read(), NonlinearComponent::Read(), DropoutMaskComponent::Read(), GeneralDropoutComponent::Read(), GeneralDropoutComponentPrecomputedIndexes::Read(), FixedAffineComponent::Read(), SumGroupComponent::Read(), FixedScaleComponent::Read(), FixedBiasComponent::Read(), SumBlockComponent::Read(), ClipGradientComponent::Read(), PermuteComponent::Read(), and MaxpoolingComponent::Read().

226  {
227  KALDI_ASSERT(token1 != token2);
228  std::string temp;
229  ReadToken(is, binary, &temp);
230  if (temp == token1) {
231  ExpectToken(is, binary, token2);
232  } else {
233  if (temp != token2) {
234  KALDI_ERR << "Expecting token " << token1 << " or " << token2
235  << " but got " << temp;
236  }
237  }
238 }
void ReadToken(std::istream &is, bool binary, std::string *str)
ReadToken gets the next token and puts it in str (exception on failure).
Definition: io-funcs.cc:154
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:194
#define KALDI_ERR
Definition: kaldi-error.h:127
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
static void kaldi::nnet3::ExpectToken ( const std::string &  token,
const std::string &  what_we_are_parsing,
const std::string **  next_token 
)
static

Definition at line 45 of file nnet-descriptor.cc.

References KALDI_ERR, and ParsingContext().

Referenced by ExpectOneOrTwoTokens(), GeneralDescriptor::Parse(), GeneralDescriptor::ParseConst(), GeneralDescriptor::ParseFailover(), GeneralDescriptor::ParseIfDefined(), GeneralDescriptor::ParseOffset(), GeneralDescriptor::ParseReplaceIndex(), GeneralDescriptor::ParseRound(), GeneralDescriptor::ParseScale(), AmNnetSimple::Read(), PnormComponent::Read(), Index::Read(), DistributeComponent::Read(), NnetIo::Read(), NnetDiscriminativeSupervision::Read(), NnetChainSupervision::Read(), IoSpecification::Read(), NormalizeComponent::Read(), NnetDiscriminativeExample::Read(), NnetChainExample::Read(), NnetExample::Read(), DropoutComponent::Read(), NnetOptimizeOptions::Read(), DistributeComponentPrecomputedIndexes::Read(), RestrictedAttentionComponent::Read(), ComputationRequest::Read(), ElementwiseProductComponent::Read(), RestrictedAttentionComponent::PrecomputedIndexes::Read(), BatchNormComponent::Read(), Nnet::Read(), StatisticsExtractionComponent::Read(), ConvolutionModel::Read(), TimeHeightConvolutionComponent::Read(), TimeHeightConvolutionComponent::PrecomputedIndexes::Read(), StatisticsExtractionComponentPrecomputedIndexes::Read(), NnetComputation::MatrixInfo::Read(), NnetComputation::MatrixDebugInfo::Read(), NnetComputation::SubMatrixInfo::Read(), ConvolutionComputation::Read(), NnetComputation::Command::Read(), StatisticsPoolingComponent::Read(), ConvolutionComputationIo::Read(), AffineComponent::Read(), StatisticsPoolingComponentPrecomputedIndexes::Read(), NnetComputation::Read(), BackpropTruncationComponent::Read(), BlockAffineComponent::Read(), RepeatedAffineComponent::Read(), BackpropTruncationComponentPrecomputedIndexes::Read(), NonlinearComponent::Read(), ComputationCache::Read(), ConstantComponent::Read(), DropoutMaskComponent::Read(), NaturalGradientAffineComponent::Read(), GeneralDropoutComponent::Read(), LinearComponent::Read(), GeneralDropoutComponentPrecomputedIndexes::Read(), FixedAffineComponent::Read(), FixedScaleComponent::Read(), FixedBiasComponent::Read(), SumBlockComponent::Read(), ClipGradientComponent::Read(), PermuteComponent::Read(), PerElementScaleComponent::Read(), PerElementOffsetComponent::Read(), ConstantFunctionComponent::Read(), NaturalGradientPerElementScaleComponent::Read(), ScaleAndOffsetComponent::Read(), ConvolutionComponent::Read(), LstmNonlinearityComponent::Read(), MaxpoolingComponent::Read(),