All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
ivector-extractor-acc-stats.cc File Reference
Include dependency graph for ivector-extractor-acc-stats.cc:

Go to the source code of this file.

Classes

class  IvectorTask
 

Namespaces

 kaldi
 Relabels neural network egs with the read pdf-id alignments.
 

Functions

int main (int argc, char *argv[])
 

Function Documentation

int main ( int  argc,
char *  argv[] 
)

Definition at line 62 of file ivector-extractor-acc-stats.cc.

References SequentialTableReader< Holder >::Done(), kaldi::g_num_threads, ParseOptions::GetArg(), RandomAccessTableReader< Holder >::HasKey(), KALDI_LOG, KALDI_WARN, SequentialTableReader< Holder >::Key(), SequentialTableReader< Holder >::Next(), TaskSequencerConfig::num_threads, ParseOptions::NumArgs(), MatrixBase< Real >::NumRows(), ParseOptions::PrintUsage(), ParseOptions::Read(), kaldi::ReadKaldiObject(), ParseOptions::Register(), TaskSequencerConfig::Register(), IvectorExtractorStatsOptions::Register(), TaskSequencer< C >::Run(), Output::Stream(), RandomAccessTableReader< Holder >::Value(), SequentialTableReader< Holder >::Value(), and IvectorExtractorStats::Write().

62  {
63  using namespace kaldi;
64  typedef kaldi::int32 int32;
65  typedef kaldi::int64 int64;
66  try {
67  const char *usage =
68  "Accumulate stats for iVector extractor training\n"
69  "Reads in features and Gaussian-level posteriors (typically from a full GMM)\n"
70  "Supports multiple threads, but won't be able to make use of too many at a time\n"
71  "(e.g. more than about 4)\n"
72  "Usage: ivector-extractor-acc-stats [options] <model-in> <feature-rspecifier>"
73  "<posteriors-rspecifier> <stats-out>\n"
74  "e.g.: \n"
75  " fgmm-global-gselect-to-post 1.fgmm '$feats' 'ark:gunzip -c gselect.1.gz|' ark:- | \\\n"
76  " ivector-extractor-acc-stats 2.ie '$feats' ark,s,cs:- 2.1.acc\n";
77 
78  ParseOptions po(usage);
79  bool binary = true;
81  TaskSequencerConfig sequencer_opts;
82  po.Register("binary", &binary, "Write output in binary mode");
83  stats_opts.Register(&po);
84  sequencer_opts.Register(&po);
85 
86  po.Read(argc, argv);
87 
88  if (po.NumArgs() != 4) {
89  po.PrintUsage();
90  exit(1);
91  }
92 
93  std::string ivector_extractor_rxfilename = po.GetArg(1),
94  feature_rspecifier = po.GetArg(2),
95  posteriors_rspecifier = po.GetArg(3),
96  accs_wxfilename = po.GetArg(4);
97 
98 
99  // Initialize these Reader objects before reading the IvectorExtractor,
100  // because it uses up a lot of memory and any fork() after that will
101  // be in danger of causing an allocation failure.
102  SequentialBaseFloatMatrixReader feature_reader(feature_rspecifier);
103  RandomAccessPosteriorReader posteriors_reader(posteriors_rspecifier);
104 
105 
106  // This is a bit of a mess... the code that reads in the extractor calls
107  // ComputeDerivedVars, and it can do this multi-threaded, controlled by
108  // g_num_threads. So if the user specified the --num-threads option, which
109  // goes to sequencer_opts in this case, copy it to g_num_threads.
110  g_num_threads = sequencer_opts.num_threads;
111 
112  IvectorExtractor extractor;
113  ReadKaldiObject(ivector_extractor_rxfilename, &extractor);
114 
115  IvectorExtractorStats stats(extractor, stats_opts);
116 
117 
118  int64 tot_t = 0;
119  int32 num_done = 0, num_err = 0;
120 
121  {
122  TaskSequencer<IvectorTask> sequencer(sequencer_opts);
123 
124  for (; !feature_reader.Done(); feature_reader.Next()) {
125  std::string key = feature_reader.Key();
126  if (!posteriors_reader.HasKey(key)) {
127  KALDI_WARN << "No posteriors for utterance " << key;
128  num_err++;
129  continue;
130  }
131  const Matrix<BaseFloat> &mat = feature_reader.Value();
132  const Posterior &posterior = posteriors_reader.Value(key);
133 
134  if (static_cast<int32>(posterior.size()) != mat.NumRows()) {
135  KALDI_WARN << "Size mismatch between posterior " << (posterior.size())
136  << " and features " << (mat.NumRows()) << " for utterance "
137  << key;
138  num_err++;
139  continue;
140  }
141 
142  sequencer.Run(new IvectorTask(extractor, mat, posterior, &stats));
143 
144  tot_t += posterior.size();
145  num_done++;
146  }
147  // destructor of "sequencer" will wait for any remaining tasks that
148  // have not yet completed.
149  }
150 
151  KALDI_LOG << "Done " << num_done << " files, " << num_err
152  << " with errors. Total frames " << tot_t;
153 
154  {
155  Output ko(accs_wxfilename, binary);
156  stats.Write(ko.Stream(), binary);
157  }
158 
159  KALDI_LOG << "Wrote stats to " << accs_wxfilename;
160 
161  return (num_done != 0 ? 0 : 1);
162  } catch(const std::exception &e) {
163  std::cerr << e.what();
164  return -1;
165  }
166 }
Relabels neural network egs with the read pdf-id alignments.
Definition: chain.dox:20
int32 g_num_threads
Definition: kaldi-thread.cc:25
IvectorExtractorStats is a class used to update the parameters of the ivector extractor.
void ReadKaldiObject(const std::string &filename, Matrix< float > *m)
Definition: kaldi-io.cc:818
Allows random access to a collection of objects in an archive or script file; see The Table concept...
Definition: kaldi-table.h:233
std::vector< std::vector< std::pair< int32, BaseFloat > > > Posterior
Posterior is a typedef for storing acoustic-state (actually, transition-id) posteriors over an uttera...
Definition: posterior.h:43
The class ParseOptions is for parsing command-line options; see Parsing command-line options for more...
Definition: parse-options.h:36
Options for IvectorExtractorStats, which is used to update the parameters of IvectorExtractor.
A templated class for reading objects sequentially from an archive or script file; see The Table conc...
Definition: kaldi-table.h:287
#define KALDI_WARN
Definition: kaldi-error.h:130
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:58
#define KALDI_LOG
Definition: kaldi-error.h:133
void Register(OptionsItf *opts)
Definition: kaldi-thread.h:159