All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
TrainingGraphCompiler Class Reference

#include <training-graph-compiler.h>

Collaboration diagram for TrainingGraphCompiler:

Public Member Functions

 TrainingGraphCompiler (const TransitionModel &trans_model, const ContextDependency &ctx_dep, fst::VectorFst< fst::StdArc > *lex_fst, const std::vector< int32 > &disambig_syms, const TrainingGraphCompilerOptions &opts)
 
bool CompileGraph (const fst::VectorFst< fst::StdArc > &word_grammar, fst::VectorFst< fst::StdArc > *out_fst)
 CompileGraph compiles a single training graph its input is a. More...
 
bool CompileGraphs (const std::vector< const fst::VectorFst< fst::StdArc > * > &word_fsts, std::vector< fst::VectorFst< fst::StdArc > * > *out_fsts)
 
bool CompileGraphFromText (const std::vector< int32 > &transcript, fst::VectorFst< fst::StdArc > *out_fst)
 
bool CompileGraphsFromText (const std::vector< std::vector< int32 > > &word_grammar, std::vector< fst::VectorFst< fst::StdArc > * > *out_fsts)
 
 ~TrainingGraphCompiler ()
 

Private Attributes

const TransitionModeltrans_model_
 
const ContextDependencyctx_dep_
 
fst::VectorFst< fst::StdArc > * lex_fst_
 
std::vector< int32 > disambig_syms_
 
fst::TableComposeCache
< fst::Fst< fst::StdArc > > 
lex_cache_
 
TrainingGraphCompilerOptions opts_
 

Detailed Description

Definition at line 57 of file training-graph-compiler.h.

Constructor & Destructor Documentation

TrainingGraphCompiler ( const TransitionModel trans_model,
const ContextDependency ctx_dep,
fst::VectorFst< fst::StdArc > *  lex_fst,
const std::vector< int32 > &  disambig_syms,
const TrainingGraphCompilerOptions opts 
)

Definition at line 24 of file training-graph-compiler.cc.

References fst::AddSubsequentialLoop(), ContextDependency::CentralPosition(), ContextDependency::ContextWidth(), TrainingGraphCompiler::disambig_syms_, TransitionModel::GetPhones(), rnnlm::i, kaldi::IsSortedAndUniq(), KALDI_ASSERT, KALDI_ERR, TrainingGraphCompiler::lex_fst_, kaldi::SortAndUniq(), and TrainingGraphCompiler::trans_model_.

28  :
29  trans_model_(trans_model), ctx_dep_(ctx_dep), lex_fst_(lex_fst),
30  disambig_syms_(disambig_syms), opts_(opts) {
31  using namespace fst;
32  const std::vector<int32> &phone_syms = trans_model_.GetPhones(); // needed to create context fst.
33 
34  KALDI_ASSERT(!phone_syms.empty());
35  KALDI_ASSERT(IsSortedAndUniq(phone_syms));
37  for (int32 i = 0; i < disambig_syms_.size(); i++)
38  if (std::binary_search(phone_syms.begin(), phone_syms.end(),
39  disambig_syms_[i]))
40  KALDI_ERR << "Disambiguation symbol " << disambig_syms_[i]
41  << " is also a phone.";
42 
43  int32 subseq_symbol = 1 + phone_syms.back();
44  if (!disambig_syms_.empty() && subseq_symbol <= disambig_syms_.back())
45  subseq_symbol = 1 + disambig_syms_.back();
46 
47  {
48  int32 N = ctx_dep.ContextWidth(),
49  P = ctx_dep.CentralPosition();
50  if (P != N-1)
51  AddSubsequentialLoop(subseq_symbol, lex_fst_); // This is needed for
52  // systems with right-context or we will not successfully compose
53  // with C.
54  }
55 
56  { // make sure lexicon is olabel sorted.
57  fst::OLabelCompare<fst::StdArc> olabel_comp;
58  fst::ArcSort(lex_fst_, olabel_comp);
59  }
60 }
const ContextDependency & ctx_dep_
Definition: graph.dox:21
void AddSubsequentialLoop(typename Arc::Label subseq_symbol, MutableFst< Arc > *fst)
Modifies an FST so that it transuces the same paths, but the input side of the paths can all have the...
const TransitionModel & trans_model_
const std::vector< int32 > & GetPhones() const
Returns a sorted, unique list of phones.
void SortAndUniq(std::vector< T > *vec)
Sorts and uniq's (removes duplicates) from a vector.
Definition: stl-utils.h:51
fst::VectorFst< fst::StdArc > * lex_fst_
#define KALDI_ERR
Definition: kaldi-error.h:127
TrainingGraphCompilerOptions opts_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool IsSortedAndUniq(const std::vector< T > &vec)
Returns true if the vector is sorted and contains each element only once.
Definition: stl-utils.h:75
~TrainingGraphCompiler ( )
inline

Definition at line 92 of file training-graph-compiler.h.

References TrainingGraphCompiler::lex_fst_.

92 { delete lex_fst_; }
fst::VectorFst< fst::StdArc > * lex_fst_

Member Function Documentation

bool CompileGraph ( const fst::VectorFst< fst::StdArc > &  word_grammar,
fst::VectorFst< fst::StdArc > *  out_fst 
)

CompileGraph compiles a single training graph its input is a.

Definition at line 71 of file training-graph-compiler.cc.

References kaldi::AddSelfLoops(), ContextDependency::CentralPosition(), fst::ComposeContextFst(), ContextDependency::ContextWidth(), TrainingGraphCompiler::ctx_dep_, fst::DeterminizeStarInLog(), TrainingGraphCompiler::disambig_syms_, kaldi::GetHTransducer(), TransitionModel::GetPhones(), KALDI_ASSERT, TrainingGraphCompiler::lex_cache_, TrainingGraphCompiler::lex_fst_, fst::MinimizeEncoded(), TrainingGraphCompiler::opts_, fst::RemoveEpsLocal(), fst::RemoveSomeInputSymbols(), TrainingGraphCompilerOptions::reorder, TrainingGraphCompilerOptions::rm_eps, TrainingGraphCompilerOptions::self_loop_scale, fst::TableCompose(), TrainingGraphCompiler::trans_model_, TrainingGraphCompilerOptions::transition_scale, and HTransducerConfig::transition_scale.

Referenced by TrainingGraphCompiler::CompileGraphFromText(), and main().

72  {
73  using namespace fst;
74  KALDI_ASSERT(lex_fst_ !=NULL);
75  KALDI_ASSERT(out_fst != NULL);
76 
77  VectorFst<StdArc> phone2word_fst;
78  // TableCompose more efficient than compose.
79  TableCompose(*lex_fst_, word_fst, &phone2word_fst, &lex_cache_);
80 
81  KALDI_ASSERT(phone2word_fst.Start() != kNoStateId);
82 
83  ContextFst<StdArc> *cfst = NULL;
84  { // make cfst [ it's expanded on the fly ]
85  const std::vector<int32> &phone_syms = trans_model_.GetPhones(); // needed to create context fst.
86  int32 subseq_symbol = phone_syms.back() + 1;
87  if (!disambig_syms_.empty() && subseq_symbol <= disambig_syms_.back())
88  subseq_symbol = 1 + disambig_syms_.back();
89 
90  cfst = new ContextFst<StdArc>(subseq_symbol,
91  phone_syms,
95  }
96 
97  VectorFst<StdArc> ctx2word_fst;
98  ComposeContextFst(*cfst, phone2word_fst, &ctx2word_fst);
99  // ComposeContextFst is like Compose but faster for this particular Fst type.
100  // [and doesn't expand too many arcs in the ContextFst.]
101 
102  KALDI_ASSERT(ctx2word_fst.Start() != kNoStateId);
103 
104  HTransducerConfig h_cfg;
105  h_cfg.transition_scale = opts_.transition_scale;
106 
107  std::vector<int32> disambig_syms_h; // disambiguation symbols on
108  // input side of H.
109  VectorFst<StdArc> *H = GetHTransducer(cfst->ILabelInfo(),
110  ctx_dep_,
111  trans_model_,
112  h_cfg,
113  &disambig_syms_h);
114 
115  VectorFst<StdArc> &trans2word_fst = *out_fst; // transition-id to word.
116  TableCompose(*H, ctx2word_fst, &trans2word_fst);
117 
118  KALDI_ASSERT(trans2word_fst.Start() != kNoStateId);
119 
120  // Epsilon-removal and determinization combined. This will fail if not determinizable.
121  DeterminizeStarInLog(&trans2word_fst);
122 
123  if (!disambig_syms_h.empty()) {
124  RemoveSomeInputSymbols(disambig_syms_h, &trans2word_fst);
125  // we elect not to remove epsilons after this phase, as it is
126  // a little slow.
127  if (opts_.rm_eps)
128  RemoveEpsLocal(&trans2word_fst);
129  }
130 
131 
132  // Encoded minimization.
133  MinimizeEncoded(&trans2word_fst);
134 
135  std::vector<int32> disambig;
136  AddSelfLoops(trans_model_,
137  disambig,
139  opts_.reorder,
140  &trans2word_fst);
141 
142  delete H;
143  delete cfst;
144  return true;
145 }
const ContextDependency & ctx_dep_
void TableCompose(const Fst< Arc > &ifst1, const Fst< Arc > &ifst2, MutableFst< Arc > *ofst, const TableComposeOptions &opts=TableComposeOptions())
void RemoveEpsLocal(MutableFst< Arc > *fst)
RemoveEpsLocal remove some (but not necessarily all) epsilons in an FST, using an algorithm that is g...
virtual int32 ContextWidth() const
ContextWidth() returns the value N (e.g.
Definition: context-dep.h:61
Definition: graph.dox:21
void AddSelfLoops(const TransitionModel &trans_model, const std::vector< int32 > &disambig_syms, BaseFloat self_loop_scale, bool reorder, fst::VectorFst< fst::StdArc > *fst)
For context, see AddSelfLoops().
Definition: hmm-utils.cc:559
fst::TableComposeCache< fst::Fst< fst::StdArc > > lex_cache_
void RemoveSomeInputSymbols(const vector< I > &to_remove, MutableFst< Arc > *fst)
RemoveSomeInputSymbols removes any symbol that appears in "to_remove", from the input side of the FST...
void MinimizeEncoded(VectorFst< Arc > *fst, float delta=kDelta)
Definition: fstext-utils.h:114
const TransitionModel & trans_model_
const std::vector< int32 > & GetPhones() const
Returns a sorted, unique list of phones.
fst::VectorFst< fst::StdArc > * lex_fst_
void DeterminizeStarInLog(VectorFst< StdArc > *fst, float delta, bool *debug_ptr, int max_states)
virtual int32 CentralPosition() const
Central position P of the phone context, in 0-based numbering, e.g.
Definition: context-dep.h:62
fst::VectorFst< fst::StdArc > * GetHTransducer(const std::vector< std::vector< int32 > > &ilabel_info, const ContextDependencyInterface &ctx_dep, const TransitionModel &trans_model, const HTransducerConfig &config, std::vector< int32 > *disambig_syms_left)
Returns the H tranducer; result owned by caller.
Definition: hmm-utils.cc:238
TrainingGraphCompilerOptions opts_
void ComposeContextFst(const ContextFst< Arc, LabelT > &ifst1, const Fst< Arc > &ifst2, MutableFst< Arc > *ofst, const ComposeOptions &opts=ComposeOptions())
Definition: context-fst.h:491
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool CompileGraphFromText ( const std::vector< int32 > &  transcript,
fst::VectorFst< fst::StdArc > *  out_fst 
)

Definition at line 62 of file training-graph-compiler.cc.

References TrainingGraphCompiler::CompileGraph(), and fst::MakeLinearAcceptor().

Referenced by main().

64  {
65  using namespace fst;
66  VectorFst<StdArc> word_fst;
67  MakeLinearAcceptor(transcript, &word_fst);
68  return CompileGraph(word_fst, out_fst);
69 }
Definition: graph.dox:21
void MakeLinearAcceptor(const vector< I > &labels, MutableFst< Arc > *ofst)
Creates unweighted linear acceptor from symbol sequence.
bool CompileGraph(const fst::VectorFst< fst::StdArc > &word_grammar, fst::VectorFst< fst::StdArc > *out_fst)
CompileGraph compiles a single training graph its input is a.
bool CompileGraphs ( const std::vector< const fst::VectorFst< fst::StdArc > * > &  word_fsts,
std::vector< fst::VectorFst< fst::StdArc > * > *  out_fsts 
)

Definition at line 164 of file training-graph-compiler.cc.

References kaldi::AddSelfLoops(), ContextDependency::CentralPosition(), fst::ComposeContextFst(), ContextDependency::ContextWidth(), TrainingGraphCompiler::ctx_dep_, fst::DeterminizeStarInLog(), TrainingGraphCompiler::disambig_syms_, kaldi::GetHTransducer(), TransitionModel::GetPhones(), rnnlm::i, ContextFst< Arc, LabelT >::ILabelInfo(), KALDI_ASSERT, TrainingGraphCompiler::lex_cache_, TrainingGraphCompiler::lex_fst_, fst::MinimizeEncoded(), TrainingGraphCompiler::opts_, fst::RemoveEpsLocal(), fst::RemoveSomeInputSymbols(), TrainingGraphCompilerOptions::reorder, TrainingGraphCompilerOptions::rm_eps, TrainingGraphCompilerOptions::self_loop_scale, fst::TableCompose(), TrainingGraphCompiler::trans_model_, TrainingGraphCompilerOptions::transition_scale, and HTransducerConfig::transition_scale.

Referenced by TrainingGraphCompiler::CompileGraphsFromText(), and main().

166  {
167 
168  using namespace fst;
169  KALDI_ASSERT(lex_fst_ !=NULL);
170  KALDI_ASSERT(out_fsts != NULL && out_fsts->empty());
171  out_fsts->resize(word_fsts.size(), NULL);
172  if (word_fsts.empty()) return true;
173 
174  ContextFst<StdArc> *cfst = NULL;
175  { // make cfst [ it's expanded on the fly ]
176  const std::vector<int32> &phone_syms = trans_model_.GetPhones(); // needed to create context fst.
177  int32 subseq_symbol = phone_syms.back() + 1;
178  if (!disambig_syms_.empty() && subseq_symbol <= disambig_syms_.back())
179  subseq_symbol = 1 + disambig_syms_.back();
180 
181  cfst = new ContextFst<StdArc>(subseq_symbol,
182  phone_syms,
186  }
187 
188  for (size_t i = 0; i < word_fsts.size(); i++) {
189  VectorFst<StdArc> phone2word_fst;
190  // TableCompose more efficient than compose.
191  TableCompose(*lex_fst_, *(word_fsts[i]), &phone2word_fst, &lex_cache_);
192 
193  KALDI_ASSERT(phone2word_fst.Start() != kNoStateId &&
194  "Perhaps you have words missing in your lexicon?");
195 
196  VectorFst<StdArc> ctx2word_fst;
197  ComposeContextFst(*cfst, phone2word_fst, &ctx2word_fst);
198  // ComposeContextFst is like Compose but faster for this particular Fst type.
199  // [and doesn't expand too many arcs in the ContextFst.]
200 
201  KALDI_ASSERT(ctx2word_fst.Start() != kNoStateId);
202 
203  (*out_fsts)[i] = ctx2word_fst.Copy(); // For now this contains the FST with symbols
204  // representing phones-in-context.
205  }
206 
207  HTransducerConfig h_cfg;
208  h_cfg.transition_scale = opts_.transition_scale;
209 
210  std::vector<int32> disambig_syms_h;
211  VectorFst<StdArc> *H = GetHTransducer(cfst->ILabelInfo(),
212  ctx_dep_,
213  trans_model_,
214  h_cfg,
215  &disambig_syms_h);
216 
217  for (size_t i = 0; i < out_fsts->size(); i++) {
218  VectorFst<StdArc> &ctx2word_fst = *((*out_fsts)[i]);
219  VectorFst<StdArc> trans2word_fst;
220  TableCompose(*H, ctx2word_fst, &trans2word_fst);
221 
222  DeterminizeStarInLog(&trans2word_fst);
223 
224  if (!disambig_syms_h.empty()) {
225  RemoveSomeInputSymbols(disambig_syms_h, &trans2word_fst);
226  if (opts_.rm_eps)
227  RemoveEpsLocal(&trans2word_fst);
228  }
229 
230  // Encoded minimization.
231  MinimizeEncoded(&trans2word_fst);
232 
233  std::vector<int32> disambig;
234  AddSelfLoops(trans_model_,
235  disambig,
237  opts_.reorder,
238  &trans2word_fst);
239 
240  KALDI_ASSERT(trans2word_fst.Start() != kNoStateId);
241 
242  *((*out_fsts)[i]) = trans2word_fst;
243  }
244 
245  delete H;
246  delete cfst;
247  return true;
248 }
const ContextDependency & ctx_dep_
void TableCompose(const Fst< Arc > &ifst1, const Fst< Arc > &ifst2, MutableFst< Arc > *ofst, const TableComposeOptions &opts=TableComposeOptions())
void RemoveEpsLocal(MutableFst< Arc > *fst)
RemoveEpsLocal remove some (but not necessarily all) epsilons in an FST, using an algorithm that is g...
virtual int32 ContextWidth() const
ContextWidth() returns the value N (e.g.
Definition: context-dep.h:61
Definition: graph.dox:21
void AddSelfLoops(const TransitionModel &trans_model, const std::vector< int32 > &disambig_syms, BaseFloat self_loop_scale, bool reorder, fst::VectorFst< fst::StdArc > *fst)
For context, see AddSelfLoops().
Definition: hmm-utils.cc:559
fst::TableComposeCache< fst::Fst< fst::StdArc > > lex_cache_
void RemoveSomeInputSymbols(const vector< I > &to_remove, MutableFst< Arc > *fst)
RemoveSomeInputSymbols removes any symbol that appears in "to_remove", from the input side of the FST...
void MinimizeEncoded(VectorFst< Arc > *fst, float delta=kDelta)
Definition: fstext-utils.h:114
const TransitionModel & trans_model_
const std::vector< int32 > & GetPhones() const
Returns a sorted, unique list of phones.
fst::VectorFst< fst::StdArc > * lex_fst_
void DeterminizeStarInLog(VectorFst< StdArc > *fst, float delta, bool *debug_ptr, int max_states)
virtual int32 CentralPosition() const
Central position P of the phone context, in 0-based numbering, e.g.
Definition: context-dep.h:62
fst::VectorFst< fst::StdArc > * GetHTransducer(const std::vector< std::vector< int32 > > &ilabel_info, const ContextDependencyInterface &ctx_dep, const TransitionModel &trans_model, const HTransducerConfig &config, std::vector< int32 > *disambig_syms_left)
Returns the H tranducer; result owned by caller.
Definition: hmm-utils.cc:238
TrainingGraphCompilerOptions opts_
void ComposeContextFst(const ContextFst< Arc, LabelT > &ifst1, const Fst< Arc > &ifst2, MutableFst< Arc > *ofst, const ComposeOptions &opts=ComposeOptions())
Definition: context-fst.h:491
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
const vector< vector< LabelT > > & ILabelInfo()
Definition: context-fst.h:286
bool CompileGraphsFromText ( const std::vector< std::vector< int32 > > &  word_grammar,
std::vector< fst::VectorFst< fst::StdArc > * > *  out_fsts 
)

Definition at line 148 of file training-graph-compiler.cc.

References TrainingGraphCompiler::CompileGraphs(), rnnlm::i, and fst::MakeLinearAcceptor().

Referenced by main().

150  {
151  using namespace fst;
152  std::vector<const VectorFst<StdArc>* > word_fsts(transcripts.size());
153  for (size_t i = 0; i < transcripts.size(); i++) {
154  VectorFst<StdArc> *word_fst = new VectorFst<StdArc>();
155  MakeLinearAcceptor(transcripts[i], word_fst);
156  word_fsts[i] = word_fst;
157  }
158  bool ans = CompileGraphs(word_fsts, out_fsts);
159  for (size_t i = 0; i < transcripts.size(); i++)
160  delete word_fsts[i];
161  return ans;
162 }
Definition: graph.dox:21
void MakeLinearAcceptor(const vector< I > &labels, MutableFst< Arc > *ofst)
Creates unweighted linear acceptor from symbol sequence.
bool CompileGraphs(const std::vector< const fst::VectorFst< fst::StdArc > * > &word_fsts, std::vector< fst::VectorFst< fst::StdArc > * > *out_fsts)

Member Data Documentation


The documentation for this class was generated from the following files: