Classes and functions for creating FSTs from HMMs

Collaboration diagram for Classes and functions for creating FSTs from HMMs:

Classes
struct	HTransducerConfig
	Configuration class for the GetHTransducer() function; see The HTransducerConfig configuration class for context. More...
struct	HmmCacheHash

Typedefs
typedef unordered_map< std::pair< int32, std::vector< int32 > >, fst::VectorFst< fst::StdArc > *, HmmCacheHash >	HmmCacheType
	HmmCacheType is a map from (central-phone, sequence of pdf-ids) to FST, used as cache in GetHmmAsFsa, as an optimization. More...

Functions
fst::VectorFst< fst::StdArc > *	GetHmmAsFsa (std::vector< int32 > context_window, const ContextDependencyInterface &ctx_dep, const TransitionModel &trans_model, const HTransducerConfig &config, HmmCacheType *cache=NULL)
	Called by GetHTransducer() and probably will not need to be called directly; it creates and returns the FST corresponding to the phone. More...
fst::VectorFst< fst::StdArc > *	GetHmmAsFsaSimple (std::vector< int32 > context_window, const ContextDependencyInterface &ctx_dep, const TransitionModel &trans_model, BaseFloat prob_scale)
	Included mainly as a form of documentation, not used in any other code currently. More...
fst::VectorFst< fst::StdArc > *	GetHTransducer (const std::vector< std::vector< int32 > > &ilabel_info, const ContextDependencyInterface &ctx_dep, const TransitionModel &trans_model, const HTransducerConfig &config, std::vector< int32 > *disambig_syms_left)
	Returns the H tranducer; result owned by caller. More...
void	GetIlabelMapping (const std::vector< std::vector< int32 > > &ilabel_info_old, const ContextDependencyInterface &ctx_dep, const TransitionModel &trans_model, std::vector< int32 > *old2new_map)
	GetIlabelMapping produces a mapping that's similar to HTK's logical-to-physical model mapping (i.e. More...
void	AddSelfLoops (const TransitionModel &trans_model, const std::vector< int32 > &disambig_syms, BaseFloat self_loop_scale, bool reorder, bool check_no_self_loops, fst::VectorFst< fst::StdArc > *fst)
	For context, see AddSelfLoops(). More...
void	AddTransitionProbs (const TransitionModel &trans_model, const std::vector< int32 > &disambig_syms, BaseFloat transition_scale, BaseFloat self_loop_scale, fst::VectorFst< fst::StdArc > *fst)
	Adds transition-probs, with the supplied scales (see Scaling of transition and acoustic probabilities), to the graph. More...
void	AddTransitionProbs (const TransitionModel &trans_model, BaseFloat transition_scale, BaseFloat self_loop_scale, Lattice *lat)
	This is as AddSelfLoops(), but operates on a Lattice, where it affects the graph part of the weight (the first element of the pair). More...
fst::VectorFst< fst::StdArc > *	GetPdfToTransitionIdTransducer (const TransitionModel &trans_model)
	Returns a transducer from pdfs plus one (input) to transition-ids (output). More...
void	ConvertTransitionIdsToPdfs (const TransitionModel &trans_model, const std::vector< int32 > &disambig_syms, fst::VectorFst< fst::StdArc > *fst)
	Converts all transition-ids in the FST to pdfs plus one. More...

Detailed Description

Typedef Documentation

HmmCacheType

typedef unordered_map<std::pair<int32, std::vector<int32> >, fst::VectorFst<fst::StdArc>*, HmmCacheHash> HmmCacheType

HmmCacheType is a map from (central-phone, sequence of pdf-ids) to FST, used as cache in GetHmmAsFsa, as an optimization.

Definition at line 70 of file hmm-utils.h.

Function Documentation

AddSelfLoops()

void AddSelfLoops	(	const TransitionModel &	trans_model,
		const std::vector< int32 > &	disambig_syms,
		BaseFloat	self_loop_scale,
		bool	reorder,
		bool	check_no_self_loops,
		fst::VectorFst< fst::StdArc > *	fst
	)

For context, see AddSelfLoops().

Expands an FST that has been built without self-loops, and adds the self-loops (it also needs to modify the probability of the non-self-loop ones, as the graph without self-loops was created in such a way that it was stochastic). Note that the disambig_syms will be empty in some recipes (e.g. if you already removed the disambiguation symbols). This function will treat numbers over 10000000 (kNontermBigNumber) the same as disambiguation symbols, assuming they are special symbols for grammar decoding.

Parameters

trans_model	[in] Transition model
disambig_syms	[in] Sorted, uniq list of disambiguation symbols, required if the graph contains disambiguation symbols but only needed for sanity checks.
self_loop_scale	[in] Transition-probability scale for self-loops; c.f. Scaling of transition and acoustic probabilities
reorder	[in] If true, reorders the transitions (see Reordering transitions). You'll normally want this to be true.
check_no_self_loops	[in] If true, it will check that there are no self-loops in the original graph; you'll normally want this to be true. If false, it will allow them, and will add self-loops after the original self-loop transitions, assuming reorder==true... this happens to be what we want when converting normal to unconstrained chain examples. WARNING: this was added in 2018; if you get a compilation error, add this as 'true', which emulates the behavior of older code.
fst	[in, out] The FST to be modified.

Definition at line 602 of file hmm-utils.cc.

References kaldi::AddSelfLoopsNoReorder(), kaldi::AddSelfLoopsReorder(), and KALDI_ASSERT.

Referenced by TrainingGraphCompiler::CompileGraph(), TrainingGraphCompiler::CompileGraphs(), kaldi::CoverageTest(), main(), and kaldi::ScoringTest().

                                                {
  KALDI_ASSERT(fst->Start() != fst::kNoStateId);
  if (reorder)
    AddSelfLoopsReorder(trans_model, disambig_syms, self_loop_scale,
                        check_no_self_loops, fst);
  else
    AddSelfLoopsNoReorder(trans_model, disambig_syms, self_loop_scale,
                          check_no_self_loops, fst);
}

AddTransitionProbs() [1/2]

void AddTransitionProbs	(	const TransitionModel &	trans_model,
		const std::vector< int32 > &	disambig_syms,
		BaseFloat	transition_scale,
		BaseFloat	self_loop_scale,
		fst::VectorFst< fst::StdArc > *	fst
	)

Adds transition-probs, with the supplied scales (see Scaling of transition and acoustic probabilities), to the graph.

Useful if you want to create a graph without transition probs, then possibly train the model (including the transition probs) but keep the graph fixed, and add back in the transition probs. It assumes the fst has transition-ids on it. It is not an error if the FST has no states (nothing will be done).

Parameters

trans_model	[in] The transition model
disambig_syms	[in] A list of disambiguation symbols, required if the graph has disambiguation symbols on its input but only used for checks.
transition_scale	[in] A scale on transition-probabilities apart from those involving self-loops; see Scaling of transition and acoustic probabilities.
self_loop_scale	[in] A scale on self-loop transition probabilities; see Scaling of transition and acoustic probabilities.
fst	[in, out] The FST to be modified.

Definition at line 1088 of file hmm-utils.cc.

References kaldi::GetScaledTransitionLogProb(), kaldi::IsSortedAndUniq(), KALDI_ASSERT, KALDI_ERR, TransitionModel::NumTransitionIds(), and fst::Times().

Referenced by main().

                                                      {
  using namespace fst;
  KALDI_ASSERT(IsSortedAndUniq(disambig_syms));
  int num_tids = trans_model.NumTransitionIds();
  for (StateIterator<VectorFst<StdArc> > siter(*fst);
      !siter.Done();
      siter.Next()) {
    for (MutableArcIterator<VectorFst<StdArc> > aiter(fst, siter.Value());
         !aiter.Done();
         aiter.Next()) {
      StdArc arc = aiter.Value();
      StdArc::Label l = arc.ilabel;
      if (l >= 1 && l <= num_tids) {  // a transition-id.
        BaseFloat scaled_log_prob = GetScaledTransitionLogProb(trans_model,
                                                               l,
                                                               transition_scale,
                                                               self_loop_scale);
        arc.weight = Times(arc.weight, TropicalWeight(-scaled_log_prob));
      } else if (l != 0) {
        if (!std::binary_search(disambig_syms.begin(), disambig_syms.end(),
                               arc.ilabel))
          KALDI_ERR << "AddTransitionProbs: invalid symbol " << arc.ilabel
                    << " on graph input side.";
      }
      aiter.SetValue(arc);
    }
  }
}

AddTransitionProbs() [2/2]

void AddTransitionProbs	(	const TransitionModel &	trans_model,
		BaseFloat	transition_scale,
		BaseFloat	self_loop_scale,
		Lattice *	lat
	)

This is as AddSelfLoops(), but operates on a Lattice, where it affects the graph part of the weight (the first element of the pair).

Definition at line 1121 of file hmm-utils.cc.

References kaldi::GetScaledTransitionLogProb(), KALDI_ERR, and TransitionModel::NumTransitionIds().

                                      {
  using namespace fst;
  int num_tids = trans_model.NumTransitionIds();
  for (fst::StateIterator<Lattice> siter(*lat);
       !siter.Done();
       siter.Next()) {
    for (MutableArcIterator<Lattice> aiter(lat, siter.Value());
         !aiter.Done();
         aiter.Next()) {
      LatticeArc arc = aiter.Value();
      LatticeArc::Label l = arc.ilabel;
      if (l >= 1 && l <= num_tids) {  // a transition-id.
        BaseFloat scaled_log_prob = GetScaledTransitionLogProb(trans_model,
                                                               l,
                                                               transition_scale,
                                                               self_loop_scale);
        // cost is negated log prob.
        arc.weight.SetValue1(arc.weight.Value1() - scaled_log_prob);
      } else if (l != 0) {
        KALDI_ERR << "AddTransitionProbs: invalid symbol " << arc.ilabel
                  << " on lattice input side.";
      }
      aiter.SetValue(arc);
    }
  }
}

ConvertTransitionIdsToPdfs()

void kaldi::ConvertTransitionIdsToPdfs	(	const TransitionModel &	trans_model,
		const std::vector< int32 > &	disambig_syms,
		fst::VectorFst< fst::StdArc > *	fst
	)

Converts all transition-ids in the FST to pdfs plus one.

Placeholder: not implemented yet!

GetHmmAsFsa()

fst::VectorFst< fst::StdArc > * GetHmmAsFsa	(	std::vector< int32 >	context_window,
		const ContextDependencyInterface &	ctx_dep,
		const TransitionModel &	trans_model,
		const HTransducerConfig &	config,
		HmmCacheType *	cache = NULL
	)

Called by GetHTransducer() and probably will not need to be called directly; it creates and returns the FST corresponding to the phone.

It's actually an acceptor (ilabels equal to olabels), which is why this is called "Fsa" not "Fst". This acceptor does not include self-loops; you have to call AddSelfLoops() for that. (We do that at a later graph compilation phase, for efficiency). The labels on the FSA correspond to transition-ids.

as the symbols. For documentation in context, see The function GetHmmAsFst()

Parameters

context_window	A vector representing the phonetic context; see here for explanation.
ctx_dep	The object that contains the phonetic decision-tree
trans_model	The transition-model object, which provides the mappings to transition-ids and also the transition probabilities.
config	Configuration object, see HTransducerConfig.
cache	Object used as a lookaside buffer to save computation; if it finds that the object it needs is already there, it will just return a pointer value from "cache"– not that this means you have to be careful not to delete things twice.

Definition at line 32 of file hmm-utils.cc.

References fst::ApplyProbabilityScale(), ContextDependencyInterface::CentralPosition(), ContextDependencyInterface::Compute(), ContextDependencyInterface::ContextWidth(), TransitionModel::GetTopo(), TransitionModel::GetTransitionLogProbIgnoringSelfLoops(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, kaldi::kNoPdf, kaldi::Log(), HmmTopology::NumPdfClasses(), TransitionModel::PairToTransitionId(), fst::RemoveEpsLocal(), HmmTopology::TopologyForPhone(), HTransducerConfig::transition_scale, and TransitionModel::TupleToTransitionState().

Referenced by kaldi::GetHTransducer().

                         {
  using namespace fst;

  if (static_cast<int32>(phone_window.size()) != ctx_dep.ContextWidth())
    KALDI_ERR << "Context size mismatch, ilabel-info [from context FST is "
              << phone_window.size() << ", context-dependency object "
        "expects " << ctx_dep.ContextWidth();

  int P = ctx_dep.CentralPosition();
  int32 phone = phone_window[P];
  if (phone == 0)
    KALDI_ERR << "phone == 0.  Some mismatch happened, or there is "
          "a code error.";

  const HmmTopology &topo = trans_model.GetTopo();
  const HmmTopology::TopologyEntry &entry  = topo.TopologyForPhone(phone);

  // vector of the pdfs, indexed by pdf-class (pdf-classes must start from zero
  // and be contiguous).
  std::vector<int32> pdfs(topo.NumPdfClasses(phone));
  for (int32 pdf_class = 0;
       pdf_class < static_cast<int32>(pdfs.size());
       pdf_class++) {
    if (! ctx_dep.Compute(phone_window, pdf_class, &(pdfs[pdf_class])) ) {
      std::ostringstream ctx_ss;
      for (size_t i = 0; i < phone_window.size(); i++)
        ctx_ss << phone_window[i] << ' ';
      KALDI_ERR << "GetHmmAsFsa: context-dependency object could not produce "
                << "an answer: pdf-class = " << pdf_class << " ctx-window = "
                << ctx_ss.str() << ".  This probably points "
          "to either a coding error in some graph-building process, "
          "a mismatch of topology with context-dependency object, the "
          "wrong FST being passed on a command-line, or something of "
          " that general nature.";
    }
  }
  std::pair<int32, std::vector<int32> > cache_index(phone, pdfs);
  if (cache != NULL) {
    HmmCacheType::iterator iter = cache->find(cache_index);
    if (iter != cache->end())
      return iter->second;
  }

  VectorFst<StdArc> *ans = new VectorFst<StdArc>;

  typedef StdArc Arc;
  typedef Arc::Weight Weight;
  typedef Arc::StateId StateId;
  typedef Arc::Label Label;

  std::vector<StateId> state_ids;
  for (size_t i = 0; i < entry.size(); i++)
    state_ids.push_back(ans->AddState());
  KALDI_ASSERT(state_ids.size() != 0);  // Or empty topology entry.
  ans->SetStart(state_ids[0]);
  StateId final = state_ids.back();
  ans->SetFinal(final, Weight::One());

  for (int32 hmm_state = 0;
       hmm_state < static_cast<int32>(entry.size());
       hmm_state++) {
    int32 forward_pdf_class = entry[hmm_state].forward_pdf_class, forward_pdf;
    int32 self_loop_pdf_class = entry[hmm_state].self_loop_pdf_class, self_loop_pdf;
    if (forward_pdf_class == kNoPdf) {  // nonemitting state.
      forward_pdf = kNoPdf;
      self_loop_pdf = kNoPdf;
    } else {
      KALDI_ASSERT(forward_pdf_class < static_cast<int32>(pdfs.size()));
      KALDI_ASSERT(self_loop_pdf_class < static_cast<int32>(pdfs.size()));
      forward_pdf = pdfs[forward_pdf_class];
      self_loop_pdf = pdfs[self_loop_pdf_class];
    }
    int32 trans_idx;
    for (trans_idx = 0;
        trans_idx < static_cast<int32>(entry[hmm_state].transitions.size());
        trans_idx++) {
      BaseFloat log_prob;
      Label label;
      int32 dest_state = entry[hmm_state].transitions[trans_idx].first;
      bool is_self_loop = (dest_state == hmm_state);
      if (is_self_loop)
        continue; // We will add self-loops in at a later stage of processing,
      // not in this function.
      if (forward_pdf_class == kNoPdf) {
        // no pdf, hence non-estimated probability.
        // [would not happen with normal topology] .  There is no transition-state
        // involved in this case.
        log_prob = Log(entry[hmm_state].transitions[trans_idx].second);
        label = 0;
      } else {  // normal probability.
        int32 trans_state =
            trans_model.TupleToTransitionState(phone, hmm_state, forward_pdf, self_loop_pdf);
        int32 trans_id =
            trans_model.PairToTransitionId(trans_state, trans_idx);
        log_prob = trans_model.GetTransitionLogProbIgnoringSelfLoops(trans_id);
        // log_prob is a negative number (or zero)...
        label = trans_id;
      }
      // Will add probability-scale later (we may want to push first).
      ans->AddArc(state_ids[hmm_state],
                  Arc(label, label, Weight(-log_prob), state_ids[dest_state]));
    }
  }

  fst::RemoveEpsLocal(ans);  // this is safe and will not blow up.

  // Now apply probability scale.
  // We waited till after the possible weight-pushing steps,
  // because weight-pushing needs "real" weights in order to work.
  ApplyProbabilityScale(config.transition_scale, ans);
  if (cache != NULL)
    (*cache)[cache_index] = ans;
  return ans;
}

GetHmmAsFsaSimple()

fst::VectorFst< fst::StdArc > * GetHmmAsFsaSimple	(	std::vector< int32 >	context_window,
		const ContextDependencyInterface &	ctx_dep,
		const TransitionModel &	trans_model,
		BaseFloat	prob_scale
	)

Included mainly as a form of documentation, not used in any other code currently.

Creates the acceptor FST with self-loops, and with fewer options.

Definition at line 155 of file hmm-utils.cc.

References ContextDependencyInterface::CentralPosition(), ContextDependencyInterface::Compute(), ContextDependencyInterface::ContextWidth(), TransitionModel::GetTopo(), TransitionModel::GetTransitionLogProb(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, kaldi::kNoPdf, kaldi::Log(), TransitionModel::PairToTransitionId(), HmmTopology::TopologyForPhone(), and TransitionModel::TupleToTransitionState().

Referenced by kaldi::GetRandomAlignmentForPhone().

                                        {
  using namespace fst;

  if (static_cast<int32>(phone_window.size()) != ctx_dep.ContextWidth())
    KALDI_ERR <<"Context size mismatch, ilabel-info [from context FST is "
              <<(phone_window.size())<<", context-dependency object "
        "expects "<<(ctx_dep.ContextWidth());

  int P = ctx_dep.CentralPosition();
  int32 phone = phone_window[P];
  KALDI_ASSERT(phone != 0);

  const HmmTopology &topo = trans_model.GetTopo();
  const HmmTopology::TopologyEntry &entry  = topo.TopologyForPhone(phone);

  VectorFst<StdArc> *ans = new VectorFst<StdArc>;

  // Create a mini-FST with a superfinal state [in case we have emitting
  // final-states, which we usually will.]
  typedef StdArc Arc;
  typedef Arc::Weight Weight;
  typedef Arc::StateId StateId;
  typedef Arc::Label Label;

  std::vector<StateId> state_ids;
  for (size_t i = 0; i < entry.size(); i++)
    state_ids.push_back(ans->AddState());
  KALDI_ASSERT(state_ids.size() > 1);  // Or invalid topology entry.
  ans->SetStart(state_ids[0]);
  StateId final = state_ids.back();
  ans->SetFinal(final, Weight::One());

  for (int32 hmm_state = 0;
       hmm_state < static_cast<int32>(entry.size());
       hmm_state++) {
    int32 forward_pdf_class = entry[hmm_state].forward_pdf_class, forward_pdf;
    int32 self_loop_pdf_class = entry[hmm_state].self_loop_pdf_class, self_loop_pdf;
    if (forward_pdf_class == kNoPdf) {   // nonemitting state; not generally used.
      forward_pdf = kNoPdf;
      self_loop_pdf = kNoPdf;
    } else {
      bool ans = ctx_dep.Compute(phone_window, forward_pdf_class, &forward_pdf);
      KALDI_ASSERT(ans && "Context-dependency computation failed.");
      ans = ctx_dep.Compute(phone_window, self_loop_pdf_class, &self_loop_pdf);
      KALDI_ASSERT(ans && "Context-dependency computation failed.");
    }
    int32 trans_idx;
    for (trans_idx = 0;
        trans_idx < static_cast<int32>(entry[hmm_state].transitions.size());
        trans_idx++) {
      BaseFloat log_prob;
      Label label;
      int32 dest_state = entry[hmm_state].transitions[trans_idx].first;
      if (forward_pdf_class == kNoPdf) {
        // no pdf, hence non-estimated probability.  very unusual case.  [would
        // not happen with normal topology] .  There is no transition-state
        // involved in this case.
        KALDI_ASSERT(hmm_state != dest_state);
        log_prob = Log(entry[hmm_state].transitions[trans_idx].second);
        label = 0;
      } else {  // normal probability.
        int32 trans_state =
            trans_model.TupleToTransitionState(phone, hmm_state, forward_pdf, self_loop_pdf);
        int32 trans_id =
            trans_model.PairToTransitionId(trans_state, trans_idx);
        log_prob = prob_scale * trans_model.GetTransitionLogProb(trans_id);
        // log_prob is a negative number (or zero)...
        label = trans_id;
      }
      ans->AddArc(state_ids[hmm_state],
                  Arc(label, label, Weight(-log_prob), state_ids[dest_state]));
    }
  }
  return ans;
}

GetHTransducer()

fst::VectorFst< fst::StdArc > * GetHTransducer	(	const std::vector< std::vector< int32 > > &	ilabel_info,
		const ContextDependencyInterface &	ctx_dep,
		const TransitionModel &	trans_model,
		const HTransducerConfig &	config,
		std::vector< int32 > *	disambig_syms_left
	)

Returns the H tranducer; result owned by caller.

Caution: our version of the H transducer does not include self-loops; you have to add those later. See GetHTransducer(). The H transducer has on the input transition-ids, and also possibly some disambiguation symbols, which will be put in disambig_syms. The output side contains the identifiers that are indexes into "ilabel_info" (these represent phones-in-context or disambiguation symbols). The ilabel_info vector allows GetHTransducer to map from symbols to phones-in-context (i.e. phonetic context windows). Any singleton symbols in the ilabel_info vector which are not phones, will be treated as disambiguation symbols. [Not all recipes use these]. The output "disambig_syms_left" will be set to a list of the disambiguation symbols on the input of the transducer (i.e. same symbol type as whatever is on the input of the transducer

Definition at line 254 of file hmm-utils.cc.

References kaldi::DeletePointers(), fst::GetEncodingMultiple(), kaldi::GetHmmAsFsa(), TransitionModel::GetPhones(), rnnlm::j, KALDI_ASSERT, KALDI_ERR, fst::kNontermBigNumber, fst::MakeLoopFst(), kaldi::MakeTrivialAcceptor(), HTransducerConfig::nonterm_phones_offset, TransitionModel::NumTransitionIds(), and kaldi::SortAndUniq().

Referenced by TrainingGraphCompiler::CompileGraph(), TrainingGraphCompiler::CompileGraphs(), and main().

                                                                                  {
  KALDI_ASSERT(ilabel_info.size() >= 1 && ilabel_info[0].size() == 0);  // make sure that eps == eps.
  HmmCacheType cache;
  // "cache" is an optimization that prevents GetHmmAsFsa repeating work
  // unnecessarily.
  using namespace fst;
  typedef StdArc Arc;
  typedef Arc::Weight Weight;
  typedef Arc::StateId StateId;
  typedef Arc::Label Label;

  std::vector<const ExpandedFst<Arc>* > fsts(ilabel_info.size(), NULL);
  std::vector<int32> phones = trans_model.GetPhones();

  KALDI_ASSERT(disambig_syms_left != 0);
  disambig_syms_left->clear();

  int32 first_disambig_sym = trans_model.NumTransitionIds() + 1;  // First disambig symbol we can have on the input side.
  int32 next_disambig_sym = first_disambig_sym;

  if (ilabel_info.size() > 0)
    KALDI_ASSERT(ilabel_info[0].size() == 0);  // make sure epsilon is epsilon...

  for (int32 j = 1; j < static_cast<int32>(ilabel_info.size()); j++) {  // zero is eps.
    KALDI_ASSERT(!ilabel_info[j].empty());
    if (ilabel_info[j][0] < 0 ||
        (ilabel_info[j][0] == 0 && ilabel_info[j].size() == 1)) {
      // disambig symbol or special symbol for grammar FSTs.
      if (ilabel_info[j].size() == 1) {
        // disambiguation symbol.
        int32 disambig_sym_left = next_disambig_sym++;
        disambig_syms_left->push_back(disambig_sym_left);
        fsts[j] = MakeTrivialAcceptor(disambig_sym_left);
      } else if (ilabel_info[j].size() == 2) {
        if (config.nonterm_phones_offset <= 0) {
          KALDI_ERR << "ilabel-info seems to be for grammar-FST.  You need to "
              "supply the --nonterm-phones-offset option.";
        }
        int32 nonterm_phones_offset = config.nonterm_phones_offset,
            nonterminal = -ilabel_info[j][0],
            left_context_phone = ilabel_info[j][1];
        if (nonterminal <= nonterm_phones_offset ||
            left_context_phone <= 0 ||
            left_context_phone > nonterm_phones_offset) {
          KALDI_ERR << "Could not interpret this ilabel-info with "
              "--nonterm-phones-offset=" << nonterm_phones_offset
                    << ": nonterminal,left-context-phone="
                    << nonterminal << ',' << left_context_phone;
        }
        int32 big_number = static_cast<int32>(fst::kNontermBigNumber),
            encoding_multiple = fst::GetEncodingMultiple(nonterm_phones_offset);
        int32 encoded_symbol = big_number + nonterminal * encoding_multiple +
            left_context_phone;
        fsts[j] = MakeTrivialAcceptor(encoded_symbol);
      } else {
        KALDI_ERR << "Could not decode this ilabel_info entry.";
      }
    } else {  // Real phone-in-context.
      std::vector<int32> phone_window = ilabel_info[j];

      VectorFst<Arc> *fst = GetHmmAsFsa(phone_window,
                                        ctx_dep,
                                        trans_model,
                                        config,
                                        &cache);
      fsts[j] = fst;
    }
  }

  VectorFst<Arc> *ans = MakeLoopFst(fsts);
  SortAndUniq(&fsts); // remove duplicate pointers, which we will have
  // in general, since we used the cache.
  DeletePointers(&fsts);
  return ans;
}

GetIlabelMapping()

void GetIlabelMapping	(	const std::vector< std::vector< int32 > > &	ilabel_info_old,
		const ContextDependencyInterface &	ctx_dep,
		const TransitionModel &	trans_model,
		std::vector< int32 > *	old2new_map
	)

GetIlabelMapping produces a mapping that's similar to HTK's logical-to-physical model mapping (i.e.

the xwrd.clustered.mlist files). It groups together "logical HMMs" (i.e. in our world, phonetic context windows) that share the same sequence of transition-ids. This can be used in an optional graph-creation step that produces a remapped form of CLG that can be more productively determinized and minimized. This is used in the command-line program make-ilabel-transducer.cc.

Parameters

ilabel_info_old	[in] The original ilabel_info vector
ctx_dep	[in] The tree
trans_model	[in] The transition-model object
old2new_map	[out] The output; this vector, which is of size equal to the number of new labels, is a mapping to the old labels such that we could create a vector ilabel_info_new such that ilabel_info_new[i] == ilabel_info_old[old2new_map[i]]

The next variable maps from the (central-phone, pdf-sequence) to the index in ilabel_info_old corresponding to the first phone-in-context that we saw for it. We use this to work out the logical-to-physical mapping. Each time we handle a phone in context, we see if its (central-phone, pdf-sequence) has already been seen; if yes, we map to the original phone-sequence, if no, we create a new "phyiscal-HMM" and there is no mapping.

old2old_map is a map from the old ilabels to themselves (but duplicates are mapped to one unique one.

Definition at line 335 of file hmm-utils.cc.

References ContextDependencyInterface::CentralPosition(), ContextDependencyInterface::Compute(), ContextDependencyInterface::ContextWidth(), TransitionModel::GetTopo(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, HmmTopology::NumPdfClasses(), and kaldi::WriteIntegerVector().

Referenced by main().

                                                      {
  KALDI_ASSERT(old2new_map != NULL);

  std::map<std::pair<int32, std::vector<int32> >, int32 >
      pair_to_physical;

  int32 N = ctx_dep.ContextWidth(),
      P = ctx_dep.CentralPosition();
  int32 num_syms_old = ilabel_info_old.size();

  std::vector<int32> old2old_map(num_syms_old);
  old2old_map[0] = 0;
  for (int32 i = 1; i < num_syms_old; i++) {
    const std::vector<int32> &vec = ilabel_info_old[i];
    if (vec.size() == 1 && vec[0] <= 0) {  // disambig.
      old2old_map[i] = i;
    } else {
      KALDI_ASSERT(vec.size() == static_cast<size_t>(N));
      // work out the vector of context-dependent phone
      int32 central_phone = vec[P];
      int32 num_pdf_classes = trans_model.GetTopo().NumPdfClasses(central_phone);
      std::vector<int32> state_seq(num_pdf_classes);  // Indexed by pdf-class
      for (int32 pdf_class = 0; pdf_class < num_pdf_classes; pdf_class++) {
        if (!ctx_dep.Compute(vec, pdf_class, &(state_seq[pdf_class]))) {
          std::ostringstream ss;
          WriteIntegerVector(ss, false, vec);
          KALDI_ERR << "tree did not succeed in converting phone window "<<ss.str();
        }
      }
      std::pair<int32, std::vector<int32> > pr(central_phone, state_seq);
      std::map<std::pair<int32, std::vector<int32> >, int32 >::iterator iter
          = pair_to_physical.find(pr);
      if (iter == pair_to_physical.end()) {  // first time we saw something like this.
        pair_to_physical[pr] = i;
        old2old_map[i] = i;
      } else {  // seen it before.  look up in the map, the index we point to.
        old2old_map[i] = iter->second;
      }
    }
  }

  std::vector<bool> seen(num_syms_old, false);
  for (int32 i = 0; i < num_syms_old; i++)
    seen[old2old_map[i]] = true;

  // Now work out the elements of old2new_map corresponding to
  // things that are first in their equivalence class.  We're just
  // compacting the labels to those for which seen[i] == true.
  int32 cur_id = 0;
  old2new_map->resize(num_syms_old);
  for (int32 i = 0; i < num_syms_old; i++)
    if (seen[i])
      (*old2new_map)[i] = cur_id++;
  // fill in the other elements of old2new_map.
  for (int32 i = 0; i < num_syms_old; i++)
    (*old2new_map)[i] = (*old2new_map)[old2old_map[i]];
}

GetPdfToTransitionIdTransducer()

fst::VectorFst< fst::StdArc > * GetPdfToTransitionIdTransducer

(

const TransitionModel &

trans_model

)

Returns a transducer from pdfs plus one (input) to transition-ids (output).

Currenly of use only for testing.

Definition at line 407 of file hmm-utils.cc.

References TransitionModel::NumTransitionIds(), and TransitionModel::TransitionIdToPdf().

Referenced by main().

                                                                                            {
  using namespace fst;
  VectorFst<StdArc> *ans = new VectorFst<StdArc>;
  typedef VectorFst<StdArc>::Weight Weight;
  typedef StdArc Arc;
  ans->AddState();
  ans->SetStart(0);
  ans->SetFinal(0, Weight::One());
  for (int32 tid = 1; tid <= trans_model.NumTransitionIds(); tid++) {
    int32 pdf = trans_model.TransitionIdToPdf(tid);
    ans->AddArc(0, Arc(pdf+1, tid, Weight::One(), 0));  // note the offset of 1 on the pdfs.
    // it's because 0 is a valid pdf.
  }
  return ans;
}