LatticeLexiconWordAligner::ComputationState Class Reference

Collaboration diagram for LatticeLexiconWordAligner::ComputationState:

Public Member Functions
void	Advance (const CompactLatticeArc &arc, const TransitionModel &tmodel, LatticeWeight *leftover_weight)
	The state of the computation in which, along a single path in the lattice, we work out the word boundaries and output aligned arcs. More...
bool	ViableIfAdvanced (const ViabilityMap &viability_map) const
	Returns true if, assuming we were to add one or more phones by calling Advance one or more times on this, we might be able later to successfully call TakeTransition. More...
int32	NumPhones () const
int32	NumWords () const
int32	PendingWord () const
Freshness	WordFreshness () const
Freshness	PhoneFreshness () const
void	TakeForcedTransition (int32 partial_word_label, ComputationState *next_state, CompactLatticeArc *arc_out) const
	This may be called at the end of a lattice, if it was forced out. More...
bool	TakeTransition (const LexiconMap &lexicon_map, int32 word_id, int32 num_phones, ComputationState *next_state, CompactLatticeArc *arc_out) const
	Take a transition, if possible; consume "num_phones" phones and (if word_id != 0) the word "word_id" which must be the first word in words_. More...
bool	IsEmpty () const
LatticeWeight	FinalWeight () const
	FinalWeight() will return "weight" if both transition_ids and word_labels are empty, otherwise it will return Weight::Zero(). More...
size_t	Hash () const
bool	operator== (const ComputationState &other) const
	ComputationState ()
	ComputationState (const ComputationState &other)

Private Attributes
std::vector< int32 >	phones_
std::vector< int32 >	words_
Freshness	phone_fresh_
Freshness	word_fresh_
std::vector< std::vector< int32 > >	transition_ids_
LatticeWeight	weight_

Detailed Description

Definition at line 62 of file word-align-lattice-lexicon.cc.

Constructor & Destructor Documentation

ComputationState() [1/2]

ComputationState ( )

inline

Definition at line 142 of file word-align-lattice-lexicon.cc.

                      : phone_fresh_(kNotFresh), word_fresh_(kNotFresh),
                        weight_(LatticeWeight::One()) { } // initial state.

ComputationState() [2/2]

ComputationState ( const ComputationState &  other )

inline

Definition at line 145 of file word-align-lattice-lexicon.cc.

                                                   :
        phones_(other.phones_), words_(other.words_),
        phone_fresh_(other.phone_fresh_), word_fresh_(other.word_fresh_),
        transition_ids_(other.transition_ids_), weight_(other.weight_) { }

Member Function Documentation

Advance()

void Advance	(	const CompactLatticeArc &	arc,
		const TransitionModel &	tmodel,
		LatticeWeight *	leftover_weight
	)

The state of the computation in which, along a single path in the lattice, we work out the word boundaries and output aligned arcs.

Advance the computation state by adding the symbols and weights from this arc. Outputs weight to "leftover_weight" and sets the weight to 1.0 (this helps keep the state space small). Note: because we previously did PhoneAlignLattice, we can assume this arc corresponds to exactly one or zero phones.

Definition at line 573 of file word-align-lattice-lexicon.cc.

References KALDI_ASSERT, LatticeLexiconWordAligner::kFresh, LatticeLexiconWordAligner::kNotFresh, LatticeWeightTpl< BaseFloat >::One(), LatticeLexiconWordAligner::ComputationState::phone_fresh_, LatticeLexiconWordAligner::ComputationState::phones_, fst::Times(), LatticeLexiconWordAligner::ComputationState::transition_ids_, TransitionModel::TransitionIdToPhone(), LatticeLexiconWordAligner::ComputationState::weight_, LatticeLexiconWordAligner::ComputationState::word_fresh_, and LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::PossiblyAdvanceArc().

                                                                                        {
  const std::vector<int32> &tids = arc.weight.String();
  int32 phone;
  if (tids.empty()) phone = 0;
  else {
    phone = tmodel.TransitionIdToPhone(tids.front());
    KALDI_ASSERT(phone == tmodel.TransitionIdToPhone(tids.back()) &&
                 "Error: lattice is not phone-aligned.");
  }
  if (arc.ilabel != 0) { // note: arc.ilabel==arc.olabel (acceptor)
    words_.push_back(arc.ilabel);
    // Note: the word freshness only applies to the word in position 0,
    // so only if the word-sequence is now of size 1, is it fresh.
    if (words_.size() == 1) word_fresh_ = kFresh;
    else word_fresh_ = kNotFresh;
  } else { // No word added -> word not fresh.
    word_fresh_ = kNotFresh;
  }
  if (phone != 0) {
    phones_.push_back(phone);
    transition_ids_.push_back(tids);
    phone_fresh_ = kFresh;
  } else {
    phone_fresh_ = kNotFresh;
  }
  *weight = Times(weight_, arc.weight.Weight()); // will go on arc in output lattice
  weight_ = LatticeWeight::One();
}

FinalWeight()

LatticeWeight FinalWeight ( ) const

inline

FinalWeight() will return "weight" if both transition_ids and word_labels are empty, otherwise it will return Weight::Zero().

Definition at line 112 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::IsEmpty(), LatticeLexiconWordAligner::ComputationState::weight_, and LatticeWeightTpl< BaseFloat >::Zero().

Referenced by LatticeLexiconWordAligner::ProcessFinal().

                                      {
      return (IsEmpty() ? weight_ : LatticeWeight::Zero());
    }

Hash()

size_t Hash ( ) const

inline

Definition at line 116 of file word-align-lattice-lexicon.cc.

References rnnlm::i, LatticeLexiconWordAligner::ComputationState::phone_fresh_, LatticeLexiconWordAligner::ComputationState::transition_ids_, LatticeLexiconWordAligner::ComputationState::word_fresh_, and LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::TupleHash::operator()().

                        {
      VectorHasher<int32> vh;
      const int32 p1 = 11117, p2 = 90647, p3 = 3967, p4 = 3557; // primes.
      size_t ans = 0;
      for (int32 i = 0; i < static_cast<int32>(transition_ids_.size()); i++) {
        ans *= p1;
        ans += vh(transition_ids_[i]);
      }
      ans += p2 * vh(words_)
          + static_cast<int32>(word_fresh_) * p3
          + static_cast<int32>(phone_fresh_) * p4;
      // phones_ is determined by transition-id sequence so we don't
      // need to include it in the hash.
      return ans;
    }

IsEmpty()

bool IsEmpty ( ) const

inline

Definition at line 107 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::transition_ids_, and LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::ComputationState::FinalWeight(), and LatticeLexiconWordAligner::ComputationState::TakeForcedTransition().

{ return (transition_ids_.empty() && words_.empty()); }

NumPhones()

int32 NumPhones ( ) const

inline

Definition at line 84 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::phones_.

Referenced by LatticeLexiconWordAligner::ProcessEpsilonTransitions(), and LatticeLexiconWordAligner::ProcessWordTransitions().

{ return phones_.size(); }

NumWords()

int32 NumWords ( ) const

inline

Definition at line 85 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::ProcessWordTransitions().

{ return words_.size(); }

operator==()

bool operator== ( const ComputationState &  other ) const

inline

Definition at line 132 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::phone_fresh_, LatticeLexiconWordAligner::ComputationState::transition_ids_, LatticeLexiconWordAligner::ComputationState::weight_, LatticeLexiconWordAligner::ComputationState::word_fresh_, and LatticeLexiconWordAligner::ComputationState::words_.

                                                           {
      // phones_ is determined by transition-id sequence so don't
      // need to compare it.
      return (transition_ids_ == other.transition_ids_ &&
              words_ == other.words_ &&
              weight_ == other.weight_ &&
              phone_fresh_ == other.phone_fresh_ &&
              word_fresh_ == other.word_fresh_);
    }

PendingWord()

int32 PendingWord ( ) const

inline

Definition at line 86 of file word-align-lattice-lexicon.cc.

References KALDI_ASSERT, and LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::ProcessWordTransitions().

{ KALDI_ASSERT(!words_.empty()); return words_[0]; }

PhoneFreshness()

Freshness PhoneFreshness ( ) const

inline

Definition at line 88 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::phone_fresh_, LatticeLexiconWordAligner::ComputationState::TakeForcedTransition(), and LatticeLexiconWordAligner::ComputationState::TakeTransition().

Referenced by LatticeLexiconWordAligner::ProcessEpsilonTransitions(), and LatticeLexiconWordAligner::ProcessWordTransitions().

{ return phone_fresh_; }

TakeForcedTransition()

void TakeForcedTransition	(	int32	partial_word_label,
		ComputationState *	next_state,
		CompactLatticeArc *	arc_out
	)		const

This may be called at the end of a lattice, if it was forced out.

Note: we will only use "partial_word_label" if there are phones without corresponding words; otherwise we'll use the word label that was there.

Definition at line 634 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::AppendVectors(), LatticeLexiconWordAligner::ComputationState::IsEmpty(), KALDI_ASSERT, KALDI_WARN, LatticeLexiconWordAligner::kFresh, LatticeWeightTpl< BaseFloat >::One(), LatticeLexiconWordAligner::ComputationState::phone_fresh_, LatticeLexiconWordAligner::ComputationState::phones_, LatticeLexiconWordAligner::ComputationState::transition_ids_, LatticeLexiconWordAligner::ComputationState::weight_, LatticeLexiconWordAligner::ComputationState::word_fresh_, and LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::ComputationState::PhoneFreshness(), and LatticeLexiconWordAligner::ProcessFinalForceOut().

                                      {
  KALDI_ASSERT(!IsEmpty());

  next_state->phones_.clear();
  next_state->words_.clear();
  next_state->transition_ids_.clear();
  // neither of the following variables should matter, actually,
  // they will never be inspected.  So just set them to kFresh for consistency,
  // so they end up at the same place in the tuple-map_.
  next_state->word_fresh_ = kFresh;
  next_state->phone_fresh_ = kFresh;
  next_state->weight_ = LatticeWeight::One();

  int32 word_id;
  if (words_.size() >= 1) {
    word_id = words_[0];
    if (words_.size() > 1)
      KALDI_WARN << "Word-aligning lattice: discarding extra word at end of lattice"
                 << "(forced-out).";
  } else {
    word_id = partial_word_label;
  }
  KALDI_ASSERT(word_id != 0);  // any zeros would have been replaced with
                               // 'temporary epsilon' = 2.
  std::vector<int32> appended_transition_ids;
  AppendVectors(transition_ids_.begin(),
                transition_ids_.end(),
                &appended_transition_ids);
  arc_out->ilabel = word_id;
  arc_out->olabel = word_id;
  arc_out->weight = CompactLatticeWeight(weight_,
                                         appended_transition_ids);
  // arc_out->nextstate will be set by the calling code.
}

TakeTransition()

bool TakeTransition	(	const LexiconMap &	lexicon_map,
		int32	word_id,
		int32	num_phones,
		ComputationState *	next_state,
		CompactLatticeArc *	arc_out
	)		const

Take a transition, if possible; consume "num_phones" phones and (if word_id != 0) the word "word_id" which must be the first word in words_.

Returns true if we could take the transition.

Definition at line 673 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::AppendVectors(), kaldi::GetVerboseLevel(), rnnlm::i, KALDI_ASSERT, KALDI_VLOG, LatticeLexiconWordAligner::kAllFresh, LatticeLexiconWordAligner::kFresh, LatticeLexiconWordAligner::kNotFresh, LatticeWeightTpl< BaseFloat >::One(), LatticeLexiconWordAligner::ComputationState::phone_fresh_, LatticeLexiconWordAligner::ComputationState::phones_, LatticeLexiconWordAligner::ComputationState::transition_ids_, LatticeLexiconWordAligner::ComputationState::weight_, LatticeLexiconWordAligner::ComputationState::word_fresh_, and LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::ComputationState::PhoneFreshness(), LatticeLexiconWordAligner::ProcessEpsilonTransitions(), and LatticeLexiconWordAligner::ProcessWordTransitions().

                                                                    {
  KALDI_ASSERT(word_id == 0 || (!words_.empty() && word_id == words_[0]));
  KALDI_ASSERT(num_phones <= static_cast<int32>(phones_.size()));

  std::vector<int32> lexicon_key;
  lexicon_key.reserve(1 + num_phones);
  lexicon_key.push_back(word_id); // put 1st word in lexicon_key.
  lexicon_key.insert(lexicon_key.end(),
                     phones_.begin(), phones_.begin() + num_phones);
  LexiconMap::const_iterator iter = lexicon_map.find(lexicon_key);
  if (iter == lexicon_map.end()) { // no such entry
    return false;
  } else { // Entry exists.  We'll create an arc.
    next_state->phones_.assign(phones_.begin() + num_phones, phones_.end());
    next_state->words_.assign(words_.begin() + (word_id == 0 ? 0 : 1),
                              words_.end());
    next_state->transition_ids_.assign(transition_ids_.begin() + num_phones,
                                       transition_ids_.end());
    next_state->word_fresh_ =
        (word_id != 0 && !next_state->words_.empty()) ? kFresh : kNotFresh;
    next_state->phone_fresh_ =
        (next_state->phones_.empty() || num_phones == 0) ? kNotFresh : kAllFresh;

    // this next thing is a bit hard to explain.  If we just consumed a word with
    // no phones, we treat the phones as fresh.  The idea is that if we need to
    // both consume a word with no phones and a phone with no words (e.g.
    // an empty word and then silence), we need to have the phones marked
    // as fresh in order for this to be possible.
    if (num_phones == 0 && word_id != 0 && !next_state->phones_.empty())
      next_state->phone_fresh_ = kAllFresh;

    next_state->weight_ = LatticeWeight::One();

    if (GetVerboseLevel() >= 5) {
      std::ostringstream ostr;
      for (size_t i = 0; i < num_phones; i++)
        ostr << phones_[i] << " ";
      KALDI_VLOG(5) << "Taking arc with word = " << word_id
                    << " and phones = " << ostr.str()
                    << ", output-word = " << iter->second
                    << ", dest-state has num-words = " << next_state->words_.size()
                    << " and num-phones = " << next_state->phones_.size();
    }

    // Set arc_out:
    Label word_id = iter->second; // word_id will typically be
    // the same as words_[0], i.e. the
    // word we consumed.

    KALDI_ASSERT(word_id != 0);  // we replaced zeros with 'temporary epsilon' = -2.

    std::vector<int32> appended_transition_ids;
    AppendVectors(transition_ids_.begin(),
                  transition_ids_.begin() + num_phones,
                  &appended_transition_ids);
    arc_out->ilabel = word_id;
    arc_out->olabel = word_id;
    arc_out->weight = CompactLatticeWeight(weight_,
                                           appended_transition_ids);
    // arc_out->nextstate will be set in the calling code.
    return true;
  }
}

ViableIfAdvanced()

bool ViableIfAdvanced

(

const ViabilityMap &

viability_map

)

const

Returns true if, assuming we were to add one or more phones by calling Advance one or more times on this, we might be able later to successfully call TakeTransition.

It's a kind of co-accessibility test that avoids us creating an exponentially large number of states that would contribute nothing to the final output.

Definition at line 604 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::phones_, and LatticeLexiconWordAligner::ComputationState::words_.

Referenced by LatticeLexiconWordAligner::PossiblyAdvanceArc().

                                             {
  /* This will ideally to return true if and only if we can ever take
     any kind of transition out of this state after "advancing" it by adding
     words and/or phones.  It's OK to return true in some cases where the
     condition is false, though, if it's a pain to check, because the result
     will just be doing extra work for nothing (those states won't be
     co-accessible in the output).
  */
  if (phones_.empty()) return true;
  if (words_.empty()) return true;
  else {
    // neither phones_ or words_ is empty.  Return true if a longer sequence
    // than this phone sequence can have either zero (<eps>/epsilon) or the
    // first element of words_, as an entry in the lexicon with that phone
    // sequence.
    ViabilityMap::const_iterator iter = viability_map.find(phones_);
    if (iter == viability_map.end()) return false;
    else {
      const std::vector<int32> &this_set = iter->second; // sorted vector.
      // Return true if either 0 or words_[0] is in the set.  If 0 is
      // in the set, it will be the 1st element of the vector, because it's
      // the lowest element.
      return (this_set.front() == 0 ||
              std::binary_search(this_set.begin(), this_set.end(), words_[0]));
    }
  }
}

WordFreshness()

Freshness WordFreshness ( ) const

inline

Definition at line 87 of file word-align-lattice-lexicon.cc.

References LatticeLexiconWordAligner::ComputationState::word_fresh_.

Referenced by LatticeLexiconWordAligner::ProcessWordTransitions().

{ return word_fresh_; }

Member Data Documentation

phone_fresh_

Freshness phone_fresh_

private

Definition at line 159 of file word-align-lattice-lexicon.cc.

Referenced by LatticeLexiconWordAligner::ComputationState::Advance(), LatticeLexiconWordAligner::ComputationState::Hash(), LatticeLexiconWordAligner::ComputationState::operator==(), LatticeLexiconWordAligner::ComputationState::PhoneFreshness(), LatticeLexiconWordAligner::ComputationState::TakeForcedTransition(), and LatticeLexiconWordAligner::ComputationState::TakeTransition().

phones_

std::vector<int32> phones_

private

Definition at line 150 of file word-align-lattice-lexicon.cc.

Referenced by LatticeLexiconWordAligner::ComputationState::Advance(), LatticeLexiconWordAligner::ComputationState::NumPhones(), LatticeLexiconWordAligner::ComputationState::TakeForcedTransition(), LatticeLexiconWordAligner::ComputationState::TakeTransition(), and LatticeLexiconWordAligner::ComputationState::ViableIfAdvanced().

transition_ids_

std::vector<std::vector<int32> > transition_ids_

private

Definition at line 162 of file word-align-lattice-lexicon.cc.

Referenced by LatticeLexiconWordAligner::ComputationState::Advance(), LatticeLexiconWordAligner::ComputationState::Hash(), LatticeLexiconWordAligner::ComputationState::IsEmpty(), LatticeLexiconWordAligner::ComputationState::operator==(), LatticeLexiconWordAligner::ComputationState::TakeForcedTransition(), and LatticeLexiconWordAligner::ComputationState::TakeTransition().

weight_

LatticeWeight weight_

private

Definition at line 164 of file word-align-lattice-lexicon.cc.

Referenced by LatticeLexiconWordAligner::ComputationState::Advance(), LatticeLexiconWordAligner::ComputationState::FinalWeight(), LatticeLexiconWordAligner::ComputationState::operator==(), LatticeLexiconWordAligner::ComputationState::TakeForcedTransition(), and LatticeLexiconWordAligner::ComputationState::TakeTransition().

word_fresh_

Freshness word_fresh_

private

Definition at line 160 of file word-align-lattice-lexicon.cc.

Referenced by LatticeLexiconWordAligner::ComputationState::Advance(), LatticeLexiconWordAligner::ComputationState::Hash(), LatticeLexiconWordAligner::ComputationState::operator==(), LatticeLexiconWordAligner::ComputationState::TakeForcedTransition(), LatticeLexiconWordAligner::ComputationState::TakeTransition(), and LatticeLexiconWordAligner::ComputationState::WordFreshness().

words_

std::vector<int32> words_

private

Definition at line 151 of file word-align-lattice-lexicon.cc.

Referenced by LatticeLexiconWordAligner::ComputationState::Advance(), LatticeLexiconWordAligner::ComputationState::Hash(), LatticeLexiconWordAligner::ComputationState::IsEmpty(), LatticeLexiconWordAligner::ComputationState::NumWords(), LatticeLexiconWordAligner::ComputationState::operator==(), LatticeLexiconWordAligner::ComputationState::PendingWord(), LatticeLexiconWordAligner::ComputationState::TakeForcedTransition(), LatticeLexiconWordAligner::ComputationState::TakeTransition(), and LatticeLexiconWordAligner::ComputationState::ViableIfAdvanced().

---

The documentation for this class was generated from the following file:

• lat/word-align-lattice-lexicon.cc