LmExampleDeterministicOnDemandFst< Arc > Class Template Reference

This class is for didactic purposes, it does not really do anything. More...

#include <deterministic-fst.h>

Inheritance diagram for LmExampleDeterministicOnDemandFst< Arc >:

Collaboration diagram for LmExampleDeterministicOnDemandFst< Arc >:

Public Types
typedef Arc::StateId	StateId
typedef Arc::Weight	Weight
typedef Arc::Label	Label
Public Types inherited from DeterministicOnDemandFst< Arc >
typedef Arc::StateId	StateId
typedef Arc::Weight	Weight
typedef Arc::Label	Label

Public Member Functions
	LmExampleDeterministicOnDemandFst (void *lm, Label bos_symbol, Label eos_symbol)
virtual StateId	Start ()
virtual Weight	Final (StateId s)
	We don't bother caching the final-probs, just the arcs. More...
virtual bool	GetArc (StateId s, Label ilabel, Arc *oarc)
	Note: ilabel must not be epsilon. More...
Public Member Functions inherited from DeterministicOnDemandFst< Arc >
virtual	~DeterministicOnDemandFst ()

Private Types
typedef unordered_map< std::vector< Label >, StateId, kaldi::VectorHasher< Label > >	MapType

Private Member Functions
size_t	GetIndex (StateId src_state, Label ilabel)

Private Attributes
void *	lm_
Label	bos_symbol_
Label	eos_symbol_
MapType	state_map_
StateId	start_state_
std::vector< std::vector< Label > >	state_vec_
void *	lm

Detailed Description

template<class Arc>
class fst::LmExampleDeterministicOnDemandFst< Arc >

This class is for didactic purposes, it does not really do anything.

It shows how you would wrap a language model. Note: you should probably have <s> and </s> not be real words in your LM, but <s> correspond somehow to the initial-state of the LM, and </s> be encoded in the final-probs.

Definition at line 262 of file deterministic-fst.h.

Member Typedef Documentation

Label

typedef Arc::Label Label

Definition at line 266 of file deterministic-fst.h.

MapType

typedef unordered_map<std::vector<Label>, StateId, kaldi::VectorHasher<Label> > MapType

private

Definition at line 284 of file deterministic-fst.h.

StateId

typedef Arc::StateId StateId

Definition at line 264 of file deterministic-fst.h.

Weight

typedef Arc::Weight Weight

Definition at line 265 of file deterministic-fst.h.

Constructor & Destructor Documentation

LmExampleDeterministicOnDemandFst()

LmExampleDeterministicOnDemandFst	(	void *	lm,
		Label	bos_symbol,
		Label	eos_symbol
	)

Definition at line 259 of file deterministic-fst-inl.h.

References LmExampleDeterministicOnDemandFst< Arc >::start_state_, LmExampleDeterministicOnDemandFst< Arc >::state_map_, and LmExampleDeterministicOnDemandFst< Arc >::state_vec_.

                                                 :
    lm_(lm), bos_symbol_(bos_symbol), eos_symbol_(eos_symbol) {
  std::vector<Label> begin_state; // history state corresponding to beginning of sentence
  begin_state.push_back(bos_symbol); // Depending how your LM is set up, you might
  // want to have a history vector with more than one bos_symbol on it.

  state_vec_.push_back(begin_state);
  start_state_ = 0;
  state_map_[begin_state] = 0;
}

Member Function Documentation

Final()

Arc::Weight Final ( StateId  s )

virtual

We don't bother caching the final-probs, just the arcs.

Implements DeterministicOnDemandFst< Arc >.

Definition at line 272 of file deterministic-fst-inl.h.

References KALDI_ASSERT, and LmExampleDeterministicOnDemandFst< Arc >::state_vec_.

Referenced by fst::TestCompose().

                                                                          {
  KALDI_ASSERT(static_cast<size_t>(s) < state_vec_.size());
  // In a real version you would probably use the following variable somehow
  // (commenting it because it's generating warnings).
  // const std::vector<Label> &wseq = state_vec_[s];
  float log_prob = -0.5; // e.g. log_prob = lm->GetLogProb(wseq, eos_symbol_);
  return Weight(-log_prob); // assuming weight is FloatWeight.
}

GetArc()

bool GetArc ( StateId  s, Label  ilabel, Arc *  oarc  )

virtual

Note: ilabel must not be epsilon.

Implements DeterministicOnDemandFst< Arc >.

Definition at line 282 of file deterministic-fst-inl.h.

References KALDI_ASSERT, LmExampleDeterministicOnDemandFst< Arc >::state_map_, and LmExampleDeterministicOnDemandFst< Arc >::state_vec_.

Referenced by fst::TestCompose().

                                        {
  KALDI_ASSERT(static_cast<size_t>(s) < state_vec_.size());
  std::vector<Label> wseq = state_vec_[s];
  float log_prob = -0.25; // e.g. log_prob = lm->GetLogProb(wseq, ilabel);
  wseq.push_back(ilabel); // the code might be different if your histories are the
  // other way around.

  while (0) { // e.g. while !lm->HistoryStateExists(wseq)
    wseq.erase(wseq.begin(), wseq.begin() + 1); // remove most distant element of history.
    // note: if your histories are the other way round, you might just do
    // wseq.pop() here.
  }
  if (log_prob == -std::numeric_limits<float>::infinity()) { // assume this
    // is what happens if prob of the word is zero.  Some LMs will never
    // return zero.
    return false; // no arc.
  }
  std::pair<const std::vector<Label>, StateId> new_value(
      wseq,
      static_cast<Label>(state_vec_.size()));

  // Now get state id for destination state.
  typedef typename MapType::iterator IterType;
  std::pair<IterType, bool> result = state_map_.insert(new_value);
  if (result.second == true) // was inserted
    state_vec_.push_back(wseq);
  oarc->ilabel = ilabel;
  oarc->olabel = ilabel;
  oarc->nextstate = result.first->second; // the next-state id.
  oarc->weight = Weight(-log_prob);
  return true;
}

GetIndex()

size_t GetIndex ( StateId  src_state, Label  ilabel  )

inlineprivate

Start()

virtual StateId Start ( )

inlinevirtual

Implements DeterministicOnDemandFst< Arc >.

Definition at line 273 of file deterministic-fst.h.

References DeterministicOnDemandFst< Arc >::Final(), and DeterministicOnDemandFst< Arc >::GetArc().

Referenced by fst::TestCompose().

{ return start_state_; }

Member Data Documentation

bos_symbol_

Label bos_symbol_

private

Definition at line 286 of file deterministic-fst.h.

eos_symbol_

Label eos_symbol_

private

Definition at line 287 of file deterministic-fst.h.

lm

void* lm

private

Definition at line 294 of file deterministic-fst.h.

lm_

void* lm_

private

Definition at line 285 of file deterministic-fst.h.

start_state_

StateId start_state_

private

Definition at line 291 of file deterministic-fst.h.

Referenced by LmExampleDeterministicOnDemandFst< Arc >::LmExampleDeterministicOnDemandFst().

state_map_

MapType state_map_

private

Definition at line 290 of file deterministic-fst.h.

Referenced by LmExampleDeterministicOnDemandFst< Arc >::GetArc(), and LmExampleDeterministicOnDemandFst< Arc >::LmExampleDeterministicOnDemandFst().

state_vec_

std::vector<std::vector<Label> > state_vec_

private

Definition at line 292 of file deterministic-fst.h.

Referenced by LmExampleDeterministicOnDemandFst< Arc >::Final(), LmExampleDeterministicOnDemandFst< Arc >::GetArc(), and LmExampleDeterministicOnDemandFst< Arc >::LmExampleDeterministicOnDemandFst().

---

The documentation for this class was generated from the following files:

• fstext/deterministic-fst.h
• fstext/deterministic-fst-inl.h