LmExampleDeterministicOnDemandFst< Arc > Class Template Reference

This class is for didactic purposes, it does not really do anything. More...

#include <deterministic-fst.h>

Inheritance diagram for LmExampleDeterministicOnDemandFst< Arc >:
Collaboration diagram for LmExampleDeterministicOnDemandFst< Arc >:

Public Types

typedef Arc::StateId StateId
 
typedef Arc::Weight Weight
 
typedef Arc::Label Label
 
- Public Types inherited from DeterministicOnDemandFst< Arc >
typedef Arc::StateId StateId
 
typedef Arc::Weight Weight
 
typedef Arc::Label Label
 

Public Member Functions

 LmExampleDeterministicOnDemandFst (void *lm, Label bos_symbol, Label eos_symbol)
 
virtual StateId Start ()
 
virtual Weight Final (StateId s)
 We don't bother caching the final-probs, just the arcs. More...
 
virtual bool GetArc (StateId s, Label ilabel, Arc *oarc)
 Note: ilabel must not be epsilon. More...
 
- Public Member Functions inherited from DeterministicOnDemandFst< Arc >
virtual ~DeterministicOnDemandFst ()
 

Private Types

typedef unordered_map< std::vector< Label >, StateId, kaldi::VectorHasher< Label > > MapType
 

Private Member Functions

size_t GetIndex (StateId src_state, Label ilabel)
 

Private Attributes

void * lm_
 
Label bos_symbol_
 
Label eos_symbol_
 
MapType state_map_
 
StateId start_state_
 
std::vector< std::vector< Label > > state_vec_
 
void * lm
 

Detailed Description

template<class Arc>
class fst::LmExampleDeterministicOnDemandFst< Arc >

This class is for didactic purposes, it does not really do anything.

It shows how you would wrap a language model. Note: you should probably have <s> and </s> not be real words in your LM, but <s> correspond somehow to the initial-state of the LM, and </s> be encoded in the final-probs.

Definition at line 262 of file deterministic-fst.h.

Member Typedef Documentation

◆ Label

typedef Arc::Label Label

Definition at line 266 of file deterministic-fst.h.

◆ MapType

typedef unordered_map<std::vector<Label>, StateId, kaldi::VectorHasher<Label> > MapType
private

Definition at line 284 of file deterministic-fst.h.

◆ StateId

typedef Arc::StateId StateId

Definition at line 264 of file deterministic-fst.h.

◆ Weight

typedef Arc::Weight Weight

Definition at line 265 of file deterministic-fst.h.

Constructor & Destructor Documentation

◆ LmExampleDeterministicOnDemandFst()

LmExampleDeterministicOnDemandFst ( void *  lm,
Label  bos_symbol,
Label  eos_symbol 
)

Definition at line 259 of file deterministic-fst-inl.h.

References LmExampleDeterministicOnDemandFst< Arc >::start_state_, LmExampleDeterministicOnDemandFst< Arc >::state_map_, and LmExampleDeterministicOnDemandFst< Arc >::state_vec_.

260  :
261  lm_(lm), bos_symbol_(bos_symbol), eos_symbol_(eos_symbol) {
262  std::vector<Label> begin_state; // history state corresponding to beginning of sentence
263  begin_state.push_back(bos_symbol); // Depending how your LM is set up, you might
264  // want to have a history vector with more than one bos_symbol on it.
265 
266  state_vec_.push_back(begin_state);
267  start_state_ = 0;
268  state_map_[begin_state] = 0;
269 }
std::vector< std::vector< Label > > state_vec_

Member Function Documentation

◆ Final()

Arc::Weight Final ( StateId  s)
virtual

We don't bother caching the final-probs, just the arcs.

Implements DeterministicOnDemandFst< Arc >.

Definition at line 272 of file deterministic-fst-inl.h.

References KALDI_ASSERT, and LmExampleDeterministicOnDemandFst< Arc >::state_vec_.

Referenced by fst::TestCompose().

272  {
273  KALDI_ASSERT(static_cast<size_t>(s) < state_vec_.size());
274  // In a real version you would probably use the following variable somehow
275  // (commenting it because it's generating warnings).
276  // const std::vector<Label> &wseq = state_vec_[s];
277  float log_prob = -0.5; // e.g. log_prob = lm->GetLogProb(wseq, eos_symbol_);
278  return Weight(-log_prob); // assuming weight is FloatWeight.
279 }
std::vector< std::vector< Label > > state_vec_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ GetArc()

bool GetArc ( StateId  s,
Label  ilabel,
Arc *  oarc 
)
virtual

Note: ilabel must not be epsilon.

Implements DeterministicOnDemandFst< Arc >.

Definition at line 282 of file deterministic-fst-inl.h.

References KALDI_ASSERT, LmExampleDeterministicOnDemandFst< Arc >::state_map_, and LmExampleDeterministicOnDemandFst< Arc >::state_vec_.

Referenced by fst::TestCompose().

283  {
284  KALDI_ASSERT(static_cast<size_t>(s) < state_vec_.size());
285  std::vector<Label> wseq = state_vec_[s];
286  float log_prob = -0.25; // e.g. log_prob = lm->GetLogProb(wseq, ilabel);
287  wseq.push_back(ilabel); // the code might be different if your histories are the
288  // other way around.
289 
290  while (0) { // e.g. while !lm->HistoryStateExists(wseq)
291  wseq.erase(wseq.begin(), wseq.begin() + 1); // remove most distant element of history.
292  // note: if your histories are the other way round, you might just do
293  // wseq.pop() here.
294  }
295  if (log_prob == -std::numeric_limits<float>::infinity()) { // assume this
296  // is what happens if prob of the word is zero. Some LMs will never
297  // return zero.
298  return false; // no arc.
299  }
300  std::pair<const std::vector<Label>, StateId> new_value(
301  wseq,
302  static_cast<Label>(state_vec_.size()));
303 
304  // Now get state id for destination state.
305  typedef typename MapType::iterator IterType;
306  std::pair<IterType, bool> result = state_map_.insert(new_value);
307  if (result.second == true) // was inserted
308  state_vec_.push_back(wseq);
309  oarc->ilabel = ilabel;
310  oarc->olabel = ilabel;
311  oarc->nextstate = result.first->second; // the next-state id.
312  oarc->weight = Weight(-log_prob);
313  return true;
314 }
std::vector< std::vector< Label > > state_vec_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ GetIndex()

size_t GetIndex ( StateId  src_state,
Label  ilabel 
)
inlineprivate

◆ Start()

Member Data Documentation

◆ bos_symbol_

Label bos_symbol_
private

Definition at line 286 of file deterministic-fst.h.

◆ eos_symbol_

Label eos_symbol_
private

Definition at line 287 of file deterministic-fst.h.

◆ lm

void* lm
private

Definition at line 294 of file deterministic-fst.h.

◆ lm_

void* lm_
private

Definition at line 285 of file deterministic-fst.h.

◆ start_state_

◆ state_map_

◆ state_vec_


The documentation for this class was generated from the following files: