All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
LmExampleDeterministicOnDemandFst< Arc > Class Template Reference

This class is for didactic purposes, it does not really do anything. More...

#include <deterministic-fst.h>

Inheritance diagram for LmExampleDeterministicOnDemandFst< Arc >:
Collaboration diagram for LmExampleDeterministicOnDemandFst< Arc >:

Public Types

typedef Arc::StateId StateId
 
typedef Arc::Weight Weight
 
typedef Arc::Label Label
 
- Public Types inherited from DeterministicOnDemandFst< Arc >
typedef Arc::StateId StateId
 
typedef Arc::Weight Weight
 
typedef Arc::Label Label
 

Public Member Functions

 LmExampleDeterministicOnDemandFst (void *lm, Label bos_symbol, Label eos_symbol)
 
virtual StateId Start ()
 
virtual Weight Final (StateId s)
 We don't bother caching the final-probs, just the arcs. More...
 
virtual bool GetArc (StateId s, Label ilabel, Arc *oarc)
 Note: ilabel must not be epsilon. More...
 
- Public Member Functions inherited from DeterministicOnDemandFst< Arc >
virtual ~DeterministicOnDemandFst ()
 

Private Types

typedef unordered_map
< std::vector< Label >
, StateId, kaldi::VectorHasher
< Label > > 
MapType
 

Private Member Functions

size_t GetIndex (StateId src_state, Label ilabel)
 

Private Attributes

void * lm_
 
Label bos_symbol_
 
Label eos_symbol_
 
MapType state_map_
 
StateId start_state_
 
std::vector< std::vector< Label > > state_vec_
 
void * lm
 

Detailed Description

template<class Arc>
class fst::LmExampleDeterministicOnDemandFst< Arc >

This class is for didactic purposes, it does not really do anything.

It shows how you would wrap a language model. Note: you should probably have <s> and </s> not be real words in your LM, but <s> correspond somehow to the initial-state of the LM, and </s> be encoded in the final-probs.

Definition at line 218 of file deterministic-fst.h.

Member Typedef Documentation

typedef Arc::Label Label

Definition at line 222 of file deterministic-fst.h.

typedef unordered_map<std::vector<Label>, StateId, kaldi::VectorHasher<Label> > MapType
private

Definition at line 240 of file deterministic-fst.h.

typedef Arc::StateId StateId

Definition at line 220 of file deterministic-fst.h.

typedef Arc::Weight Weight

Definition at line 221 of file deterministic-fst.h.

Constructor & Destructor Documentation

LmExampleDeterministicOnDemandFst ( void *  lm,
Label  bos_symbol,
Label  eos_symbol 
)

Definition at line 258 of file deterministic-fst-inl.h.

References LmExampleDeterministicOnDemandFst< Arc >::start_state_, LmExampleDeterministicOnDemandFst< Arc >::state_map_, and LmExampleDeterministicOnDemandFst< Arc >::state_vec_.

259  :
260  lm_(lm), bos_symbol_(bos_symbol), eos_symbol_(eos_symbol) {
261  std::vector<Label> begin_state; // history state corresponding to beginning of sentence
262  begin_state.push_back(bos_symbol); // Depending how your LM is set up, you might
263  // want to have a history vector with more than one bos_symbol on it.
264 
265  state_vec_.push_back(begin_state);
266  start_state_ = 0;
267  state_map_[begin_state] = 0;
268 }
std::vector< std::vector< Label > > state_vec_

Member Function Documentation

Arc::Weight Final ( StateId  s)
virtual

We don't bother caching the final-probs, just the arcs.

Implements DeterministicOnDemandFst< Arc >.

Definition at line 271 of file deterministic-fst-inl.h.

References KALDI_ASSERT.

Referenced by fst::TestCompose().

271  {
272  KALDI_ASSERT(static_cast<size_t>(s) < state_vec_.size());
273  // In a real version you would probably use the following variable somehow
274  // (commenting it because it's generating warnings).
275  // const std::vector<Label> &wseq = state_vec_[s];
276  float log_prob = -0.5; // e.g. log_prob = lm->GetLogProb(wseq, eos_symbol_);
277  return Weight(-log_prob); // assuming weight is FloatWeight.
278 }
std::vector< std::vector< Label > > state_vec_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool GetArc ( StateId  s,
Label  ilabel,
Arc *  oarc 
)
virtual

Note: ilabel must not be epsilon.

Implements DeterministicOnDemandFst< Arc >.

Definition at line 281 of file deterministic-fst-inl.h.

References KALDI_ASSERT.

Referenced by fst::TestCompose().

282  {
283  KALDI_ASSERT(static_cast<size_t>(s) < state_vec_.size());
284  std::vector<Label> wseq = state_vec_[s];
285  float log_prob = -0.25; // e.g. log_prob = lm->GetLogProb(wseq, ilabel);
286  wseq.push_back(ilabel); // the code might be different if your histories are the
287  // other way around.
288 
289  while (0) { // e.g. while !lm->HistoryStateExists(wseq)
290  wseq.erase(wseq.begin(), wseq.begin() + 1); // remove most distant element of history.
291  // note: if your histories are the other way round, you might just do
292  // wseq.pop() here.
293  }
294  if (log_prob == -numeric_limits<float>::infinity()) { // assume this
295  // is what happens if prob of the word is zero. Some LMs will never
296  // return zero.
297  return false; // no arc.
298  }
299  std::pair<const std::vector<Label>, StateId> new_value(
300  wseq,
301  static_cast<Label>(state_vec_.size()));
302 
303  // Now get state id for destination state.
304  typedef typename MapType::iterator IterType;
305  std::pair<IterType, bool> result = state_map_.insert(new_value);
306  if (result.second == true) // was inserted
307  state_vec_.push_back(wseq);
308  oarc->ilabel = ilabel;
309  oarc->olabel = ilabel;
310  oarc->nextstate = result.first->second; // the next-state id.
311  oarc->weight = Weight(-log_prob);
312  return true;
313 }
std::vector< std::vector< Label > > state_vec_
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
size_t GetIndex ( StateId  src_state,
Label  ilabel 
)
inlineprivate
virtual StateId Start ( )
inlinevirtual

Member Data Documentation

Label bos_symbol_
private

Definition at line 242 of file deterministic-fst.h.

Label eos_symbol_
private

Definition at line 243 of file deterministic-fst.h.

void* lm
private

Definition at line 250 of file deterministic-fst.h.

void* lm_
private

Definition at line 241 of file deterministic-fst.h.

std::vector<std::vector<Label> > state_vec_
private

The documentation for this class was generated from the following files: