#include <grammar-context-fst.h>

Inheritance diagram for InverseLeftBiphoneContextFst:

Collaboration diagram for InverseLeftBiphoneContextFst:

Public Types
typedef StdArc	Arc

typedef StdArc::StateId	StateId

typedef StdArc::Weight	Weight

typedef StdArc::Label	Label

Public Types inherited from DeterministicOnDemandFst< StdArc >
typedef StdArc ::StateId	StateId

typedef StdArc ::Weight	Weight

typedef StdArc ::Label	Label

Public Member Functions
	InverseLeftBiphoneContextFst (Label nonterm_phones_offset, const std::vector< int32 > &phones, const std::vector< int32 > &disambig_syms)
	Constructor. More...

virtual StateId	Start ()
	Here is a note on the state space of InverseLeftBiphoneContextFst; see Special symbols in C.fst which has some documentation on this. More...

virtual Weight	Final (StateId s)

virtual bool	GetArc (StateId s, Label ilabel, Arc *arc)
	Note: ilabel must not be epsilon. More...

	~InverseLeftBiphoneContextFst ()

const std::vector< std::vector< int32 > > &	IlabelInfo () const

void	SwapIlabelInfo (std::vector< std::vector< int32 > > *vec)

Public Member Functions inherited from DeterministicOnDemandFst< StdArc >
virtual Weight	Final (StateId s)=0

virtual bool	GetArc (StateId s, Label ilabel, StdArc *oarc)=0
	Note: ilabel must not be epsilon. More...

virtual	~DeterministicOnDemandFst ()

Private Types
typedef unordered_map< std::vector< int32 >, Label, kaldi::VectorHasher< int32 > >	VectorToLabelMap

Private Member Functions
int32	GetPhoneSymbolFor (enum NonterminalValues n)

Label	FindLabel (const std::vector< int32 > &label_info)
	Finds the label index corresponding to this context-window of phones (likely of width context_width_). More...

Private Attributes
int32	nonterm_phones_offset_

kaldi::ConstIntegerSet< Label >	phone_syms_

kaldi::ConstIntegerSet< Label >	disambig_syms_

VectorToLabelMap	ilabel_map_

std::vector< std::vector< int32 > >	ilabel_info_

Detailed Description

Definition at line 146 of file grammar-context-fst.h.

Member Typedef Documentation

◆ Arc

typedef StdArc Arc

Definition at line 148 of file grammar-context-fst.h.

◆ Label

typedef StdArc::Label Label

Definition at line 151 of file grammar-context-fst.h.

◆ StateId

typedef StdArc::StateId StateId

Definition at line 149 of file grammar-context-fst.h.

◆ VectorToLabelMap

typedef unordered_map<std::vector<int32>, Label, kaldi::VectorHasher<int32> > VectorToLabelMap

private

Definition at line 248 of file grammar-context-fst.h.

◆ Weight

typedef StdArc::Weight Weight

Definition at line 150 of file grammar-context-fst.h.

Constructor & Destructor Documentation

◆ InverseLeftBiphoneContextFst()

InverseLeftBiphoneContextFst	(	Label	nonterm_phones_offset,
		const std::vector< int32 > &	phones,
		const std::vector< int32 > &	disambig_syms
	)

Constructor.

This does not take the arguments 'context_width' or 'central_position' because they are assumed to be (2, 1) meaning a system with left-biphone context; and there is no subsequential symbol because it is not needed in systems without right context.

Parameters

[in]	nonterm_phones_offset	The integer id of the symbol #nonterm_bos in the phones.txt file. You can just set this to a large value (like 1 million) if you are not actually using nonterminals (e.g. for testing purposes).
[in]	phones	List of integer ids of phones, as you would see in phones.txt
[in]	disambig_syms	List of integer ids of disambiguation symbols, e.g. the ids of #0, #1, #2 in phones.txt

See The ContextFst object for more details.

Definition at line 27 of file grammar-context-fst.cc.

References ConstIntegerSet< I >::empty(), InverseLeftBiphoneContextFst::FindLabel(), rnnlm::i, KALDI_ASSERT, KALDI_ERR, KALDI_WARN, InverseLeftBiphoneContextFst::phone_syms_, and kaldi::SortAndUniq().

                                        :
     nonterm_phones_offset_(nonterm_phones_offset),
     phone_syms_(phones),
     disambig_syms_(disambig_syms) {
 
   { // This block does some checks.
     std::vector<int32> all_inputs(phones);
     all_inputs.insert(all_inputs.end(), disambig_syms.begin(),
                       disambig_syms.end());
     all_inputs.push_back(nonterm_phones_offset);
     size_t size = all_inputs.size();
     kaldi::SortAndUniq(&all_inputs);
     if (all_inputs.size() != size) {
       KALDI_ERR << "There was overlap between disambig symbols, phones, "
           "and/or --nonterm-phones-offset";
     }
     if (all_inputs.front() <= 0)
       KALDI_ERR << "Symbols <= 0 were passed in as phones, disambig-syms, "
           "or nonterm-phones-offset.";
     if (all_inputs.back() != nonterm_phones_offset) {
       // the value passed --nonterm-phones-offset is not higher numbered
       // than all the phones and disambig syms... do some more checking.
       for (int32 i = 1; i < 4; i++) {
         int32 symbol = nonterm_phones_offset + i;
         // None of the symbols --nonterm-phones-offset + {kNontermBos, kNontermBegin,
         //                  kNontermEnd, kNontermReenter, kNontermUserDefined}
         // (i.e. the special symbols plus the first user-defined symbol) may be
         // listed as phones or disambig symbols... this doesn't make sense.  We
         // do allow disambig symbols to be higher-numbered than the nonterminal
         // sybols, just in case that happens to be needed, but they can't overlap.
         if (std::binary_search(all_inputs.begin(), all_inputs.end(), symbol)) {
           KALDI_ERR << "The symbol " << symbol
                     << " = --nonterm-phones-offset + " << i
                     << " was listed as a phone or disambig symbol.";
         }
       }
     }
     if (phone_syms_.empty())
       KALDI_WARN << "Context FST created but there are no phone symbols: probably "
           "input FST was empty.";
   }
 
   // empty vector, will be the ilabel_info vector that corresponds to epsilon,
   // in case our FST needs to output epsilons.
   vector<int32> empty_vec;
   Label epsilon_label = FindLabel(empty_vec);
   // Make sure that a label is assigned for epsilon.
   KALDI_ASSERT(epsilon_label == 0);
 }

◆ ~InverseLeftBiphoneContextFst()

~InverseLeftBiphoneContextFst ( )

inline

Definition at line 218 of file grammar-context-fst.h.

218 { }

Member Function Documentation

◆ Final()

InverseLeftBiphoneContextFst::Weight Final ( StateId s )

virtual

Definition at line 81 of file grammar-context-fst.cc.

References ConstIntegerSet< I >::count(), InverseLeftBiphoneContextFst::GetPhoneSymbolFor(), fst::kNontermEnd, and InverseLeftBiphoneContextFst::phone_syms_.

Referenced by InverseLeftBiphoneContextFst::Start().

                                                                                 {
   if (s == 0 || phone_syms_.count(s) != 0 ||
       s == GetPhoneSymbolFor(kNontermEnd))
     return Weight::One();
   else
     return Weight::Zero();
 }

◆ FindLabel()

StdArc::Label FindLabel ( const std::vector< int32 > & label_info )

private

Finds the label index corresponding to this context-window of phones (likely of width context_width_).

Inserts it into the ilabel_info_/ilabel_map_ tables if necessary.

Definition at line 181 of file grammar-context-fst.cc.

References InverseLeftBiphoneContextFst::ilabel_info_, and InverseLeftBiphoneContextFst::ilabel_map_.

Referenced by InverseLeftBiphoneContextFst::GetArc(), InverseLeftBiphoneContextFst::GetPhoneSymbolFor(), and InverseLeftBiphoneContextFst::InverseLeftBiphoneContextFst().

                                                                                   {
   // Finds the ilabel corresponding to this vector (creates a new ilabel if
   // necessary).
   VectorToLabelMap::const_iterator iter = ilabel_map_.find(label_vec);
   if (iter == ilabel_map_.end()) {  // Not already in map.
     Label this_label = ilabel_info_.size();
     ilabel_info_.push_back(label_vec);
     ilabel_map_[label_vec] = this_label;
     return this_label;
   } else {
     return iter->second;
   }
 }

◆ GetArc()

bool GetArc	(	StateId	s,
		Label	ilabel,
		Arc *	arc
	)

virtual

Note: ilabel must not be epsilon.

Definition at line 89 of file grammar-context-fst.cc.

References ConstIntegerSet< I >::count(), InverseLeftBiphoneContextFst::disambig_syms_, InverseLeftBiphoneContextFst::FindLabel(), InverseLeftBiphoneContextFst::GetPhoneSymbolFor(), KALDI_ASSERT, KALDI_ERR, fst::kNontermBegin, fst::kNontermBos, fst::kNontermEnd, fst::kNontermReenter, fst::kNontermUserDefined, and InverseLeftBiphoneContextFst::phone_syms_.

Referenced by InverseLeftBiphoneContextFst::Start().

                                        {
   // it's a rule of the DeterministicOnDemandFst that the ilabel cannot be zero.q
   KALDI_ASSERT(ilabel != 0);
 
   arc->ilabel = ilabel;
   arc->weight = Weight::One();
 
   if (s == 0 || phone_syms_.count(s) != 0) {
     // This is an epsilon or phone state.
     if (phone_syms_.count(ilabel) != 0) {
       // The ilabel is a phone.
       std::vector<int32> context_window(2);
       context_window[0] = s;
       context_window[1] = ilabel;
       arc->olabel = FindLabel(context_window);
       arc->nextstate = ilabel;
       return true;
     } else if (disambig_syms_.count(ilabel) != 0) {
       // the ilabel is a disambiguation symbol.  Make a self-loop arc that
       // replicates the disambiguation symbol on the input.
       // The ilabel-info vector for disambig symbols is just a single element
       // consisting of the negative of the disambig symbols (for easier
       // identification from code).
       std::vector<int32> this_ilabel_info(1);
       this_ilabel_info[0] = -ilabel;
       arc->olabel = FindLabel(this_ilabel_info);
       arc->nextstate = s;
       return true;
     } else if (ilabel == GetPhoneSymbolFor(kNontermBegin) &&
                s == 0) {
       // We were at the start state and saw the symbol #nonterm_begin.
       // Output nothing, but transition to the special #nonterm_begin state.
       // when we're in that state, arcs for phones generate special
       // osymbols corresponding to pairs like (#nonterm_begin, p1).
       arc->olabel = 0;
       arc->nextstate = GetPhoneSymbolFor(kNontermBegin);
       return true;
     } else if (ilabel == GetPhoneSymbolFor(kNontermEnd)) {
       // we saw #nonterm_end.
       std::vector<int32> this_ilabel_info(2);
       this_ilabel_info[0] = -(GetPhoneSymbolFor(kNontermEnd));
       this_ilabel_info[1] = (s != 0 ? s : GetPhoneSymbolFor(kNontermBos));
       arc->olabel = FindLabel(this_ilabel_info);
       arc->nextstate = GetPhoneSymbolFor(kNontermEnd);
       return true;
     } else if (ilabel >= GetPhoneSymbolFor(kNontermUserDefined)) {
       // Assume this ilabel is a user-defined nonterminal.
       // Transition to the state kNontermUserDefined, with an olabel
       // (#nonterm:foo, p1) where 'p1' is the current left-context.
       std::vector<int32> this_ilabel_info(2);
       this_ilabel_info[0] = -ilabel;
       this_ilabel_info[1] = (s != 0 ? s : GetPhoneSymbolFor(kNontermBos));
       arc->olabel = FindLabel(this_ilabel_info);
       // the destination state is not specific to this user-defined symbol, it's
       // a generic destination state.
       arc->nextstate = GetPhoneSymbolFor(kNontermUserDefined);
       return true;
     } else {
       return false;
     }
   } else if (s == GetPhoneSymbolFor(kNontermBegin)) {
     if (phone_syms_.count(ilabel) != 0 || ilabel == GetPhoneSymbolFor(kNontermBos)) {
       std::vector<int32> this_ilabel_info(2);
       this_ilabel_info[0] = -GetPhoneSymbolFor(kNontermBegin);
       this_ilabel_info[1] = ilabel;
       arc->nextstate = (ilabel == GetPhoneSymbolFor(kNontermBos) ? 0 : ilabel);
       arc->olabel = FindLabel(this_ilabel_info);
       return true;
     } else {
       return false;
     }
   } else if (s == GetPhoneSymbolFor(kNontermEnd)) {
     return false;
   } else if (s == GetPhoneSymbolFor(kNontermUserDefined)) {
     if (phone_syms_.count(ilabel) != 0 || ilabel == GetPhoneSymbolFor(kNontermBos)) {
       std::vector<int32> this_ilabel_info(2);
       this_ilabel_info[0] = -GetPhoneSymbolFor(kNontermReenter);
       this_ilabel_info[1] = ilabel;
       arc->nextstate = (ilabel == GetPhoneSymbolFor(kNontermBos) ? 0 : ilabel);
       arc->olabel = FindLabel(this_ilabel_info);
       return true;
     } else {
       return false;
     }
   } else {
     // likely code error.
     KALDI_ERR << "Invalid state encountered";
     return false;  // won't get here.  suppress compiler error.
   }
 }

◆ GetPhoneSymbolFor()

int32 GetPhoneSymbolFor ( enum NonterminalValues n )

inlineprivate

Definition at line 234 of file grammar-context-fst.h.

References InverseLeftBiphoneContextFst::FindLabel(), rnnlm::n, and InverseLeftBiphoneContextFst::nonterm_phones_offset_.

Referenced by InverseLeftBiphoneContextFst::Final(), and InverseLeftBiphoneContextFst::GetArc().

                                                            {
     return nonterm_phones_offset_ + static_cast<int32>(n);
   }

◆ IlabelInfo()

const std::vector<std::vector<int32> >& IlabelInfo ( ) const

inline

Definition at line 224 of file grammar-context-fst.h.

References InverseLeftBiphoneContextFst::ilabel_info_.

                                                          {
     return ilabel_info_;
   }

◆ Start()

virtual StateId Start ( )

inlinevirtual

Here is a note on the state space of InverseLeftBiphoneContextFst; see Special symbols in C.fst which has some documentation on this.

The state space uses the same numbering as phones.txt.

State 0 means the beginning-of-sequence state, where there is no left context.

For each phone p in the list 'phones' passed to the constructor (i.e. in the set passed to the constructor), the state 'p' corresponds to a left-context of that phone.

If p is equal to nonterm_phones_offset_ + kNontermBegin (i.e. the integer form of `#nonterm_begin`), then this is the state we transition to when we see that symbol starting from left-context==0 (no context). The transition to this special state will have epsilon on the output. (talking here about inv(C), not C, so input/output are reversed). The state is nonfinal and when we see a regular phone p1 or #nonterm_bos, instead of outputting that phone in context, we output the pair (#nonterm_begin,p1) or (#nonterm_begin,#nonterm_bos). This state is not final.

If p is equal to nonterm_phones_offset_ + kNontermUserDefined, then this is the state we transition to when we see any user-defined nonterminal. Transitions to this special state have olabels of the form (#nonterm:foo,p1) where p1 is the preceding context (with #nonterm_begin if that context was 0); transitions out of it have olabels of the form (#nonterm_reenter,p2), where p2 is the phone on the ilabel of that transition. Again: talking about inv(C). This state is not final.

If p is equal to nonterm_phones_offset_ + kNontermEnd, then this is the state we transition to when we see the ilabel #nonterm_end. The olabels on the transitions to it (talking here about inv(C), so ilabels and olabels are reversed) are of the form (#nonterm_end, p1) where p1 corresponds to the context we were in. This state is final.

Implements DeterministicOnDemandFst< StdArc >.

Definition at line 211 of file grammar-context-fst.h.

References InverseLeftBiphoneContextFst::Final(), and InverseLeftBiphoneContextFst::GetArc().

211 { return 0; }