A class for estimating Maximum Likelihood Linear Transform, also known as global Semi-tied Covariance (STC), for GMMs. More...

#include <mllt.h>

Collaboration diagram for MlltAccs:

[legend]

Public Member Functions
	MlltAccs ()

	MlltAccs (int32 dim, BaseFloat rand_prune=0.25)
	Need rand_prune >= 0. More...

void	Init (int32 dim, BaseFloat rand_prune=0.25)
	initializes (destroys anything that was there before). More...

void	Read (std::istream &is, bool binary, bool add=false)

void	Write (std::ostream &os, bool binary) const

int32	Dim ()

void	Update (MatrixBase< BaseFloat > M, BaseFloat objf_impr_out, BaseFloat *count_out) const
	The Update function does the ML update; it requires that M has the right size. More...

void	AccumulateFromPosteriors (const DiagGmm &gmm, const VectorBase< BaseFloat > &data, const VectorBase< BaseFloat > &posteriors)

BaseFloat	AccumulateFromGmm (const DiagGmm &gmm, const VectorBase< BaseFloat > &data, BaseFloat weight)

BaseFloat	AccumulateFromGmmPreselect (const DiagGmm &gmm, const std::vector< int32 > &gselect, const VectorBase< BaseFloat > &data, BaseFloat weight)

Static Public Member Functions
static void	Update (double beta, const std::vector< SpMatrix< double > > &G, MatrixBase< BaseFloat > M, BaseFloat objf_impr_out, BaseFloat *count_out)

Public Attributes
BaseFloat	rand_prune_
	rand_prune_ controls randomized pruning; the larger it is, the more pruning we do. More...

double	beta_

std::vector< SpMatrix< double > >	G_

Detailed Description

A class for estimating Maximum Likelihood Linear Transform, also known as global Semi-tied Covariance (STC), for GMMs.

The resulting transform left-multiplies the feature vector.

Definition at line 42 of file mllt.h.

Constructor & Destructor Documentation

◆ MlltAccs() [1/2]

MlltAccs ( )

inline

Definition at line 44 of file mllt.h.

44 : rand_prune_(0.0), beta_(0.0) { }

kaldi::MlltAccs::beta_

double beta_

Definition: mllt.h:108

kaldi::MlltAccs::rand_prune_

BaseFloat rand_prune_

rand_prune_ controls randomized pruning; the larger it is, the more pruning we do.

Definition: mllt.h:107

◆ MlltAccs() [2/2]

MlltAccs	(	int32	dim,
		BaseFloat	rand_prune = `0.25`
	)

inline

Need rand_prune >= 0.

The larger it is, the faster it will be. Zero is exact. If a posterior p < rand_prune, will set p to rand_prune with probability (p/rand_prune), otherwise zero. E.g. 10 will give 10x speedup.

Definition at line 51 of file mllt.h.

References MlltAccs::Init(), MlltAccs::Read(), and MlltAccs::Write().

51 { Init(dim, rand_prune); }

kaldi::MlltAccs::Init

void Init(int32 dim, BaseFloat rand_prune=0.25)

initializes (destroys anything that was there before).

Definition: mllt.cc:25

Member Function Documentation

◆ AccumulateFromGmm()

BaseFloat AccumulateFromGmm	(	const DiagGmm &	gmm,
		const VectorBase< BaseFloat > &	data,
		BaseFloat	weight
	)

Definition at line 162 of file mllt.cc.

References MlltAccs::AccumulateFromPosteriors(), DiagGmm::ComponentPosteriors(), and DiagGmm::NumGauss().

Referenced by MlltAccs::Update().

                                                         {  // e.g. weight = 1.0
   Vector<BaseFloat> posteriors(gmm.NumGauss());
   BaseFloat ans = gmm.ComponentPosteriors(data, &posteriors);
   posteriors.Scale(weight);
   AccumulateFromPosteriors(gmm, data, posteriors);
   return ans;
 }

◆ AccumulateFromGmmPreselect()

BaseFloat AccumulateFromGmmPreselect	(	const DiagGmm &	gmm,
		const std::vector< int32 > &	gselect,
		const VectorBase< BaseFloat > &	data,
		BaseFloat	weight
	)

Definition at line 173 of file mllt.cc.

References MlltAccs::AccumulateFromPosteriors(), rnnlm::i, KALDI_ASSERT, DiagGmm::LogLikelihoodsPreselect(), and DiagGmm::NumGauss().

Referenced by MlltAccs::Update().

                       {  // e.g. weight = 1.0
   KALDI_ASSERT(!gselect.empty());
   Vector<BaseFloat> loglikes;
   gmm.LogLikelihoodsPreselect(data, gselect, &loglikes);
   BaseFloat loglike = loglikes.ApplySoftMax();
   // now "loglikes" is a vector of posteriors, indexed
   // by the same index as gselect.
   Vector<BaseFloat> posteriors(gmm.NumGauss());
   for (size_t i = 0; i < gselect.size(); i++)
     posteriors(gselect[i]) = loglikes(i) * weight;
   AccumulateFromPosteriors(gmm, data, posteriors);
   return loglike;
 }

◆ AccumulateFromPosteriors()

void AccumulateFromPosteriors	(	const DiagGmm &	gmm,
		const VectorBase< BaseFloat > &	data,
		const VectorBase< BaseFloat > &	posteriors
	)

Definition at line 131 of file mllt.cc.

References MlltAccs::beta_, MlltAccs::Dim(), VectorBase< Real >::Dim(), DiagGmm::Dim(), MlltAccs::G_, rnnlm::i, DiagGmm::inv_vars(), rnnlm::j, KALDI_ASSERT, DiagGmm::means_invvars(), DiagGmm::NumGauss(), MlltAccs::rand_prune_, and kaldi::RandPrune().

Referenced by MlltAccs::AccumulateFromGmm(), MlltAccs::AccumulateFromGmmPreselect(), and MlltAccs::Update().

                                                                                  {
   KALDI_ASSERT(data.Dim() == gmm.Dim());
   KALDI_ASSERT(data.Dim() == Dim());
   KALDI_ASSERT(posteriors.Dim() == gmm.NumGauss());
   const Matrix<BaseFloat> &means_invvars = gmm.means_invvars();
   const Matrix<BaseFloat> &inv_vars = gmm.inv_vars();
   Vector<BaseFloat> mean(data.Dim());
   SpMatrix<double> tmp(data.Dim());
   Vector<double> offset_dbl(data.Dim());
   double this_beta_ = 0.0;
   KALDI_ASSERT(rand_prune_ >= 0.0);
   for (int32 i = 0; i < posteriors.Dim(); i++) {  // for each mixcomp..
     BaseFloat posterior = RandPrune(posteriors(i), rand_prune_);
     if (posterior == 0.0) continue;
     SubVector<BaseFloat> mean_invvar(means_invvars, i);
     SubVector<BaseFloat> inv_var(inv_vars, i);
     mean.AddVecDivVec(1.0, mean_invvar, inv_var, 0.0);  // get mean.
     mean.AddVec(-1.0, data);  // get offset
     offset_dbl.CopyFromVec(mean);  // make it double.
     tmp.SetZero();
     tmp.AddVec2(1.0, offset_dbl);
     for (int32 j = 0; j < data.Dim(); j++)
       G_[j].AddSp(inv_var(j)*posterior, tmp);
     this_beta_ += posterior;
   }
   beta_ += this_beta_;
   Vector<double> data_dbl(data);
 }

◆ Dim()

int32 Dim ( )

inline

Definition at line 60 of file mllt.h.

References MlltAccs::G_.

Referenced by MlltAccs::AccumulateFromPosteriors(), and main().

60 { return G_.size(); }; // returns model dimension.

kaldi::MlltAccs::G_

std::vector< SpMatrix< double > > G_

Definition: mllt.h:109

◆ Init()

void Init	(	int32	dim,
		BaseFloat	rand_prune = `0.25`
	)

initializes (destroys anything that was there before).

Definition at line 25 of file mllt.cc.

References MlltAccs::beta_, MlltAccs::G_, rnnlm::i, KALDI_ASSERT, and MlltAccs::rand_prune_.

Referenced by MlltAccs::MlltAccs().

                                                    {  // initializes (destroys anything that was there before).
   KALDI_ASSERT(dim > 0);
   beta_ = 0;
   rand_prune_ = rand_prune;
   G_.resize(dim);
   for (int32 i = 0; i < dim; i++)
     G_[i].Resize(dim);  // will zero it too.
 }

◆ Read()

void Read	(	std::istream &	is,
		bool	binary,
		bool	add = `false`
	)

Definition at line 34 of file mllt.cc.

References MlltAccs::beta_, kaldi::ExpectToken(), MlltAccs::G_, rnnlm::i, KALDI_ERR, and kaldi::ReadBasicType().

Referenced by main(), and MlltAccs::MlltAccs().

                                                          {
   ExpectToken(is, binary, "<MlltAccs>");
   double beta;
   int32 dim;
   ReadBasicType(is, binary, &beta);
   if (!add) beta_ = beta;
   else beta_ += beta;
   ReadBasicType(is, binary, &dim);
   if (add && G_.size() != 0 && static_cast<size_t>(dim) != G_.size())
     KALDI_ERR << "MlltAccs::Read, summing accs of different size.";
   if (!add || G_.empty()) G_.resize(dim);
   ExpectToken(is, binary, "<G>");
   for (size_t i = 0; i < G_.size(); i++)
     G_[i].Read(is, binary, add);
   ExpectToken(is, binary, "</MlltAccs>");
 }

◆ Update() [1/2]

void Update	(	MatrixBase< BaseFloat > *	M,
		BaseFloat *	objf_impr_out,
		BaseFloat *	count_out
	)		const

inline

The Update function does the ML update; it requires that M has the right size.

Parameters

[in,out]	M	The output transform, will be of dimension Dim() x Dim(). At input, should be the unit transform (the objective function improvement is measured relative to this value).
[out]	objf_impr_out	The objective function improvement
[out]	count_out	The data-count

Definition at line 69 of file mllt.h.

References MlltAccs::AccumulateFromGmm(), MlltAccs::AccumulateFromGmmPreselect(), MlltAccs::AccumulateFromPosteriors(), MlltAccs::beta_, and MlltAccs::G_.

Referenced by main().

                                           {
     Update(beta_, G_, M, objf_impr_out, count_out);
   }

◆ Update() [2/2]

void Update	(	double	beta,
		const std::vector< SpMatrix< double > > &	G,
		MatrixBase< BaseFloat > *	M,
		BaseFloat *	objf_impr_out,
		BaseFloat *	count_out
	)

static

Definition at line 66 of file mllt.cc.

References VectorBase< Real >::AddSpVec(), MatrixBase< Real >::CopyFromMat(), rnnlm::i, MatrixBase< Real >::Invert(), KALDI_ASSERT, KALDI_ERR, KALDI_LOG, KALDI_WARN, kaldi::Log(), MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), Matrix< Real >::Transpose(), kaldi::VecSpVec(), and kaldi::VecVec().

                                             {
   int32 dim = G.size();
   KALDI_ASSERT(dim != 0 && M_ptr != NULL
                && M_ptr->NumRows() == dim
                && M_ptr->NumCols() == dim);
   if (beta < 10*dim) {  // not really enough data to estimate.
     // don't bother with min-count parameter etc., as MLLT is typically
     // global.
     if (beta > 2*dim)
       KALDI_WARN << "Mllt:Update, very small count " << beta;
     else
       KALDI_WARN << "Mllt:Update, insufficient count " << beta;
   }
   int32 num_iters = 200;  // may later make this an option.
   Matrix<double> M(dim, dim), Minv(dim, dim);
   M.CopyFromMat(*M_ptr);
   std::vector<SpMatrix<double> > Ginv(dim);
   for (int32 i = 0; i < dim;  i++) {
     Ginv[i].Resize(dim);
     Ginv[i].CopyFromSp(G[i]);
     Ginv[i].Invert();
   }
 
   double tot_objf_impr = 0.0;
   for (int32 p = 0; p < num_iters; p++) {
     for (int32 i = 0; i < dim; i++) {  // for each row
       SubVector<double> row(M, i);
       // work out cofactor (actually cofactor times a constant which
       // doesn't affect anything):
       Minv.CopyFromMat(M);
       Minv.Invert();
       Minv.Transpose();
       SubVector<double> cofactor(Minv, i);
       // Objf is: beta log(|row . cofactor|) -0.5 row^T G[i] row
       // optimized by (c.f. Mark Gales's techreport "semitied covariance matrices
       // for hidden markov models, eq.  (22)),
       // row = G_i^{-1} cofactor sqrt(beta / cofactor^T G_i^{-1} cofactor). (1)
       // here, "row" and "cofactor" are considered as column vectors.
       double objf_before = beta * Log(std::abs(VecVec(row, cofactor)))
           -0.5 * VecSpVec(row, G[i], row);
       // do eq. (1) above:
       row.AddSpVec(std::sqrt(beta / VecSpVec(cofactor, Ginv[i], cofactor)),
                    Ginv[i], cofactor, 0.0);
       double objf_after = beta * Log(std::abs(VecVec(row, cofactor)))
           -0.5 * VecSpVec(row, G[i], row);
       if (objf_after < objf_before - fabs(objf_before)*0.00001)
         KALDI_ERR << "Objective decrease in MLLT update.";
       tot_objf_impr += objf_after - objf_before;
     }
     if (p < 10 || p % 10 == 0)
       KALDI_LOG << "MLLT objective improvement per frame by " << p
                 << "'th iteration is " << (tot_objf_impr/beta) << " per frame "
                 << "over " << beta << " frames.";
   }
   if (objf_impr_out)
     *objf_impr_out = tot_objf_impr;
   if (count_out)
     *count_out = beta;
   M_ptr->CopyFromMat(M);
 }

◆ Write()

void Write	(	std::ostream &	os,
		bool	binary
	)		const

Definition at line 51 of file mllt.cc.

References MlltAccs::beta_, MlltAccs::G_, rnnlm::i, kaldi::WriteBasicType(), and kaldi::WriteToken().

Referenced by main(), and MlltAccs::MlltAccs().

                                                       {
   WriteToken(os, binary, "<MlltAccs>");
   if(!binary) os << '\n';
   WriteBasicType(os, binary, beta_);
   int32 dim = G_.size();
   WriteBasicType(os, binary, dim);
   WriteToken(os, binary, "<G>");
   if(!binary) os << '\n';
   for (size_t i = 0; i < G_.size(); i++)
     G_[i].Write(os, binary);
   WriteToken(os, binary, "</MlltAccs>");
   if(!binary) os << '\n';
 }

Member Data Documentation

◆ beta_

double beta_

Definition at line 108 of file mllt.h.

Referenced by MlltAccs::AccumulateFromPosteriors(), MlltAccs::Init(), MlltAccs::Read(), MlltAccs::Update(), and MlltAccs::Write().

◆ G_

std::vector<SpMatrix<double> > G_

Definition at line 109 of file mllt.h.

Referenced by MlltAccs::AccumulateFromPosteriors(), MlltAccs::Dim(), MlltAccs::Init(), MlltAccs::Read(), MlltAccs::Update(), and MlltAccs::Write().

◆ rand_prune_

BaseFloat rand_prune_

rand_prune_ controls randomized pruning; the larger it is, the more pruning we do.

Typical value is 0.1.

Definition at line 107 of file mllt.h.

Referenced by MlltAccs::AccumulateFromPosteriors(), and MlltAccs::Init().

The documentation for this class was generated from the following files:

transform/mllt.h
transform/mllt.cc

Public Member Functions

Static Public Member Functions

Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ MlltAccs() [1/2]

◆ MlltAccs() [2/2]

Member Function Documentation

◆ AccumulateFromGmm()

◆ AccumulateFromGmmPreselect()

◆ AccumulateFromPosteriors()

◆ Dim()

◆ Init()

◆ Read()

◆ Update() [1/2]

◆ Update() [2/2]

◆ Write()

Member Data Documentation

◆ beta_

◆ G_

◆ rand_prune_