IvectorExtractorStats Class Reference

IvectorExtractorStats is a class used to update the parameters of the ivector extractor. More...

#include <ivector-extractor.h>

Collaboration diagram for IvectorExtractorStats:

Public Member Functions
	IvectorExtractorStats ()
	IvectorExtractorStats (const IvectorExtractor &extractor, const IvectorExtractorStatsOptions &stats_opts)
void	Add (const IvectorExtractorStats &other)
void	AccStatsForUtterance (const IvectorExtractor &extractor, const MatrixBase< BaseFloat > &feats, const Posterior &post)
double	AccStatsForUtterance (const IvectorExtractor &extractor, const MatrixBase< BaseFloat > &feats, const FullGmm &fgmm)
void	Read (std::istream &is, bool binary, bool add=false)
void	Write (std::ostream &os, bool binary)
void	Write (std::ostream &os, bool binary) const
double	Update (const IvectorExtractorEstimationOptions &opts, IvectorExtractor *extractor) const
	Returns the objf improvement per frame. More...
double	AuxfPerFrame ()
void	IvectorVarianceDiagnostic (const IvectorExtractor &extractor)
	Prints the proportion of the variance explained by the Ivector model versus the Gaussians. More...
	IvectorExtractorStats (const IvectorExtractorStats &other)

Protected Member Functions
void	CommitStatsForUtterance (const IvectorExtractor &extractor, const IvectorExtractorUtteranceStats &utt_stats)
void	CommitStatsForM (const IvectorExtractor &extractor, const IvectorExtractorUtteranceStats &utt_stats, const VectorBase< double > &ivec_mean, const SpMatrix< double > &ivec_var)
	This is called by CommitStatsForUtterance. More...
void	FlushCache ()
	Flushes the cache for the R_ stats. More...
void	CommitStatsForSigma (const IvectorExtractor &extractor, const IvectorExtractorUtteranceStats &utt_stats)
	Commit the stats used to update the variance. More...
void	CommitStatsForWPoint (const IvectorExtractor &extractor, const IvectorExtractorUtteranceStats &utt_stats, const VectorBase< double > &ivector, double weight)
	Commit the stats used to update the weight-projection w_– this one takes a point sample, it's called from CommitStatsForW(). More...
void	CommitStatsForW (const IvectorExtractor &extractor, const IvectorExtractorUtteranceStats &utt_stats, const VectorBase< double > &ivec_mean, const SpMatrix< double > &ivec_var)
	Commit the stats used to update the weight-projection w_. More...
void	CommitStatsForPrior (const VectorBase< double > &ivec_mean, const SpMatrix< double > &ivec_var)
	Commit the stats used to update the prior distribution. More...
double	UpdateProjections (const IvectorExtractorEstimationOptions &opts, IvectorExtractor *extractor) const
double	UpdateProjection (const IvectorExtractorEstimationOptions &opts, int32 gaussian, IvectorExtractor *extractor) const
double	UpdateWeights (const IvectorExtractorEstimationOptions &opts, IvectorExtractor *extractor) const
double	UpdateWeight (const IvectorExtractorEstimationOptions &opts, int32 gaussian, IvectorExtractor *extractor) const
double	UpdateVariances (const IvectorExtractorEstimationOptions &opts, IvectorExtractor *extractor) const
double	UpdatePrior (const IvectorExtractorEstimationOptions &opts, IvectorExtractor *extractor) const
double	PriorDiagnostics (double old_prior_offset) const
void	CheckDims (const IvectorExtractor &extractor) const

Protected Attributes
IvectorExtractorStatsOptions	config_
double	tot_auxf_
	Caution: if we read from disk, this. More...
std::mutex	gamma_Y_lock_
	This mutex guards gamma_ and Y_ (for multi-threaded update) More...
Vector< double >	gamma_
	Total occupation count for each Gaussian index (zeroth-order stats) More...
std::vector< Matrix< double > >	Y_
	Stats Y_i for estimating projections M. More...
std::mutex	R_lock_
	This mutex guards R_ (for multi-threaded update) More...
Matrix< double >	R_
	R_i, quadratic term for ivector subspace (M matrix)estimation. More...
std::mutex	R_cache_lock_
	This mutex guards R_num_cached_, R_gamma_cache_, R_ivec_cache_ (for multi-threaded update) More...
int32	R_num_cached_
	To avoid too-frequent rank-1 update of R, which is slow, we cache some quantities here. More...
Matrix< double >	R_gamma_cache_
	dimension: [num-to-cache][I] More...
Matrix< double >	R_ivec_scatter_cache_
	dimension: [num-to-cache][S*(S+1)/2] More...
std::mutex	weight_stats_lock_
	This mutex guards Q_ and G_ (for multi-threaded update) More...
Matrix< double >	Q_
	Q_ is like R_ (with same dimensions), except used for weight estimation; the scatter of ivectors is weighted by the coefficient of the quadratic term in the expansion for w (the "safe" one, with the max expression). More...
Matrix< double >	G_
	G_ is the linear term in the weight projection matrix w_. More...
std::mutex	variance_stats_lock_
	This mutex guards S_ (for multi-threaded update) More...
std::vector< SpMatrix< double > >	S_
	S_{i}, raw second-order stats per Gaussian which we will use to update the variances Sigma_inv_. More...
std::mutex	prior_stats_lock_
	This mutex guards num_ivectors_, ivector_sum_ and ivector_scatter_ (for multi-threaded update) More...
double	num_ivectors_
	Count of the number of iVectors we trained on. More...
Vector< double >	ivector_sum_
	Sum of all the iVector means. Needed for prior re-estimation. More...
SpMatrix< double >	ivector_scatter_
	Second-order stats for the iVectors. Needed for prior re-estimation. More...

Private Member Functions
void	GetOrthogonalIvectorTransform (const SubMatrix< double > &T, IvectorExtractor *extractor, Matrix< double > *A) const
	Computes an orthogonal matrix A from the iVector transform T such that T' = A*T is an alternative transform which diagonalizes the quadratic_term_ in the iVector estimation objective function. More...
IvectorExtractorStats &	operator= (const IvectorExtractorStats &other)

Friends
class	IvectorExtractor
class	IvectorExtractorUpdateProjectionClass
class	IvectorExtractorUpdateWeightClass

Detailed Description

IvectorExtractorStats is a class used to update the parameters of the ivector extractor.

Definition at line 481 of file ivector-extractor.h.

Constructor & Destructor Documentation

IvectorExtractorStats() [1/3]

IvectorExtractorStats ( )

inline

Definition at line 485 of file ivector-extractor.h.

References kaldi::AccStatsForUtterance().

: tot_auxf_(0.0), R_num_cached_(0), num_ivectors_(0) { }

IvectorExtractorStats() [2/3]

IvectorExtractorStats	(	const IvectorExtractor &	extractor,
		const IvectorExtractorStatsOptions &	stats_opts
	)

Definition at line 891 of file ivector-extractor.cc.

References IvectorExtractorStatsOptions::cache_size, IvectorExtractorStats::config_, IvectorExtractor::FeatDim(), IvectorExtractorStats::G_, IvectorExtractorStats::gamma_, rnnlm::i, IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, IvectorExtractor::IvectorDependentWeights(), IvectorExtractor::IvectorDim(), KALDI_ASSERT, IvectorExtractorStats::num_ivectors_, IvectorExtractorStatsOptions::num_samples_for_weights, IvectorExtractor::NumGauss(), IvectorExtractorStats::Q_, IvectorExtractorStats::R_, IvectorExtractorStats::R_gamma_cache_, IvectorExtractorStats::R_ivec_scatter_cache_, IvectorExtractorStats::R_num_cached_, SpMatrix< Real >::Resize(), Vector< Real >::Resize(), Matrix< Real >::Resize(), IvectorExtractorStats::S_, IvectorExtractorStats::tot_auxf_, IvectorExtractorStatsOptions::update_variances, and IvectorExtractorStats::Y_.

                                                   :
    config_(stats_opts) {
  int32 S = extractor.IvectorDim(), D = extractor.FeatDim(),
      I = extractor.NumGauss();

  KALDI_ASSERT(config_.num_samples_for_weights > 1);
  tot_auxf_ = 0.0;
  gamma_.Resize(I);
  Y_.resize(I);
  for (int32 i = 0; i < I; i++)
    Y_[i].Resize(D, S);
  R_.Resize(I, S * (S + 1) / 2);
  R_num_cached_ = 0;
  KALDI_ASSERT(stats_opts.cache_size > 0 && "--cache-size=0 not allowed");

  R_gamma_cache_.Resize(stats_opts.cache_size, I);
  R_ivec_scatter_cache_.Resize(stats_opts.cache_size, S*(S+1)/2);

  if (extractor.IvectorDependentWeights()) {
    Q_.Resize(I, S * (S + 1) / 2);
    G_.Resize(I, S);
  }
  if (stats_opts.update_variances) {
    S_.resize(I);
    for (int32 i = 0; i < I; i++)
      S_[i].Resize(D);
  }
  num_ivectors_ = 0;
  ivector_sum_.Resize(S);
  ivector_scatter_.Resize(S);
}

IvectorExtractorStats() [3/3]

IvectorExtractorStats ( const IvectorExtractorStats &  other )

explicit

Definition at line 1766 of file ivector-extractor.cc.

                                       :
    config_(other.config_), tot_auxf_(other.tot_auxf_), gamma_(other.gamma_),
    Y_(other.Y_), R_(other.R_), R_num_cached_(other.R_num_cached_),
    R_gamma_cache_(other.R_gamma_cache_),
    R_ivec_scatter_cache_(other.R_ivec_scatter_cache_),
    Q_(other.Q_), G_(other.G_), S_(other.S_), num_ivectors_(other.num_ivectors_),
    ivector_sum_(other.ivector_sum_), ivector_scatter_(other.ivector_scatter_) {
}

Member Function Documentation

AccStatsForUtterance() [1/2]

void AccStatsForUtterance	(	const IvectorExtractor &	extractor,
		const MatrixBase< BaseFloat > &	feats,
		const Posterior &	post
	)

Definition at line 1132 of file ivector-extractor.cc.

References IvectorExtractorUtteranceStats::AccStats(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForUtterance(), IvectorExtractor::FeatDim(), KALDI_ASSERT, KALDI_ERR, MatrixBase< Real >::NumCols(), IvectorExtractor::NumGauss(), MatrixBase< Real >::NumRows(), and IvectorExtractorStats::S_.

Referenced by IvectorExtractorStats::AccStatsForUtterance(), IvectorTask::operator()(), and kaldi::UnitTestIvectorExtractor().

                           {
  typedef std::vector<std::pair<int32, BaseFloat> > VecType;

  CheckDims(extractor);

  int32 num_gauss = extractor.NumGauss(), feat_dim = extractor.FeatDim();

  if (feat_dim != feats.NumCols()) {
    KALDI_ERR << "Feature dimension mismatch, expected " << feat_dim
              << ", got " << feats.NumCols();
  }
  KALDI_ASSERT(static_cast<int32>(post.size()) == feats.NumRows());

  bool update_variance = (!S_.empty());

  // The zeroth and 1st-order stats are in "utt_stats".
  IvectorExtractorUtteranceStats utt_stats(num_gauss, feat_dim,
                                           update_variance);

  utt_stats.AccStats(feats, post);

  CommitStatsForUtterance(extractor, utt_stats);
}

AccStatsForUtterance() [2/2]

double AccStatsForUtterance	(	const IvectorExtractor &	extractor,
		const MatrixBase< BaseFloat > &	feats,
		const FullGmm &	fgmm
	)

Definition at line 1159 of file ivector-extractor.cc.

References IvectorExtractorStats::AccStatsForUtterance(), FullGmm::ComponentPosteriors(), rnnlm::i, kaldi::kUndefined, FullGmm::NumGauss(), and MatrixBase< Real >::NumRows().

                         {
  int32 num_frames = feats.NumRows();
  Posterior post(num_frames);

  double tot_log_like = 0.0;
  for (int32 t = 0; t < num_frames; t++) {
    SubVector<BaseFloat> frame(feats, t);
    Vector<BaseFloat> posterior(fgmm.NumGauss(), kUndefined);
    tot_log_like += fgmm.ComponentPosteriors(frame, &posterior);
    for (int32 i = 0; i < posterior.Dim(); i++)
      post[t].push_back(std::make_pair(i, posterior(i)));
  }
  AccStatsForUtterance(extractor, feats, post);
  return tot_log_like;
}

Add()

void Add

(

const IvectorExtractorStats &

other

)

Definition at line 1178 of file ivector-extractor.cc.

References MatrixBase< Real >::AddMat(), SpMatrix< Real >::AddSp(), VectorBase< Real >::AddVec(), IvectorExtractorStats::config_, IvectorExtractorStats::G_, IvectorExtractorStats::gamma_, rnnlm::i, IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, KALDI_ASSERT, IvectorExtractorStats::num_ivectors_, IvectorExtractorStatsOptions::num_samples_for_weights, IvectorExtractorStats::Q_, IvectorExtractorStats::R_, IvectorExtractorStats::S_, IvectorExtractorStats::tot_auxf_, and IvectorExtractorStats::Y_.

Referenced by kaldi::TestIvectorExtractorStatsIO().

                                                                  {
  KALDI_ASSERT(config_.num_samples_for_weights ==
               other.config_.num_samples_for_weights);
  double weight = 1.0; // will later make this configurable if needed.
  tot_auxf_ += weight * other.tot_auxf_;
  gamma_.AddVec(weight, other.gamma_);
  KALDI_ASSERT(Y_.size() == other.Y_.size());
  for (size_t i = 0; i < Y_.size(); i++)
    Y_[i].AddMat(weight, other.Y_[i]);
  R_.AddMat(weight, other.R_);
  Q_.AddMat(weight, other.Q_);
  G_.AddMat(weight, other.G_);
  KALDI_ASSERT(S_.size() == other.S_.size());
  for (size_t i = 0; i < S_.size(); i++)
    S_[i].AddSp(weight, other.S_[i]);
  num_ivectors_ += weight * other.num_ivectors_;
  ivector_sum_.AddVec(weight, other.ivector_sum_);
  ivector_scatter_.AddSp(weight, other.ivector_scatter_);
}

AuxfPerFrame()

double AuxfPerFrame ( )

inline

Definition at line 515 of file ivector-extractor.h.

Referenced by kaldi::UnitTestIvectorExtractor().

{ return tot_auxf_ / gamma_.Sum(); }

CheckDims()

void CheckDims ( const IvectorExtractor &  extractor ) const

protected

Definition at line 1104 of file ivector-extractor.cc.

References IvectorExtractorStats::config_, VectorBase< Real >::Dim(), IvectorExtractor::FeatDim(), IvectorExtractorStats::G_, IvectorExtractorStats::gamma_, rnnlm::i, IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, IvectorExtractor::IvectorDependentWeights(), IvectorExtractor::IvectorDim(), KALDI_ASSERT, IvectorExtractorStats::num_ivectors_, IvectorExtractorStatsOptions::num_samples_for_weights, MatrixBase< Real >::NumCols(), IvectorExtractor::NumGauss(), MatrixBase< Real >::NumRows(), PackedMatrix< Real >::NumRows(), IvectorExtractorStats::Q_, IvectorExtractorStats::R_, IvectorExtractorStats::S_, and IvectorExtractorStats::Y_.

Referenced by IvectorExtractorStats::AccStatsForUtterance(), and IvectorExtractorStats::Update().

                                                                             {
  int32 S = extractor.IvectorDim(), D = extractor.FeatDim(),
      I = extractor.NumGauss();
  KALDI_ASSERT(config_.num_samples_for_weights > 0);
  KALDI_ASSERT(gamma_.Dim() == I);
  KALDI_ASSERT(static_cast<int32>(Y_.size()) == I);
  for (int32 i = 0; i < I; i++)
    KALDI_ASSERT(Y_[i].NumRows() == D && Y_[i].NumCols() == S);
  KALDI_ASSERT(R_.NumRows() == I && R_.NumCols() == S*(S+1)/2);
  if (extractor.IvectorDependentWeights()) {
    KALDI_ASSERT(Q_.NumRows() == I && Q_.NumCols() == S*(S+1)/2);
    KALDI_ASSERT(G_.NumRows() == I && G_.NumCols() == S);
  } else {
    KALDI_ASSERT(Q_.NumRows() == 0);
    KALDI_ASSERT(G_.NumRows() == 0);
  }
  // S_ may be empty or not, depending on whether update_variances == true in
  // the options.
  if (!S_.empty()) {
    KALDI_ASSERT(static_cast<int32>(S_.size() == I));
    for (int32 i = 0; i < I; i++)
      KALDI_ASSERT(S_[i].NumRows() == D);
  }
  KALDI_ASSERT(num_ivectors_ >= 0);
  KALDI_ASSERT(ivector_sum_.Dim() == S);
  KALDI_ASSERT(ivector_scatter_.NumRows() == S);
}

CommitStatsForM()

void CommitStatsForM ( const IvectorExtractor &  extractor, const IvectorExtractorUtteranceStats &  utt_stats, const VectorBase< double > &  ivec_mean, const SpMatrix< double > &  ivec_var  )

protected

This is called by CommitStatsForUtterance.

We commit the stats used to update the M matrix.

Definition at line 926 of file ivector-extractor.cc.

References VectorBase< Real >::AddVec(), SpMatrix< Real >::AddVec2(), PackedMatrix< Real >::Data(), VectorBase< Real >::Dim(), IvectorExtractorStats::FlushCache(), IvectorExtractorUtteranceStats::gamma_, IvectorExtractorStats::gamma_, IvectorExtractorStats::gamma_Y_lock_, rnnlm::i, IvectorExtractor::NumGauss(), MatrixBase< Real >::NumRows(), IvectorExtractorStats::R_cache_lock_, IvectorExtractorStats::R_gamma_cache_, IvectorExtractorStats::R_ivec_scatter_cache_, IvectorExtractorStats::R_num_cached_, MatrixBase< Real >::Row(), IvectorExtractorUtteranceStats::X_, and IvectorExtractorStats::Y_.

Referenced by IvectorExtractorStats::CommitStatsForUtterance().

                                      {

  gamma_Y_lock_.lock();

  // We do the occupation stats here also.
  gamma_.AddVec(1.0, utt_stats.gamma_);

  // Stats for the linear term in M:
  for  (int32 i = 0; i < extractor.NumGauss(); i++) {
    Y_[i].AddVecVec(1.0, utt_stats.X_.Row(i),
                    Vector<double>(ivec_mean));
  }
  gamma_Y_lock_.unlock();

  SpMatrix<double> ivec_scatter(ivec_var);
  ivec_scatter.AddVec2(1.0, ivec_mean);

  R_cache_lock_.lock();
  while (R_num_cached_ == R_gamma_cache_.NumRows()) {
    // Cache full.  The "while" statement is in case of certain race conditions.
    R_cache_lock_.unlock();
    FlushCache();
    R_cache_lock_.lock();
  }
  R_gamma_cache_.Row(R_num_cached_).CopyFromVec(utt_stats.gamma_);
  int32 ivector_dim = ivec_mean.Dim();
  SubVector<double> ivec_scatter_vec(ivec_scatter.Data(),
                                     ivector_dim * (ivector_dim + 1) / 2);
  R_ivec_scatter_cache_.Row(R_num_cached_).CopyFromVec(ivec_scatter_vec);
  R_num_cached_++;
  R_cache_lock_.unlock();
}

CommitStatsForPrior()

void CommitStatsForPrior ( const VectorBase< double > &  ivec_mean, const SpMatrix< double > &  ivec_var  )

protected

Commit the stats used to update the prior distribution.

Definition at line 1067 of file ivector-extractor.cc.

References SpMatrix< Real >::AddSp(), VectorBase< Real >::AddVec(), SpMatrix< Real >::AddVec2(), IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, IvectorExtractorStats::num_ivectors_, and IvectorExtractorStats::prior_stats_lock_.

Referenced by IvectorExtractorStats::CommitStatsForUtterance().

                                      {
  SpMatrix<double> ivec_scatter(ivec_var);
  ivec_scatter.AddVec2(1.0, ivec_mean);
  prior_stats_lock_.lock();
  num_ivectors_ += 1.0;
  ivector_sum_.AddVec(1.0, ivec_mean);
  ivector_scatter_.AddSp(1.0, ivec_scatter);
  prior_stats_lock_.unlock();
}

CommitStatsForSigma()

void CommitStatsForSigma ( const IvectorExtractor &  extractor, const IvectorExtractorUtteranceStats &  utt_stats  )

protected

Commit the stats used to update the variance.

Definition at line 988 of file ivector-extractor.cc.

References rnnlm::i, IvectorExtractor::NumGauss(), IvectorExtractorUtteranceStats::S_, IvectorExtractorStats::S_, and IvectorExtractorStats::variance_stats_lock_.

Referenced by IvectorExtractorStats::CommitStatsForUtterance().

                                                     {
  variance_stats_lock_.lock();
  // Storing the raw scatter statistics per Gaussian.  In the update phase we'll
  // take into account some other terms relating to the model means and their
  // correlation with the data.
  for (int32 i = 0; i < extractor.NumGauss(); i++)
    S_[i].AddSp(1.0, utt_stats.S_[i]);
  variance_stats_lock_.unlock();
}

CommitStatsForUtterance()

void CommitStatsForUtterance ( const IvectorExtractor &  extractor, const IvectorExtractorUtteranceStats &  utt_stats  )

protected

Definition at line 1080 of file ivector-extractor.cc.

References IvectorExtractorStats::CommitStatsForM(), IvectorExtractorStats::CommitStatsForPrior(), IvectorExtractorStats::CommitStatsForSigma(), IvectorExtractorStats::CommitStatsForW(), IvectorExtractorStatsOptions::compute_auxf, IvectorExtractorStats::config_, IvectorExtractor::GetAuxf(), IvectorExtractor::GetIvectorDistribution(), IvectorExtractor::IvectorDependentWeights(), IvectorExtractor::IvectorDim(), IvectorExtractorStats::S_, and IvectorExtractorStats::tot_auxf_.

Referenced by IvectorExtractorStats::AccStatsForUtterance().

                                                     {

  int32 ivector_dim = extractor.IvectorDim();
  Vector<double> ivec_mean(ivector_dim);
  SpMatrix<double> ivec_var(ivector_dim);

  extractor.GetIvectorDistribution(utt_stats,
                                   &ivec_mean,
                                   &ivec_var);

  if (config_.compute_auxf)
    tot_auxf_ += extractor.GetAuxf(utt_stats, ivec_mean, &ivec_var);

  CommitStatsForM(extractor, utt_stats, ivec_mean, ivec_var);
  if (extractor.IvectorDependentWeights())
    CommitStatsForW(extractor, utt_stats, ivec_mean, ivec_var);
  CommitStatsForPrior(ivec_mean, ivec_var);
  if (!S_.empty())
    CommitStatsForSigma(extractor, utt_stats);
}

CommitStatsForW()

void CommitStatsForW ( const IvectorExtractor &  extractor, const IvectorExtractorUtteranceStats &  utt_stats, const VectorBase< double > &  ivec_mean, const SpMatrix< double > &  ivec_var  )

protected

Commit the stats used to update the weight-projection w_.

Definition at line 1038 of file ivector-extractor.cc.

References MatrixBase< Real >::AddMatTp(), VectorBase< Real >::AddRowSumMat(), TpMatrix< Real >::Cholesky(), IvectorExtractorStats::CommitStatsForWPoint(), IvectorExtractorStats::config_, IvectorExtractor::IvectorDim(), KALDI_ASSERT, kaldi::kNoTrans, kaldi::kTrans, IvectorExtractorStatsOptions::num_samples_for_weights, and MatrixBase< Real >::SetRandn().

Referenced by IvectorExtractorStats::CommitStatsForUtterance().

                                      {
  KALDI_ASSERT(config_.num_samples_for_weights > 1);

  Matrix<double> rand(config_.num_samples_for_weights, extractor.IvectorDim());
  rand.SetRandn();
  TpMatrix<double> ivec_stddev(extractor.IvectorDim());
  ivec_stddev.Cholesky(ivec_var);
  Matrix<double> ivecs(config_.num_samples_for_weights, extractor.IvectorDim());
  ivecs.AddMatTp(1.0, rand, kNoTrans, ivec_stddev, kTrans, 0.0);
  // Now make the ivecs zero-mean
  Vector<double> avg_ivec(extractor.IvectorDim());
  avg_ivec.AddRowSumMat(1.0 / config_.num_samples_for_weights, ivecs);
  ivecs.AddVecToRows(-1.0, avg_ivec);
  // Correct the variance for what we just did, so the expected
  // variance still has the correct value.
  ivecs.Scale(sqrt(config_.num_samples_for_weights / (config_.num_samples_for_weights - 1.0)));
  // Add the mean of the distribution to "ivecs".
  ivecs.AddVecToRows(1.0, ivec_mean);
  // "ivecs" is now a sample from the iVector distribution.
  for (int32 samp = 0; samp < config_.num_samples_for_weights; samp++)
    CommitStatsForWPoint(extractor, utt_stats,
                         ivecs.Row(samp),
                         1.0 / config_.num_samples_for_weights);
}

CommitStatsForWPoint()

void CommitStatsForWPoint ( const IvectorExtractor &  extractor, const IvectorExtractorUtteranceStats &  utt_stats, const VectorBase< double > &  ivector, double  weight  )

protected

Commit the stats used to update the weight-projection w_– this one takes a point sample, it's called from CommitStatsForW().

Definition at line 1003 of file ivector-extractor.cc.

References VectorBase< Real >::AddMatVec(), SpMatrix< Real >::AddVec2(), MatrixBase< Real >::AddVecVec(), VectorBase< Real >::ApplySoftMax(), PackedMatrix< Real >::Data(), IvectorExtractorStats::G_, IvectorExtractorUtteranceStats::gamma_, rnnlm::i, IvectorExtractor::IvectorDim(), kaldi::kNoTrans, IvectorExtractor::NumGauss(), IvectorExtractorStats::Q_, VectorBase< Real >::Sum(), IvectorExtractor::w_, and IvectorExtractorStats::weight_stats_lock_.

Referenced by IvectorExtractorStats::CommitStatsForW().

                   {
  int32 num_gauss = extractor.NumGauss();
  // Compare this function with GetIvectorDistWeight(), from which it
  // was derived.
  Vector<double> logw_unnorm(num_gauss);
  logw_unnorm.AddMatVec(1.0, extractor.w_, kNoTrans, ivector, 0.0);

  Vector<double> w(logw_unnorm);
  w.ApplySoftMax(); // now w is the weights.

  Vector<double> linear_coeff(num_gauss);
  Vector<double> quadratic_coeff(num_gauss);
  double gamma = utt_stats.gamma_.Sum();
  for (int32 i = 0; i < num_gauss; i++) {
    double gamma_i = utt_stats.gamma_(i);
    double max_term = std::max(gamma_i, gamma * w(i));
    linear_coeff(i) = gamma_i - gamma * w(i) + max_term * logw_unnorm(i);
    quadratic_coeff(i) = max_term;
  }
  weight_stats_lock_.lock();
  G_.AddVecVec(weight, linear_coeff, Vector<double>(ivector));

  int32 ivector_dim = extractor.IvectorDim();
  SpMatrix<double> outer_prod(ivector_dim);
  outer_prod.AddVec2(1.0, ivector);
  SubVector<double> outer_prod_vec(outer_prod.Data(),
                                   ivector_dim * (ivector_dim + 1) / 2);
  Q_.AddVecVec(weight, quadratic_coeff, outer_prod_vec);
  weight_stats_lock_.unlock();
}

FlushCache()

void FlushCache ( )

protected

Flushes the cache for the R_ stats.

Definition at line 963 of file ivector-extractor.cc.

References MatrixBase< Real >::AddMatMat(), KALDI_VLOG, kaldi::kNoTrans, kaldi::kTrans, MatrixBase< Real >::NumCols(), IvectorExtractorStats::R_, IvectorExtractorStats::R_cache_lock_, IvectorExtractorStats::R_gamma_cache_, IvectorExtractorStats::R_ivec_scatter_cache_, IvectorExtractorStats::R_lock_, IvectorExtractorStats::R_num_cached_, and MatrixBase< Real >::Range().

Referenced by IvectorExtractorStats::CommitStatsForM(), and IvectorExtractorStats::Write().

                                       {
  R_cache_lock_.lock();
  if (R_num_cached_ > 0) {
    KALDI_VLOG(1) << "Flushing cache for IvectorExtractorStats";
    // Store these quantities as copies in memory so other threads can use the
    // cache while we update R_ from the cache.
    Matrix<double> R_gamma_cache(
        R_gamma_cache_.Range(0, R_num_cached_,
                             0, R_gamma_cache_.NumCols()));
    Matrix<double> R_ivec_scatter_cache(
        R_ivec_scatter_cache_.Range(0, R_num_cached_,
                                    0, R_ivec_scatter_cache_.NumCols()));
    R_num_cached_ = 0; // As far as other threads are concerned, the cache is
                       // cleared and they may write to it.
    R_cache_lock_.unlock();
    R_lock_.lock();
    R_.AddMatMat(1.0, R_gamma_cache, kTrans,
                 R_ivec_scatter_cache, kNoTrans, 1.0);
    R_lock_.unlock();
  } else {
    R_cache_lock_.unlock();
  }
}

GetOrthogonalIvectorTransform()

void GetOrthogonalIvectorTransform ( const SubMatrix< double > &  T, IvectorExtractor *  extractor, Matrix< double > *  A  ) const

private

Computes an orthogonal matrix A from the iVector transform T such that T' = A*T is an alternative transform which diagonalizes the quadratic_term_ in the iVector estimation objective function.

This reorders the dimensions of the iVector from most to least important, which may be more convenient to view. The transform should not affect the performance of systems which use iVectors.

Definition at line 1353 of file ivector-extractor.cc.

References MatrixBase< Real >::AddMat(), VectorBase< Real >::AddMatVec(), IvectorExtractor::ComputeDerivedVars(), rnnlm::i, MatrixBase< Real >::Invert(), IvectorExtractor::IvectorDependentWeights(), IvectorExtractor::IvectorDim(), KALDI_LOG, kaldi::kNoTrans, kaldi::kTrans, MatrixBase< Real >::NumCols(), IvectorExtractor::NumGauss(), MatrixBase< Real >::NumRows(), Matrix< Real >::Resize(), MatrixBase< Real >::SetZero(), kaldi::SortSvd(), IvectorExtractor::U_, and IvectorExtractor::w_vec_.

Referenced by IvectorExtractorStats::UpdatePrior().

                                                       {
  extractor->ComputeDerivedVars(); // Update the extractor->U_ matrix.
  int32 ivector_dim = extractor->IvectorDim(),
        num_gauss = extractor->NumGauss();
  int32 quad_dim = ivector_dim*(ivector_dim + 1)/2;

  // Each row of extractor->U_ is an SpMatrix. We can compute the weighted
  // avg of these rows in a SubVector that updates the data of the SpMatrix
  // Uavg.
  SpMatrix<double> Uavg(ivector_dim), Vavg(ivector_dim - 1);
  SubVector<double> uavg_vec(Uavg.Data(), quad_dim);
  if (extractor->IvectorDependentWeights()) {
    Vector<double> w_uniform(num_gauss);
    for (int32 i = 0; i < num_gauss; i++) w_uniform(i) = 1.0;
    uavg_vec.AddMatVec(1.0/num_gauss, extractor->U_, kTrans, w_uniform, 0.0);
  } else {
    uavg_vec.AddMatVec(1.0, extractor->U_, kTrans, extractor->w_vec_, 0.0);
  }

  Matrix<double> Tinv(T);
  Tinv.Invert();
  Matrix<double> Vavg_temp(Vavg), Uavg_temp(Uavg);

  Vavg_temp.AddMatMatMat(1.0, Tinv, kTrans, SubMatrix<double>(Uavg_temp,
                           1, ivector_dim-1, 1, ivector_dim-1),
                         kNoTrans, Tinv, kNoTrans, 0.0);
  Vavg.CopyFromMat(Vavg_temp);

  Vector<double> s(ivector_dim-1);
  Matrix<double> P(ivector_dim-1, ivector_dim-1);
  Vavg.Eig(&s, &P);
  SortSvd(&s, &P);
  A->Resize(P.NumCols(), P.NumRows());
  A->SetZero();
  A->AddMat(1.0, P, kTrans);
  KALDI_LOG << "Eigenvalues of Vavg: " << s;
}

IvectorVarianceDiagnostic()

void IvectorVarianceDiagnostic

(

const IvectorExtractor &

extractor

)

Prints the proportion of the variance explained by the Ivector model versus the Gaussians.

Definition at line 1297 of file ivector-extractor.cc.

References IvectorExtractor::FeatDim(), IvectorExtractorStats::gamma_, rnnlm::i, IvectorExtractor::InvertWithFlooring(), KALDI_LOG, kaldi::kNoTrans, IvectorExtractor::M_, IvectorExtractor::NumGauss(), PackedMatrix< Real >::Scale(), IvectorExtractor::Sigma_inv_, and VectorBase< Real >::Sum().

Referenced by main().

                                     {

  // W is an estimate of the total residual variance explained by the
  // speaker-adapated model.  B is an estimate of the total variance
  // explained by the Ivector-subspace.
  SpMatrix<double> W(extractor.Sigma_inv_[0].NumRows()),
                      B(extractor.M_[0].NumRows());
  Vector<double> w(gamma_);
  w.Scale(1.0 / gamma_.Sum());
  for (int32 i = 0; i < extractor.NumGauss(); i++) {
    SpMatrix<double> Sigma_i(extractor.FeatDim());
    extractor.InvertWithFlooring(extractor.Sigma_inv_[i], &Sigma_i);
    W.AddSp(w(i), Sigma_i);
    B.AddMat2(w(i), extractor.M_[i], kNoTrans, 1.0);
  }
  double trace_W = W.Trace(),
         trace_B = B.Trace();
  KALDI_LOG << "The proportion of within-Gaussian variance explained by "
            << "the iVectors is " << trace_B / (trace_B + trace_W) << ".";
}

operator=()

IvectorExtractorStats& operator= ( const IvectorExtractorStats &  other )

private

PriorDiagnostics()

double PriorDiagnostics ( double  old_prior_offset ) const

protected

Definition at line 1613 of file ivector-extractor.cc.

References SpMatrix< Real >::AddVec2(), VectorBase< Real >::Dim(), IvectorExtractorStats::gamma_, IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, KALDI_LOG, SpMatrix< Real >::LogPosDefDet(), IvectorExtractorStats::num_ivectors_, PackedMatrix< Real >::Scale(), VectorBase< Real >::Scale(), VectorBase< Real >::Sum(), and SpMatrix< Real >::Trace().

Referenced by IvectorExtractorStats::UpdatePrior().

                                                                            {
  // The iVectors had a centered covariance "covar"; we want to figure out
  // the objective-function change from rescaling.  It's as if we were
  // formerly modeling "covar" with the unit matrix, and we're now modeling
  // it with "covar" itself.  This is ignoring flooring issues.  Of course,
  // we implement it through rescaling the space, but it has the same effect.
  // We also need to take into account that before the rescaling etc., the
  // old mean might have been wrong.

  int32 ivector_dim = ivector_sum_.Dim();
  Vector<double> sum(ivector_sum_);
  sum.Scale(1.0 / num_ivectors_);
  SpMatrix<double> covar(ivector_scatter_);
  covar.Scale(1.0 / num_ivectors_);
  covar.AddVec2(-1.0, sum); // Get the centered covariance.

  // Now work out the offset from the old prior's mean.
  Vector<double> mean_offset(sum);
  mean_offset(0) -= old_prior_offset;

  SpMatrix<double> old_covar(covar); // the covariance around the old mean.
  old_covar.AddVec2(1.0, mean_offset);
  // old likelihood = -0.5 * (Trace(I old_covar) + logdet(I) + [ignored])
  double old_like = -0.5 * old_covar.Trace();
  // new likelihood is if we updated the variance to equal "covar"... this isn't
  // how we did it (we use rescaling of the ivectors) but it has the same
  // effect.  -0.5 * (Trace(covar^{-1} covar)  + logdet(covar))
  double new_like = -0.5 * (ivector_dim + covar.LogPosDefDet()),
      like_change = new_like - old_like,
      like_change_per_frame = like_change * num_ivectors_ / gamma_.Sum();

  KALDI_LOG << "Overall auxf improvement from prior is " << like_change_per_frame
            << " per frame, or " << like_change << " per iVector.";
  return like_change_per_frame; // we'll be adding this to other per-frame
                                // quantities.
}

Read()

void Read	(	std::istream &	is,
		bool	binary,
		bool	add = false
	)

Definition at line 1240 of file ivector-extractor.cc.

References kaldi::ExpectToken(), IvectorExtractorStats::G_, IvectorExtractorStats::gamma_, rnnlm::i, IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, IvectorExtractorStats::num_ivectors_, IvectorExtractorStats::Q_, IvectorExtractorStats::R_, PackedMatrix< Real >::Read(), Vector< Real >::Read(), Matrix< Real >::Read(), kaldi::ReadBasicType(), IvectorExtractorStats::S_, IvectorExtractorStats::tot_auxf_, and IvectorExtractorStats::Y_.

Referenced by main(), and kaldi::TestIvectorExtractorStatsIO().

                                                                      {
  ExpectToken(is, binary, "<IvectorExtractorStats>");
  ExpectToken(is, binary, "<TotAuxf>");
  ReadBasicType(is, binary, &tot_auxf_, add);
  ExpectToken(is, binary, "<gamma>");
  gamma_.Read(is, binary, add);
  ExpectToken(is, binary, "<Y>");
  int32 size;
  ReadBasicType(is, binary, &size);
  Y_.resize(size);
  for (int32 i = 0; i < size; i++)
    Y_[i].Read(is, binary, add);
  ExpectToken(is, binary, "<R>");
  R_.Read(is, binary, add);
  ExpectToken(is, binary, "<Q>");
  Q_.Read(is, binary, add);
  ExpectToken(is, binary, "<G>");
  G_.Read(is, binary, add);
  ExpectToken(is, binary, "<S>");
  ReadBasicType(is, binary, &size);
  S_.resize(size);
  for (int32 i = 0; i < size; i++)
    S_[i].Read(is, binary, add);
  ExpectToken(is, binary, "<NumIvectors>");
  ReadBasicType(is, binary, &num_ivectors_, add);
  ExpectToken(is, binary, "<IvectorSum>");
  ivector_sum_.Read(is, binary, add);
  ExpectToken(is, binary, "<IvectorScatter>");
  ivector_scatter_.Read(is, binary, add);
  ExpectToken(is, binary, "</IvectorExtractorStats>");
}

Update()

double Update	(	const IvectorExtractorEstimationOptions &	opts,
		IvectorExtractor *	extractor
	)		const

Returns the objf improvement per frame.

Definition at line 1272 of file ivector-extractor.cc.

References IvectorExtractorStats::CheckDims(), IvectorExtractor::ComputeDerivedVars(), IvectorExtractorStats::gamma_, IvectorExtractor::IvectorDependentWeights(), KALDI_LOG, IvectorExtractorStats::S_, VectorBase< Real >::Sum(), IvectorExtractorStats::tot_auxf_, IvectorExtractorStats::UpdatePrior(), IvectorExtractorStats::UpdateProjections(), IvectorExtractorStats::UpdateVariances(), and IvectorExtractorStats::UpdateWeights().

Referenced by main(), and kaldi::UnitTestIvectorExtractor().

                                       {
  CheckDims(*extractor);
  if (tot_auxf_ != 0.0) {
    KALDI_LOG << "Overall auxf/frame on training data was "
              << (tot_auxf_/gamma_.Sum()) << " per frame over "
              << gamma_.Sum() << " frames.";
  }

  double ans = 0.0;
  ans += UpdateProjections(opts, extractor);
  if (extractor->IvectorDependentWeights())
    ans += UpdateWeights(opts, extractor);
  if (!S_.empty())
    ans += UpdateVariances(opts, extractor);
  ans += UpdatePrior(opts, extractor); // This will also transform the ivector
                                       // space.  Note: this must be done as the
                                       // last stage, because it will make the
                                       // stats invalid for that model.
  KALDI_LOG << "Overall objective-function improvement per frame was " << ans;
  extractor->ComputeDerivedVars();
  return ans;
}

UpdatePrior()

double UpdatePrior ( const IvectorExtractorEstimationOptions &  opts, IvectorExtractor *  extractor  ) const

protected

Definition at line 1651 of file ivector-extractor.cc.

References SpMatrix< Real >::AddMat2Sp(), MatrixBase< Real >::AddMatMat(), SpMatrix< Real >::AddVec2(), MatrixBase< Real >::AddVecVec(), VectorBase< Real >::ApplyFloor(), VectorBase< Real >::ApplyPow(), kaldi::ApproxEqual(), IvectorExtractorEstimationOptions::diagonalize, SpMatrix< Real >::Eig(), IvectorExtractorStats::GetOrthogonalIvectorTransform(), SpMatrix< Real >::IsUnit(), IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, IvectorExtractor::IvectorDim(), KALDI_ASSERT, KALDI_LOG, KALDI_WARN, kaldi::kNoTrans, kaldi::kTrans, M_SQRT2, VectorBase< Real >::Max(), VectorBase< Real >::Min(), MatrixBase< Real >::MulRowsVec(), VectorBase< Real >::Norm(), IvectorExtractorStats::num_ivectors_, MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), IvectorExtractor::prior_offset_, IvectorExtractorStats::PriorDiagnostics(), PackedMatrix< Real >::Scale(), VectorBase< Real >::Scale(), MatrixBase< Real >::SetUnit(), and IvectorExtractor::TransformIvectors().

Referenced by IvectorExtractorStats::Update().

                                       {

  KALDI_ASSERT(num_ivectors_ > 0.0);
  Vector<double> sum(ivector_sum_);
  sum.Scale(1.0 / num_ivectors_);
  SpMatrix<double> covar(ivector_scatter_);
  covar.Scale(1.0 / num_ivectors_);
  covar.AddVec2(-1.0, sum); // Get the centered covariance.

  int32 ivector_dim = extractor->IvectorDim();
  Vector<double> s(ivector_dim);
  Matrix<double> P(ivector_dim, ivector_dim);
  // decompose covar = P diag(s) P^T:
  covar.Eig(&s, &P);
  KALDI_LOG << "Eigenvalues of iVector covariance range from "
            << s.Min() << " to " << s.Max();
  int32 num_floored;
  s.ApplyFloor(1.0e-07, &num_floored);
  if (num_floored > 0)
    KALDI_WARN << "Floored " << num_floored << " eigenvalues of covar "
               << "of iVectors.";

  Matrix<double> T(P, kTrans);
  { // set T to a transformation that makes covar unit
    // (modulo floored eigenvalues).
    Vector<double> scales(s);
    scales.ApplyPow(-0.5);
    T.MulRowsVec(scales);
    if (num_floored == 0) { // a check..
      SpMatrix<double> Tproj(ivector_dim);
      Tproj.AddMat2Sp(1.0, T, kNoTrans, covar, 0.0);
      KALDI_ASSERT(Tproj.IsUnit(1.0e-06));
    }
  }

  Vector<double> sum_proj(ivector_dim);
  sum_proj.AddMatVec(1.0, T, kNoTrans, sum, 0.0);

  KALDI_ASSERT(sum_proj.Norm(2.0) != 0.0);

  // We need a projection that (like T) makes "covar" unit,
  // but also that sends "sum" to a multiple of the vector e0 = [ 1 0 0 0 .. ].
  // We'll do this by a transform that follows T, of the form
  // (I - 2 a a^T), where a is unit.  [i.e. a Householder reflection].
  // Firstly, let x equal sum_proj normalized to unit length.
  // We'll let a = alpha x + beta e0, for suitable coefficients alpha and beta,
  // To project sum_proj (or equivalenty, x) to a multiple of e0, we'll need that
  // the x term in
  //  (I - 2(alpha x + beta e0)(alpha x + beta e0)  x
  // equals zero., i.e. 1 - 2 alpha (alpha x^T x + beta e0^T x) == 0,
  //    (1 - 2 alpha^2 - 2 alpha beta x0) = 0
  // To ensure that a is unit, we require that
  // (alpha x + beta e0).(alpha x + beta e0) = 1, i.e.
  //    alpha^2 + beta^2 + 2 alpha beta x0 = 1
  // at wolframalpha.com,
  // Solve[ {a^2 + b^2 + 2 a b x = 1}, {1 - 2 a^2 - 2 a b x = 0}, {a, b} ]
  // gives different solutions, but the one that keeps the offset positive
  // after projection seems to be:
  //    alpha = 1/(sqrt(2)sqrt(1 - x0)), beta = -alpha

  Matrix<double> U(ivector_dim, ivector_dim);
  U.SetUnit();
  Vector<double> x(sum_proj);
  x.Scale(1.0 / x.Norm(2.0));
  double x0 = x(0), alpha, beta;
  alpha = 1.0 / (M_SQRT2 * sqrt(1.0 - x0));
  beta = -alpha;
  Vector<double> a(x);
  a.Scale(alpha);
  a(0) += beta;
  U.AddVecVec(-2.0, a, a);

  Matrix<double> V(ivector_dim, ivector_dim);
  V.AddMatMat(1.0, U, kNoTrans, T, kNoTrans, 0.0);

  // Optionally replace transform V with V' such that V' makes the
  // covariance unit and additionally diagonalizes the quadratic
  // term.
  if (opts.diagonalize) {

    SubMatrix<double> Vsub(V, 1, V.NumRows()-1, 0, V.NumCols());
    Matrix<double> Vtemp(SubMatrix<double>(V, 1, V.NumRows()-1,
                         0, V.NumCols())),
                   A;
    GetOrthogonalIvectorTransform(SubMatrix<double>(Vtemp, 0,
                                  Vtemp.NumRows(), 1, Vtemp.NumCols()-1),
                                  extractor, &A);

    // It is necessary to exclude the first row of V in this transformation
    // so that the sum_vproj has the form [ x 0 0 0 .. ], where x > 0.
    Vsub.AddMatMat(1.0, A, kNoTrans, Vtemp, kNoTrans, 0.0);
  }

  if (num_floored == 0) { // a check..
    SpMatrix<double> Vproj(ivector_dim);
    Vproj.AddMat2Sp(1.0, V, kNoTrans, covar, 0.0);
    KALDI_ASSERT(Vproj.IsUnit(1.0e-04));
  }


  Vector<double> sum_vproj(ivector_dim);
  sum_vproj.AddMatVec(1.0, V, kNoTrans, sum, 0.0);
  // Make sure sum_vproj is of the form [ x 0 0 0 .. ] with x > 0.
  // (the x > 0 part isn't really necessary, it's just nice to know.)
  KALDI_ASSERT(ApproxEqual(sum_vproj(0), sum_vproj.Norm(2.0)));

  double ans = PriorDiagnostics(extractor->prior_offset_);

  extractor->TransformIvectors(V, sum_vproj(0));

  return ans;
}

UpdateProjection()

double UpdateProjection ( const IvectorExtractorEstimationOptions &  opts, int32  gaussian, IvectorExtractor *  extractor  ) const

protected

Definition at line 1319 of file ivector-extractor.cc.

References VectorBase< Real >::CopyFromVec(), SolverOptions::diagonal_precondition, IvectorExtractorStats::gamma_, IvectorExtractorEstimationOptions::gaussian_min_count, rnnlm::i, IvectorExtractor::IvectorDim(), KALDI_ASSERT, KALDI_VLOG, KALDI_WARN, kaldi::kUndefined, IvectorExtractor::M_, SolverOptions::name, IvectorExtractor::NumGauss(), IvectorExtractorStats::R_, IvectorExtractor::Sigma_inv_, kaldi::SolveQuadraticMatrixProblem(), and IvectorExtractorStats::Y_.

                                       {
  int32 I = extractor->NumGauss(), S = extractor->IvectorDim();
  KALDI_ASSERT(i >= 0 && i < I);
  /*
    For Gaussian index i, maximize the auxiliary function
       Q_i(x) = tr(M_i^T Sigma_i^{-1} Y_i)  - 0.5 tr(Sigma_i^{-1} M_i R_i M_i^T)
   */
  if (gamma_(i) < opts.gaussian_min_count) {
    KALDI_WARN << "Skipping Gaussian index " << i << " because count "
               << gamma_(i) << " is below min-count.";
    return 0.0;
  }
  SpMatrix<double> R(S, kUndefined), SigmaInv(extractor->Sigma_inv_[i]);
  SubVector<double> R_vec(R_, i); // i'th row of R; vectorized form of SpMatrix.
  SubVector<double> R_sp(R.Data(), S * (S+1) / 2);
  R_sp.CopyFromVec(R_vec); // copy to SpMatrix's memory.

  Matrix<double> M(extractor->M_[i]);
  SolverOptions solver_opts;
  solver_opts.name = "M";
  solver_opts.diagonal_precondition = true;
  double impr = SolveQuadraticMatrixProblem(R, Y_[i], SigmaInv, solver_opts, &M),
      gamma = gamma_(i);
  if (i < 4) {
    KALDI_VLOG(1) << "Objf impr for M for Gaussian index " << i << " is "
                  << (impr / gamma) << " per frame over " << gamma << " frames.";
  }
  extractor->M_[i].CopyFromMat(M);
  return impr;
}

UpdateProjections()

double UpdateProjections ( const IvectorExtractorEstimationOptions &  opts, IvectorExtractor *  extractor  ) const

protected

Definition at line 1416 of file ivector-extractor.cc.

References count, kaldi::g_num_threads, IvectorExtractorStats::gamma_, rnnlm::i, KALDI_LOG, TaskSequencerConfig::num_threads, IvectorExtractor::NumGauss(), TaskSequencer< C >::Run(), and VectorBase< Real >::Sum().

Referenced by IvectorExtractorStats::Update().

                                       {
  int32 I = extractor->NumGauss();
  double tot_impr = 0.0;
  {
    TaskSequencerConfig sequencer_opts;
    sequencer_opts.num_threads = g_num_threads;
    TaskSequencer<IvectorExtractorUpdateProjectionClass> sequencer(
        sequencer_opts);
    for (int32 i = 0; i < I; i++)
      sequencer.Run(new IvectorExtractorUpdateProjectionClass(
          *this, opts, i, extractor, &tot_impr));
  }
  double count = gamma_.Sum();
  KALDI_LOG << "Overall objective function improvement for M (mean projections) "
            << "was " << (tot_impr / count) << " per frame over "
            << count << " frames.";
  return tot_impr / count;
}

UpdateVariances()

double UpdateVariances ( const IvectorExtractorEstimationOptions &  opts, IvectorExtractor *  extractor  ) const

protected

Definition at line 1437 of file ivector-extractor.cc.

References MatrixBase< Real >::AddMat(), SpMatrix< Real >::AddMat2Sp(), MatrixBase< Real >::AddMatMat(), SpMatrix< Real >::AddSp(), SpMatrix< Real >::ApplyFloor(), VectorBase< Real >::CopyFromVec(), PackedMatrix< Real >::Data(), IvectorExtractor::FeatDim(), IvectorExtractorStats::gamma_, IvectorExtractorEstimationOptions::gaussian_min_count, rnnlm::i, SpMatrix< Real >::Invert(), IvectorExtractor::IvectorDim(), KALDI_ASSERT, KALDI_LOG, KALDI_VLOG, KALDI_WARN, kaldi::kNoTrans, kaldi::kTakeMeanAndCheck, kaldi::kTrans, SpMatrix< Real >::LogPosDefDet(), IvectorExtractor::M_, SpMatrix< Real >::MaxAbsEig(), IvectorExtractor::NumGauss(), PackedMatrix< Real >::NumRows(), IvectorExtractorStats::R_, MatrixBase< Real >::Row(), IvectorExtractorStats::S_, PackedMatrix< Real >::Scale(), IvectorExtractor::Sigma_inv_, VectorBase< Real >::Sum(), kaldi::TraceSpSp(), IvectorExtractorEstimationOptions::variance_floor_factor, and IvectorExtractorStats::Y_.

Referenced by IvectorExtractorStats::Update().

                                       {
  int32 num_gauss = extractor->NumGauss(),
      feat_dim = extractor->FeatDim(),
      ivector_dim = extractor->IvectorDim();
  KALDI_ASSERT(!S_.empty());
  double tot_objf_impr = 0.0;

  // "raw_variances" will be the variances directly from
  // the stats, without any flooring.
  std::vector<SpMatrix<double> > raw_variances(num_gauss);
  SpMatrix<double> var_floor(feat_dim);
  double var_floor_count = 0.0;

  for (int32 i = 0; i < num_gauss; i++) {
    if (gamma_(i) < opts.gaussian_min_count) continue; // warned in UpdateProjections
    SpMatrix<double> &S(raw_variances[i]);
    S = S_[i]; // Set it to the raw scatter statistics.

    // The equations for estimating the variance are similar to
    // those used in SGMMs.  We need to convert it to a centered
    // covariance, and for this we can use a combination of other
    // stats and the model parameters.

    Matrix<double> M(extractor->M_[i]);
    // Y * M^T.
    Matrix<double> YM(feat_dim, feat_dim);
    YM.AddMatMat(1.0, Y_[i], kNoTrans, M, kTrans, 0.0);
    Matrix<double> YMMY(YM, kTrans);
    YMMY.AddMat(1.0, YM);
    // Now, YMMY = Y * M^T + M * Y^T.  This is a kind of cross-term
    // between the mean and the data, which we subtract.
    SpMatrix<double> YMMY_sp(YMMY, kTakeMeanAndCheck);
    S.AddSp(-1.0, YMMY_sp);

    // Add in a mean-squared term.
    SpMatrix<double> R(ivector_dim); // will be scatter of iVectors, weighted
                                     // by count for this Gaussian.
    SubVector<double> R_vec(R.Data(),
                            ivector_dim * (ivector_dim + 1) / 2);
    R_vec.CopyFromVec(R_.Row(i)); //

    S.AddMat2Sp(1.0, M, kNoTrans, R, 1.0);

    var_floor.AddSp(1.0, S);
    var_floor_count += gamma_(i);
    S.Scale(1.0 / gamma_(i));
  }
  KALDI_ASSERT(var_floor_count > 0.0);
  KALDI_ASSERT(opts.variance_floor_factor > 0.0 &&
               opts.variance_floor_factor <= 1.0);

  var_floor.Scale(opts.variance_floor_factor / var_floor_count);

  // var_floor should not be singular in any normal case, but previously
  // we've had situations where cholesky on it failed (perhaps due to
  // people using linearly dependent features).  So we floor its
  // singular values.
  int eig_floored = var_floor.ApplyFloor(var_floor.MaxAbsEig() * 1.0e-04);
  if (eig_floored > 0) {
    KALDI_WARN << "Floored " << eig_floored << " eigenvalues of the "
               << "variance floor matrix.  This is not expected.  Maybe your "
               << "feature data is linearly dependent.";
  }

  int32 tot_num_floored = 0;
  for (int32 i = 0; i < num_gauss; i++) {
    SpMatrix<double> &S(raw_variances[i]); // un-floored variance.
    if (S.NumRows() == 0) continue; // due to low count.
    SpMatrix<double> floored_var(S);
    SpMatrix<double> old_inv_var(extractor->Sigma_inv_[i]);

    int32 num_floored = floored_var.ApplyFloor(var_floor);
    tot_num_floored += num_floored;
    if (num_floored > 0)
      KALDI_LOG << "For Gaussian index " << i << ", floored "
                << num_floored << " eigenvalues of variance.";
    // this objf is per frame;
    double old_objf = -0.5 * (TraceSpSp(S, old_inv_var) -
                              old_inv_var.LogPosDefDet());

    SpMatrix<double> new_inv_var(floored_var);
    new_inv_var.Invert();

    double new_objf = -0.5 * (TraceSpSp(S, new_inv_var) -
                                 new_inv_var.LogPosDefDet());
    if (i < 4) {
      KALDI_VLOG(1) << "Objf impr/frame for variance for Gaussian index "
                    << i << " was " << (new_objf - old_objf);
    }
    tot_objf_impr += gamma_(i) * (new_objf - old_objf);
    extractor->Sigma_inv_[i].CopyFromSp(new_inv_var);
  }
  double floored_percent = tot_num_floored * 100.0 / (num_gauss * feat_dim);
  KALDI_LOG << "Floored " << floored_percent << "% of all Gaussian eigenvalues";

  KALDI_LOG << "Overall objf impr/frame for variances was "
            << (tot_objf_impr / gamma_.Sum()) << " over "
            << gamma_.Sum() << " frames.";
  return tot_objf_impr / gamma_.Sum();
}

UpdateWeight()

double UpdateWeight ( const IvectorExtractorEstimationOptions &  opts, int32  gaussian, IvectorExtractor *  extractor  ) const

protected

Definition at line 1540 of file ivector-extractor.cc.

References VectorBase< Real >::CopyFromVec(), PackedMatrix< Real >::Data(), SolverOptions::diagonal_precondition, IvectorExtractorStats::G_, IvectorExtractorStats::gamma_, IvectorExtractor::IvectorDim(), KALDI_ASSERT, KALDI_VLOG, SolverOptions::name, IvectorExtractor::NumGauss(), IvectorExtractorStats::Q_, MatrixBase< Real >::Row(), kaldi::SolveQuadraticProblem(), and IvectorExtractor::w_.

                                       {

  int32 num_gauss = extractor->NumGauss(),
      ivector_dim = extractor->IvectorDim();
  KALDI_ASSERT(i >= 0 && i < num_gauss);

  SolverOptions solver_opts;
  solver_opts.diagonal_precondition = true;
  solver_opts.name = "w";

  SubVector<double> w_i(extractor->w_, i);
  SubVector<double> g_i(G_, i);
  SpMatrix<double> Q(ivector_dim);
  SubVector<double> Q_vec(Q.Data(), ivector_dim * (ivector_dim + 1) / 2);
  Q_vec.CopyFromVec(Q_.Row(i));
  double objf_impr = SolveQuadraticProblem(Q, g_i, solver_opts, &w_i);
  if (i < 4 && gamma_(i) != 0.0) {
    KALDI_VLOG(1) << "Auxf impr/frame for Gaussian index " << i
                  << " for weights is " << (objf_impr / gamma_(i))
                  << " over " << gamma_(i) << " frames.";
  }
  return objf_impr;
}

UpdateWeights()

double UpdateWeights ( const IvectorExtractorEstimationOptions &  opts, IvectorExtractor *  extractor  ) const

protected

Definition at line 1589 of file ivector-extractor.cc.

References kaldi::g_num_threads, IvectorExtractorStats::gamma_, rnnlm::i, KALDI_LOG, TaskSequencerConfig::num_threads, IvectorExtractor::NumGauss(), TaskSequencer< C >::Run(), and VectorBase< Real >::Sum().

Referenced by IvectorExtractorStats::Update().

                                       {

  int32 I = extractor->NumGauss();
  double tot_impr = 0.0;
  {
    TaskSequencerConfig sequencer_opts;
    sequencer_opts.num_threads = g_num_threads;
    TaskSequencer<IvectorExtractorUpdateWeightClass> sequencer(
        sequencer_opts);
    for (int32 i = 0; i < I; i++)
      sequencer.Run(new IvectorExtractorUpdateWeightClass(
          *this, opts, i, extractor, &tot_impr));
  }

  double num_frames = gamma_.Sum();
  KALDI_LOG << "Overall auxf impr/frame from weight update is "
            << (tot_impr / num_frames) << " over "
            << num_frames << " frames.";
  return tot_impr / num_frames;
}

Write() [1/2]

void Write	(	std::ostream &	os,
		bool	binary
	)

Definition at line 1199 of file ivector-extractor.cc.

References IvectorExtractorStats::FlushCache().

Referenced by main(), kaldi::TestIvectorExtractorStatsIO(), and IvectorExtractorStats::Write().

                                                             {
  FlushCache(); // for R stats.
  ((const IvectorExtractorStats&)(*this)).Write(os, binary); // call const version.
}

Write() [2/2]

void Write	(	std::ostream &	os,
		bool	binary
	)		const

Definition at line 1205 of file ivector-extractor.cc.

References IvectorExtractorStats::G_, IvectorExtractorStats::gamma_, rnnlm::i, IvectorExtractorStats::ivector_scatter_, IvectorExtractorStats::ivector_sum_, KALDI_ASSERT, IvectorExtractorStats::num_ivectors_, IvectorExtractorStats::Q_, IvectorExtractorStats::R_, IvectorExtractorStats::R_num_cached_, IvectorExtractorStats::S_, IvectorExtractorStats::tot_auxf_, PackedMatrix< Real >::Write(), VectorBase< Real >::Write(), IvectorExtractorStats::Write(), MatrixBase< Real >::Write(), kaldi::WriteBasicType(), kaldi::WriteToken(), and IvectorExtractorStats::Y_.

                                                                   {
  KALDI_ASSERT(R_num_cached_ == 0 && "Please use the non-const Write().");
  WriteToken(os, binary, "<IvectorExtractorStats>");
  WriteToken(os, binary, "<TotAuxf>");
  WriteBasicType(os, binary, tot_auxf_);
  WriteToken(os, binary, "<gamma>");
  gamma_.Write(os, binary);
  WriteToken(os, binary, "<Y>");
  int32 size = Y_.size();
  WriteBasicType(os, binary, size);
  for (int32 i = 0; i < size; i++)
    Y_[i].Write(os, binary);
  WriteToken(os, binary, "<R>");
  Matrix<BaseFloat> R_float(R_);
  R_float.Write(os, binary);
  WriteToken(os, binary, "<Q>");
  Matrix<BaseFloat> Q_float(Q_);
  Q_float.Write(os, binary);
  WriteToken(os, binary, "<G>");
  G_.Write(os, binary);
  WriteToken(os, binary, "<S>");
  size = S_.size();
  WriteBasicType(os, binary, size);
  for (int32 i = 0; i < size; i++)
    S_[i].Write(os, binary);
  WriteToken(os, binary, "<NumIvectors>");
  WriteBasicType(os, binary, num_ivectors_);
  WriteToken(os, binary, "<IvectorSum>");
  ivector_sum_.Write(os, binary);
  WriteToken(os, binary, "<IvectorScatter>");
  ivector_scatter_.Write(os, binary);
  WriteToken(os, binary, "</IvectorExtractorStats>");
}

Friends And Related Function Documentation

IvectorExtractor

friend class IvectorExtractor

friend

Definition at line 483 of file ivector-extractor.h.

IvectorExtractorUpdateProjectionClass

friend class IvectorExtractorUpdateProjectionClass

friend

Definition at line 525 of file ivector-extractor.h.

IvectorExtractorUpdateWeightClass

friend class IvectorExtractorUpdateWeightClass

friend

Definition at line 526 of file ivector-extractor.h.

Member Data Documentation

config_

IvectorExtractorStatsOptions config_

protected

Definition at line 603 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForUtterance(), IvectorExtractorStats::CommitStatsForW(), and IvectorExtractorStats::IvectorExtractorStats().

G_

Matrix<double> G_

protected

G_ is the linear term in the weight projection matrix w_.

It has the same dim as w_, i.e. [I][S]

Definition at line 654 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForWPoint(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::Read(), IvectorExtractorStats::UpdateWeight(), and IvectorExtractorStats::Write().

gamma_

Vector<double> gamma_

protected

Total occupation count for each Gaussian index (zeroth-order stats)

Definition at line 616 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForM(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::IvectorVarianceDiagnostic(), IvectorExtractorStats::PriorDiagnostics(), IvectorExtractorStats::Read(), IvectorExtractorStats::Update(), IvectorExtractorStats::UpdateProjection(), IvectorExtractorStats::UpdateProjections(), IvectorExtractorStats::UpdateVariances(), IvectorExtractorStats::UpdateWeight(), IvectorExtractorStats::UpdateWeights(), and IvectorExtractorStats::Write().

gamma_Y_lock_

std::mutex gamma_Y_lock_

protected

This mutex guards gamma_ and Y_ (for multi-threaded update)

Definition at line 613 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForM().

ivector_scatter_

SpMatrix<double> ivector_scatter_

protected

Second-order stats for the iVectors. Needed for prior re-estimation.

Definition at line 676 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForPrior(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::PriorDiagnostics(), IvectorExtractorStats::Read(), IvectorExtractorStats::UpdatePrior(), and IvectorExtractorStats::Write().

ivector_sum_

Vector<double> ivector_sum_

protected

Sum of all the iVector means. Needed for prior re-estimation.

Definition at line 673 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForPrior(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::PriorDiagnostics(), IvectorExtractorStats::Read(), IvectorExtractorStats::UpdatePrior(), and IvectorExtractorStats::Write().

num_ivectors_

double num_ivectors_

protected

Count of the number of iVectors we trained on.

Need for prior re-estimation. (make it double not int64 to more easily support weighting later.)

Definition at line 670 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForPrior(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::PriorDiagnostics(), IvectorExtractorStats::Read(), IvectorExtractorStats::UpdatePrior(), and IvectorExtractorStats::Write().

prior_stats_lock_

std::mutex prior_stats_lock_

protected

This mutex guards num_ivectors_, ivector_sum_ and ivector_scatter_ (for multi-threaded update)

Definition at line 666 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForPrior().

Q_

Matrix<double> Q_

protected

Q_ is like R_ (with same dimensions), except used for weight estimation; the scatter of ivectors is weighted by the coefficient of the quadratic term in the expansion for w (the "safe" one, with the max expression).

Definition at line 650 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForWPoint(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::Read(), IvectorExtractorStats::UpdateWeight(), and IvectorExtractorStats::Write().

R_

Matrix<double> R_

protected

R_i, quadratic term for ivector subspace (M matrix)estimation.

This is a kind of scatter of ivectors of training speakers, weighted by count for each Gaussian. Conceptually vector<SpMatrix<double> >, but we store each SpMatrix as a row of R_. Conceptually, the dim is [I][S][S]; the actual dim is [I][S*(S+1)/2].

Definition at line 630 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::FlushCache(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::Read(), IvectorExtractorStats::UpdateProjection(), IvectorExtractorStats::UpdateVariances(), and IvectorExtractorStats::Write().

R_cache_lock_

std::mutex R_cache_lock_

protected

This mutex guards R_num_cached_, R_gamma_cache_, R_ivec_cache_ (for multi-threaded update)

Definition at line 634 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForM(), and IvectorExtractorStats::FlushCache().

R_gamma_cache_

Matrix<double> R_gamma_cache_

protected

dimension: [num-to-cache][I]

Definition at line 640 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForM(), IvectorExtractorStats::FlushCache(), and IvectorExtractorStats::IvectorExtractorStats().

R_ivec_scatter_cache_

Matrix<double> R_ivec_scatter_cache_

protected

dimension: [num-to-cache][S*(S+1)/2]

Definition at line 642 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForM(), IvectorExtractorStats::FlushCache(), and IvectorExtractorStats::IvectorExtractorStats().

R_lock_

std::mutex R_lock_

protected

This mutex guards R_ (for multi-threaded update)

Definition at line 623 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::FlushCache().

R_num_cached_

int32 R_num_cached_

protected

To avoid too-frequent rank-1 update of R, which is slow, we cache some quantities here.

Definition at line 638 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForM(), IvectorExtractorStats::FlushCache(), IvectorExtractorStats::IvectorExtractorStats(), and IvectorExtractorStats::Write().

S_

std::vector< SpMatrix<double> > S_

protected

S_{i}, raw second-order stats per Gaussian which we will use to update the variances Sigma_inv_.

Definition at line 661 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::AccStatsForUtterance(), IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForSigma(), IvectorExtractorStats::CommitStatsForUtterance(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::Read(), IvectorExtractorStats::Update(), IvectorExtractorStats::UpdateVariances(), and IvectorExtractorStats::Write().

tot_auxf_

double tot_auxf_

protected

Caution: if we read from disk, this.

is not recovered. Options will not be used during the update phase anyway, so this should not matter. Total auxiliary function over the training data– can be used to check convergence, etc.

Definition at line 610 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CommitStatsForUtterance(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::Read(), IvectorExtractorStats::Update(), and IvectorExtractorStats::Write().

variance_stats_lock_

std::mutex variance_stats_lock_

protected

This mutex guards S_ (for multi-threaded update)

Definition at line 657 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForSigma().

weight_stats_lock_

std::mutex weight_stats_lock_

protected

This mutex guards Q_ and G_ (for multi-threaded update)

Definition at line 645 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::CommitStatsForWPoint().

Y_

std::vector<Matrix<double> > Y_

protected

Stats Y_i for estimating projections M.

Dimension is [I][D][S]. The linear term in M.

Definition at line 620 of file ivector-extractor.h.

Referenced by IvectorExtractorStats::Add(), IvectorExtractorStats::CheckDims(), IvectorExtractorStats::CommitStatsForM(), IvectorExtractorStats::IvectorExtractorStats(), IvectorExtractorStats::Read(), IvectorExtractorStats::UpdateProjection(), IvectorExtractorStats::UpdateVariances(), and IvectorExtractorStats::Write().

---

The documentation for this class was generated from the following files:

• ivector/ivector-extractor.h
• ivector/ivector-extractor.cc