All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
DiagGmm Class Reference

Definition for Gaussian Mixture Model with diagonal covariances. More...

#include <diag-gmm.h>

Collaboration diagram for DiagGmm:

Public Member Functions

 DiagGmm ()
 Empty constructor. More...
 
 DiagGmm (const DiagGmm &gmm)
 
 DiagGmm (const GaussClusterable &gc, BaseFloat var_floor)
 Initializer from GaussClusterable initializes the DiagGmm as a single Gaussian from tree stats. More...
 
void CopyFromNormal (const DiagGmmNormal &diag_gmm_normal)
 Copies from DiagGmmNormal; does not resize. More...
 
 DiagGmm (int32 nMix, int32 dim)
 
 DiagGmm (const std::vector< std::pair< BaseFloat, const DiagGmm * > > &gmms)
 Constructor that allows us to merge GMMs with weights. More...
 
void Resize (int32 nMix, int32 dim)
 Resizes arrays to this dim. Does not initialize data. More...
 
int32 NumGauss () const
 Returns the number of mixture components in the GMM. More...
 
int32 Dim () const
 Returns the dimensionality of the Gaussian mean vectors. More...
 
void CopyFromDiagGmm (const DiagGmm &diaggmm)
 Copies from given DiagGmm. More...
 
void CopyFromFullGmm (const FullGmm &fullgmm)
 Copies from given FullGmm. More...
 
BaseFloat LogLikelihood (const VectorBase< BaseFloat > &data) const
 Returns the log-likelihood of a data point (vector) given the GMM. More...
 
void LogLikelihoods (const VectorBase< BaseFloat > &data, Vector< BaseFloat > *loglikes) const
 Outputs the per-component log-likelihoods. More...
 
void LogLikelihoods (const MatrixBase< BaseFloat > &data, Matrix< BaseFloat > *loglikes) const
 This version of the LogLikelihoods function operates on a sequence of frames simultaneously; the row index of both "data" and "loglikes" is the frame index. More...
 
void LogLikelihoodsPreselect (const VectorBase< BaseFloat > &data, const std::vector< int32 > &indices, Vector< BaseFloat > *loglikes) const
 Outputs the per-component log-likelihoods of a subset of mixture components. More...
 
BaseFloat GaussianSelection (const VectorBase< BaseFloat > &data, int32 num_gselect, std::vector< int32 > *output) const
 Get gaussian selection information for one frame. More...
 
BaseFloat GaussianSelection (const MatrixBase< BaseFloat > &data, int32 num_gselect, std::vector< std::vector< int32 > > *output) const
 This version of the Gaussian selection function works for a sequence of frames rather than just a single frame. More...
 
BaseFloat GaussianSelectionPreselect (const VectorBase< BaseFloat > &data, const std::vector< int32 > &preselect, int32 num_gselect, std::vector< int32 > *output) const
 Get gaussian selection information for one frame. More...
 
BaseFloat ComponentPosteriors (const VectorBase< BaseFloat > &data, Vector< BaseFloat > *posteriors) const
 Computes the posterior probabilities of all Gaussian components given a data point. More...
 
BaseFloat ComponentLogLikelihood (const VectorBase< BaseFloat > &data, int32 comp_id) const
 Computes the log-likelihood of a data point given a single Gaussian component. More...
 
int32 ComputeGconsts ()
 Sets the gconsts. More...
 
void Generate (VectorBase< BaseFloat > *output)
 Generates a random data-point from this distribution. More...
 
void Split (int32 target_components, float perturb_factor, std::vector< int32 > *history=NULL)
 Split the components and remember the order in which the components were split. More...
 
void Perturb (float perturb_factor)
 Perturbs the component means with a random vector multiplied by the pertrub factor. More...
 
void Merge (int32 target_components, std::vector< int32 > *history=NULL)
 Merge the components and remember the order in which the components were merged (flat list of pairs) More...
 
void MergeKmeans (int32 target_components, ClusterKMeansOptions cfg=ClusterKMeansOptions())
 
void Write (std::ostream &os, bool binary) const
 
void Read (std::istream &in, bool binary)
 
void Interpolate (BaseFloat rho, const DiagGmm &source, GmmFlagsType flags=kGmmAll)
 this = rho x source + (1-rho) x this More...
 
void Interpolate (BaseFloat rho, const FullGmm &source, GmmFlagsType flags=kGmmAll)
 this = rho x source + (1-rho) x this More...
 
const Vector< BaseFloat > & gconsts () const
 Const accessors. More...
 
const Vector< BaseFloat > & weights () const
 
const Matrix< BaseFloat > & means_invvars () const
 
const Matrix< BaseFloat > & inv_vars () const
 
bool valid_gconsts () const
 
void RemoveComponent (int32 gauss, bool renorm_weights)
 Removes single component from model. More...
 
void RemoveComponents (const std::vector< int32 > &gauss, bool renorm_weights)
 Removes multiple components from model; "gauss" must not have dups. More...
 
template<class Real >
void SetWeights (const VectorBase< Real > &w)
 Mutators for both float or double. More...
 
template<class Real >
void SetMeans (const MatrixBase< Real > &m)
 Use SetMeans to update only the Gaussian means (and not variances) More...
 
template<class Real >
void SetInvVarsAndMeans (const MatrixBase< Real > &invvars, const MatrixBase< Real > &means)
 Use SetInvVarsAndMeans if updating both means and (inverse) variances. More...
 
template<class Real >
void SetInvVars (const MatrixBase< Real > &v)
 Set the (inverse) variances and recompute means_invvars_. More...
 
template<class Real >
void GetVars (Matrix< Real > *v) const
 Accessor for covariances. More...
 
template<class Real >
void GetMeans (Matrix< Real > *m) const
 Accessor for means. More...
 
template<class Real >
void SetComponentMean (int32 gauss, const VectorBase< Real > &in)
 Mutators for single component, supports float or double Set mean for a single component - internally multiplies with inv(var) More...
 
template<class Real >
void SetComponentInvVar (int32 gauss, const VectorBase< Real > &in)
 Set inv-var for single component (recommend to do this before setting the mean, if doing both, for numerical reasons). More...
 
void SetComponentWeight (int32 gauss, BaseFloat weight)
 Set weight for single component. More...
 
template<class Real >
void GetComponentMean (int32 gauss, VectorBase< Real > *out) const
 Accessor for single component mean. More...
 
template<class Real >
void GetComponentVariance (int32 gauss, VectorBase< Real > *out) const
 Accessor for single component variance. More...
 

Private Member Functions

BaseFloat merged_components_logdet (BaseFloat w1, BaseFloat w2, const VectorBase< BaseFloat > &f1, const VectorBase< BaseFloat > &f2, const VectorBase< BaseFloat > &s1, const VectorBase< BaseFloat > &s2) const
 
const DiagGmmoperator= (const DiagGmm &other)
 

Private Attributes

Vector< BaseFloatgconsts_
 Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var)) More...
 
bool valid_gconsts_
 Recompute gconsts_ if false. More...
 
Vector< BaseFloatweights_
 weights (not log). More...
 
Matrix< BaseFloatinv_vars_
 Inverted (diagonal) variances. More...
 
Matrix< BaseFloatmeans_invvars_
 Means times inverted variance. More...
 

Friends

class DiagGmmNormal
 this makes it a little easier to modify the internals More...
 

Detailed Description

Definition for Gaussian Mixture Model with diagonal covariances.

Definition at line 42 of file diag-gmm.h.

Constructor & Destructor Documentation

DiagGmm ( )
inline

Empty constructor.

Definition at line 48 of file diag-gmm.h.

Referenced by DiagGmm::Split().

48 : valid_gconsts_(false) { }
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
DiagGmm ( const DiagGmm gmm)
inlineexplicit

Definition at line 50 of file diag-gmm.h.

References DiagGmm::CopyFromDiagGmm().

50  : valid_gconsts_(false) {
51  CopyFromDiagGmm(gmm);
52  }
void CopyFromDiagGmm(const DiagGmm &diaggmm)
Copies from given DiagGmm.
Definition: diag-gmm.cc:83
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
DiagGmm ( const GaussClusterable gc,
BaseFloat  var_floor 
)

Initializer from GaussClusterable initializes the DiagGmm as a single Gaussian from tree stats.

Definition at line 944 of file diag-gmm.cc.

References DiagGmm::ComputeGconsts(), count, GaussClusterable::count(), KALDI_ASSERT, DiagGmm::Resize(), MatrixBase< Real >::Row(), DiagGmm::SetInvVarsAndMeans(), DiagGmm::SetWeights(), DiagGmm::weights(), GaussClusterable::x2_stats(), and GaussClusterable::x_stats().

945  : valid_gconsts_(false) {
946  Vector<BaseFloat> x (gc.x_stats());
947  Vector<BaseFloat> x2 (gc.x2_stats());
948  BaseFloat count = gc.count();
949  KALDI_ASSERT(count > 0.0);
950  this->Resize(1, x.Dim());
951  x.Scale(1.0/count);
952  x2.Scale(1.0/count);
953  x2.AddVec2(-1.0, x); // subtract mean^2.
954  x2.ApplyFloor(var_floor);
955  x2.InvertElements(); // get inv-var.
956  KALDI_ASSERT(x2.Min() > 0);
957  Matrix<BaseFloat> mean(1, x.Dim());
958  mean.Row(0).CopyFromVec(x);
959  Matrix<BaseFloat> inv_var(1, x.Dim());
960  inv_var.Row(0).CopyFromVec(x2);
961  this->SetInvVarsAndMeans(inv_var, mean);
962  Vector<BaseFloat> weights(1);
963  weights(0) = 1.0;
964  this->SetWeights(weights);
965  this->ComputeGconsts();
966 }
void SetInvVarsAndMeans(const MatrixBase< Real > &invvars, const MatrixBase< Real > &means)
Use SetInvVarsAndMeans if updating both means and (inverse) variances.
Definition: diag-gmm-inl.h:63
void Resize(int32 nMix, int32 dim)
Resizes arrays to this dim. Does not initialize data.
Definition: diag-gmm.cc:66
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
const size_t count
float BaseFloat
Definition: kaldi-types.h:29
const Vector< BaseFloat > & weights() const
Definition: diag-gmm.h:178
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void SetWeights(const VectorBase< Real > &w)
Mutators for both float or double.
Definition: diag-gmm-inl.h:28
DiagGmm ( int32  nMix,
int32  dim 
)
inline

Definition at line 61 of file diag-gmm.h.

References DiagGmm::Resize().

61 : valid_gconsts_(false) { Resize(nMix, dim); }
void Resize(int32 nMix, int32 dim)
Resizes arrays to this dim. Does not initialize data.
Definition: diag-gmm.cc:66
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
DiagGmm ( const std::vector< std::pair< BaseFloat, const DiagGmm * > > &  gmms)
explicit

Constructor that allows us to merge GMMs with weights.

Weights must sum to one, or this GMM will not be properly normalized (we don't check this). Weights must be positive (we check this).

Definition at line 39 of file diag-gmm.cc.

References DiagGmm::ComputeGconsts(), rnnlm::i, DiagGmm::inv_vars(), DiagGmm::inv_vars_, KALDI_ASSERT, DiagGmm::means_invvars(), DiagGmm::means_invvars_, DiagGmm::NumGauss(), DiagGmm::Resize(), MatrixBase< Real >::Row(), DiagGmm::weights(), and DiagGmm::weights_.

40  : valid_gconsts_(false) {
41  if (gmms.empty()) {
42  return; // GMM will be empty.
43  } else {
44  int32 num_gauss = 0, dim = gmms[0].second->Dim();
45  for (size_t i = 0; i < gmms.size(); i++)
46  num_gauss += gmms[i].second->NumGauss();
47  Resize(num_gauss, dim);
48  int32 cur_gauss = 0;
49  for (size_t i = 0; i < gmms.size(); i++) {
50  BaseFloat weight = gmms[i].first;
51  KALDI_ASSERT(weight > 0.0);
52  const DiagGmm &gmm = *(gmms[i].second);
53  for (int32 g = 0; g < gmm.NumGauss(); g++, cur_gauss++) {
54  means_invvars_.Row(cur_gauss).CopyFromVec(gmm.means_invvars().Row(g));
55  inv_vars_.Row(cur_gauss).CopyFromVec(gmm.inv_vars().Row(g));
56  weights_(cur_gauss) = weight * gmm.weights()(g);
57  }
58  }
59  KALDI_ASSERT(cur_gauss == NumGauss());
61  }
62 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
void Resize(int32 nMix, int32 dim)
Resizes arrays to this dim. Does not initialize data.
Definition: diag-gmm.cc:66
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
const SubVector< Real > Row(MatrixIndexT i) const
Return specific row of matrix [const].
Definition: kaldi-matrix.h:182
float BaseFloat
Definition: kaldi-types.h:29
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
DiagGmm()
Empty constructor.
Definition: diag-gmm.h:48
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236

Member Function Documentation

BaseFloat ComponentLogLikelihood ( const VectorBase< BaseFloat > &  data,
int32  comp_id 
) const

Computes the log-likelihood of a data point given a single Gaussian component.

NOTE: Currently we make no guarantees about what happens if one of the variances is zero.

Definition at line 497 of file diag-gmm.cc.

References VectorBase< Real >::ApplyPow(), VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ERR, DiagGmm::means_invvars_, MatrixBase< Real >::Row(), DiagGmm::valid_gconsts_, and kaldi::VecVec().

498  {
499  if (!valid_gconsts_)
500  KALDI_ERR << "Must call ComputeGconsts() before computing likelihood";
501  if (static_cast<int32>(data.Dim()) != Dim()) {
502  KALDI_ERR << "DiagGmm::ComponentLogLikelihood, dimension "
503  << "mismatch " << (data.Dim()) << " vs. "<< (Dim());
504  }
505  BaseFloat loglike;
506  Vector<BaseFloat> data_sq(data);
507  data_sq.ApplyPow(2.0);
508 
509  // loglike = means * inv(vars) * data.
510  loglike = VecVec(means_invvars_.Row(comp_id), data);
511  // loglike += -0.5 * inv(vars) * data_sq.
512  loglike -= 0.5 * VecVec(inv_vars_.Row(comp_id), data_sq);
513  return loglike + gconsts_(comp_id);
514 }
const SubVector< Real > Row(MatrixIndexT i) const
Return specific row of matrix [const].
Definition: kaldi-matrix.h:182
float BaseFloat
Definition: kaldi-types.h:29
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:36
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
BaseFloat ComponentPosteriors ( const VectorBase< BaseFloat > &  data,
Vector< BaseFloat > *  posteriors 
) const

Computes the posterior probabilities of all Gaussian components given a data point.

Returns the log-likehood of the data given the GMM.

Definition at line 601 of file diag-gmm.cc.

References VectorBase< Real >::ApplySoftMax(), VectorBase< Real >::CopyFromVec(), VectorBase< Real >::Dim(), KALDI_ERR, KALDI_ISINF, KALDI_ISNAN, DiagGmm::LogLikelihoods(), Vector< Real >::Resize(), and DiagGmm::valid_gconsts_.

Referenced by FmllrDiagGmmAccs::AccumulateForGmm(), FmllrRawAccs::AccumulateForGmm(), RegtreeMllrDiagGmmAccs::AccumulateForGmm(), RegtreeFmllrDiagGmmAccs::AccumulateForGmm(), AccumFullGmm::AccumulateFromDiag(), AccumDiagGmm::AccumulateFromDiag(), MlltAccs::AccumulateFromGmm(), kaldi::ComputeAmGmmFeatureDeriv(), kaldi::GetFeatDeriv(), main(), TestComponentAcc(), kaldi::UnitTestDiagGmm(), kaldi::UnitTestFmllrDiagGmm(), kaldi::UnitTestFmllrDiagGmmDiagonal(), and kaldi::UnitTestFmllrDiagGmmOffset().

602  {
603  if (!valid_gconsts_)
604  KALDI_ERR << "Must call ComputeGconsts() before computing likelihood";
605  if (posterior == NULL) KALDI_ERR << "NULL pointer passed as return argument.";
606  Vector<BaseFloat> loglikes;
607  LogLikelihoods(data, &loglikes);
608  BaseFloat log_sum = loglikes.ApplySoftMax();
609  if (KALDI_ISNAN(log_sum) || KALDI_ISINF(log_sum))
610  KALDI_ERR << "Invalid answer (overflow or invalid variances/features?)";
611  if (posterior->Dim() != loglikes.Dim())
612  posterior->Resize(loglikes.Dim());
613  posterior->CopyFromVec(loglikes);
614  return log_sum;
615 }
#define KALDI_ISINF
Definition: kaldi-math.h:73
float BaseFloat
Definition: kaldi-types.h:29
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
#define KALDI_ERR
Definition: kaldi-error.h:127
void LogLikelihoods(const VectorBase< BaseFloat > &data, Vector< BaseFloat > *loglikes) const
Outputs the per-component log-likelihoods.
Definition: diag-gmm.cc:528
#define KALDI_ISNAN
Definition: kaldi-math.h:72
int32 ComputeGconsts ( )

Sets the gconsts.

Returns the number that are "invalid" e.g. because of zero weights or variances.

Definition at line 114 of file diag-gmm.cc.

References rnnlm::d, VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ASSERT, KALDI_ERR, KALDI_ISINF, KALDI_ISNAN, kaldi::Log(), M_LOG_2PI, DiagGmm::means_invvars_, DiagGmm::NumGauss(), Vector< Real >::Resize(), DiagGmm::valid_gconsts_, and DiagGmm::weights_.

Referenced by Sgmm2Project::ApplyProjection(), DiagGmm::CopyFromFullGmm(), DiagGmm::DiagGmm(), init_rand_diag_gmm(), kaldi::InitGmmFromRandomFrames(), kaldi::unittest::InitRandDiagGmm(), kaldi::InitRandomGmm(), DiagGmm::Interpolate(), main(), kaldi::MapDiagGmmUpdate(), DiagGmm::Merge(), DiagGmm::MergeKmeans(), kaldi::MleDiagGmmUpdate(), DiagGmm::Perturb(), rand_diag_gmm(), DiagGmm::Read(), kaldi::ResizeModel(), DiagGmm::Split(), test_flags_driven_update(), TestXformMean(), kaldi::UnitTestDiagGmm(), UnitTestEstimateDiagGmm(), kaldi::UnitTestEstimateMmieDiagGmm(), kaldi::UnitTestRegtreeFmllrDiagGmm(), HldaAccsDiagGmm::Update(), kaldi::UpdateEbwDiagGmm(), and kaldi::UpdateEbwWeightsDiagGmm().

114  {
115  int32 num_mix = NumGauss();
116  int32 dim = Dim();
117  BaseFloat offset = -0.5 * M_LOG_2PI * dim; // constant term in gconst.
118  int32 num_bad = 0;
119 
120  // Resize if Gaussians have been removed during Update()
121  if (num_mix != static_cast<int32>(gconsts_.Dim()))
122  gconsts_.Resize(num_mix);
123 
124  for (int32 mix = 0; mix < num_mix; mix++) {
125  KALDI_ASSERT(weights_(mix) >= 0); // Cannot have negative weights.
126  BaseFloat gc = Log(weights_(mix)) + offset; // May be -inf if weights == 0
127  for (int32 d = 0; d < dim; d++) {
128  gc += 0.5 * Log(inv_vars_(mix, d)) - 0.5 * means_invvars_(mix, d)
129  * means_invvars_(mix, d) / inv_vars_(mix, d);
130  }
131  // Change sign for logdet because var is inverted. Also, note that
132  // mean_invvars(mix, d)*mean_invvars(mix, d)/inv_vars(mix, d) is the
133  // mean-squared times inverse variance, since mean_invvars(mix, d) contains
134  // the mean times inverse variance.
135  // So gc is the likelihood at zero feature value.
136 
137  if (KALDI_ISNAN(gc)) { // negative infinity is OK but NaN is not acceptable
138  KALDI_ERR << "At component " << mix
139  << ", not a number in gconst computation";
140  }
141  if (KALDI_ISINF(gc)) {
142  num_bad++;
143  // If positive infinity, make it negative infinity.
144  // Want to make sure the answer becomes -inf in the end, not NaN.
145  if (gc > 0) gc = -gc;
146  }
147  gconsts_(mix) = gc;
148  }
149 
150  valid_gconsts_ = true;
151  return num_bad;
152 }
#define M_LOG_2PI
Definition: kaldi-math.h:60
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
#define KALDI_ISINF
Definition: kaldi-math.h:73
void Resize(MatrixIndexT length, MatrixResizeType resize_type=kSetZero)
Set vector to a specified size (can be zero).
float BaseFloat
Definition: kaldi-types.h:29
double Log(double x)
Definition: kaldi-math.h:100
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ISNAN
Definition: kaldi-math.h:72
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void CopyFromDiagGmm ( const DiagGmm diaggmm)

Copies from given DiagGmm.

Definition at line 83 of file diag-gmm.cc.

References MatrixBase< Real >::CopyFromMat(), VectorBase< Real >::CopyFromVec(), VectorBase< Real >::Dim(), DiagGmm::gconsts_, DiagGmm::inv_vars_, DiagGmm::means_invvars_, MatrixBase< Real >::NumCols(), DiagGmm::Resize(), DiagGmm::valid_gconsts_, and DiagGmm::weights_.

Referenced by AmDiagGmm::AddPdf(), kaldi::ClusterGaussiansToUbm(), AmSgmm2::CopyFromSgmm2(), AmSgmm2::CopyGlobalsInitVecs(), DiagGmm::DiagGmm(), DiagGmm::Split(), test_flags_driven_update(), test_io(), TestXformMean(), kaldi::UnitTestDiagGmm(), and UnitTestRegressionTree().

83  {
84  Resize(diaggmm.weights_.Dim(), diaggmm.means_invvars_.NumCols());
85  gconsts_.CopyFromVec(diaggmm.gconsts_);
86  weights_.CopyFromVec(diaggmm.weights_);
87  inv_vars_.CopyFromMat(diaggmm.inv_vars_);
88  means_invvars_.CopyFromMat(diaggmm.means_invvars_);
89  valid_gconsts_ = diaggmm.valid_gconsts_;
90 }
void Resize(int32 nMix, int32 dim)
Resizes arrays to this dim. Does not initialize data.
Definition: diag-gmm.cc:66
void CopyFromMat(const MatrixBase< OtherReal > &M, MatrixTransposeType trans=kNoTrans)
Copy given matrix. (no resize is done).
void CopyFromVec(const VectorBase< Real > &v)
Copy data from another vector (must match own size).
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void CopyFromFullGmm ( const FullGmm fullgmm)

Copies from given FullGmm.

Definition at line 92 of file diag-gmm.cc.

References DiagGmm::ComputeGconsts(), VectorBase< Real >::CopyDiagFromPacked(), MatrixBase< Real >::CopyFromMat(), SpMatrix< Real >::CopyFromSp(), VectorBase< Real >::CopyFromVec(), kaldi::diag, FullGmm::Dim(), FullGmm::gconsts(), DiagGmm::gconsts_, FullGmm::GetMeans(), FullGmm::inv_covars(), DiagGmm::inv_vars_, SpMatrix< Real >::Invert(), VectorBase< Real >::InvertElements(), DiagGmm::means_invvars_, MatrixBase< Real >::MulElements(), FullGmm::NumGauss(), DiagGmm::NumGauss(), DiagGmm::Resize(), MatrixBase< Real >::Row(), FullGmm::weights(), and DiagGmm::weights_.

Referenced by Sgmm2Project::ApplyProjection(), AmSgmm2::ComputeDerivedVars(), AmSgmm2::InitializeFromFullGmm(), main(), and UnitTestFullGmm().

92  {
93  int32 num_comp = fullgmm.NumGauss(), dim = fullgmm.Dim();
94  Resize(num_comp, dim);
95  gconsts_.CopyFromVec(fullgmm.gconsts());
96  weights_.CopyFromVec(fullgmm.weights());
97  Matrix<BaseFloat> means(num_comp, dim);
98  fullgmm.GetMeans(&means);
99  int32 ncomp = NumGauss();
100  for (int32 mix = 0; mix < ncomp; mix++) {
101  SpMatrix<double> covar(dim);
102  covar.CopyFromSp(fullgmm.inv_covars()[mix]);
103  covar.Invert();
104  Vector<double> diag(dim);
105  diag.CopyDiagFromPacked(covar);
106  diag.InvertElements();
107  inv_vars_.Row(mix).CopyFromVec(diag);
108  }
111  ComputeGconsts();
112 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
void Resize(int32 nMix, int32 dim)
Resizes arrays to this dim. Does not initialize data.
Definition: diag-gmm.cc:66
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
void CopyFromMat(const MatrixBase< OtherReal > &M, MatrixTransposeType trans=kNoTrans)
Copy given matrix. (no resize is done).
const SubVector< Real > Row(MatrixIndexT i) const
Return specific row of matrix [const].
Definition: kaldi-matrix.h:182
void CopyFromVec(const VectorBase< Real > &v)
Copy data from another vector (must match own size).
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
void MulElements(const MatrixBase< Real > &A)
Element by element multiplication with a given matrix.
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void CopyFromNormal ( const DiagGmmNormal diag_gmm_normal)

Copies from DiagGmmNormal; does not resize.

Definition at line 918 of file diag-gmm.cc.

References DiagGmmNormal::CopyToDiagGmm().

Referenced by kaldi::InitGmmFromRandomFrames().

918  {
919  diag_gmm_normal.CopyToDiagGmm(this);
920 }
int32 Dim ( ) const
inline

Returns the dimensionality of the Gaussian mean vectors.

Definition at line 74 of file diag-gmm.h.

References DiagGmm::means_invvars_, and MatrixBase< Real >::NumCols().

Referenced by AccumFullGmm::AccumulateFromDiag(), AccumDiagGmm::AccumulateFromDiag(), MlltAccs::AccumulateFromPosteriors(), AmDiagGmm::AddPdf(), AmSgmm2::Check(), DiagGmm::ComponentLogLikelihood(), Fmpe::ComputeC(), AmSgmm2::ComputeDerivedVars(), DiagGmm::ComputeGconsts(), DiagGmmNormal::CopyFromDiagGmm(), FullGmm::CopyFromDiagGmm(), DiagGmmNormal::CopyToDiagGmm(), kaldi::DiagGmmToStats(), kaldi::DoRescalingUpdate(), Fmpe::FeatDim(), FmllrDiagGmmAccs::FmllrDiagGmmAccs(), AmSgmm2::GaussianSelection(), DiagGmm::Generate(), DiagGmm::GetComponentMean(), DiagGmm::GetComponentVariance(), DiagGmm::GetMeans(), kaldi::GetStatsDerivative(), DiagGmm::GetVars(), init_rand_diag_gmm(), DiagGmm::Interpolate(), DiagGmm::LogLikelihoods(), DiagGmm::LogLikelihoodsPreselect(), DecodableAmDiagGmmRegtreeFmllr::LogLikelihoodZeroBased(), DecodableAmDiagGmmUnmapped::LogLikelihoodZeroBased(), DecodableAmDiagGmmRegtreeMllr::LogLikelihoodZeroBased(), main(), kaldi::MapDiagGmmUpdate(), DiagGmm::Merge(), DiagGmm::MergeKmeans(), kaldi::MleDiagGmmUpdate(), DiagGmm::Perturb(), AccumDiagGmm::Resize(), DiagGmm::SetComponentInvVar(), DiagGmm::SetComponentMean(), DiagGmm::SetInvVars(), AccumDiagGmm::SmoothWithModel(), DiagGmm::Split(), test_flags_driven_update(), TestComponentAcc(), kaldi::UnitTestDiagGmm(), kaldi::UnitTestDiagGmmGenerate(), UnitTestEstimateDiagGmm(), kaldi::UnitTestEstimateMmieDiagGmm(), kaldi::UnitTestFmllrDiagGmm(), kaldi::UnitTestFmllrDiagGmmDiagonal(), kaldi::UnitTestFmllrDiagGmmOffset(), kaldi::UnitTestFmllrRaw(), and kaldi::UpdateEbwDiagGmm().

74 { return means_invvars_.NumCols(); }
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
BaseFloat GaussianSelection ( const VectorBase< BaseFloat > &  data,
int32  num_gselect,
std::vector< int32 > *  output 
) const

Get gaussian selection information for one frame.

Returns og-like this frame. Output is the best "num_gselect" indices, sorted from best to worst likelihood. If "num_gselect" > NumGauss(), sets it to NumGauss().

Definition at line 765 of file diag-gmm.cc.

References VectorBase< Real >::Data(), rnnlm::j, KALDI_ASSERT, kaldi::kUndefined, kaldi::LogAdd(), DiagGmm::LogLikelihoods(), and DiagGmm::NumGauss().

Referenced by DiagGmm::GaussianSelection(), and main().

767  {
768  int32 num_gauss = NumGauss();
769  Vector<BaseFloat> loglikes(num_gauss, kUndefined);
770  output->clear();
771  this->LogLikelihoods(data, &loglikes);
772 
773  BaseFloat thresh;
774  if (num_gselect < num_gauss) {
775  Vector<BaseFloat> loglikes_copy(loglikes);
776  BaseFloat *ptr = loglikes_copy.Data();
777  std::nth_element(ptr, ptr+num_gauss-num_gselect, ptr+num_gauss);
778  thresh = ptr[num_gauss-num_gselect];
779  } else {
780  thresh = -std::numeric_limits<BaseFloat>::infinity();
781  }
782  BaseFloat tot_loglike = -std::numeric_limits<BaseFloat>::infinity();
783  std::vector<std::pair<BaseFloat, int32> > pairs;
784  for (int32 p = 0; p < num_gauss; p++) {
785  if (loglikes(p) >= thresh) {
786  pairs.push_back(std::make_pair(loglikes(p), p));
787  }
788  }
789  std::sort(pairs.begin(), pairs.end(),
790  std::greater<std::pair<BaseFloat, int32> >());
791  for (int32 j = 0;
792  j < num_gselect && j < static_cast<int32>(pairs.size());
793  j++) {
794  output->push_back(pairs[j].second);
795  tot_loglike = LogAdd(tot_loglike, pairs[j].first);
796  }
797  KALDI_ASSERT(!output->empty());
798  return tot_loglike;
799 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
float BaseFloat
Definition: kaldi-types.h:29
void LogLikelihoods(const VectorBase< BaseFloat > &data, Vector< BaseFloat > *loglikes) const
Outputs the per-component log-likelihoods.
Definition: diag-gmm.cc:528
double LogAdd(double x, double y)
Definition: kaldi-math.h:184
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
BaseFloat GaussianSelection ( const MatrixBase< BaseFloat > &  data,
int32  num_gselect,
std::vector< std::vector< int32 > > *  output 
) const

This version of the Gaussian selection function works for a sequence of frames rather than just a single frame.

Returns sum of the log-likes over all frames.

Definition at line 801 of file diag-gmm.cc.

References VectorBase< Real >::Data(), DiagGmm::GaussianSelection(), rnnlm::i, rnnlm::j, KALDI_ASSERT, kaldi::kUndefined, kaldi::LogAdd(), DiagGmm::LogLikelihoods(), MatrixBase< Real >::NumCols(), DiagGmm::NumGauss(), and MatrixBase< Real >::NumRows().

803  {
804  double ans = 0.0;
805  int32 num_frames = data.NumRows(), num_gauss = NumGauss();
806 
807  int32 max_mem = 10000000; // Don't devote more than 10Mb to loglikes_mat;
808  // break up the utterance if needed.
809  int32 mem_needed = num_frames * num_gauss * sizeof(BaseFloat);
810  if (mem_needed > max_mem) {
811  // Break into parts and recurse, we don't want to consume too
812  // much memory.
813  int32 num_parts = (mem_needed + max_mem - 1) / max_mem;
814  int32 part_frames = (data.NumRows() + num_parts - 1) / num_parts;
815  double tot_ans = 0.0;
816  std::vector<std::vector<int32> > part_output;
817  output->clear();
818  output->resize(num_frames);
819  for (int32 p = 0; p < num_parts; p++) {
820  int32 start_frame = p * part_frames,
821  this_num_frames = std::min(num_frames - start_frame, part_frames);
822  SubMatrix<BaseFloat> data_part(data, start_frame, this_num_frames,
823  0, data.NumCols());
824  tot_ans += GaussianSelection(data_part, num_gselect, &part_output);
825  for (int32 t = 0; t < this_num_frames; t++)
826  (*output)[start_frame + t].swap(part_output[t]);
827  }
828  KALDI_ASSERT(!output->back().empty());
829  return tot_ans;
830  }
831 
832  KALDI_ASSERT(num_frames != 0);
833  Matrix<BaseFloat> loglikes_mat(num_frames, num_gauss, kUndefined);
834  this->LogLikelihoods(data, &loglikes_mat);
835 
836  output->clear();
837  output->resize(num_frames);
838 
839  for (int32 i = 0; i < num_frames; i++) {
840  SubVector<BaseFloat> loglikes(loglikes_mat, i);
841 
842  BaseFloat thresh;
843  if (num_gselect < num_gauss) {
844  Vector<BaseFloat> loglikes_copy(loglikes);
845  BaseFloat *ptr = loglikes_copy.Data();
846  std::nth_element(ptr, ptr+num_gauss-num_gselect, ptr+num_gauss);
847  thresh = ptr[num_gauss-num_gselect];
848  } else {
849  thresh = -std::numeric_limits<BaseFloat>::infinity();
850  }
851  BaseFloat tot_loglike = -std::numeric_limits<BaseFloat>::infinity();
852  std::vector<std::pair<BaseFloat, int32> > pairs;
853  for (int32 p = 0; p < num_gauss; p++) {
854  if (loglikes(p) >= thresh) {
855  pairs.push_back(std::make_pair(loglikes(p), p));
856  }
857  }
858  std::sort(pairs.begin(), pairs.end(),
859  std::greater<std::pair<BaseFloat, int32> >());
860  std::vector<int32> &this_output = (*output)[i];
861  for (int32 j = 0;
862  j < num_gselect && j < static_cast<int32>(pairs.size());
863  j++) {
864  this_output.push_back(pairs[j].second);
865  tot_loglike = LogAdd(tot_loglike, pairs[j].first);
866  }
867  KALDI_ASSERT(!this_output.empty());
868  ans += tot_loglike;
869  }
870  return ans;
871 }
BaseFloat GaussianSelection(const VectorBase< BaseFloat > &data, int32 num_gselect, std::vector< int32 > *output) const
Get gaussian selection information for one frame.
Definition: diag-gmm.cc:765
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
float BaseFloat
Definition: kaldi-types.h:29
void LogLikelihoods(const VectorBase< BaseFloat > &data, Vector< BaseFloat > *loglikes) const
Outputs the per-component log-likelihoods.
Definition: diag-gmm.cc:528
double LogAdd(double x, double y)
Definition: kaldi-math.h:184
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:58
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
BaseFloat GaussianSelectionPreselect ( const VectorBase< BaseFloat > &  data,
const std::vector< int32 > &  preselect,
int32  num_gselect,
std::vector< int32 > *  output 
) const

Get gaussian selection information for one frame.

Returns log-like for this frame. Output is the best "num_gselect" indices that were preselected, sorted from best to worst likelihood. If "num_gselect" > NumGauss(), sets it to NumGauss().

Definition at line 875 of file diag-gmm.cc.

References VectorBase< Real >::Data(), rnnlm::j, KALDI_ASSERT, KALDI_WARN, kaldi::LogAdd(), and DiagGmm::LogLikelihoodsPreselect().

Referenced by main().

879  {
880  static bool warned_size = false;
881  int32 preselect_sz = preselect.size();
882  int32 this_num_gselect = std::min(num_gselect, preselect_sz);
883  if (preselect_sz <= num_gselect && !warned_size) {
884  warned_size = true;
885  KALDI_WARN << "Preselect size is less or equal to than final size, "
886  << "doing nothing: " << preselect_sz << " < " << num_gselect
887  << " [won't warn again]";
888  }
889  Vector<BaseFloat> loglikes(preselect_sz);
890  LogLikelihoodsPreselect(data, preselect, &loglikes);
891 
892  Vector<BaseFloat> loglikes_copy(loglikes);
893  BaseFloat *ptr = loglikes_copy.Data();
894  std::nth_element(ptr, ptr+preselect_sz-this_num_gselect,
895  ptr+preselect_sz);
896  BaseFloat thresh = ptr[preselect_sz-this_num_gselect];
897 
898  BaseFloat tot_loglike = -std::numeric_limits<BaseFloat>::infinity();
899  // we want the output sorted from best likelihood to worse
900  // (so we can prune further without the model)...
901  std::vector<std::pair<BaseFloat, int32> > pairs;
902  for (int32 p = 0; p < preselect_sz; p++)
903  if (loglikes(p) >= thresh)
904  pairs.push_back(std::make_pair(loglikes(p), preselect[p]));
905  std::sort(pairs.begin(), pairs.end(),
906  std::greater<std::pair<BaseFloat, int32> >());
907  output->clear();
908  for (int32 j = 0;
909  j < this_num_gselect && j < static_cast<int32>(pairs.size());
910  j++) {
911  output->push_back(pairs[j].second);
912  tot_loglike = LogAdd(tot_loglike, pairs[j].first);
913  }
914  KALDI_ASSERT(!output->empty());
915  return tot_loglike;
916 }
void LogLikelihoodsPreselect(const VectorBase< BaseFloat > &data, const std::vector< int32 > &indices, Vector< BaseFloat > *loglikes) const
Outputs the per-component log-likelihoods of a subset of mixture components.
Definition: diag-gmm.cc:566
float BaseFloat
Definition: kaldi-types.h:29
#define KALDI_WARN
Definition: kaldi-error.h:130
double LogAdd(double x, double y)
Definition: kaldi-math.h:184
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
const Vector<BaseFloat>& gconsts ( ) const
inline

Const accessors.

Definition at line 174 of file diag-gmm.h.

References DiagGmm::gconsts_, KALDI_ASSERT, and DiagGmm::valid_gconsts_.

Referenced by FullGmm::CopyFromDiagGmm(), HldaAccsDiagGmm::Init(), DecodableAmDiagGmmRegtreeFmllr::LogLikelihoodZeroBased(), DecodableAmDiagGmmUnmapped::LogLikelihoodZeroBased(), kaldi::MlObjective(), and HldaAccsDiagGmm::Update().

174  {
176  return gconsts_;
177  }
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
void Generate ( VectorBase< BaseFloat > *  output)

Generates a random data-point from this distribution.

Definition at line 922 of file diag-gmm.cc.

References rnnlm::d, VectorBase< Real >::Dim(), DiagGmm::Dim(), rnnlm::i, DiagGmm::inv_vars_, KALDI_ASSERT, DiagGmm::means_invvars_, kaldi::RandGauss(), kaldi::RandUniform(), VectorBase< Real >::Sum(), and DiagGmm::weights_.

Referenced by TestMllrAccsIO(), kaldi::UnitTestDiagGmmGenerate(), kaldi::UnitTestFmllrDiagGmm(), kaldi::UnitTestFmllrDiagGmmDiagonal(), kaldi::UnitTestFmllrDiagGmmOffset(), and UnitTestRegtreeMllrDiagGmm().

922  {
923  KALDI_ASSERT(static_cast<int32>(output->Dim()) == Dim());
924  BaseFloat tot = weights_.Sum();
925  KALDI_ASSERT(tot > 0.0);
926  double r = tot * RandUniform() * 0.99999;
927  int32 i = 0;
928  double sum = 0.0;
929  while (sum + weights_(i) < r) {
930  sum += weights_(i);
931  i++;
932  KALDI_ASSERT(i < static_cast<int32>(weights_.Dim()));
933  }
934  // now i is the index of the Gaussian we chose.
935  SubVector<BaseFloat> inv_var(inv_vars_, i),
936  mean_invvar(means_invvars_, i);
937  for (int32 d = 0; d < inv_var.Dim(); d++) {
938  BaseFloat stddev = 1.0 / sqrt(inv_var(d)),
939  mean = mean_invvar(d) / inv_var(d);
940  (*output)(d) = mean + RandGauss() * stddev;
941  }
942 }
float RandUniform(struct RandomState *state=NULL)
Returns a random number strictly between 0 and 1.
Definition: kaldi-math.h:151
float RandGauss(struct RandomState *state=NULL)
Definition: kaldi-math.h:155
Real Sum() const
Returns sum of the elements.
float BaseFloat
Definition: kaldi-types.h:29
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void GetComponentMean ( int32  gauss,
VectorBase< Real > *  out 
) const

Accessor for single component mean.

Definition at line 135 of file diag-gmm-inl.h.

References VectorBase< Real >::CopyRowFromMat(), VectorBase< Real >::Dim(), DiagGmm::Dim(), VectorBase< Real >::DivElements(), DiagGmm::inv_vars_, KALDI_ASSERT, DiagGmm::means_invvars_, and DiagGmm::NumGauss().

Referenced by RegtreeMllrDiagGmmAccs::AccumulateForGaussian(), RegtreeMllrDiagGmmAccs::AccumulateForGmm(), kaldi::ClusterGaussiansToUbm(), kaldi::UnitTestDiagGmm(), and UnitTestRegressionTree().

135  {
136  KALDI_ASSERT(gauss < NumGauss());
137  KALDI_ASSERT(static_cast<int32>(out->Dim()) == Dim());
138  Vector<Real> tmp(Dim());
139  tmp.CopyRowFromMat(inv_vars_, gauss);
140  out->CopyRowFromMat(means_invvars_, gauss);
141  out->DivElements(tmp);
142 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void GetComponentVariance ( int32  gauss,
VectorBase< Real > *  out 
) const

Accessor for single component variance.

Definition at line 145 of file diag-gmm-inl.h.

References VectorBase< Real >::CopyRowFromMat(), VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::inv_vars_, VectorBase< Real >::InvertElements(), KALDI_ASSERT, and DiagGmm::NumGauss().

Referenced by kaldi::ClusterGaussiansToUbm().

145  {
146  KALDI_ASSERT(gauss < NumGauss());
147  KALDI_ASSERT(static_cast<int32>(out->Dim()) == Dim());
148  out->CopyRowFromMat(inv_vars_, gauss);
149  out->InvertElements();
150 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void GetMeans ( Matrix< Real > *  m) const

Accessor for means.

Definition at line 123 of file diag-gmm-inl.h.

References MatrixBase< Real >::CopyFromMat(), DiagGmm::Dim(), DiagGmm::inv_vars_, MatrixBase< Real >::InvertElements(), KALDI_ASSERT, DiagGmm::means_invvars_, MatrixBase< Real >::MulElements(), DiagGmm::NumGauss(), and Matrix< Real >::Resize().

Referenced by BasisFmllrEstimate::ComputeAmDiagPrecond(), main(), AccumDiagGmm::SmoothWithModel(), test_flags_driven_update(), kaldi::UnitTestDiagGmm(), and UnitTestRegressionTree().

123  {
124  KALDI_ASSERT(m != NULL);
125  m->Resize(NumGauss(), Dim());
126  Matrix<Real> vars(NumGauss(), Dim());
127  vars.CopyFromMat(inv_vars_);
128  vars.InvertElements();
129  m->CopyFromMat(means_invvars_);
130  m->MulElements(vars);
131 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void GetVars ( Matrix< Real > *  v) const

Accessor for covariances.

Definition at line 115 of file diag-gmm-inl.h.

References MatrixBase< Real >::CopyFromMat(), DiagGmm::Dim(), DiagGmm::inv_vars_, MatrixBase< Real >::InvertElements(), KALDI_ASSERT, DiagGmm::NumGauss(), and Matrix< Real >::Resize().

Referenced by BasisFmllrEstimate::ComputeAmDiagPrecond(), AccumDiagGmm::SmoothWithModel(), test_flags_driven_update(), and kaldi::UnitTestDiagGmm().

115  {
116  KALDI_ASSERT(v != NULL);
117  v->Resize(NumGauss(), Dim());
118  v->CopyFromMat(inv_vars_);
119  v->InvertElements();
120 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Interpolate ( BaseFloat  rho,
const DiagGmm source,
GmmFlagsType  flags = kGmmAll 
)

this = rho x source + (1-rho) x this

Definition at line 645 of file diag-gmm.cc.

References DiagGmm::ComputeGconsts(), DiagGmm::Dim(), KALDI_ASSERT, kaldi::kGmmMeans, kaldi::kGmmVariances, kaldi::kGmmWeights, DiagGmmNormal::means_, DiagGmm::NumGauss(), DiagGmmNormal::vars_, and DiagGmmNormal::weights_.

646  {
647  KALDI_ASSERT(NumGauss() == source.NumGauss());
648  KALDI_ASSERT(Dim() == source.Dim());
649 
650  DiagGmmNormal us(*this);
651  DiagGmmNormal them(source);
652 
653  if (flags & kGmmWeights) {
654  us.weights_.Scale(1.0 - rho);
655  us.weights_.AddVec(rho, them.weights_);
656  us.weights_.Scale(1.0 / us.weights_.Sum());
657  }
658 
659  if (flags & kGmmMeans) {
660  us.means_.Scale(1.0 - rho);
661  us.means_.AddMat(rho, them.means_);
662  }
663 
664  if (flags & kGmmVariances) {
665  us.vars_.Scale(1.0 - rho);
666  us.vars_.AddMat(rho, them.vars_);
667  }
668 
669  us.CopyToDiagGmm(this);
670  ComputeGconsts();
671 }
friend class DiagGmmNormal
this makes it a little easier to modify the internals
Definition: diag-gmm.h:44
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void Interpolate ( BaseFloat  rho,
const FullGmm source,
GmmFlagsType  flags = kGmmAll 
)

this = rho x source + (1-rho) x this

Definition at line 673 of file diag-gmm.cc.

References DiagGmm::ComputeGconsts(), kaldi::diag, FullGmm::Dim(), DiagGmm::Dim(), rnnlm::i, rnnlm::j, KALDI_ASSERT, kaldi::kGmmMeans, kaldi::kGmmVariances, kaldi::kGmmWeights, FullGmmNormal::means_, FullGmm::NumGauss(), DiagGmm::NumGauss(), FullGmmNormal::vars_, and FullGmmNormal::weights_.

674  {
675  KALDI_ASSERT(NumGauss() == source.NumGauss());
676  KALDI_ASSERT(Dim() == source.Dim());
677  DiagGmmNormal us(*this);
678  FullGmmNormal them(source);
679 
680  if (flags & kGmmWeights) {
681  us.weights_.Scale(1.0 - rho);
682  us.weights_.AddVec(rho, them.weights_);
683  us.weights_.Scale(1.0 / us.weights_.Sum());
684  }
685 
686  if (flags & kGmmMeans) {
687  us.means_.Scale(1.0 - rho);
688  us.means_.AddMat(rho, them.means_);
689  }
690 
691  if (flags & kGmmVariances) {
692  for (int32 i = 0; i < NumGauss(); i++) {
693  us.vars_.Scale(1. - rho);
694  Vector<double> diag(Dim());
695  for (int32 j = 0; j < Dim(); j++)
696  diag(j) = them.vars_[i](j, j);
697  us.vars_.Row(i).AddVec(rho, diag);
698  }
699  }
700 
701  us.CopyToDiagGmm(this);
702  ComputeGconsts();
703 }
friend class DiagGmmNormal
this makes it a little easier to modify the internals
Definition: diag-gmm.h:44
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
BaseFloat LogLikelihood ( const VectorBase< BaseFloat > &  data) const

Returns the log-likelihood of a data point (vector) given the GMM.

Definition at line 517 of file diag-gmm.cc.

References KALDI_ERR, KALDI_ISINF, KALDI_ISNAN, DiagGmm::LogLikelihoods(), VectorBase< Real >::LogSumExp(), and DiagGmm::valid_gconsts_.

Referenced by kaldi::GetGmmLike(), main(), test_flags_driven_update(), test_io(), TestComponentAcc(), TestXformMean(), kaldi::UnitTestDiagGmm(), and UnitTestFullGmm().

517  {
518  if (!valid_gconsts_)
519  KALDI_ERR << "Must call ComputeGconsts() before computing likelihood";
520  Vector<BaseFloat> loglikes;
521  LogLikelihoods(data, &loglikes);
522  BaseFloat log_sum = loglikes.LogSumExp();
523  if (KALDI_ISNAN(log_sum) || KALDI_ISINF(log_sum))
524  KALDI_ERR << "Invalid answer (overflow or invalid variances/features?)";
525  return log_sum;
526 }
#define KALDI_ISINF
Definition: kaldi-math.h:73
float BaseFloat
Definition: kaldi-types.h:29
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
#define KALDI_ERR
Definition: kaldi-error.h:127
void LogLikelihoods(const VectorBase< BaseFloat > &data, Vector< BaseFloat > *loglikes) const
Outputs the per-component log-likelihoods.
Definition: diag-gmm.cc:528
#define KALDI_ISNAN
Definition: kaldi-math.h:72
void LogLikelihoods ( const VectorBase< BaseFloat > &  data,
Vector< BaseFloat > *  loglikes 
) const

Outputs the per-component log-likelihoods.

Definition at line 528 of file diag-gmm.cc.

References VectorBase< Real >::AddMatVec(), VectorBase< Real >::ApplyPow(), VectorBase< Real >::CopyFromVec(), VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ERR, kaldi::kNoTrans, kaldi::kUndefined, DiagGmm::means_invvars_, and Vector< Real >::Resize().

Referenced by DiagGmm::ComponentPosteriors(), DiagGmm::GaussianSelection(), AmSgmm2::GaussianSelection(), DiagGmm::LogLikelihood(), main(), and kaldi::UnitTestDiagGmm().

529  {
530  loglikes->Resize(gconsts_.Dim(), kUndefined);
531  loglikes->CopyFromVec(gconsts_);
532  if (data.Dim() != Dim()) {
533  KALDI_ERR << "DiagGmm::ComponentLogLikelihood, dimension "
534  << "mismatch " << data.Dim() << " vs. "<< Dim();
535  }
536  Vector<BaseFloat> data_sq(data);
537  data_sq.ApplyPow(2.0);
538 
539  // loglikes += means * inv(vars) * data.
540  loglikes->AddMatVec(1.0, means_invvars_, kNoTrans, data, 1.0);
541  // loglikes += -0.5 * inv(vars) * data_sq.
542  loglikes->AddMatVec(-0.5, inv_vars_, kNoTrans, data_sq, 1.0);
543 }
void Resize(MatrixIndexT length, MatrixResizeType resize_type=kSetZero)
Set vector to a specified size (can be zero).
void CopyFromVec(const VectorBase< Real > &v)
Copy data from another vector (must match own size).
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
void AddMatVec(const Real alpha, const MatrixBase< Real > &M, const MatrixTransposeType trans, const VectorBase< Real > &v, const Real beta)
Add matrix times vector : this <– beta*this + alpha*M*v.
Definition: kaldi-vector.cc:91
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void LogLikelihoods ( const MatrixBase< BaseFloat > &  data,
Matrix< BaseFloat > *  loglikes 
) const

This version of the LogLikelihoods function operates on a sequence of frames simultaneously; the row index of both "data" and "loglikes" is the frame index.

Definition at line 546 of file diag-gmm.cc.

References MatrixBase< Real >::AddMatMat(), MatrixBase< Real >::ApplyPow(), MatrixBase< Real >::CopyRowsFromVec(), VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ASSERT, KALDI_ERR, kaldi::kNoTrans, kaldi::kTrans, kaldi::kUndefined, DiagGmm::means_invvars_, MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), and Matrix< Real >::Resize().

547  {
548  KALDI_ASSERT(data.NumRows() != 0);
549  loglikes->Resize(data.NumRows(), gconsts_.Dim(), kUndefined);
550  loglikes->CopyRowsFromVec(gconsts_);
551  if (data.NumCols() != Dim()) {
552  KALDI_ERR << "DiagGmm::ComponentLogLikelihood, dimension "
553  << "mismatch " << data.NumCols() << " vs. "<< Dim();
554  }
555  Matrix<BaseFloat> data_sq(data);
556  data_sq.ApplyPow(2.0);
557 
558  // loglikes += means * inv(vars) * data.
559  loglikes->AddMatMat(1.0, data, kNoTrans, means_invvars_, kTrans, 1.0);
560  // loglikes += -0.5 * inv(vars) * data_sq.
561  loglikes->AddMatMat(-0.5, data_sq, kNoTrans, inv_vars_, kTrans, 1.0);
562 }
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
void AddMatMat(const Real alpha, const MatrixBase< Real > &A, MatrixTransposeType transA, const MatrixBase< Real > &B, MatrixTransposeType transB, const Real beta)
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:58
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
void Resize(const MatrixIndexT r, const MatrixIndexT c, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Sets matrix to a specified size (zero is OK as long as both r and c are zero).
void CopyRowsFromVec(const VectorBase< Real > &v)
This function has two modes of operation.
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void LogLikelihoodsPreselect ( const VectorBase< BaseFloat > &  data,
const std::vector< int32 > &  indices,
Vector< BaseFloat > *  loglikes 
) const

Outputs the per-component log-likelihoods of a subset of mixture components.

Note: at output, loglikes->Dim() will equal indices.size(). loglikes[i] will correspond to the log-likelihood of the Gaussian indexed indices[i], including the mixture weight.

Definition at line 566 of file diag-gmm.cc.

References VectorBase< Real >::AddMatVec(), VectorBase< Real >::CopyFromVec(), VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::gconsts_, rnnlm::i, DiagGmm::inv_vars_, KALDI_ASSERT, kaldi::kNoTrans, kaldi::kUndefined, DiagGmm::means_invvars_, Vector< Real >::Resize(), MatrixBase< Real >::Row(), and kaldi::VecVec().

Referenced by FmllrDiagGmmAccs::AccumulateForGmmPreselect(), kaldi::AccumulateForUtterance(), MlltAccs::AccumulateFromGmmPreselect(), Fmpe::ApplyProjection(), Fmpe::ApplyProjectionReverse(), DiagGmm::GaussianSelectionPreselect(), main(), and kaldi::UnitTestDiagGmm().

568  {
569  KALDI_ASSERT(data.Dim() == Dim());
570  Vector<BaseFloat> data_sq(data);
571  data_sq.ApplyPow(2.0);
572 
573  int32 num_indices = static_cast<int32>(indices.size());
574  loglikes->Resize(num_indices, kUndefined);
575  if (indices.back() + 1 - indices.front() == num_indices) {
576  // A special (but common) case when the indices form a contiguous range.
577  int32 start_idx = indices.front();
578  loglikes->CopyFromVec(SubVector<BaseFloat>(gconsts_, start_idx, num_indices));
579  // loglikes += means * inv(vars) * data.
580  SubMatrix<BaseFloat> means_invvars_sub(means_invvars_, start_idx, num_indices,
581  0, Dim());
582  loglikes->AddMatVec(1.0, means_invvars_sub, kNoTrans, data, 1.0);
583  SubMatrix<BaseFloat> inv_vars_sub(inv_vars_, start_idx, num_indices,
584  0, Dim());
585  // loglikes += -0.5 * inv(vars) * data_sq.
586  loglikes->AddMatVec(-0.5, inv_vars_sub, kNoTrans, data_sq, 1.0);
587  } else {
588  for (int32 i = 0; i < num_indices; i++) {
589  int32 idx = indices[i]; // The Gaussian index.
590  BaseFloat this_loglike =
591  gconsts_(idx) + VecVec(means_invvars_.Row(idx), data)
592  - 0.5*VecVec(inv_vars_.Row(idx), data_sq);
593  (*loglikes)(i) = this_loglike;
594  }
595  }
596 }
void Resize(MatrixIndexT length, MatrixResizeType resize_type=kSetZero)
Set vector to a specified size (can be zero).
const SubVector< Real > Row(MatrixIndexT i) const
Return specific row of matrix [const].
Definition: kaldi-matrix.h:182
void CopyFromVec(const VectorBase< Real > &v)
Copy data from another vector (must match own size).
float BaseFloat
Definition: kaldi-types.h:29
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
void AddMatVec(const Real alpha, const MatrixBase< Real > &M, const MatrixTransposeType trans, const VectorBase< Real > &v, const Real beta)
Add matrix times vector : this <– beta*this + alpha*M*v.
Definition: kaldi-vector.cc:91
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
Real VecVec(const VectorBase< Real > &a, const VectorBase< Real > &b)
Returns dot product between v1 and v2.
Definition: kaldi-vector.cc:36
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void Merge ( int32  target_components,
std::vector< int32 > *  history = NULL 
)

Merge the components and remember the order in which the components were merged (flat list of pairs)

Definition at line 295 of file diag-gmm.cc.

References kaldi::ApproxEqual(), DiagGmm::ComputeGconsts(), rnnlm::d, DiagGmm::Dim(), DiagGmm::gconsts_, rnnlm::i, DiagGmm::inv_vars_, MatrixBase< Real >::InvertElements(), rnnlm::j, KALDI_ASSERT, KALDI_ERR, KALDI_VLOG, KALDI_WARN, kaldi::Log(), DiagGmm::means_invvars_, DiagGmm::merged_components_logdet(), MatrixBase< Real >::MulElements(), DiagGmm::NumGauss(), Vector< Real >::RemoveElement(), Matrix< Real >::RemoveRow(), Vector< Real >::Resize(), Matrix< Real >::Resize(), MatrixBase< Real >::Row(), MatrixBase< Real >::Scale(), DiagGmm::weights(), and DiagGmm::weights_.

Referenced by kaldi::ClusterGaussiansToUbm(), and kaldi::UnitTestDiagGmm().

295  {
296  if (target_components <= 0 || NumGauss() < target_components) {
297  KALDI_ERR << "Invalid argument for target number of Gaussians (="
298  << target_components << "), #Gauss = " << NumGauss();
299  }
300  if (NumGauss() == target_components) {
301  KALDI_VLOG(2) << "No components merged, as target (" << target_components
302  << ") = total.";
303  return; // Nothing to do.
304  }
305 
306  int32 num_comp = NumGauss(), dim = Dim();
307 
308  if (target_components == 1) { // global mean and variance
309  Vector<BaseFloat> weights(weights_);
310  // Undo variance inversion and multiplication of mean by inv var.
311  Matrix<BaseFloat> vars(inv_vars_);
312  Matrix<BaseFloat> means(means_invvars_);
313  vars.InvertElements();
314  means.MulElements(vars);
315  // add means square to variances; get second-order stats
316  for (int32 i = 0; i < num_comp; i++) {
317  vars.Row(i).AddVec2(1.0, means.Row(i));
318  }
319 
320  // Slightly more efficient than calling this->Resize(1, dim)
321  gconsts_.Resize(1);
322  weights_.Resize(1);
323  means_invvars_.Resize(1, dim);
324  inv_vars_.Resize(1, dim);
325 
326  for (int32 i = 0; i < num_comp; i++) {
327  weights_(0) += weights(i);
328  means_invvars_.Row(0).AddVec(weights(i), means.Row(i));
329  inv_vars_.Row(0).AddVec(weights(i), vars.Row(i));
330  }
331  if (!ApproxEqual(weights_(0), 1.0, 1e-6)) {
332  KALDI_WARN << "Weights sum to " << weights_(0) << ": rescaling.";
335  weights_(0) = 1.0;
336  }
337  inv_vars_.Row(0).AddVec2(-1.0, means_invvars_.Row(0));
340  ComputeGconsts();
341  return;
342  }
343 
344  // If more than 1 merged component is required, use the hierarchical
345  // clustering of components that lead to the smallest decrease in likelihood.
346  std::vector<bool> discarded_component(num_comp);
347  Vector<BaseFloat> logdet(num_comp); // logdet for each component
348  for (int32 i = 0; i < num_comp; i++) {
349  discarded_component[i] = false;
350  for (int32 d = 0; d < dim; d++) {
351  logdet(i) += 0.5 * Log(inv_vars_(i, d)); // +0.5 because var is inverted
352  }
353  }
354 
355  // Undo variance inversion and multiplication of mean by this
356  // Makes copy of means and vars for all components - memory inefficient?
357  Matrix<BaseFloat> vars(inv_vars_);
358  Matrix<BaseFloat> means(means_invvars_);
359  vars.InvertElements();
360  means.MulElements(vars);
361 
362  // add means square to variances; get second-order stats
363  // (normalized by zero-order stats)
364  for (int32 i = 0; i < num_comp; i++) {
365  vars.Row(i).AddVec2(1.0, means.Row(i));
366  }
367 
368  // compute change of likelihood for all combinations of components
369  SpMatrix<BaseFloat> delta_like(num_comp);
370  for (int32 i = 0; i < num_comp; i++) {
371  for (int32 j = 0; j < i; j++) {
372  BaseFloat w1 = weights_(i), w2 = weights_(j), w_sum = w1 + w2;
373  BaseFloat merged_logdet = merged_components_logdet(w1, w2,
374  means.Row(i), means.Row(j), vars.Row(i), vars.Row(j));
375  delta_like(i, j) = w_sum * merged_logdet
376  - w1 * logdet(i) - w2 * logdet(j);
377  }
378  }
379 
380  // Merge components with smallest impact on the loglike
381  for (int32 removed = 0; removed < num_comp - target_components; removed++) {
382  // Search for the least significant change in likelihood
383  // (maximum of negative delta_likes)
384  BaseFloat max_delta_like = -std::numeric_limits<BaseFloat>::max();
385  int32 max_i = -1, max_j = -1;
386  for (int32 i = 0; i < NumGauss(); i++) {
387  if (discarded_component[i]) continue;
388  for (int32 j = 0; j < i; j++) {
389  if (discarded_component[j]) continue;
390  if (delta_like(i, j) > max_delta_like) {
391  max_delta_like = delta_like(i, j);
392  max_i = i;
393  max_j = j;
394  }
395  }
396  }
397 
398  // make sure that different components will be merged
399  KALDI_ASSERT(max_i != max_j && max_i != -1 && max_j != -1);
400 
401  // remember the merge candidates
402  if (history != NULL) {
403  history->push_back(max_i);
404  history->push_back(max_j);
405  }
406 
407  // Merge components
408  BaseFloat w1 = weights_(max_i), w2 = weights_(max_j);
409  BaseFloat w_sum = w1 + w2;
410  // merge means
411  means.Row(max_i).AddVec(w2/w1, means.Row(max_j));
412  means.Row(max_i).Scale(w1/w_sum);
413  // merge vars
414  vars.Row(max_i).AddVec(w2/w1, vars.Row(max_j));
415  vars.Row(max_i).Scale(w1/w_sum);
416  // merge weights
417  weights_(max_i) = w_sum;
418 
419  // Update gmm for merged component
420  // copy second-order stats (normalized by zero-order stats)
421  inv_vars_.Row(max_i).CopyFromVec(vars.Row(max_i));
422  // centralize
423  inv_vars_.Row(max_i).AddVec2(-1.0, means.Row(max_i));
424  // invert
425  inv_vars_.Row(max_i).InvertElements();
426  // copy first-order stats (normalized by zero-order stats)
427  means_invvars_.Row(max_i).CopyFromVec(means.Row(max_i));
428  // multiply by inv_vars
429  means_invvars_.Row(max_i).MulElements(inv_vars_.Row(max_i));
430 
431  // Update logdet for merged component
432  logdet(max_i) = 0.0;
433  for (int32 d = 0; d < dim; d++) {
434  logdet(max_i) += 0.5 * Log(inv_vars_(max_i, d));
435  // +0.5 because var is inverted
436  }
437 
438  // Label the removed component as discarded
439  discarded_component[max_j] = true;
440 
441  // Update delta_like for merged component
442  for (int32 j = 0; j < num_comp; j++) {
443  if ((j == max_i) || (discarded_component[j])) continue;
444  BaseFloat w1 = weights_(max_i),
445  w2 = weights_(j),
446  w_sum = w1 + w2;
447  BaseFloat merged_logdet = merged_components_logdet(w1, w2,
448  means.Row(max_i), means.Row(j), vars.Row(max_i), vars.Row(j));
449  delta_like(max_i, j) = w_sum * merged_logdet - w1 * logdet(max_i)
450  - w2 * logdet(j);
451  // doesn't respect lower triangular indeces,
452  // relies on implicitly performed swap of coordinates if necessary
453  }
454  }
455 
456  // Remove the consumed components
457  int32 m = 0;
458  for (int32 i = 0; i < num_comp; i++) {
459  if (discarded_component[i]) {
462  inv_vars_.RemoveRow(m);
463  } else {
464  ++m;
465  }
466  }
467 
468  ComputeGconsts();
469 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
void Resize(MatrixIndexT length, MatrixResizeType resize_type=kSetZero)
Set vector to a specified size (can be zero).
const SubVector< Real > Row(MatrixIndexT i) const
Return specific row of matrix [const].
Definition: kaldi-matrix.h:182
float BaseFloat
Definition: kaldi-types.h:29
BaseFloat merged_components_logdet(BaseFloat w1, BaseFloat w2, const VectorBase< BaseFloat > &f1, const VectorBase< BaseFloat > &f2, const VectorBase< BaseFloat > &s1, const VectorBase< BaseFloat > &s2) const
Definition: diag-gmm.cc:471
double Log(double x)
Definition: kaldi-math.h:100
void Scale(Real alpha)
Multiply each element with a scalar value.
const Vector< BaseFloat > & weights() const
Definition: diag-gmm.h:178
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_WARN
Definition: kaldi-error.h:130
void MulElements(const MatrixBase< Real > &A)
Element by element multiplication with a given matrix.
void InvertElements()
Inverts all the elements of the matrix.
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
#define KALDI_VLOG(v)
Definition: kaldi-error.h:136
void Resize(const MatrixIndexT r, const MatrixIndexT c, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Sets matrix to a specified size (zero is OK as long as both r and c are zero).
void RemoveRow(MatrixIndexT i)
Remove a specified row.
void RemoveElement(MatrixIndexT i)
Remove one element and shifts later elements down.
static bool ApproxEqual(float a, float b, float relative_tolerance=0.001)
return abs(a - b) <= relative_tolerance * (abs(a)+abs(b)).
Definition: kaldi-math.h:262
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
BaseFloat merged_components_logdet ( BaseFloat  w1,
BaseFloat  w2,
const VectorBase< BaseFloat > &  f1,
const VectorBase< BaseFloat > &  f2,
const VectorBase< BaseFloat > &  s1,
const VectorBase< BaseFloat > &  s2 
) const
private

Definition at line 471 of file diag-gmm.cc.

References VectorBase< Real >::AddVec(), VectorBase< Real >::AddVec2(), VectorBase< Real >::CopyFromVec(), rnnlm::d, VectorBase< Real >::Dim(), kaldi::Log(), and VectorBase< Real >::Scale().

Referenced by DiagGmm::Merge().

476  {
477  int32 dim = f1.Dim();
478  Vector<BaseFloat> tmp_mean(dim);
479  Vector<BaseFloat> tmp_var(dim);
480 
481  BaseFloat w_sum = w1 + w2;
482  tmp_mean.CopyFromVec(f1);
483  tmp_mean.AddVec(w2/w1, f2);
484  tmp_mean.Scale(w1/w_sum);
485  tmp_var.CopyFromVec(s1);
486  tmp_var.AddVec(w2/w1, s2);
487  tmp_var.Scale(w1/w_sum);
488  tmp_var.AddVec2(-1.0, tmp_mean);
489  BaseFloat merged_logdet = 0.0;
490  for (int32 d = 0; d < dim; d++) {
491  merged_logdet -= 0.5 * Log(tmp_var(d));
492  // -0.5 because var is not inverted
493  }
494  return merged_logdet;
495 }
float BaseFloat
Definition: kaldi-types.h:29
double Log(double x)
Definition: kaldi-math.h:100
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void MergeKmeans ( int32  target_components,
ClusterKMeansOptions  cfg = ClusterKMeansOptions() 
)

Definition at line 231 of file diag-gmm.cc.

References VectorBase< Real >::AddVec2(), kaldi::ClusterKMeans(), DiagGmm::ComputeGconsts(), VectorBase< Real >::CopyFromVec(), count, GaussClusterable::count(), kaldi::DeletePointers(), DiagGmm::Dim(), DiagGmm::inv_vars_, VectorBase< Real >::InvertElements(), KALDI_ERR, KALDI_VLOG, KALDI_WARN, DiagGmm::means_invvars_, VectorBase< Real >::MulElements(), DiagGmm::NumGauss(), DiagGmm::Resize(), VectorBase< Real >::Scale(), DiagGmm::weights_, GaussClusterable::x2_stats(), and GaussClusterable::x_stats().

Referenced by kaldi::UnitTestDiagGmm().

232  {
233  if (target_components <= 0 || NumGauss() < target_components) {
234  KALDI_ERR << "Invalid argument for target number of Gaussians (="
235  << target_components << "), #Gauss = " << NumGauss();
236  }
237  if (NumGauss() == target_components) {
238  KALDI_VLOG(2) << "No components merged, as target (" << target_components
239  << ") = total.";
240  return; // Nothing to do.
241  }
242  double min_var = 1.0e-10;
243  std::vector<Clusterable*> clusterable_vec;
244  for (int32 g = 0; g < NumGauss(); g++) {
245  if (weights_(g) == 0) {
246  KALDI_WARN << "Not using zero-weight Gaussians in clustering.";
247  continue;
248  }
249  Vector<BaseFloat> x_stats(Dim()),
250  x2_stats(Dim());
251  BaseFloat count = weights_(g);
252 
253  SubVector<BaseFloat> inv_var(inv_vars_, g),
254  mean_invvar(means_invvars_, g);
255  x_stats.AddVecDivVec(1.0, mean_invvar, inv_var, count); // x_stats is now mean.
256  x2_stats.CopyFromVec(inv_var);
257  x2_stats.InvertElements(); // x2_stats is now var.
258  x2_stats.AddVec2(1.0, x_stats); // x2_stats is now var + mean^2
259  x_stats.Scale(count); // x_stats is now scaled by count.
260  x2_stats.Scale(count); // x2_stats is now scaled by count.
261  clusterable_vec.push_back(new GaussClusterable(x_stats, x2_stats, min_var,
262  count));
263  }
264  if (clusterable_vec.size() <= target_components) {
265  KALDI_WARN << "Not doing clustering phase since lost too many Gaussians "
266  << "due to zero weight. Warning: zero-weight Gaussians are "
267  << "still there.";
268  DeletePointers(&clusterable_vec);
269  return;
270  } else {
271  std::vector<Clusterable*> clusters;
272  ClusterKMeans(clusterable_vec,
273  target_components,
274  &clusters, NULL, cfg);
275  Resize(clusters.size(), Dim());
276  for (int32 g = 0; g < static_cast<int32>(clusters.size()); g++) {
277  GaussClusterable *gc = static_cast<GaussClusterable*>(clusters[g]);
278  weights_(g) = gc->count();
279  SubVector<BaseFloat> inv_var(inv_vars_, g),
280  mean_invvar(means_invvars_, g);
281  inv_var.CopyFromVec(gc->x2_stats());
282  inv_var.Scale(1.0 / gc->count()); // inv_var is now the var + mean^2
283  mean_invvar.CopyFromVec(gc->x_stats());
284  mean_invvar.Scale(1.0 / gc->count()); // mean_invvar is now the mean.
285  inv_var.AddVec2(-1.0, mean_invvar); // subtract mean^2; inv_var is now the var
286  inv_var.InvertElements(); // inv_var is now the inverse var.
287  mean_invvar.MulElements(inv_var); // mean_invvar is now mean * inverse var.
288  }
289  ComputeGconsts();
290  DeletePointers(&clusterable_vec);
291  DeletePointers(&clusters);
292  }
293 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
void Resize(int32 nMix, int32 dim)
Resizes arrays to this dim. Does not initialize data.
Definition: diag-gmm.cc:66
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
const size_t count
float BaseFloat
Definition: kaldi-types.h:29
BaseFloat ClusterKMeans(const std::vector< Clusterable * > &points, int32 num_clust, std::vector< Clusterable * > *clusters_out, std::vector< int32 > *assignments_out, ClusterKMeansOptions cfg)
ClusterKMeans is a K-means-like clustering algorithm.
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_WARN
Definition: kaldi-error.h:130
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_VLOG(v)
Definition: kaldi-error.h:136
void DeletePointers(std::vector< A * > *v)
Deletes any non-NULL pointers in the vector v, and sets the corresponding entries of v to NULL...
Definition: stl-utils.h:186
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
int32 NumGauss ( ) const
inline

Returns the number of mixture components in the GMM.

Definition at line 72 of file diag-gmm.h.

References VectorBase< Real >::Dim(), and DiagGmm::weights_.

Referenced by AccumAmDiagGmm::AccumulateForGaussian(), FmllrDiagGmmAccs::AccumulateForGmm(), FmllrRawAccs::AccumulateForGmm(), RegtreeMllrDiagGmmAccs::AccumulateForGmm(), RegtreeFmllrDiagGmmAccs::AccumulateForGmm(), AccumFullGmm::AccumulateFromDiag(), AccumDiagGmm::AccumulateFromDiag(), MlltAccs::AccumulateFromGmm(), MlltAccs::AccumulateFromGmmPreselect(), MlltAccs::AccumulateFromPosteriors(), FmllrRawAccs::AccumulateFromPosteriors(), RegressionTree::BuildTree(), AmSgmm2::Check(), kaldi::ClusterGaussiansToUbm(), BasisFmllrEstimate::ComputeAmDiagPrecond(), Fmpe::ComputeC(), AmSgmm2::ComputeDerivedVars(), DiagGmm::ComputeGconsts(), FullGmm::CopyFromDiagGmm(), DiagGmm::CopyFromFullGmm(), DiagGmm::DiagGmm(), kaldi::DiagGmmToStats(), kaldi::DoRescalingUpdate(), FmllrDiagGmmAccs::FmllrDiagGmmAccs(), DiagGmm::GaussianSelection(), AmSgmm2::GaussianSelection(), DiagGmm::GetComponentMean(), DiagGmm::GetComponentVariance(), DiagGmm::GetMeans(), kaldi::GetStatsDerivative(), RegtreeMllrDiagGmm::GetTransformedMeans(), DiagGmm::GetVars(), DecodableAmDiagGmmRegtreeMllr::GetXformedMeanInvVars(), AccumAmDiagGmm::Init(), init_rand_diag_gmm(), kaldi::InitGmmFromRandomFrames(), DiagGmm::Interpolate(), DecodableAmDiagGmmRegtreeFmllr::LogLikelihoodZeroBased(), main(), kaldi::MapDiagGmmUpdate(), DiagGmm::Merge(), DiagGmm::MergeKmeans(), kaldi::MleDiagGmmUpdate(), Fmpe::NumGauss(), DiagGmm::Perturb(), DiagGmm::RemoveComponent(), AccumDiagGmm::Resize(), kaldi::ResizeModel(), DiagGmm::SetComponentInvVar(), DiagGmm::SetComponentMean(), DiagGmm::SetComponentWeight(), DiagGmm::SetInvVars(), AccumDiagGmm::SmoothWithModel(), DiagGmm::Split(), test_flags_driven_update(), TestComponentAcc(), kaldi::TestFmpe(), TestXformMean(), kaldi::UnitTestDiagGmm(), kaldi::UnitTestDiagGmmGenerate(), UnitTestEstimateDiagGmm(), kaldi::UnitTestEstimateMmieDiagGmm(), kaldi::UnitTestFmllrDiagGmm(), kaldi::UnitTestFmllrDiagGmmDiagonal(), kaldi::UnitTestFmllrDiagGmmOffset(), UnitTestRegressionTree(), HldaAccsDiagGmm::Update(), kaldi::UpdateEbwDiagGmm(), and Fmpe::Write().

72 { return weights_.Dim(); }
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
const DiagGmm& operator= ( const DiagGmm other)
private
void Perturb ( float  perturb_factor)

Perturbs the component means with a random vector multiplied by the pertrub factor.

Definition at line 215 of file diag-gmm.cc.

References MatrixBase< Real >::AddMat(), DiagGmm::ComputeGconsts(), rnnlm::d, DiagGmm::Dim(), rnnlm::i, DiagGmm::inv_vars_, kaldi::kNoTrans, DiagGmm::means_invvars_, DiagGmm::NumGauss(), and kaldi::RandGauss().

Referenced by init_rand_diag_gmm(), kaldi::InitRandomGmm(), and main().

215  {
216  int32 num_comps = NumGauss(),
217  dim = Dim();
218  Matrix<BaseFloat> rand_mat(num_comps, dim);
219  for (int32 i = 0; i < num_comps; i++) {
220  for (int32 d = 0; d < dim; d++) {
221  rand_mat(i, d) = RandGauss() * std::sqrt(inv_vars_(i, d));
222  // as in DiagGmm::Split, we perturb the means_invvars using a random
223  // fraction of inv_vars_
224  }
225  }
226  means_invvars_.AddMat(perturb_factor, rand_mat, kNoTrans);
227  ComputeGconsts();
228 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
void AddMat(const Real alpha, const MatrixBase< Real > &M, MatrixTransposeType transA=kNoTrans)
*this += alpha * M [or M^T]
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
float RandGauss(struct RandomState *state=NULL)
Definition: kaldi-math.h:155
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void Read ( std::istream &  in,
bool  binary 
)

Definition at line 728 of file diag-gmm.cc.

References DiagGmm::ComputeGconsts(), kaldi::ExpectToken(), DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ERR, DiagGmm::means_invvars_, Vector< Real >::Read(), Matrix< Real >::Read(), kaldi::ReadToken(), and DiagGmm::weights_.

Referenced by main(), kaldi::operator>>(), Fmpe::Read(), AmSgmm2::Read(), and kaldi::UnitTestDiagGmm().

728  {
729 // ExpectToken(is, binary, "<DiagGMMBegin>");
730  std::string token;
731  ReadToken(is, binary, &token);
732  // <DiagGMMBegin> is for compatibility. Will be deleted later
733  if (token != "<DiagGMMBegin>" && token != "<DiagGMM>")
734  KALDI_ERR << "Expected <DiagGMM>, got " << token;
735  ReadToken(is, binary, &token);
736  if (token == "<GCONSTS>") { // The gconsts are optional.
737  gconsts_.Read(is, binary);
738  ExpectToken(is, binary, "<WEIGHTS>");
739  } else {
740  if (token != "<WEIGHTS>")
741  KALDI_ERR << "DiagGmm::Read, expected <WEIGHTS> or <GCONSTS>, got "
742  << token;
743  }
744  weights_.Read(is, binary);
745  ExpectToken(is, binary, "<MEANS_INVVARS>");
746  means_invvars_.Read(is, binary);
747  ExpectToken(is, binary, "<INV_VARS>");
748  inv_vars_.Read(is, binary);
749 // ExpectToken(is, binary, "<DiagGMMEnd>");
750  ReadToken(is, binary, &token);
751  // <DiagGMMEnd> is for compatibility. Will be deleted later
752  if (token != "<DiagGMMEnd>" && token != "</DiagGMM>")
753  KALDI_ERR << "Expected </DiagGMM>, got " << token;
754 
755  ComputeGconsts(); // safer option than trusting the read gconsts
756 }
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
void ReadToken(std::istream &is, bool binary, std::string *str)
ReadToken gets the next token and puts it in str (exception on failure).
Definition: io-funcs.cc:154
void Read(std::istream &in, bool binary, bool add=false)
read from stream.
void ExpectToken(std::istream &is, bool binary, const char *token)
ExpectToken tries to read in the given token, and throws an exception on failure. ...
Definition: io-funcs.cc:188
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
void Read(std::istream &in, bool binary, bool add=false)
Read function using C++ streams.
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void RemoveComponent ( int32  gauss,
bool  renorm_weights 
)

Removes single component from model.

Definition at line 617 of file diag-gmm.cc.

References DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ASSERT, KALDI_ERR, DiagGmm::means_invvars_, DiagGmm::NumGauss(), Vector< Real >::RemoveElement(), Matrix< Real >::RemoveRow(), VectorBase< Real >::Scale(), VectorBase< Real >::Sum(), DiagGmm::valid_gconsts_, and DiagGmm::weights_.

Referenced by DiagGmm::RemoveComponents().

617  {
618  KALDI_ASSERT(gauss < NumGauss());
619  if (NumGauss() == 1)
620  KALDI_ERR << "Attempting to remove the only remaining component.";
621  weights_.RemoveElement(gauss);
622  gconsts_.RemoveElement(gauss);
623  means_invvars_.RemoveRow(gauss);
624  inv_vars_.RemoveRow(gauss);
625  BaseFloat sum_weights = weights_.Sum();
626  if (renorm_weights) {
627  weights_.Scale(1.0/sum_weights);
628  valid_gconsts_ = false;
629  }
630 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
Real Sum() const
Returns sum of the elements.
float BaseFloat
Definition: kaldi-types.h:29
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
void Scale(Real alpha)
Multiplies all elements by this constant.
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
void RemoveRow(MatrixIndexT i)
Remove a specified row.
void RemoveElement(MatrixIndexT i)
Remove one element and shifts later elements down.
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void RemoveComponents ( const std::vector< int32 > &  gauss,
bool  renorm_weights 
)

Removes multiple components from model; "gauss" must not have dups.

Definition at line 632 of file diag-gmm.cc.

References rnnlm::i, kaldi::IsSortedAndUniq(), rnnlm::j, KALDI_ASSERT, and DiagGmm::RemoveComponent().

Referenced by kaldi::MleDiagGmmUpdate().

633  {
634  std::vector<int32> gauss(gauss_in);
635  std::sort(gauss.begin(), gauss.end());
637  // If efficiency is later an issue, will code this specially (unlikely).
638  for (size_t i = 0; i < gauss.size(); i++) {
639  RemoveComponent(gauss[i], renorm_weights);
640  for (size_t j = i + 1; j < gauss.size(); j++)
641  gauss[j]--;
642  }
643 }
void RemoveComponent(int32 gauss, bool renorm_weights)
Removes single component from model.
Definition: diag-gmm.cc:617
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
bool IsSortedAndUniq(const std::vector< T > &vec)
Returns true if the vector is sorted and contains each element only once.
Definition: stl-utils.h:63
void Resize ( int32  nMix,
int32  dim 
)

Resizes arrays to this dim. Does not initialize data.

Definition at line 66 of file diag-gmm.cc.

References VectorBase< Real >::Dim(), DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ASSERT, DiagGmm::means_invvars_, MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), Vector< Real >::Resize(), Matrix< Real >::Resize(), MatrixBase< Real >::Set(), DiagGmm::valid_gconsts_, and DiagGmm::weights_.

Referenced by Sgmm2Project::ApplyProjection(), kaldi::ClusterGaussiansToUbm(), DiagGmm::CopyFromDiagGmm(), DiagGmm::CopyFromFullGmm(), DiagGmm::DiagGmm(), kaldi::unittest::InitRandDiagGmm(), kaldi::InitRandomGmm(), main(), DiagGmm::MergeKmeans(), kaldi::ResizeModel(), TestComponentAcc(), kaldi::UnitTestDiagGmm(), UnitTestEstimateDiagGmm(), kaldi::UnitTestEstimateMmieDiagGmm(), UnitTestFullGmm(), UnitTestRegressionTree(), and kaldi::UnitTestRegtreeFmllrDiagGmm().

66  {
67  KALDI_ASSERT(nmix > 0 && dim > 0);
68  if (gconsts_.Dim() != nmix) gconsts_.Resize(nmix);
69  if (weights_.Dim() != nmix) weights_.Resize(nmix);
70  if (inv_vars_.NumRows() != nmix ||
71  inv_vars_.NumCols() != dim) {
72  inv_vars_.Resize(nmix, dim);
73  inv_vars_.Set(1.0);
74  // must be initialized to unit for case of calling SetMeans while having
75  // covars/invcovars that are not set yet (i.e. zero)
76  }
77  if (means_invvars_.NumRows() != nmix ||
78  means_invvars_.NumCols() != dim)
79  means_invvars_.Resize(nmix, dim);
80  valid_gconsts_ = false;
81 }
void Resize(MatrixIndexT length, MatrixResizeType resize_type=kSetZero)
Set vector to a specified size (can be zero).
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:58
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
void Resize(const MatrixIndexT r, const MatrixIndexT c, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Sets matrix to a specified size (zero is OK as long as both r and c are zero).
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void Set(Real)
Sets all elements to a specific value.
void SetComponentInvVar ( int32  gauss,
const VectorBase< Real > &  in 
)

Set inv-var for single component (recommend to do this before setting the mean, if doing both, for numerical reasons).

Definition at line 97 of file diag-gmm-inl.h.

References VectorBase< Real >::CopyFromVec(), VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::inv_vars_, VectorBase< Real >::InvertElements(), KALDI_ASSERT, DiagGmm::means_invvars_, DiagGmm::NumGauss(), MatrixBase< Real >::Row(), and DiagGmm::valid_gconsts_.

97  {
98  KALDI_ASSERT(g < NumGauss() && v.Dim() == Dim());
99 
100  int32 dim = Dim();
101  Vector<Real> mean(dim), var(dim);
102 
103  var.CopyFromVec(inv_vars_.Row(g));
104  var.InvertElements(); // This inversion happens in double if Real == double
105  mean.CopyFromVec(means_invvars_.Row(g));
106  mean.MulElements(var); // This is a real mean now.
107  mean.MulElements(v); // currently, v is inverted (in double if Real == double)
108  means_invvars_.Row(g).CopyFromVec(mean); // Mean times new inverse variance
109  inv_vars_.Row(g).CopyFromVec(v);
110  valid_gconsts_ = false;
111 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
const SubVector< Real > Row(MatrixIndexT i) const
Return specific row of matrix [const].
Definition: kaldi-matrix.h:182
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void SetComponentMean ( int32  gauss,
const VectorBase< Real > &  in 
)

Mutators for single component, supports float or double Set mean for a single component - internally multiplies with inv(var)

Definition at line 52 of file diag-gmm-inl.h.

References MatrixBase< Real >::CopyRowFromVec(), VectorBase< Real >::Dim(), DiagGmm::Dim(), DiagGmm::inv_vars_, KALDI_ASSERT, DiagGmm::means_invvars_, DiagGmm::NumGauss(), and DiagGmm::valid_gconsts_.

Referenced by kaldi::UnitTestDiagGmm().

52  {
53  KALDI_ASSERT(g < NumGauss() && Dim() == in.Dim());
54  Vector<Real> tmp(Dim());
55  tmp.CopyRowFromMat(inv_vars_, g);
56  tmp.MulElements(in);
58  valid_gconsts_ = false;
59 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void CopyRowFromVec(const VectorBase< Real > &v, const MatrixIndexT row)
Copy vector into specific row of matrix.
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void SetComponentWeight ( int32  gauss,
BaseFloat  weight 
)
inline

Set weight for single component.

Definition at line 34 of file diag-gmm-inl.h.

References KALDI_ASSERT, DiagGmm::NumGauss(), DiagGmm::valid_gconsts_, and DiagGmm::weights_.

34  {
35  KALDI_ASSERT(w > 0.0);
36  KALDI_ASSERT(g < NumGauss());
37  weights_(g) = w;
38  valid_gconsts_ = false;
39 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
void SetInvVars ( const MatrixBase< Real > &  v)

Set the (inverse) variances and recompute means_invvars_.

Definition at line 78 of file diag-gmm-inl.h.

References MatrixBase< Real >::CopyFromMat(), DiagGmm::Dim(), DiagGmm::inv_vars_, MatrixBase< Real >::InvertElements(), KALDI_ASSERT, DiagGmm::means_invvars_, MatrixBase< Real >::MulElements(), MatrixBase< Real >::NumCols(), DiagGmm::NumGauss(), MatrixBase< Real >::NumRows(), and DiagGmm::valid_gconsts_.

Referenced by kaldi::ResizeModel(), test_flags_driven_update(), and kaldi::UnitTestDiagGmm().

78  {
79  KALDI_ASSERT(inv_vars_.NumRows() == v.NumRows()
80  && inv_vars_.NumCols() == v.NumCols());
81 
82  int32 num_comp = NumGauss(), dim = Dim();
83  Matrix<Real> means(num_comp, dim);
84  Matrix<Real> vars(num_comp, dim);
85 
86  vars.CopyFromMat(inv_vars_);
87  vars.InvertElements(); // This inversion happens in double if Real == double
88  means.CopyFromMat(means_invvars_);
89  means.MulElements(vars); // These are real means now
90  means.MulElements(v); // v is inverted (in double if Real == double)
91  means_invvars_.CopyFromMat(means); // Means times new inverse variance
93  valid_gconsts_ = false;
94 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
void CopyFromMat(const MatrixBase< OtherReal > &M, MatrixTransposeType trans=kNoTrans)
Copy given matrix. (no resize is done).
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:58
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void SetInvVarsAndMeans ( const MatrixBase< Real > &  invvars,
const MatrixBase< Real > &  means 
)

Use SetInvVarsAndMeans if updating both means and (inverse) variances.

Definition at line 63 of file diag-gmm-inl.h.

References MatrixBase< Real >::CopyFromMat(), DiagGmm::inv_vars_, KALDI_ASSERT, DiagGmm::means_invvars_, MatrixBase< Real >::MulElements(), MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), and DiagGmm::valid_gconsts_.

Referenced by kaldi::ClusterGaussiansToUbm(), DiagGmm::DiagGmm(), init_rand_diag_gmm(), kaldi::unittest::InitRandDiagGmm(), kaldi::InitRandomGmm(), main(), rand_diag_gmm(), TestXformMean(), kaldi::UnitTestDiagGmm(), UnitTestEstimateDiagGmm(), kaldi::UnitTestEstimateMmieDiagGmm(), and kaldi::UnitTestRegtreeFmllrDiagGmm().

64  {
65  KALDI_ASSERT(means_invvars_.NumRows() == means.NumRows()
66  && means_invvars_.NumCols() == means.NumCols()
67  && inv_vars_.NumRows() == invvars.NumRows()
68  && inv_vars_.NumCols() == invvars.NumCols());
69 
70  inv_vars_.CopyFromMat(invvars);
71  Matrix<Real> new_means_invvars(means);
72  new_means_invvars.MulElements(invvars);
73  means_invvars_.CopyFromMat(new_means_invvars);
74  valid_gconsts_ = false;
75 }
void CopyFromMat(const MatrixBase< OtherReal > &M, MatrixTransposeType trans=kNoTrans)
Copy given matrix. (no resize is done).
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:58
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void SetMeans ( const MatrixBase< Real > &  m)

Use SetMeans to update only the Gaussian means (and not variances)

Definition at line 43 of file diag-gmm-inl.h.

References MatrixBase< Real >::CopyFromMat(), DiagGmm::inv_vars_, KALDI_ASSERT, DiagGmm::means_invvars_, MatrixBase< Real >::MulElements(), MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), and DiagGmm::valid_gconsts_.

Referenced by main(), test_flags_driven_update(), kaldi::UnitTestDiagGmm(), and UnitTestRegressionTree().

43  {
44  KALDI_ASSERT(means_invvars_.NumRows() == m.NumRows()
45  && means_invvars_.NumCols() == m.NumCols());
48  valid_gconsts_ = false;
49 }
void CopyFromMat(const MatrixBase< OtherReal > &M, MatrixTransposeType trans=kNoTrans)
Copy given matrix. (no resize is done).
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
void MulElements(const MatrixBase< Real > &A)
Element by element multiplication with a given matrix.
MatrixIndexT NumRows() const
Returns number of rows (or zero for emtpy matrix).
Definition: kaldi-matrix.h:58
MatrixIndexT NumCols() const
Returns number of columns (or zero for emtpy matrix).
Definition: kaldi-matrix.h:61
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
void SetWeights ( const VectorBase< Real > &  w)

Mutators for both float or double.

Set mixure weights

Definition at line 28 of file diag-gmm-inl.h.

References VectorBase< Real >::CopyFromVec(), VectorBase< Real >::Dim(), KALDI_ASSERT, DiagGmm::valid_gconsts_, and DiagGmm::weights_.

Referenced by kaldi::ClusterGaussiansToUbm(), DiagGmm::DiagGmm(), init_rand_diag_gmm(), kaldi::unittest::InitRandDiagGmm(), kaldi::InitRandomGmm(), main(), rand_diag_gmm(), test_flags_driven_update(), kaldi::UnitTestDiagGmm(), UnitTestEstimateDiagGmm(), kaldi::UnitTestEstimateMmieDiagGmm(), and kaldi::UnitTestRegtreeFmllrDiagGmm().

28  {
29  KALDI_ASSERT(weights_.Dim() == w.Dim());
31  valid_gconsts_ = false;
32 }
void CopyFromVec(const VectorBase< Real > &v)
Copy data from another vector (must match own size).
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:169
MatrixIndexT Dim() const
Returns the dimension of the vector.
Definition: kaldi-vector.h:62
void Split ( int32  target_components,
float  perturb_factor,
std::vector< int32 > *  history = NULL 
)

Split the components and remember the order in which the components were split.

Definition at line 154 of file diag-gmm.cc.

References DiagGmm::ComputeGconsts(), DiagGmm::CopyFromDiagGmm(), DiagGmm::DiagGmm(), DiagGmm::Dim(), DiagGmm::gconsts_, rnnlm::i, DiagGmm::inv_vars_, KALDI_ERR, KALDI_WARN, DiagGmm::means_invvars_, DiagGmm::NumGauss(), kaldi::RandGauss(), VectorBase< Real >::Range(), MatrixBase< Real >::Range(), Vector< Real >::Resize(), Matrix< Real >::Resize(), MatrixBase< Real >::Row(), and DiagGmm::weights_.

Referenced by main(), kaldi::UnitTestDiagGmm(), UnitTestEstimateDiagGmm(), and kaldi::UnitTestEstimateMmieDiagGmm().

155  {
156  if (target_components < NumGauss() || NumGauss() == 0) {
157  KALDI_ERR << "Cannot split from " << NumGauss() << " to "
158  << target_components << " components";
159  }
160  if (target_components == NumGauss()) {
161  KALDI_WARN << "Already have the target # of Gaussians. Doing nothing.";
162  return;
163  }
164 
165  int32 current_components = NumGauss(), dim = Dim();
166  DiagGmm *tmp = new DiagGmm;
167  tmp->CopyFromDiagGmm(*this); // so we have copies of matrices
168  // First do the resize:
169  weights_.Resize(target_components);
170  weights_.Range(0, current_components).CopyFromVec(tmp->weights_);
171  means_invvars_.Resize(target_components, dim);
172  means_invvars_.Range(0, current_components, 0, dim).CopyFromMat(
173  tmp->means_invvars_);
174  inv_vars_.Resize(target_components, dim);
175  inv_vars_.Range(0, current_components, 0, dim).CopyFromMat(tmp->inv_vars_);
176  gconsts_.Resize(target_components);
177 
178  delete tmp;
179 
180  // future work(arnab): Use a priority queue instead?
181  while (current_components < target_components) {
182  BaseFloat max_weight = weights_(0);
183  int32 max_idx = 0;
184  for (int32 i = 1; i < current_components; i++) {
185  if (weights_(i) > max_weight) {
186  max_weight = weights_(i);
187  max_idx = i;
188  }
189  }
190 
191  // remember what component was split
192  if (history != NULL)
193  history->push_back(max_idx);
194 
195  weights_(max_idx) /= 2;
196  weights_(current_components) = weights_(max_idx);
197  Vector<BaseFloat> rand_vec(dim);
198  for (int32 i = 0; i < dim; i++) {
199  rand_vec(i) = RandGauss() * std::sqrt(inv_vars_(max_idx, i));
200  // note, this looks wrong but is really right because it's the
201  // means_invvars we're multiplying and they have the dimension
202  // of an inverse standard variance. [dan]
203  }
204  inv_vars_.Row(current_components).CopyFromVec(inv_vars_.Row(max_idx));
205  means_invvars_.Row(current_components).CopyFromVec(means_invvars_.Row(
206  max_idx));
207  means_invvars_.Row(current_components).AddVec(perturb_factor, rand_vec);
208  means_invvars_.Row(max_idx).AddVec(-perturb_factor, rand_vec);
209  current_components++;
210  }
211  ComputeGconsts();
212 }
int32 NumGauss() const
Returns the number of mixture components in the GMM.
Definition: diag-gmm.h:72
int32 ComputeGconsts()
Sets the gconsts.
Definition: diag-gmm.cc:114
float RandGauss(struct RandomState *state=NULL)
Definition: kaldi-math.h:155
void Resize(MatrixIndexT length, MatrixResizeType resize_type=kSetZero)
Set vector to a specified size (can be zero).
const SubVector< Real > Row(MatrixIndexT i) const
Return specific row of matrix [const].
Definition: kaldi-matrix.h:182
float BaseFloat
Definition: kaldi-types.h:29
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
SubMatrix< Real > Range(const MatrixIndexT row_offset, const MatrixIndexT num_rows, const MatrixIndexT col_offset, const MatrixIndexT num_cols) const
Return a sub-part of matrix.
Definition: kaldi-matrix.h:196
int32 Dim() const
Returns the dimensionality of the Gaussian mean vectors.
Definition: diag-gmm.h:74
#define KALDI_WARN
Definition: kaldi-error.h:130
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
void Resize(const MatrixIndexT r, const MatrixIndexT c, MatrixResizeType resize_type=kSetZero, MatrixStrideType stride_type=kDefaultStride)
Sets matrix to a specified size (zero is OK as long as both r and c are zero).
DiagGmm()
Empty constructor.
Definition: diag-gmm.h:48
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236
SubVector< Real > Range(const MatrixIndexT o, const MatrixIndexT l)
Returns a sub-vector of a vector (a range of elements).
Definition: kaldi-vector.h:92
bool valid_gconsts ( ) const
inline

Definition at line 181 of file diag-gmm.h.

References DiagGmm::valid_gconsts_.

Referenced by DecodableAmDiagGmmRegtreeFmllr::LogLikelihoodZeroBased(), and DecodableAmDiagGmmUnmapped::LogLikelihoodZeroBased().

181 { return valid_gconsts_; }
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
void Write ( std::ostream &  os,
bool  binary 
) const

Definition at line 705 of file diag-gmm.cc.

References DiagGmm::gconsts_, DiagGmm::inv_vars_, KALDI_ERR, DiagGmm::means_invvars_, DiagGmm::valid_gconsts_, DiagGmm::weights_, VectorBase< Real >::Write(), MatrixBase< Real >::Write(), and kaldi::WriteToken().

Referenced by main(), kaldi::operator<<(), kaldi::UnitTestDiagGmm(), Fmpe::Write(), and AmSgmm2::Write().

705  {
706  if (!valid_gconsts_)
707  KALDI_ERR << "Must call ComputeGconsts() before writing the model.";
708  WriteToken(out_stream, binary, "<DiagGMM>");
709  if (!binary) out_stream << "\n";
710  WriteToken(out_stream, binary, "<GCONSTS>");
711  gconsts_.Write(out_stream, binary);
712  WriteToken(out_stream, binary, "<WEIGHTS>");
713  weights_.Write(out_stream, binary);
714  WriteToken(out_stream, binary, "<MEANS_INVVARS>");
715  means_invvars_.Write(out_stream, binary);
716  WriteToken(out_stream, binary, "<INV_VARS>");
717  inv_vars_.Write(out_stream, binary);
718  WriteToken(out_stream, binary, "</DiagGMM>");
719  if (!binary) out_stream << "\n";
720 }
void Write(std::ostream &Out, bool binary) const
Writes to C++ stream (option to write in binary).
bool valid_gconsts_
Recompute gconsts_ if false.
Definition: diag-gmm.h:233
Matrix< BaseFloat > inv_vars_
Inverted (diagonal) variances.
Definition: diag-gmm.h:235
#define KALDI_ERR
Definition: kaldi-error.h:127
void Write(std::ostream &out, bool binary) const
write to stream.
void WriteToken(std::ostream &os, bool binary, const char *token)
The WriteToken functions are for writing nonempty sequences of non-space characters.
Definition: io-funcs.cc:134
Vector< BaseFloat > weights_
weights (not log).
Definition: diag-gmm.h:234
Vector< BaseFloat > gconsts_
Equals log(weight) - 0.5 * (log det(var) + mean*mean*inv(var))
Definition: diag-gmm.h:232
Matrix< BaseFloat > means_invvars_
Means times inverted variance.
Definition: diag-gmm.h:236

Friends And Related Function Documentation

friend class DiagGmmNormal
friend

this makes it a little easier to modify the internals

Definition at line 44 of file diag-gmm.h.

Member Data Documentation


The documentation for this class was generated from the following files: