FmllrRawAccs Class Reference

#include <fmllr-raw.h>

Collaboration diagram for FmllrRawAccs:

Classes
struct	SingleFrameStats

Public Member Functions
	FmllrRawAccs ()
int32	RawDim () const
	Dimension of raw MFCC (etc.) features. More...
int32	FullDim () const
	Full feature dimension after splicing. More...
int32	SpliceWidth () const
	Number of frames that are spliced together each time. More...
int32	ModelDim () const
	Dimension of the model. More...
	FmllrRawAccs (int32 raw_dim, int32 model_dim, const Matrix< BaseFloat > &full_transform)
BaseFloat	AccumulateForGmm (const DiagGmm &gmm, const VectorBase< BaseFloat > &data, BaseFloat weight)
	Accumulate stats for a single GMM in the model; returns log likelihood. More...
void	AccumulateFromPosteriors (const DiagGmm &gmm, const VectorBase< BaseFloat > &data, const VectorBase< BaseFloat > &posteriors)
	Accumulate stats for a GMM, given supplied posteriors. More...
void	Update (const FmllrRawOptions &opts, MatrixBase< BaseFloat > *raw_fmllr_mat, BaseFloat *objf_impr, BaseFloat *count)
	Update "raw_fmllr_mat"; it should have the correct dimension and reasonable values at entry (see the function InitFmllr in fmllr-diag-gmm.h for how to initialize it.) The only reason this function is not const is because we may have to call CommitSingleFrameStats(). More...
void	SetZero ()

Private Member Functions
void	CommitSingleFrameStats ()
void	InitSingleFrameStats (const VectorBase< BaseFloat > &data)
bool	DataHasChanged (const VectorBase< BaseFloat > &data) const
double	GetAuxf (const Vector< double > &simple_linear_stats, const SpMatrix< double > &simple_quadratic_stats, const Matrix< double > &fmllr_mat) const
	Compute the auxiliary function for this matrix. More...
void	ConvertToSimpleStats (Vector< double > *simple_linear_stats, SpMatrix< double > *simple_quadratic_stats) const
	Converts from the Q and S stats to a simple objective function of the form l . More...
void	ComputeM (std::vector< Matrix< double > > *M) const
	Computes the M_i matrices used in the update, see the extended comment in fmllr-raw.cc for explanation. More...
void	ConvertToPerRowStats (const Vector< double > &simple_linear_stats, const SpMatrix< double > &simple_quadratic_stats_sp, Matrix< double > *linear_stats, std::vector< SpMatrix< double > > *diag_stats, std::vector< std::vector< Matrix< double > > > *off_diag_stats) const
	Transform stats into a convenient format for the update. More...
	KALDI_DISALLOW_COPY_AND_ASSIGN (FmllrRawAccs)

Private Attributes
int32	raw_dim_
int32	model_dim_
Matrix< BaseFloat >	full_transform_
Vector< BaseFloat >	transform_offset_
SingleFrameStats	single_frame_stats_
double	count_
SpMatrix< double >	temp_
Matrix< double >	Q_
Matrix< double >	S_

Detailed Description

Definition at line 81 of file fmllr-raw.h.

Constructor & Destructor Documentation

FmllrRawAccs() [1/2]

FmllrRawAccs ( )

inline

Definition at line 83 of file fmllr-raw.h.

{ }

FmllrRawAccs() [2/2]

FmllrRawAccs	(	int32	raw_dim,
		int32	model_dim,
		const Matrix< BaseFloat > &	full_transform
	)

Definition at line 29 of file fmllr-raw.cc.

References FmllrRawAccs::SingleFrameStats::a, FmllrRawAccs::SingleFrameStats::b, FmllrRawAccs::SingleFrameStats::count, FmllrRawAccs::count_, FmllrRawAccs::full_transform_, KALDI_ERR, MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), FmllrRawAccs::Q_, MatrixBase< Real >::Range(), SpMatrix< Real >::Resize(), Vector< Real >::Resize(), Matrix< Real >::Resize(), FmllrRawAccs::SingleFrameStats::s, FmllrRawAccs::S_, FmllrRawAccs::single_frame_stats_, FmllrRawAccs::temp_, FmllrRawAccs::transform_offset_, and FmllrRawAccs::SingleFrameStats::transformed_data.

                                                                   :
    raw_dim_(raw_dim),
    model_dim_(model_dim) {
  if (full_transform.NumCols() != full_transform.NumRows() &&
      full_transform.NumCols() != full_transform.NumRows() + 1) {
    KALDI_ERR << "Expecting full LDA+MLLT transform to be square or d by d+1 "
              << "(make sure you are including rejected rows).";
  }
  if (raw_dim <= 0 || full_transform.NumRows() % raw_dim != 0)
    KALDI_ERR << "Raw feature dimension is invalid " << raw_dim
              << "(must be positive and divide feature dimension)";
  int32 full_dim = full_transform.NumRows();
  full_transform_ = full_transform.Range(0, full_dim, 0, full_dim);
  transform_offset_.Resize(full_dim);
  if (full_transform_.NumCols() == full_dim + 1)
    transform_offset_.CopyColFromMat(full_transform_, full_dim);
  
  int32 full_dim2 = ((full_dim+1)*(full_dim+2))/2;
  count_ = 0.0;

  temp_.Resize(full_dim + 1);
  Q_.Resize(model_dim + 1, full_dim + 1);
  S_.Resize(model_dim + 1, full_dim2);

  single_frame_stats_.s.Resize(full_dim + 1);
  single_frame_stats_.transformed_data.Resize(full_dim);
  single_frame_stats_.count = 0.0;
  single_frame_stats_.a.Resize(model_dim);
  single_frame_stats_.b.Resize(model_dim);
}

Member Function Documentation

AccumulateForGmm()

BaseFloat AccumulateForGmm	(	const DiagGmm &	gmm,
		const VectorBase< BaseFloat > &	data,
		BaseFloat	weight
	)

Accumulate stats for a single GMM in the model; returns log likelihood.

Here, "data" will typically be of larger dimension than the model. Note: "data" is the original, spliced features– before LDA+MLLT. Returns log-like for this data given this GMM, including rejected dimensions (not multiplied by weight).

Definition at line 107 of file fmllr-raw.cc.

References FmllrRawAccs::AccumulateFromPosteriors(), FmllrRawAccs::CommitSingleFrameStats(), DiagGmm::ComponentPosteriors(), FmllrRawAccs::DataHasChanged(), VectorBase< Real >::Dim(), FmllrRawAccs::FullDim(), FmllrRawAccs::InitSingleFrameStats(), KALDI_ASSERT, M_LOG_2PI, FmllrRawAccs::ModelDim(), DiagGmm::NumGauss(), VectorBase< Real >::Scale(), FmllrRawAccs::single_frame_stats_, FmllrRawAccs::SingleFrameStats::transformed_data, and kaldi::VecVec().

Referenced by kaldi::AccStatsForUtterance(), and kaldi::UnitTestFmllrRaw().

                                                           {
  int32 model_dim = ModelDim(), full_dim = FullDim();
  KALDI_ASSERT(data.Dim() == full_dim &&
               "Expect raw, spliced data, which should have same dimension as "
               "full transform.");
  if (DataHasChanged(data)) {
    // this is part of our mechanism to accumulate certain sub-parts of
    // the computation for each frame, to avoid excessive compute.
    CommitSingleFrameStats();
    InitSingleFrameStats(data);
  }
  SingleFrameStats &stats = single_frame_stats_;

  SubVector<BaseFloat> projected_data(stats.transformed_data, 0, model_dim);

  int32 num_gauss = gmm.NumGauss();
  Vector<BaseFloat> posterior(num_gauss);
  BaseFloat log_like = gmm.ComponentPosteriors(projected_data, &posterior);
  posterior.Scale(weight);
  // Note: AccumulateFromPosteriors takes the original, spliced data,
  // and returns the log-like of the rejected dimensions.
  AccumulateFromPosteriors(gmm, data, posterior);

  // Add the likelihood of the rejected dimensions to the objective function
  // (assume zero-mean, unit-variance Gaussian; the LDA should have any offset
  // required to ensure this).
  if (full_dim > model_dim) {
    SubVector<BaseFloat> rejected_data(stats.transformed_data,
                                       model_dim, full_dim - model_dim);
    log_like += -0.5 * (VecVec(rejected_data, rejected_data)
                        + (full_dim - model_dim) * M_LOG_2PI);
  }
  return log_like;
}

AccumulateFromPosteriors()

void AccumulateFromPosteriors	(	const DiagGmm &	gmm,
		const VectorBase< BaseFloat > &	data,
		const VectorBase< BaseFloat > &	posteriors
	)

Accumulate stats for a GMM, given supplied posteriors.

Note: "data" is the original, spliced features– before LDA+MLLT.

Definition at line 246 of file fmllr-raw.cc.

References FmllrRawAccs::SingleFrameStats::a, VectorBase< Real >::AddVec(), VectorBase< Real >::AddVecVec(), FmllrRawAccs::SingleFrameStats::b, FmllrRawAccs::CommitSingleFrameStats(), count, FmllrRawAccs::SingleFrameStats::count, FmllrRawAccs::DataHasChanged(), FmllrRawAccs::InitSingleFrameStats(), DiagGmm::inv_vars(), DiagGmm::means_invvars(), FmllrRawAccs::ModelDim(), DiagGmm::NumGauss(), MatrixBase< Real >::Row(), FmllrRawAccs::single_frame_stats_, and FmllrRawAccs::transform_offset_.

Referenced by kaldi::AccStatsForUtterance(), and FmllrRawAccs::AccumulateForGmm().

                                            {
  // The user may call this function directly, even though we also
  // call it from AccumulateForGmm(), so check again:
  if (DataHasChanged(data)) { 
    CommitSingleFrameStats();
    InitSingleFrameStats(data);
  }
  
  int32  model_dim = ModelDim();

  SingleFrameStats &stats = single_frame_stats_;
  
  // The quantities a and b describe the diagonal auxiliary function
  // for each of the retained dimensions in the transformed space--
  // in the format F = \sum_d alpha(d) x(d)  -0.5 beta(d) x(d)^2,
  // where x(d) is the d'th dimensional fully processed feature.
  // For d, see the comment-- it's alpha processed to take into
  // account any offset in the LDA.  Note that it's a reference.
  //
  Vector<double> a(model_dim), b(model_dim);
  
  int32 num_comp = diag_gmm.NumGauss();
  
  double count = 0.0; // data-count contribution from this frame.

  // Note: we could do this using matrix-matrix operations instead of
  // row by row.  In the end it won't really matter as this is not
  // the slowest part of the computation.
  for (size_t m = 0; m < num_comp; m++) {
    BaseFloat this_post = posterior(m);
    if (this_post != 0.0) {
      count += this_post;
      a.AddVec(this_post, diag_gmm.means_invvars().Row(m));
      b.AddVec(this_post, diag_gmm.inv_vars().Row(m));
    }
  }
  // Correct "a" for any offset term in the LDA transform-- we view it as
  // the opposite offset in the model [note: we'll handle the rejected dimensions
  // in update time.]  Here, multiplying the element of "b" (which is the
  // weighted inv-vars) by transform_offset_, and subtracting the result from
  // a, is like subtracting the transform-offset from the original means
  // (because a contains the means times inv-vars_.
  Vector<double> offset(transform_offset_.Range(0, model_dim));
  a.AddVecVec(-1.0, b, offset, 1.0);
  stats.a.AddVec(1.0, a);
  stats.b.AddVec(1.0, b);
  stats.count += count;
}

CommitSingleFrameStats()

void CommitSingleFrameStats ( )

private

Definition at line 68 of file fmllr-raw.cc.

References FmllrRawAccs::SingleFrameStats::a, SpMatrix< Real >::AddVec2(), MatrixBase< Real >::AddVecVec(), FmllrRawAccs::SingleFrameStats::b, FmllrRawAccs::SingleFrameStats::count, FmllrRawAccs::count_, PackedMatrix< Real >::Data(), FmllrRawAccs::FullDim(), FmllrRawAccs::ModelDim(), FmllrRawAccs::Q_, VectorBase< Real >::Range(), FmllrRawAccs::SingleFrameStats::s, FmllrRawAccs::S_, PackedMatrix< Real >::SetZero(), FmllrRawAccs::single_frame_stats_, and FmllrRawAccs::temp_.

Referenced by FmllrRawAccs::AccumulateForGmm(), FmllrRawAccs::AccumulateFromPosteriors(), and FmllrRawAccs::Update().

                                          {
  // Commit the stats for this from (in SingleFrameStats).
  int32 model_dim = ModelDim(), full_dim = FullDim();
  SingleFrameStats &stats = single_frame_stats_;
  if (stats.count == 0.0) return;

  count_ += stats.count;

  // a_ext and b_ext are a and b extended with the count,
  // which we'll later use to reconstruct the full stats for
  // the rejected dimensions.
  Vector<double> a_ext(model_dim + 1), b_ext(model_dim + 1);
  a_ext.Range(0, model_dim).CopyFromVec(stats.a);
  b_ext.Range(0, model_dim).CopyFromVec(stats.b);
  a_ext(model_dim) = stats.count;
  b_ext(model_dim) = stats.count;
  Q_.AddVecVec(1.0, a_ext, Vector<double>(stats.s));

  temp_.SetZero();
  temp_.AddVec2(1.0, stats.s);
  int32 full_dim2 = ((full_dim + 1) * (full_dim + 2)) / 2;
  SubVector<double> temp_vec(temp_.Data(), full_dim2);
  S_.AddVecVec(1.0, b_ext, temp_vec);
}

ComputeM()

void ComputeM ( std::vector< Matrix< double > > *  M ) const

private

Computes the M_i matrices used in the update, see the extended comment in fmllr-raw.cc for explanation.

Definition at line 412 of file fmllr-raw.cc.

References FmllrRawAccs::full_transform_, FmllrRawAccs::FullDim(), rnnlm::i, rnnlm::j, and FmllrRawAccs::RawDim().

Referenced by FmllrRawAccs::ConvertToSimpleStats().

                                                               {
  int32 full_dim = FullDim(), raw_dim = RawDim(),
      raw_dim2 = raw_dim * (raw_dim + 1);
  M->resize(full_dim);
  for (int32 i = 0; i < full_dim; i++)
    (*M)[i].Resize(raw_dim2, full_dim + 1);  

  // the N's are simpler matrices from which we'll interpolate the M's.
  // In this loop we imagine w are computing the vector of N's, but
  // when we get each element, if it's nonzero we propagate it straight
  // to the M's.
  for (int32 i = 0; i < full_dim; i++) {
    // i is index after fMLLR transform; i1 is splicing index,
    // i2 is cepstral index.
    int32 i1 = i / raw_dim, i2 = i % raw_dim;
    for (int32 j = 0; j < raw_dim2; j++) {
      // j1 is row-index of fMLLR transform, j2 is column-index
      int32 j1 = j / (raw_dim + 1), j2 = j % (raw_dim + 1);
      for (int32 k = 0; k < full_dim + 1; k++) {
        BaseFloat n_ijk;
        if (j1 != i2) {
          n_ijk = 0.0;
        } else if (k == full_dim) {
          if (j2 == raw_dim) // offset term in fMLLR matrix.
            n_ijk = 1.0;
          else
            n_ijk = 0.0;
        } else {
          // k1 is splicing index, k2 is cepstral idnex.
          int32 k1 = k / raw_dim, k2 = k % raw_dim;
          if (k1 != i1 || k2 != j2)
            n_ijk = 0.0;
          else
            n_ijk = 1.0;
        }
        if (n_ijk != 0.0)
          for (int32 l = 0; l < full_dim; l++)
            (*M)[l](j, k) += n_ijk * full_transform_(l, i);
      }
    }
  }
}

ConvertToPerRowStats()

void ConvertToPerRowStats ( const Vector< double > &  simple_linear_stats, const SpMatrix< double > &  simple_quadratic_stats_sp, Matrix< double > *  linear_stats, std::vector< SpMatrix< double > > *  diag_stats, std::vector< std::vector< Matrix< double > > > *  off_diag_stats  ) const

private

Transform stats into a convenient format for the update.

linear_stats is of dim RawDim() by RawDim() + 1, it's the linear term. diag_stats (of dimension RawDim(), each element of dimension RawDim() + 1 is the quadratic terms w.r.t. the diagonals. off_diag_stats contains the cross-terms between different rows; it is indexed [i][j], with 0 <= i < RawDim(), and j < i, and each element is of dimension RawDim() + 1 by RawDim() + 1. The [i][j]'th element is interpreted as follows: the inner product with the [i'th row] [element [i][j]] [j'th row] is the term in the objective function. This function resizes its output.

Definition at line 493 of file fmllr-raw.cc.

References MatrixBase< Real >::CopyRowsFromVec(), rnnlm::i, rnnlm::j, kaldi::kTakeMean, FmllrRawAccs::RawDim(), and Matrix< Real >::Resize().

Referenced by FmllrRawAccs::Update().

                                                                  {

  // get it as a Matrix, which makes it easier to extract sub-parts.
  Matrix<double> simple_quadratic_stats(simple_quadratic_stats_sp);

  linear_stats->Resize(RawDim(), RawDim() + 1);
  linear_stats->CopyRowsFromVec(simple_linear_stats);
  diag_stats->resize(RawDim());
  off_diag_stats->resize(RawDim());

  // Set *diag_stats
  int32 rd1 = RawDim() + 1;
  for (int32 i = 0; i < RawDim(); i++) {
    SubMatrix<double> this_diag(simple_quadratic_stats,
                                i * rd1, rd1,
                                i * rd1, rd1);
    (*diag_stats)[i].Resize(RawDim() + 1);
    (*diag_stats)[i].CopyFromMat(this_diag, kTakeMean);
  }    
  
  for (int32 i = 0; i < RawDim(); i++) {
    (*off_diag_stats)[i].resize(i);
    for (int32 j = 0; j < i; j++) {
      SubMatrix<double> this_off_diag(simple_quadratic_stats,
                                      i * rd1, rd1,
                                      j * rd1, rd1);
      (*off_diag_stats)[i][j] = this_off_diag;
    }
  }
}

ConvertToSimpleStats()

void ConvertToSimpleStats ( Vector< double > *  simple_linear_stats, SpMatrix< double > *  simple_quadratic_stats  ) const

private

Converts from the Q and S stats to a simple objective function of the form l .

simple_linear_stats -0.5 l^t simple_quadratic_stats l, plus the determinant term, where l is the linearized transform.

Definition at line 455 of file fmllr-raw.cc.

References VectorBase< Real >::AddMatVec(), SpMatrix< Real >::AddSmat2Sp(), FmllrRawAccs::ComputeM(), VectorBase< Real >::CopyFromVec(), PackedMatrix< Real >::Data(), FmllrRawAccs::FullDim(), rnnlm::i, kaldi::kNoTrans, FmllrRawAccs::ModelDim(), FmllrRawAccs::Q_, FmllrRawAccs::RawDim(), SpMatrix< Real >::Resize(), Vector< Real >::Resize(), MatrixBase< Real >::Row(), FmllrRawAccs::S_, VectorBase< Real >::Scale(), and FmllrRawAccs::transform_offset_.

Referenced by FmllrRawAccs::Update().

                                                    {
  std::vector<Matrix<double> > M;
  ComputeM(&M);

  int32 full_dim = FullDim(), raw_dim = RawDim(), model_dim = ModelDim(),
      raw_dim2 = raw_dim * (raw_dim + 1),
      full_dim2 = ((full_dim+1)*(full_dim+2))/2;
  simple_linear_stats->Resize(raw_dim2);
  simple_quadratic_stats->Resize(raw_dim2);
  for (int32 i = 0; i < full_dim; i++) {
    Vector<double> q_i(full_dim + 1);
    SpMatrix<double> S_i(full_dim + 1);
    SubVector<double> S_i_vec(S_i.Data(), full_dim2);
    if (i < model_dim) {
      q_i.CopyFromVec(Q_.Row(i));
      S_i_vec.CopyFromVec(S_.Row(i));
    } else {
      q_i.CopyFromVec(Q_.Row(model_dim)); // The last row contains stats proportional
      // to "count", which we need to modify to be correct.
      q_i.Scale(-transform_offset_(i)); // These stats are zero (corresponding to
      // a zero-mean model) if there is no offset in the LDA transform.  Note:
      // the two statements above are the equivalent, for the rejected dims,
      // of the statement "a.AddVecVec(-1.0, b, offset);" for the kept ones.
      // 
      S_i_vec.CopyFromVec(S_.Row(model_dim)); // these are correct, and
      // all the same (corresponds to unit variance).
    }
    // The equation v = \sum_i M_i q_i:
    simple_linear_stats->AddMatVec(1.0, M[i], kNoTrans, q_i, 1.0);
    // The equation W = \sum_i M_i S_i M_i^T
    // Here, M[i] is quite sparse, so AddSmat2Sp will be faster.
    simple_quadratic_stats->AddSmat2Sp(1.0, M[i], kNoTrans, S_i, 1.0);
  }
}

DataHasChanged()

bool DataHasChanged ( const VectorBase< BaseFloat > &  data ) const

private

Definition at line 63 of file fmllr-raw.cc.

References VectorBase< Real >::ApproxEqual(), VectorBase< Real >::Dim(), FmllrRawAccs::FullDim(), KALDI_ASSERT, FmllrRawAccs::SingleFrameStats::s, and FmllrRawAccs::single_frame_stats_.

Referenced by FmllrRawAccs::AccumulateForGmm(), and FmllrRawAccs::AccumulateFromPosteriors().

                                                                         {
  KALDI_ASSERT(data.Dim() == FullDim());
  return !data.ApproxEqual(single_frame_stats_.s.Range(0, FullDim()), 0.0);
}

FullDim()

int32 FullDim ( ) const

inline

Full feature dimension after splicing.

Definition at line 88 of file fmllr-raw.h.

Referenced by FmllrRawAccs::AccumulateForGmm(), FmllrRawAccs::CommitSingleFrameStats(), FmllrRawAccs::ComputeM(), FmllrRawAccs::ConvertToSimpleStats(), FmllrRawAccs::DataHasChanged(), and FmllrRawAccs::InitSingleFrameStats().

{ return full_transform_.NumRows(); }

GetAuxf()

double GetAuxf ( const Vector< double > &  simple_linear_stats, const SpMatrix< double > &  simple_quadratic_stats, const Matrix< double > &  fmllr_mat  ) const

private

Compute the auxiliary function for this matrix.

Definition at line 529 of file fmllr-raw.cc.

References VectorBase< Real >::CopyRowsFromMat(), FmllrRawAccs::count_, MatrixBase< Real >::LogDet(), FmllrRawAccs::RawDim(), FmllrRawAccs::SpliceWidth(), kaldi::VecSpVec(), and kaldi::VecVec().

Referenced by FmllrRawAccs::Update().

                                                                    {
  // linearize transform...
  int32 raw_dim = RawDim(), spice_width = SpliceWidth();
  Vector<double> fmllr_vec(raw_dim * (raw_dim + 1));
  fmllr_vec.CopyRowsFromMat(fmllr_mat);
  SubMatrix<double> square_part(fmllr_mat, 0, raw_dim,
                                0, raw_dim);
  double logdet = square_part.LogDet();
  return VecVec(fmllr_vec, simple_linear_stats) -
      0.5 * VecSpVec(fmllr_vec, simple_quadratic_stats, fmllr_vec) +
      logdet * spice_width * count_;
}

InitSingleFrameStats()

void InitSingleFrameStats ( const VectorBase< BaseFloat > &  data )

private

Definition at line 93 of file fmllr-raw.cc.

References FmllrRawAccs::SingleFrameStats::a, FmllrRawAccs::SingleFrameStats::b, FmllrRawAccs::SingleFrameStats::count, VectorBase< Real >::Dim(), FmllrRawAccs::full_transform_, FmllrRawAccs::FullDim(), KALDI_ASSERT, kaldi::kNoTrans, FmllrRawAccs::SingleFrameStats::s, VectorBase< Real >::SetZero(), FmllrRawAccs::single_frame_stats_, FmllrRawAccs::transform_offset_, and FmllrRawAccs::SingleFrameStats::transformed_data.

Referenced by FmllrRawAccs::AccumulateForGmm(), and FmllrRawAccs::AccumulateFromPosteriors().

                                                                         {
  SingleFrameStats &stats = single_frame_stats_;
  int32 full_dim = FullDim();
  KALDI_ASSERT(data.Dim() == full_dim);
  stats.s.Range(0, full_dim).CopyFromVec(data);
  stats.s(full_dim) = 1.0;
  stats.transformed_data.AddMatVec(1.0, full_transform_, kNoTrans, data, 0.0);
  stats.transformed_data.AddVec(1.0, transform_offset_);
  stats.count = 0.0;
  stats.a.SetZero();
  stats.b.SetZero();
}

KALDI_DISALLOW_COPY_AND_ASSIGN()

KALDI_DISALLOW_COPY_AND_ASSIGN ( FmllrRawAccs  )

private

ModelDim()

int32 ModelDim ( ) const

inline

Dimension of the model.

Definition at line 92 of file fmllr-raw.h.

References count.

Referenced by FmllrRawAccs::AccumulateForGmm(), FmllrRawAccs::AccumulateFromPosteriors(), FmllrRawAccs::CommitSingleFrameStats(), and FmllrRawAccs::ConvertToSimpleStats().

{ return model_dim_; }

RawDim()

int32 RawDim ( ) const

inline

Dimension of raw MFCC (etc.) features.

Definition at line 86 of file fmllr-raw.h.

Referenced by FmllrRawAccs::ComputeM(), FmllrRawAccs::ConvertToPerRowStats(), FmllrRawAccs::ConvertToSimpleStats(), FmllrRawAccs::GetAuxf(), and FmllrRawAccs::Update().

{ return raw_dim_; }

SetZero()

void SetZero

(

)

Definition at line 402 of file fmllr-raw.cc.

References FmllrRawAccs::SingleFrameStats::count, FmllrRawAccs::count_, FmllrRawAccs::Q_, FmllrRawAccs::SingleFrameStats::s, FmllrRawAccs::S_, MatrixBase< Real >::SetZero(), and FmllrRawAccs::single_frame_stats_.

Referenced by kaldi::UnitTestFmllrRaw().

                           {
  count_ = 0.0;
  single_frame_stats_.count = 0.0;
  single_frame_stats_.s.SetZero();
  Q_.SetZero();
  S_.SetZero();
}

SpliceWidth()

int32 SpliceWidth ( ) const

inline

Number of frames that are spliced together each time.

Definition at line 90 of file fmllr-raw.h.

Referenced by FmllrRawAccs::GetAuxf(), and FmllrRawAccs::Update().

{ return FullDim() / RawDim(); }

Update()

void Update	(	const FmllrRawOptions &	opts,
		MatrixBase< BaseFloat > *	raw_fmllr_mat,
		BaseFloat *	objf_impr,
		BaseFloat *	count
	)

Update "raw_fmllr_mat"; it should have the correct dimension and reasonable values at entry (see the function InitFmllr in fmllr-diag-gmm.h for how to initialize it.) The only reason this function is not const is because we may have to call CommitSingleFrameStats().

Definition at line 299 of file fmllr-raw.cc.

References VectorBase< Real >::AddMatVec(), FmllrRawAccs::CommitSingleFrameStats(), FmllrRawAccs::ConvertToPerRowStats(), FmllrRawAccs::ConvertToSimpleStats(), MatrixBase< Real >::CopyFromMat(), VectorBase< Real >::CopyFromVec(), FmllrRawAccs::SingleFrameStats::count, FmllrRawAccs::count_, kaldi::FmllrInnerUpdate(), FmllrRawAccs::GetAuxf(), kaldi::GetVerboseLevel(), rnnlm::i, MatrixBase< Real >::IsZero(), KALDI_ASSERT, KALDI_VLOG, KALDI_WARN, kaldi::kNoTrans, kaldi::kTrans, FmllrRawOptions::min_count, FmllrRawOptions::num_iters, MatrixBase< Real >::NumCols(), MatrixBase< Real >::NumRows(), FmllrRawAccs::RawDim(), MatrixBase< Real >::Row(), FmllrRawAccs::single_frame_stats_, and FmllrRawAccs::SpliceWidth().

Referenced by kaldi::UnitTestFmllrRaw().

                                            {
  // First commit any pending stats from the last frame.
  if (single_frame_stats_.count != 0.0)
    CommitSingleFrameStats();
  
  if (this->count_ < opts.min_count) {
    KALDI_WARN << "Not updating (raw) fMLLR since count " << this->count_
               << " is less than min count " << opts.min_count;
    *objf_impr = 0.0;
    *count = this->count_;
    return;
  }
  KALDI_ASSERT(raw_fmllr_mat->NumRows() == RawDim() &&
               raw_fmllr_mat->NumCols() == RawDim() + 1 &&
               !raw_fmllr_mat->IsZero());
  Matrix<double> fmllr_mat(*raw_fmllr_mat); // temporary, double-precision version
                                            // of matrix.


  Matrix<double> linear_stats; // like K in diagonal update.
  std::vector<SpMatrix<double> > diag_stats; // like G in diagonal update.
                                             // Note: we will invert these.
  std::vector<std::vector<Matrix<double> > > off_diag_stats; // these will
  // contribute to the linear term.

  Vector<double> simple_linear_stats;
  SpMatrix<double> simple_quadratic_stats;
  ConvertToSimpleStats(&simple_linear_stats, &simple_quadratic_stats);
  
  ConvertToPerRowStats(simple_linear_stats, simple_quadratic_stats,
                       &linear_stats, &diag_stats, &off_diag_stats);

  try {
    for (size_t i = 0; i < diag_stats.size(); i++) {
      diag_stats[i].Invert();
    }
  } catch (...) {
    KALDI_WARN << "Error inverting stats matrices for fMLLR "
               << "[min-count too small?  Bad data?], not updating.";
    return;
  }
  
  int32 raw_dim = RawDim(), splice_width = SpliceWidth();
  
  double effective_beta = count_ * splice_width; // We "count" the determinant
  // splice_width times in the objective function.

  double auxf_orig = GetAuxf(simple_linear_stats, simple_quadratic_stats,
                             fmllr_mat);
  for (int32 iter = 0; iter < opts.num_iters; iter++) {
    for (int32 row = 0; row < raw_dim; row++) {
      SubVector<double> this_row(fmllr_mat, row);
      Vector<double> this_linear(raw_dim + 1);  // Here, k_i is the linear term
      // in the auxf expressed as a function of this row.
      this_linear.CopyFromVec(linear_stats.Row(row));
      for (int32 row2 = 0; row2 < raw_dim; row2++) {
        if (row2 != row) {
          if (row2 < row) {
            this_linear.AddMatVec(-1.0, off_diag_stats[row][row2], kNoTrans,
                                  fmllr_mat.Row(row2), 1.0);
          } else {
            // We won't have the element [row][row2] stored, but use symmetry.
            this_linear.AddMatVec(-1.0, off_diag_stats[row2][row], kTrans,
                                  fmllr_mat.Row(row2), 1.0);
          }
        }
      }
      FmllrInnerUpdate(diag_stats[row],
                       this_linear,
                       effective_beta,
                       row,
                       &fmllr_mat);
    }
    if (GetVerboseLevel() >= 2) {
      double cur_auxf = GetAuxf(simple_linear_stats, simple_quadratic_stats,
                                 fmllr_mat),
          auxf_change = cur_auxf - auxf_orig;
      KALDI_VLOG(2) << "Updating raw fMLLR: objf improvement per frame was "
                    << (auxf_change / this->count_) << " over "
                    << this->count_ << " frames, by the " << iter
                    << "'th iteration";
    }
  }
  double auxf_final = GetAuxf(simple_linear_stats, simple_quadratic_stats,
                              fmllr_mat),
      auxf_change = auxf_final - auxf_orig;
  *count = this->count_;
  KALDI_VLOG(1) << "Updating raw fMLLR: objf improvement per frame was "
                << (auxf_change / this->count_) << " over "
                << this->count_ << " frames.";
  if (auxf_final > auxf_orig) {
    *objf_impr = auxf_change;
    *count = this->count_;
    raw_fmllr_mat->CopyFromMat(fmllr_mat);
  } else {
    *objf_impr = 0.0;
    // don't update "raw_fmllr_mat"
  }
}

Member Data Documentation

count_

double count_

private

Definition at line 190 of file fmllr-raw.h.

Referenced by FmllrRawAccs::CommitSingleFrameStats(), FmllrRawAccs::FmllrRawAccs(), FmllrRawAccs::GetAuxf(), FmllrRawAccs::SetZero(), and FmllrRawAccs::Update().

full_transform_

Matrix<BaseFloat> full_transform_

private

Definition at line 183 of file fmllr-raw.h.

Referenced by FmllrRawAccs::ComputeM(), FmllrRawAccs::FmllrRawAccs(), and FmllrRawAccs::InitSingleFrameStats().

model_dim_

int32 model_dim_

private

Definition at line 181 of file fmllr-raw.h.

Q_

Matrix<double> Q_

private

Definition at line 195 of file fmllr-raw.h.

Referenced by FmllrRawAccs::CommitSingleFrameStats(), FmllrRawAccs::ConvertToSimpleStats(), FmllrRawAccs::FmllrRawAccs(), and FmllrRawAccs::SetZero().

raw_dim_

int32 raw_dim_

private

Definition at line 180 of file fmllr-raw.h.

S_

Matrix<double> S_

private

Definition at line 196 of file fmllr-raw.h.

Referenced by FmllrRawAccs::CommitSingleFrameStats(), FmllrRawAccs::ConvertToSimpleStats(), FmllrRawAccs::FmllrRawAccs(), and FmllrRawAccs::SetZero().

single_frame_stats_

SingleFrameStats single_frame_stats_

private

Definition at line 188 of file fmllr-raw.h.

Referenced by FmllrRawAccs::AccumulateForGmm(), FmllrRawAccs::AccumulateFromPosteriors(), FmllrRawAccs::CommitSingleFrameStats(), FmllrRawAccs::DataHasChanged(), FmllrRawAccs::FmllrRawAccs(), FmllrRawAccs::InitSingleFrameStats(), FmllrRawAccs::SetZero(), and FmllrRawAccs::Update().

temp_

SpMatrix<double> temp_

private

Definition at line 194 of file fmllr-raw.h.

Referenced by FmllrRawAccs::CommitSingleFrameStats(), and FmllrRawAccs::FmllrRawAccs().

transform_offset_

Vector<BaseFloat> transform_offset_

private

Definition at line 185 of file fmllr-raw.h.

Referenced by FmllrRawAccs::AccumulateFromPosteriors(), FmllrRawAccs::ConvertToSimpleStats(), FmllrRawAccs::FmllrRawAccs(), and FmllrRawAccs::InitSingleFrameStats().

---

The documentation for this class was generated from the following files:

• transform/fmllr-raw.h
• transform/fmllr-raw.cc