This header implements a form of Extended Baum-Welch training for SGMMs. More...

#include <estimate-am-sgmm2-ebw.h>

Collaboration diagram for EbwAmSgmm2Options:

Public Member Functions
	EbwAmSgmm2Options ()
	for an issue in some implementations of SVD. More...

void	Register (OptionsItf *opts)

Public Attributes
BaseFloat	tau_v
	Smoothing constant for updates of sub-state vectors v_{jm}. More...

BaseFloat	lrate_v
	Learning rate used in updating v– default 0.5. More...

BaseFloat	tau_M
	Smoothing constant for the M quantities (phone-subspace projections) More...

BaseFloat	lrate_M
	Learning rate used in updating M– default 0.5. More...

BaseFloat	tau_N
	Smoothing constant for the N quantities (speaker-subspace projections) More...

BaseFloat	lrate_N
	Learning rate used in updating N– default 0.5. More...

BaseFloat	tau_c
	Tau value for smoothing substate weights (c) More...

BaseFloat	tau_w
	Tau value for smoothing update of phonetic-subspace weight projectsions (w) More...

BaseFloat	lrate_w
	Learning rate used in updating w– default 1.0. More...

BaseFloat	tau_u
	Tau value for smoothing update of speaker-subspace weight projectsions (u) More...

BaseFloat	lrate_u
	Learning rate used in updating u– default 1.0. More...

BaseFloat	max_impr_u
	Maximum improvement/frame allowed for u [0.25, carried over from ML update.]. More...

BaseFloat	tau_Sigma
	Tau value for smoothing covariance-matrices Sigma. More...

BaseFloat	lrate_Sigma
	Learning rate used in updating Sigma– default 0.5. More...

BaseFloat	min_substate_weight
	Minimum allowed weight in a sub-state. More...

BaseFloat	cov_min_value
	E.g. More...

BaseFloat	max_cond
	is allowed to change. More...

BaseFloat	epsilon
	very small value used in SolveQuadraticProblem; workaround More...

Detailed Description

This header implements a form of Extended Baum-Welch training for SGMMs.

If you are confused by this comment, see Dan Povey's thesis for an explanation of Extended Baum-Welch. A note on the EBW (Extended Baum-Welch) updates for the SGMMs... In general there is a parameter-specific value D that is similar to the D in EBW for GMMs. The value of D is generally set to: E * (denominator-count for that parameter) + tau-value for that parameter where the tau-values are user-specified parameters that are specific to the type of the parameter (e.g. phonetic vector, subspace projection, etc.). Things are a bit more complex for this update than for GMMs, because it's not just a question of picking a tau-value for smoothing: there is sometimes a scatter-matrix of some kind (e.g. an outer product of vectors, or something) that defines a quadratic objective function that we'll add as smoothing. We have to pick where to get this scatter-matrix from. We feel that it's appropriate for the "E" part of the D to get its scatter-matrix from denominator stats, and the tau part of the D to get half its scatter-matrix from the both the numerator and denominator stats, assigned a weight proportional to how much stats there were. When you see the auxiliary function written out, it's clear why this makes sense.

Definition at line 54 of file estimate-am-sgmm2-ebw.h.

Constructor & Destructor Documentation

◆ EbwAmSgmm2Options()

EbwAmSgmm2Options ( )

inline

for an issue in some implementations of SVD.

Definition at line 80 of file estimate-am-sgmm2-ebw.h.

                       {
     tau_v = 50.0;
     lrate_v = 0.5;
     tau_M = 500.0;
     lrate_M = 0.5;
     tau_N = 500.0;
     lrate_N = 0.5;
     tau_c = 10.0;
     tau_w = 50.0;
     lrate_w = 1.0;
     tau_u = 50.0;
     lrate_u = 1.0;
     max_impr_u = 0.25;
     tau_Sigma = 500.0;
     lrate_Sigma = 0.5;
 
     min_substate_weight = 1.0e-05;
     cov_min_value = 0.5;
     
     max_cond = 1.0e+05;
     epsilon = 1.0e-40;
   }

Member Function Documentation

◆ Register()

void Register ( OptionsItf * opts )

inline

Definition at line 103 of file estimate-am-sgmm2-ebw.h.

References OptionsItf::Register().

Referenced by main().

                                   {
     std::string module = "EbwAmSgmm2Options: ";
     opts->Register("tau-v", &tau_v, module+
                    "Smoothing constant for phone vector estimation.");
     opts->Register("lrate-v", &lrate_v, module+
                    "Learning rate constant for phone vector estimation.");
     opts->Register("tau-m", &tau_M, module+
                    "Smoothing constant for estimation of phonetic-subspace projections (M).");
     opts->Register("lrate-m", &lrate_M, module+
                    "Learning rate constant for phonetic-subspace projections.");
     opts->Register("tau-n", &tau_N, module+
                    "Smoothing constant for estimation of speaker-subspace projections (N).");
     opts->Register("lrate-n", &lrate_N, module+
                    "Learning rate constant for speaker-subspace projections.");
     opts->Register("tau-c", &tau_c, module+
                    "Smoothing constant for estimation of substate weights (c)");
     opts->Register("tau-w", &tau_w, module+
                    "Smoothing constant for estimation of phonetic-space weight projections (w)");
     opts->Register("lrate-w", &lrate_w, module+
                    "Learning rate constant for phonetic-space weight-projections (w)");
     opts->Register("tau-u", &tau_u, module+
                    "Smoothing constant for estimation of speaker-space weight projections (u)");
     opts->Register("lrate-u", &lrate_u, module+
                    "Learning rate constant for speaker-space weight-projections (u)");
     opts->Register("tau-sigma", &tau_Sigma, module+
                    "Smoothing constant for estimation of within-class covariances (Sigma)");
     opts->Register("lrate-sigma", &lrate_Sigma, module+
                    "Constant that controls speed of learning for variances (larger->slower)");
     opts->Register("cov-min-value", &cov_min_value, module+
                    "Minimum value that an eigenvalue of the updated covariance matrix can take, "
                    "relative to its old value (maximum is inverse of this.)");
     opts->Register("min-substate-weight", &min_substate_weight, module+
                    "Floor for weights of sub-states.");
     opts->Register("max-cond", &max_cond, module+
                    "Value used in handling singular matrices during update.");
     opts->Register("epsilon", &max_cond, module+
                    "Value used in handling singular matrices during update.");
   }

Member Data Documentation

◆ cov_min_value

BaseFloat cov_min_value

E.g.

0.5– the maximum any eigenvalue of a covariance

Definition at line 71 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateVars().

◆ epsilon

BaseFloat epsilon

very small value used in SolveQuadraticProblem; workaround

Definition at line 77 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateM(), EbwAmSgmm2Updater::UpdateN(), EbwAmSgmm2Updater::UpdatePhoneVectorsInternal(), EbwAmSgmm2Updater::UpdateU(), and EbwAmSgmm2Updater::UpdateW().

◆ lrate_M

BaseFloat lrate_M

Learning rate used in updating M– default 0.5.

Definition at line 58 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateM().

◆ lrate_N

BaseFloat lrate_N

Learning rate used in updating N– default 0.5.

Definition at line 60 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateN().

◆ lrate_Sigma

BaseFloat lrate_Sigma

Learning rate used in updating Sigma– default 0.5.

Definition at line 68 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateVars().

◆ lrate_u

BaseFloat lrate_u

Learning rate used in updating u– default 1.0.

Definition at line 65 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateU().

◆ lrate_v

BaseFloat lrate_v

Learning rate used in updating v– default 0.5.

Definition at line 56 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdatePhoneVectorsInternal().

◆ lrate_w

BaseFloat lrate_w

Learning rate used in updating w– default 1.0.

Definition at line 63 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateW().

◆ max_cond

BaseFloat max_cond

is allowed to change.

[this is the minimum; the maximum is the inverse of this, i.e. 2.0 in this case. For example, 0.9 would constrain the covariance quite tightly, 0.1 would be a loose setting. large value used in SolveQuadraticProblem.

Definition at line 76 of file estimate-am-sgmm2-ebw.h.

Referenced by EbwAmSgmm2Updater::UpdateM(), EbwAmSgmm2Updater::UpdateN(), EbwAmSgmm2Updater::UpdatePhoneVectorsInternal(), EbwAmSgmm2Updater::UpdateU(), and EbwAmSgmm2Updater::UpdateW().