This class enables you to do the compilation and optimization in one call, and also ensures that if the ComputationRequest is identical to the previous one, the compilation process is not repeated. More...

#include <nnet-optimize.h>

Collaboration diagram for CachingOptimizingCompiler:

[legend]

Public Member Functions
	CachingOptimizingCompiler (const Nnet &nnet, const CachingOptimizingCompilerOptions config=CachingOptimizingCompilerOptions())

	CachingOptimizingCompiler (const Nnet &nnet, const NnetOptimizeOptions &opt_config, const CachingOptimizingCompilerOptions config=CachingOptimizingCompilerOptions())
	Note: nnet is retained as a const reference but opt_config is copied. More...

	~CachingOptimizingCompiler ()

std::shared_ptr< const NnetComputation >	Compile (const ComputationRequest &request)
	Does the compilation and returns a const pointer to the result, which is owned by this class, not the caller. More...

void	ReadCache (std::istream &is, bool binary)

void	WriteCache (std::ostream &os, bool binary)

void	GetSimpleNnetContext (int32 nnet_left_context, int32 nnet_right_context)

Private Member Functions
std::shared_ptr< const NnetComputation >	CompileInternal (const ComputationRequest &request)

std::shared_ptr< const NnetComputation >	CompileAndCache (const ComputationRequest &request)

const NnetComputation *	CompileViaShortcut (const ComputationRequest &request)

const NnetComputation *	CompileNoShortcut (const ComputationRequest &request)

Private Attributes
const Nnet &	nnet_

CachingOptimizingCompilerOptions	config_

NnetOptimizeOptions	opt_config_

double	seconds_taken_total_

double	seconds_taken_compile_

double	seconds_taken_optimize_

double	seconds_taken_expand_

double	seconds_taken_check_

double	seconds_taken_indexes_

double	seconds_taken_io_

ComputationCache	cache_

int32	nnet_left_context_

int32	nnet_right_context_

Detailed Description

This class enables you to do the compilation and optimization in one call, and also ensures that if the ComputationRequest is identical to the previous one, the compilation process is not repeated.

It is safe to call Compile() from multiple parallel threads without additional synchronization; synchronization is managed internally by class ComputationCache.

Definition at line 219 of file nnet-optimize.h.

Constructor & Destructor Documentation

◆ CachingOptimizingCompiler() [1/2]

CachingOptimizingCompiler	(	const Nnet &	nnet,
		const CachingOptimizingCompilerOptions	config = `CachingOptimizingCompilerOptions()`
	)

Definition at line 635 of file nnet-optimize.cc.

                                                   :
     nnet_(nnet), config_(config),
     seconds_taken_total_(0.0), seconds_taken_compile_(0.0),
     seconds_taken_optimize_(0.0), seconds_taken_expand_(0.0),
     seconds_taken_check_(0.0), seconds_taken_indexes_(0.0),
     seconds_taken_io_(0.0), cache_(config.cache_capacity),
     nnet_left_context_(-1), nnet_right_context_(-1) { }

◆ CachingOptimizingCompiler() [2/2]

CachingOptimizingCompiler	(	const Nnet &	nnet,
		const NnetOptimizeOptions &	opt_config,
		const CachingOptimizingCompilerOptions	config = `CachingOptimizingCompilerOptions()`
	)

Note: nnet is retained as a const reference but opt_config is copied.

Definition at line 645 of file nnet-optimize.cc.

                                                   :
     nnet_(nnet), config_(config), opt_config_(opt_config),
     seconds_taken_total_(0.0), seconds_taken_compile_(0.0),
     seconds_taken_optimize_(0.0), seconds_taken_expand_(0.0),
     seconds_taken_check_(0.0), seconds_taken_indexes_(0.0),
     seconds_taken_io_(0.0), cache_(config.cache_capacity),
     nnet_left_context_(-1), nnet_right_context_(-1) { }

◆ ~CachingOptimizingCompiler()

~CachingOptimizingCompiler ( )

Definition at line 695 of file nnet-optimize.cc.

References KALDI_LOG, CachingOptimizingCompiler::seconds_taken_check_, CachingOptimizingCompiler::seconds_taken_compile_, CachingOptimizingCompiler::seconds_taken_expand_, CachingOptimizingCompiler::seconds_taken_indexes_, CachingOptimizingCompiler::seconds_taken_io_, CachingOptimizingCompiler::seconds_taken_optimize_, and CachingOptimizingCompiler::seconds_taken_total_.

                                                       {
   if (seconds_taken_total_ > 0.0 || seconds_taken_io_ > 0.0) {
     std::ostringstream os;
     double seconds_taken_misc = seconds_taken_total_ - seconds_taken_compile_
         - seconds_taken_optimize_ - seconds_taken_expand_
         - seconds_taken_check_ - seconds_taken_indexes_;
     os << std::setprecision(3) << seconds_taken_total_
        << " seconds taken in nnet3 compilation total (breakdown: "
        << seconds_taken_compile_ << " compilation, "
        << seconds_taken_optimize_ << " optimization, "
        << seconds_taken_expand_ << " shortcut expansion, "
        << seconds_taken_check_ << " checking, "
        << seconds_taken_indexes_ << " computing indexes, "
        << seconds_taken_misc << " misc.) + "
        << seconds_taken_io_ << " I/O.";
     KALDI_LOG << os.str();
     // note: the leftover amount is misc things like hashing and == comparisons on
     // computation-requests, and calling RequestIsDecomposable().
   }
 }

Member Function Documentation

◆ Compile()

std::shared_ptr< const NnetComputation > Compile ( const ComputationRequest & request )

Does the compilation and returns a const pointer to the result, which is owned by this class, not the caller.

It calls ComputeCudaIndexes() for you, because you wouldn't be able to do this on a const object.

Note: this used to return 'const NnetComputation*'. If you get a compilation failure, just replace 'const NnetComputation*' with 'std::shared_ptr<const NnetComputation>' in the calling code.

Definition at line 716 of file nnet-optimize.cc.

References CachingOptimizingCompiler::CompileInternal(), Timer::Elapsed(), and CachingOptimizingCompiler::seconds_taken_total_.

Referenced by NnetLdaStatsAccumulator::AccStats(), BatchedXvectorComputer::BatchedXvectorComputer(), NnetComputerFromEg::Compute(), NnetDiscriminativeComputeObjf::Compute(), NnetChainComputeProb::Compute(), NnetComputeProb::Compute(), DecodableNnetSimple::DoNnetComputation(), NnetBatchComputer::GetComputation(), kaldi::nnet3::RunNnetComputation(), NnetChainTrainer::Train(), NnetDiscriminativeTrainer::Train(), NnetTrainer::Train(), kaldi::nnet3::UnitTestNnetModelDerivatives(), and kaldi::nnet3::UnitTestNnetOptimizeWithOptions().

                                            {
   Timer timer;
   std::shared_ptr<const NnetComputation>  ans = CompileInternal(in_request);
   seconds_taken_total_ += timer.Elapsed();
   return ans;
 }

◆ CompileAndCache()

std::shared_ptr<const NnetComputation> CompileAndCache ( const ComputationRequest & request )

private

◆ CompileInternal()

std::shared_ptr< const NnetComputation > CompileInternal ( const ComputationRequest & request )

private

Definition at line 724 of file nnet-optimize.cc.

References CachingOptimizingCompiler::cache_, CachingOptimizingCompiler::CompileNoShortcut(), CachingOptimizingCompiler::CompileViaShortcut(), CachingOptimizingCompiler::config_, ComputationCache::Find(), ComputationCache::Insert(), KALDI_ASSERT, and CachingOptimizingCompilerOptions::use_shortcut.

Referenced by CachingOptimizingCompiler::Compile(), and CachingOptimizingCompiler::CompileViaShortcut().

                                         {
   std::shared_ptr<const NnetComputation> ans = cache_.Find(request);
   if (ans != NULL) {
     return ans;
   } else {
     const NnetComputation *computation = NULL;
     if (config_.use_shortcut)
       computation = CompileViaShortcut(request);
     if (computation == NULL)
       computation = CompileNoShortcut(request);
     KALDI_ASSERT(computation != NULL);
     return cache_.Insert(request, computation);
   }
 }

◆ CompileNoShortcut()

const NnetComputation * CompileNoShortcut ( const ComputationRequest & request )

private

Definition at line 741 of file nnet-optimize.cc.

References ComputationChecker::Check(), CheckComputationOptions::check_rewrite, NnetComputation::ComputeCudaIndexes(), Compiler::CreateComputation(), Timer::Elapsed(), kaldi::GetVerboseLevel(), KALDI_LOG, kaldi::nnet3::MaxOutputTimeInRequest(), CachingOptimizingCompiler::nnet_, kaldi::nnet2::NnetComputation(), CachingOptimizingCompiler::opt_config_, kaldi::nnet3::Optimize(), ComputationRequest::Print(), NnetComputation::Print(), CachingOptimizingCompiler::seconds_taken_check_, CachingOptimizingCompiler::seconds_taken_compile_, CachingOptimizingCompiler::seconds_taken_indexes_, and CachingOptimizingCompiler::seconds_taken_optimize_.

Referenced by CachingOptimizingCompiler::CompileInternal().

                                        {
 
   Compiler compiler(request, nnet_);
   // note: 'opts' only contains 'output_debug_info', which is true by default.
   // There may be situations where we'd prefer not to keep it, for speed.
   CompilerOptions opts;
   NnetComputation *computation = new NnetComputation;
 
   {
     Timer timer;
     compiler.CreateComputation(opts, computation);
     seconds_taken_compile_ += timer.Elapsed();
   }
 
   int32 verbose_cutoff = 4;
   if (GetVerboseLevel() >= verbose_cutoff) {
     std::ostringstream os1;
     request.Print(os1);
     KALDI_LOG << "Computation request is " << os1.str();
     std::ostringstream os2;
     computation->Print(os2, nnet_);
     KALDI_LOG << "Generated computation is: " << os2.str();
   }
 
   { // some checking.  Note: there may come a time when we might
     // prefer to disable this checking.
     Timer timer;
     CheckComputationOptions check_config;
     // we can do the rewrite check since it's before optimization.
     check_config.check_rewrite = true;
     ComputationChecker checker(check_config, nnet_, *computation);
     checker.Check();
     seconds_taken_check_ += timer.Elapsed();
   }
 
   {
     Timer timer;
     Optimize(opt_config_, nnet_,
              MaxOutputTimeInRequest(request),
              computation);
     seconds_taken_optimize_ += timer.Elapsed();
   }
 
   if (GetVerboseLevel() >= verbose_cutoff) {
     std::ostringstream os;
     computation->Print(os, nnet_);
     KALDI_LOG << "Optimized computation is: " << os.str();
   }
 
   {  // check the computation again.
     Timer timer;
     CheckComputationOptions check_config;
     ComputationChecker checker(check_config, nnet_, *computation);
     checker.Check();
     seconds_taken_check_ += timer.Elapsed();
   }
 
   {
     Timer timer;
     computation->ComputeCudaIndexes();
     seconds_taken_indexes_ += timer.Elapsed();
   }
   return computation;
 }

◆ CompileViaShortcut()

const NnetComputation * CompileViaShortcut ( const ComputationRequest & request )

private

Definition at line 808 of file nnet-optimize.cc.

References kaldi::nnet3::CheckComputation(), CachingOptimizingCompiler::CompileInternal(), NnetComputation::ComputeCudaIndexes(), Timer::Elapsed(), kaldi::nnet3::ExpandComputation(), kaldi::GetVerboseLevel(), ComputationRequest::misc_info, CachingOptimizingCompiler::nnet_, kaldi::nnet2::NnetComputation(), kaldi::nnet3::RequestIsDecomposable(), CachingOptimizingCompiler::seconds_taken_expand_, and CachingOptimizingCompiler::seconds_taken_indexes_.

Referenced by CachingOptimizingCompiler::CompileInternal().

                                        {
   int32 num_n_values;
   ComputationRequest mini_request;
   if (!RequestIsDecomposable(request, &mini_request, &num_n_values))
     return NULL;
 
   // By invoking CompileInternal() on the mini request, we go through the same
   // caching process as for any externally requested computation.
   std::shared_ptr<const NnetComputation> mini_computation =
       CompileInternal(mini_request);
 
   // note: by default we always create debug_info, even in regular compilation.
   // (e.g. it defaults to true in CompilerOptions).  If it really seems to be a
   // significant overhead, we can revisit this at some point in future.
   bool need_debug_info = true;
 
 
   NnetComputation *ans = new NnetComputation();
 
   {
     Timer timer;
     ExpandComputation(nnet_, request.misc_info, *mini_computation,
                       need_debug_info, num_n_values, ans);
     seconds_taken_expand_ += timer.Elapsed();
   }
   if (GetVerboseLevel() >= 3) {
     CheckComputation(nnet_, *ans, false);
   }
 
   {
     Timer timer;
     ans->ComputeCudaIndexes();
     seconds_taken_indexes_ += timer.Elapsed();
   }
   return ans;
 }

◆ GetSimpleNnetContext()

void GetSimpleNnetContext	(	int32 *	nnet_left_context,
		int32 *	nnet_right_context
	)

Definition at line 656 of file nnet-optimize.cc.

References kaldi::nnet3::ComputeSimpleNnetContext(), CachingOptimizingCompiler::nnet_, CachingOptimizingCompiler::nnet_left_context_, and CachingOptimizingCompiler::nnet_right_context_.

Referenced by DecodableNnetSimple::DecodableNnetSimple().

                                                          {
   if (nnet_left_context_ == -1) {
     ComputeSimpleNnetContext(nnet_, &nnet_left_context_,
                              &nnet_right_context_);
   }
   *nnet_left_context = nnet_left_context_;
   *nnet_right_context = nnet_right_context_;
 }

◆ ReadCache()

void ReadCache	(	std::istream &	is,
		bool	binary
	)

Definition at line 666 of file nnet-optimize.cc.

References CachingOptimizingCompiler::cache_, ComputationCache::Check(), Timer::Elapsed(), kaldi::GetVerboseLevel(), CachingOptimizingCompiler::nnet_, CachingOptimizingCompiler::opt_config_, NnetOptimizeOptions::Read(), ComputationCache::Read(), CachingOptimizingCompiler::seconds_taken_check_, CachingOptimizingCompiler::seconds_taken_io_, and CachingOptimizingCompiler::seconds_taken_total_.

Referenced by main(), NnetChainTrainer::NnetChainTrainer(), NnetDiscriminativeTrainer::NnetDiscriminativeTrainer(), and NnetTrainer::NnetTrainer().

                                                                      {
   {
     Timer timer;
     NnetOptimizeOptions opt_config_cached;
     opt_config_cached.Read(is, binary);
     // we won't read cached computations if any optimize option has been changed.
     if (!(opt_config_ == opt_config_cached))
       return;
     cache_.Read(is, binary);
     seconds_taken_io_ += timer.Elapsed();
   }
   if (GetVerboseLevel() >= 2) {
     Timer timer;
     cache_.Check(nnet_);
     seconds_taken_check_ += timer.Elapsed();
     // we consider the check time part of the total time...  this is very
     // arbitrary but it only affects printed times-taken.
     seconds_taken_total_ += timer.Elapsed();
   }
 
 }

◆ WriteCache()

void WriteCache	(	std::ostream &	os,
		bool	binary
	)

Definition at line 688 of file nnet-optimize.cc.

References CachingOptimizingCompiler::cache_, Timer::Elapsed(), CachingOptimizingCompiler::opt_config_, CachingOptimizingCompiler::seconds_taken_io_, NnetOptimizeOptions::Write(), and ComputationCache::Write().

Referenced by main(), NnetChainTrainer::~NnetChainTrainer(), NnetDiscriminativeTrainer::~NnetDiscriminativeTrainer(), and NnetTrainer::~NnetTrainer().

                                                                       {
   Timer timer;
   opt_config_.Write(os, binary);
   cache_.Write(os, binary);
   seconds_taken_io_ += timer.Elapsed();
 }