Class CuCompressedMatrix, templated on an integer type (expected to be one of: int8, uint8, int16, uint16), this provides a way to approximate a CuMatrix in a more memory-efficient format. More...

#include <cu-compressed-matrix.h>

Inheritance diagram for CuCompressedMatrix:

[legend]

Collaboration diagram for CuCompressedMatrix:

[legend]

Public Member Functions
	CuCompressedMatrix (BaseFloat range, bool truncate=true)
	Constructor which sets 'scale_' according to scale_ = range / std::numeric_limits<I>::max(). More...

virtual void	CopyFromMat (const CuMatrixBase< BaseFloat > &mat)
	Sets this to an appropriately compressed copy of 'mat', which includes resizing this. More...

virtual void	CopyToMat (CuMatrixBase< BaseFloat > *mat) const
	Copies the contents of *this to 'mat', which should be correctly sized beforehand. More...

virtual MatrixIndexT	NumRows () const

virtual MatrixIndexT	NumCols () const

virtual	~CuCompressedMatrix ()

Public Member Functions inherited from CuCompressedMatrixBase
virtual	~CuCompressedMatrixBase ()

Private Member Functions
void	Destroy ()

Private Attributes
I *	data_

BaseFloat	scale_

bool	truncate_

MatrixIndexT	num_rows_

MatrixIndexT	num_cols_

MatrixIndexT	stride_

Detailed Description

template<typename I>
class kaldi::CuCompressedMatrix

Class CuCompressedMatrix, templated on an integer type (expected to be one of: int8, uint8, int16, uint16), this provides a way to approximate a CuMatrix in a more memory-efficient format.

It's used in nnet3 to reduce memory use for large networks.

It is *not* a CUDA equivalent for class CompressedMatrix (of ../matrix/compressed-matrix.h). Note: this class is only to be used when you are using a GPU. If you didn't compile for CUDA or you are not using a GPU, you are not supposed to create an instance of this class, and doing so will cause a runtime error.

Definition at line 72 of file cu-compressed-matrix.h.

Constructor & Destructor Documentation

◆ CuCompressedMatrix()

CuCompressedMatrix	(	BaseFloat	range,
		bool	truncate = `true`
	)

Constructor which sets 'scale_' according to scale_ = range / std::numeric_limits::max().

range = 0 (only supported for I == int8) is a special case in which only the sign of the input is retained; and when we reconstruct, the output will be -1, 0 or 1.

truncate (only relevant if range != 0) should be true if it's possible that the input could exceed the allowed input range, i.e. [0, range] if I is unsigned, and [-range, range] if I is signed; and it may be false if you know that the input (the matrix given to CopyFromMat) will have elements only in the allowed range. Setting 'truncate' to false allows the compression code to avoid the bounds check.

Definition at line 38 of file cu-compressed-matrix.cc.

References KALDI_ASSERT, and KALDI_ERR.

                                                                        :
     data_(NULL), scale_(range / std::numeric_limits<I>::max()),
     truncate_(truncate), num_rows_(0), num_cols_(0), stride_(0) {
 #if HAVE_CUDA == 1
   KALDI_ASSERT(CuDevice::Instantiate().Enabled());
 #else
   KALDI_ERR << "You instantiated CuCompressedMatrix while GPU use "
       "was not compiled in.";
 #endif
 }

◆ ~CuCompressedMatrix()

virtual ~CuCompressedMatrix ( )

inlinevirtual

Definition at line 99 of file cu-compressed-matrix.h.

99 { Destroy(); }

kaldi::CuCompressedMatrix::Destroy

void Destroy()

Definition: cu-compressed-matrix.cc:50

Member Function Documentation

◆ CopyFromMat()

void CopyFromMat ( const CuMatrixBase< BaseFloat > & mat )

virtual

Sets *this to an appropriately compressed copy of 'mat', which includes resizing *this.

The details of how this is done will be different in different child classes.

Implements CuCompressedMatrixBase.

Definition at line 65 of file cu-compressed-matrix.cc.

References CuMatrixBase< Real >::Data(), CuCompressedMatrix::data_, CuCompressedMatrix::Destroy(), CuMatrixBase< Real >::Dim(), KALDI_ASSERT, CuCompressedMatrix::num_cols_, CuCompressedMatrix::num_rows_, CuCompressedMatrix::NumCols(), CuMatrixBase< Real >::NumCols(), CuCompressedMatrix::NumRows(), CuMatrixBase< Real >::NumRows(), CuCompressedMatrix::scale_, CuCompressedMatrix::stride_, and CuCompressedMatrix::truncate_.

                                         {
 #if HAVE_CUDA == 1
   KALDI_ASSERT(CuDevice::Instantiate().Enabled());
   if (mat.NumRows() == 0)
     return;
   if (num_rows_ != mat.NumRows() || num_cols_ != mat.NumCols()) {
     Destroy();
     num_rows_ = mat.NumRows();
     num_cols_ = mat.NumCols();
     data_ = static_cast<I*>(
         CuDevice::Instantiate().Malloc(sizeof(I) * num_rows_ * num_cols_));
     stride_ = num_cols_;
   }
 
   {
     CuTimer tim;
     dim3 dimGrid, dimBlock;
     GetBlockSizesForSimpleMatrixOperation(NumRows(), NumCols(),
                                           &dimGrid, &dimBlock);
 
     if (scale_ == 0.0) { // scale == 0 calls a different kernel from the others.
       cuda_mat_compress_sign(dimGrid, dimBlock, mat.Data(), mat.Dim(),
                              data_, stride_);
     } else {
       cuda_mat_compress(dimGrid, dimBlock, mat.Data(), mat.Dim(),
                         data_, stride_, float(1.0 / scale_),
                         truncate_);
     }
     CU_SAFE_CALL(cudaGetLastError());
 
     CuDevice::Instantiate().AccuProfile(__func__, tim);
   }
 #endif
 }

◆ CopyToMat()

void CopyToMat ( CuMatrixBase< BaseFloat > * mat ) const

virtual

Copies the contents of *this to 'mat', which should be correctly sized beforehand.

Implements CuCompressedMatrixBase.

Definition at line 102 of file cu-compressed-matrix.cc.

References CuMatrixBase< Real >::Data(), CuCompressedMatrix::data_, CuMatrixBase< Real >::Dim(), KALDI_ASSERT, CuCompressedMatrix::num_cols_, CuCompressedMatrix::num_rows_, CuCompressedMatrix::NumCols(), CuMatrixBase< Real >::NumCols(), CuCompressedMatrix::NumRows(), CuMatrixBase< Real >::NumRows(), CuCompressedMatrix::scale_, and CuCompressedMatrix::stride_.

                                                                         {
 #if HAVE_CUDA == 1
   KALDI_ASSERT(CuDevice::Instantiate().Enabled());
   KALDI_ASSERT(mat->NumRows() == num_rows_ && mat->NumCols() == num_cols_);
   {
     CuTimer tim;
     dim3 dimGrid, dimBlock;
     GetBlockSizesForSimpleMatrixOperation(NumRows(), NumCols(),
                                           &dimGrid, &dimBlock);
     BaseFloat scale = (scale_ == 0.0 ? 1.0 : scale_);
     cuda_mat_uncompress(dimGrid, dimBlock, mat->Data(), mat->Dim(),
                         data_, stride_, float(scale));
   }
 #endif
 }

◆ Destroy()

void Destroy ( )

private

Definition at line 50 of file cu-compressed-matrix.cc.

References CuCompressedMatrix::data_, CuCompressedMatrix::num_cols_, CuCompressedMatrix::num_rows_, and CuCompressedMatrix::stride_.

Referenced by CuCompressedMatrix::CopyFromMat().

                                     {
 #if HAVE_CUDA == 1
   if (data_ != NULL) {
     // we don't bother timing this because Free() won't normally have to
     // access the GPU at all (due to caching).
     CuDevice::Instantiate().Free(data_);
     data_ = NULL;
     num_rows_ = 0;
     num_cols_ = 0;
     stride_ = 0;
   }
 #endif
 }