"Low-level I/O functions"

These functions are provided to write fundamental types, strings, and a few STL types to and from C++ streams; see Input/output mechanisms for fundamental types and STL types for how this fits into the bigger picture of Kaldi-style I/O. More...

Functions

template<class T >
void WriteBasicType (std::ostream &os, bool binary, T t)
 WriteBasicType is the name of the write function for bool, integer types, and floating-point types. More...
 
template<class T >
void ReadBasicType (std::istream &is, bool binary, T *t)
 ReadBasicType is the name of the read function for bool, integer types, and floating-point types. More...
 
template<>
void WriteBasicType< bool > (std::ostream &os, bool binary, bool b)
 
template<>
void ReadBasicType< bool > (std::istream &is, bool binary, bool *b)
 
template<>
void WriteBasicType< float > (std::ostream &os, bool binary, float f)
 
template<>
void WriteBasicType< double > (std::ostream &os, bool binary, double f)
 
template<>
void ReadBasicType< float > (std::istream &is, bool binary, float *f)
 
template<>
void ReadBasicType< double > (std::istream &is, bool binary, double *d)
 
template<class T >
void ReadBasicType (std::istream &is, bool binary, T *t, bool add)
 
template<class T >
void WriteIntegerVector (std::ostream &os, bool binary, const std::vector< T > &v)
 Function for writing STL vectors of integer types. More...
 
template<class T >
void ReadIntegerVector (std::istream &is, bool binary, std::vector< T > *v)
 Function for reading STL vector of integer types. More...
 
template<class T >
void WriteIntegerPairVector (std::ostream &os, bool binary, const std::vector< std::pair< T, T > > &v)
 Function for writing STL vectors of pairs of integer types. More...
 
template<class T >
void ReadIntegerPairVector (std::istream &is, bool binary, std::vector< std::pair< T, T > > *v)
 Function for reading STL vector of pairs of integer types. More...
 
void WriteToken (std::ostream &os, bool binary, const char *token)
 The WriteToken functions are for writing nonempty sequences of non-space characters. More...
 
void WriteToken (std::ostream &os, bool binary, const std::string &token)
 
int Peek (std::istream &is, bool binary)
 Peek consumes whitespace (if binary == false) and then returns the peek() value of the stream. More...
 
void ReadToken (std::istream &is, bool binary, std::string *token)
 ReadToken gets the next token and puts it in str (exception on failure). More...
 
int PeekToken (std::istream &is, bool binary)
 PeekToken will return the first character of the next token, or -1 if end of file. More...
 
void ExpectToken (std::istream &is, bool binary, const char *token)
 ExpectToken tries to read in the given token, and throws an exception on failure. More...
 
void ExpectToken (std::istream &is, bool binary, const std::string &token)
 
void ExpectPretty (std::istream &is, bool binary, const char *token)
 ExpectPretty attempts to read the text in "token", but only in non-binary mode. More...
 
void ExpectPretty (std::istream &is, bool binary, const std::string &token)
 

Detailed Description

These functions are provided to write fundamental types, strings, and a few STL types to and from C++ streams; see Input/output mechanisms for fundamental types and STL types for how this fits into the bigger picture of Kaldi-style I/O.

Function Documentation

◆ ExpectPretty() [1/2]

void kaldi::ExpectPretty ( std::istream &  is,
bool  binary,
const char *  token 
)

ExpectPretty attempts to read the text in "token", but only in non-binary mode.

Throws exception on failure. It expects an exact match except that arbitrary whitespace matches arbitrary whitespace.

Referenced by kaldi::ReadBasicType().

◆ ExpectPretty() [2/2]

void kaldi::ExpectPretty ( std::istream &  is,
bool  binary,
const std::string &  token 
)

◆ ExpectToken() [1/2]

void ExpectToken ( std::istream &  is,
bool  binary,
const char *  token 
)

ExpectToken tries to read in the given token, and throws an exception on failure.

Definition at line 191 of file io-funcs.cc.

References kaldi::CheckToken(), KALDI_ASSERT, and KALDI_ERR.

Referenced by kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectOneOrTwoTokens(), kaldi::ExpectToken(), Component::Init(), AccumAmDiagGmm::Read(), AffineXformStats::Read(), CompressedAffineXformStats::Read(), MlltAccs::Read(), ScalarClusterable::Read(), NnetExample::Read(), RegressionTree::Read(), QuestionsForKey::Read(), CuBlockMatrix< Real >::Read(), RegtreeMllrDiagGmm::Read(), LogisticRegression::Read(), LinearVtln::Read(), AmDiagGmm::Read(), ContextDependency::Read(), BasisFmllrAccus::Read(), AccumFullGmm::Read(), LdaEstimate::Read(), RegtreeFmllrDiagGmm::Read(), Sgmm2FmllrGlobalParams::Read(), GaussClusterable::Read(), SparseVector< Real >::Read(), AccumDiagGmm::Read(), BasisFmllrEstimate::Read(), Questions::Read(), RegtreeMllrDiagGmmAccs::Read(), FmllrSgmm2Accs::Read(), DiscriminativeSupervision::Read(), MleAmSgmm2Accs::Read(), HmmTopology::Read(), FullGmm::Read(), TransitionModel::Read(), RefineClustersOptions::Read(), Plda::Read(), VectorClusterable::Read(), Component::Read(), EventMap::Read(), DiagGmm::Read(), GrammarFst::Read(), DiscriminativeNnetExample::Read(), RegtreeFmllrDiagGmmAccs::Read(), SparseMatrix< float >::Read(), ConstantEventMap::Read(), OnlineGmmAdaptationState::Read(), Nnet::Read(), AmSgmm2::Read(), OnlineIvectorExtractorAdaptationState::Read(), IvectorExtractor::Read(), TableEventMap::Read(), OnlineCmvnState::Read(), SplitEventMap::Read(), NonlinearComponent::Read(), OnlineIvectorEstimationStats::Read(), MaxoutComponent::Read(), MaxpoolingComponent::Read(), IvectorExtractorStats::Read(), PnormComponent::Read(), Sgmm2GauPost::Read(), PowerComponent::Read(), ScaleComponent::Read(), AffineComponent::Read(), AffineComponentPreconditioned::Read(), AffineComponentPreconditionedOnline::Read(), SpliceComponent::Read(), SpliceMaxComponent::Read(), BlockAffineComponent::Read(), BlockAffineComponentPreconditioned::Read(), PermuteComponent::Read(), DctComponent::Read(), FixedLinearComponent::Read(), FixedAffineComponent::Read(), FixedScaleComponent::Read(), FixedBiasComponent::Read(), DropoutComponent::Read(), AdditiveNoiseComponent::Read(), Convolutional1dComponent::Read(), kaldi::ReadBasicType(), kaldi::ReadBuildTreeStats(), AveragePoolingComponent::ReadData(), MaxPoolingComponent::ReadData(), ParametricRelu::ReadData(), SimpleSentenceAveragingComponent::ReadData(), AffineTransform::ReadData(), RecurrentComponent::ReadData(), LinearTransform::ReadData(), LstmProjected::ReadData(), ParallelComponent::ReadData(), FramePoolingComponent::ReadData(), ConvolutionalComponent::ReadData(), BlstmProjected::ReadData(), MultiBasisComponent::ReadData(), Dropout::ReadData(), AddShift::ReadData(), Rescale::ReadData(), kaldi::ReadEventType(), ConstArpaLm::ReadInternal(), TreeRenderer::Render(), TreeRenderer::RenderConstant(), TreeRenderer::RenderSplit(), TreeRenderer::RenderSubTree(), TreeRenderer::RenderTable(), kaldi::UnitTestIo(), kaldi::UnitTestIoNew(), and kaldi::UnitTestIoPipe().

191  {
192  int pos_at_start = is.tellg();
193  KALDI_ASSERT(token != NULL);
194  CheckToken(token); // make sure it's valid (can be read back)
195  if (!binary) is >> std::ws; // consume whitespace.
196  std::string str;
197  is >> str;
198  is.get(); // consume the space.
199  if (is.fail()) {
200  KALDI_ERR << "Failed to read token [started at file position "
201  << pos_at_start << "], expected " << token;
202  }
203  // The second half of the '&&' expression below is so that if we're expecting
204  // "<Foo>", we will accept "Foo>" instead. This is so that the model-reading
205  // code will tolerate errors in PeekToken where is.unget() failed; search for
206  // is.clear() in PeekToken() for an explanation.
207  if (strcmp(str.c_str(), token) != 0 &&
208  !(token[0] == '<' && strcmp(str.c_str(), token + 1) == 0)) {
209  KALDI_ERR << "Expected token \"" << token << "\", got instead \""
210  << str <<"\".";
211  }
212 }
void CheckToken(const char *token)
Definition: io-funcs.cc:122
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ ExpectToken() [2/2]

void ExpectToken ( std::istream &  is,
bool  binary,
const std::string &  token 
)

Definition at line 214 of file io-funcs.cc.

References kaldi::ExpectToken().

214  {
215  ExpectToken(is, binary, token.c_str());
216 }
void ExpectToken(std::istream &is, bool binary, const std::string &token)
Definition: io-funcs.cc:214

◆ Peek()

◆ PeekToken()

int PeekToken ( std::istream &  is,
bool  binary 
)

PeekToken will return the first character of the next token, or -1 if end of file.

It's the same as Peek(), except if the first character is '<' it will skip over it and will return the next character. It will attempt to unget the '<' so the stream is where it was before you did PeekToken(), however, this is not guaranteed (see ReadToken()).

Definition at line 170 of file io-funcs.cc.

Referenced by NnetOptimizeOptions::Read(), Component::Read(), Nnet::Read(), AffineComponent::Read(), BlockAffineComponent::Read(), RepeatedAffineComponent::Read(), NonlinearComponent::Read(), DropoutMaskComponent::Read(), NaturalGradientAffineComponent::Read(), GeneralDropoutComponent::Read(), LinearComponent::Read(), SpecAugmentTimeMaskComponent::Read(), NoOpComponent::Read(), PerElementScaleComponent::Read(), PerElementOffsetComponent::Read(), kaldi::ReadBasicType(), ParametricRelu::ReadData(), AffineTransform::ReadData(), SimpleSentenceAveragingComponent::ReadData(), RecurrentComponent::ReadData(), LinearTransform::ReadData(), LstmProjected::ReadData(), ConvolutionalComponent::ReadData(), BlstmProjected::ReadData(), MultiBasisComponent::ReadData(), Dropout::ReadData(), kaldi::UnitTestIo(), and kaldi::UnitTestIoPipe().

170  {
171  if (!binary) is >> std::ws; // consume whitespace.
172  bool read_bracket;
173  if (static_cast<char>(is.peek()) == '<') {
174  read_bracket = true;
175  is.get();
176  } else {
177  read_bracket = false;
178  }
179  int ans = is.peek();
180  if (read_bracket) {
181  if (!is.unget()) {
182  // Clear the bad bit. This code can be (and is in fact) reached, since the
183  // C++ standard does not guarantee that a call to unget() must succeed.
184  is.clear();
185  }
186  }
187  return ans;
188 }

◆ ReadBasicType() [1/2]

void ReadBasicType ( std::istream &  is,
bool  binary,
T *  t 
)
inline

ReadBasicType is the name of the read function for bool, integer types, and floating-point types.

They all throw on error.

Definition at line 55 of file io-funcs-inl.h.

References rnnlm::i, KALDI_ASSERT_IS_INTEGER_TYPE, KALDI_ERR, and KALDI_PARANOID_ASSERT.

Referenced by LstmNonlinearityComponent::ConsolidateMemory(), Component::Init(), AffineTransform::InitData(), LinearTransform::InitData(), MultiBasisComponent::InitData(), ParametricRelu::InitData(), RecurrentComponent::InitData(), AveragePoolingComponent::InitData(), MaxPoolingComponent::InitData(), FramePoolingComponent::InitData(), SimpleSentenceAveragingComponent::InitData(), LstmProjected::InitData(), BlstmProjected::InitData(), ConvolutionalComponent::InitData(), Rbm::InitData(), SentenceAveragingComponent::InitData(), Dropout::InitData(), AddShift::InitData(), Rescale::InitData(), AccumAmDiagGmm::Read(), AffineXformStats::Read(), CompressedAffineXformStats::Read(), MlltAccs::Read(), ScalarClusterable::Read(), NnetExample::Read(), RegressionTree::Read(), AmNnetSimple::Read(), QuestionsForKey::Read(), CuBlockMatrix< Real >::Read(), RegtreeMllrDiagGmm::Read(), LinearVtln::Read(), AmDiagGmm::Read(), Index::Read(), DistributeComponent::Read(), ContextDependency::Read(), FmpeOptions::Read(), BasisFmllrAccus::Read(), AccumFullGmm::Read(), LdaEstimate::Read(), RegtreeFmllrDiagGmm::Read(), Sgmm2FmllrGlobalParams::Read(), IoSpecification::Read(), GaussClusterable::Read(), NormalizeComponent::Read(), SparseVector< Real >::Read(), GaussPostHolder::Read(), AccumDiagGmm::Read(), NnetDiscriminativeExample::Read(), BasisFmllrEstimate::Read(), NnetChainExample::Read(), Questions::Read(), RegtreeMllrDiagGmmAccs::Read(), FmllrSgmm2Accs::Read(), DiscriminativeSupervision::Read(), MleAmSgmm2Accs::Read(), HmmTopology::Read(), TransitionModel::Read(), RefineClustersOptions::Read(), VectorClusterable::Read(), BasicHolder< BasicType >::Read(), PackedMatrix< float >::Read(), NnetOptimizeOptions::Read(), Component::Read(), RestrictedAttentionComponent::Read(), ComputationRequest::Read(), ConvolutionComponent::Read(), GrammarFst::Read(), DiscriminativeNnetExample::Read(), RegtreeFmllrDiagGmmAccs::Read(), ElementwiseProductComponent::Read(), SparseMatrix< float >::Read(), ConstantEventMap::Read(), BatchNormComponent::Read(), StatisticsExtractionComponent::Read(), Nnet::Read(), AmSgmm2::Read(), ConvolutionModel::Read(), TimeHeightConvolutionComponent::Read(), IvectorExtractor::Read(), TableEventMap::Read(), BasicVectorHolder< BasicType >::Read(), SplitEventMap::Read(), NnetComputation::MatrixInfo::Read(), NnetComputation::MatrixDebugInfo::Read(), NnetComputation::SubMatrixInfo::Read(), ConvolutionComputation::Read(), LstmNonlinearityComponent::Read(), NnetComputation::Command::Read(), NonlinearComponent::Read(), StatisticsPoolingComponent::Read(), OnlineIvectorEstimationStats::Read(), BasicVectorVectorHolder< BasicType >::Read(), ConvolutionComputationIo::Read(), MaxoutComponent::Read(), Vector< float >::Read(), NnetComputation::Read(), TdnnComponent::Read(), MaxpoolingComponent::Read(), IvectorExtractorStats::Read(), BackpropTruncationComponent::Read(), TdnnComponent::PrecomputedIndexes::Read(), BasicPairVectorHolder< BasicType >::Read(), PnormComponent::Read(), Sgmm2GauPost::Read(), BackpropTruncationComponentPrecomputedIndexes::Read(), RepeatedAffineComponent::Read(), ComputationCache::Read(), PowerComponent::Read(), ConstantComponent::Read(), ScaleComponent::Read(), DropoutMaskComponent::Read(), NaturalGradientAffineComponent::Read(), Matrix< BaseFloat >::Read(), AffineComponent::Read(), GeneralDropoutComponent::Read(), LinearComponent::Read(), AffineComponentPreconditioned::Read(), GeneralDropoutComponentPrecomputedIndexes::Read(), AffineComponentPreconditionedOnline::Read(), SpecAugmentTimeMaskComponent::Read(), SpecAugmentTimeMaskComponentPrecomputedIndexes::Read(), SpliceComponent::Read(), SpliceMaxComponent::Read(), NoOpComponent::Read(), BlockAffineComponent::Read(), SumBlockComponent::Read(), BlockAffineComponentPreconditioned::Read(), ClipGradientComponent::Read(), DctComponent::Read(), PerElementScaleComponent::Read(), DropoutComponent::Read(), PerElementOffsetComponent::Read(), AdditiveNoiseComponent::Read(), ConstantFunctionComponent::Read(), Convolutional1dComponent::Read(), NaturalGradientPerElementScaleComponent::Read(), ScaleAndOffsetComponent::Read(), CompositeComponent::Read(), kaldi::ReadBasicType(), kaldi::ReadBasicType< double >(), kaldi::ReadBasicType< float >(), kaldi::ReadBuildTreeStats(), kaldi::nnet3::ReadCindexVector(), kaldi::nnet3::ReadCindexVectorElementBinary(), AveragePoolingComponent::ReadData(), MaxPoolingComponent::ReadData(), ParametricRelu::ReadData(), AffineTransform::ReadData(), SimpleSentenceAveragingComponent::ReadData(), RecurrentComponent::ReadData(), LinearTransform::ReadData(), LstmProjected::ReadData(), ParallelComponent::ReadData(), FramePoolingComponent::ReadData(), ConvolutionalComponent::ReadData(), BlstmProjected::ReadData(), MultiBasisComponent::ReadData(), Dropout::ReadData(), AddShift::ReadData(), Rescale::ReadData(), kaldi::ReadEventType(), fst::ReadILabelInfo(), kaldi::nnet3::ReadIndexVector(), kaldi::nnet3::ReadIndexVectorElementBinary(), ConstArpaLm::ReadInternal(), ConstArpaLm::ReadInternalOldFormat(), kaldi::ReadPosterior(), UpdatableComponent::ReadUpdatableCommon(), TreeRenderer::Render(), TreeRenderer::RenderConstant(), TreeRenderer::RenderSplit(), TreeRenderer::RenderTable(), kaldi::UnitTestIo(), kaldi::UnitTestIoNew(), and kaldi::UnitTestIoPipe().

56  {
57  KALDI_PARANOID_ASSERT(t != NULL);
58  // Compile time assertion that this is not called with a wrong type.
60  if (binary) {
61  int len_c_in = is.get();
62  if (len_c_in == -1)
63  KALDI_ERR << "ReadBasicType: encountered end of stream.";
64  char len_c = static_cast<char>(len_c_in), len_c_expected
65  = (std::numeric_limits<T>::is_signed ? 1 : -1)
66  * static_cast<char>(sizeof(*t));
67  if (len_c != len_c_expected) {
68  KALDI_ERR << "ReadBasicType: did not get expected integer type, "
69  << static_cast<int>(len_c)
70  << " vs. " << static_cast<int>(len_c_expected)
71  << ". You can change this code to successfully"
72  << " read it later, if needed.";
73  // insert code here to read "wrong" type. Might have a switch statement.
74  }
75  is.read(reinterpret_cast<char *>(t), sizeof(*t));
76  } else {
77  if (sizeof(*t) == 1) {
78  int16 i;
79  is >> i;
80  *t = i;
81  } else {
82  is >> *t;
83  }
84  }
85  if (is.fail()) {
86  KALDI_ERR << "Read failure in ReadBasicType, file position is "
87  << is.tellg() << ", next char is " << is.peek();
88  }
89 }
#define KALDI_ASSERT_IS_INTEGER_TYPE(I)
Definition: kaldi-utils.h:133
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_PARANOID_ASSERT(cond)
Definition: kaldi-error.h:206

◆ ReadBasicType() [2/2]

void kaldi::ReadBasicType ( std::istream &  is,
bool  binary,
T *  t,
bool  add 
)
inline

Definition at line 169 of file io-funcs.h.

References kaldi::ExpectPretty(), kaldi::ExpectToken(), kaldi::InitKaldiInputStream(), kaldi::InitKaldiOutputStream(), kaldi::Peek(), kaldi::PeekToken(), kaldi::ReadBasicType(), kaldi::ReadIntegerPairVector(), kaldi::ReadIntegerVector(), kaldi::ReadToken(), kaldi::WriteIntegerPairVector(), kaldi::WriteIntegerVector(), and kaldi::WriteToken().

169  {
170  if (!add) {
171  ReadBasicType(is, binary, t);
172  } else {
173  T tmp = T(0);
174  ReadBasicType(is, binary, &tmp);
175  *t += tmp;
176  }
177 }
void ReadBasicType(std::istream &is, bool binary, T *t, bool add)
Definition: io-funcs.h:169

◆ ReadBasicType< bool >()

void ReadBasicType< bool > ( std::istream &  is,
bool  binary,
bool b 
)

Definition at line 34 of file io-funcs.cc.

References kaldi::CharToString(), KALDI_ERR, and KALDI_PARANOID_ASSERT.

34  {
35  KALDI_PARANOID_ASSERT(b != NULL);
36  if (!binary) is >> std::ws; // eat up whitespace.
37  char c = is.peek();
38  if (c == 'T') {
39  *b = true;
40  is.get();
41  } else if (c == 'F') {
42  *b = false;
43  is.get();
44  } else {
45  KALDI_ERR << "Read failure in ReadBasicType<bool>, file position is "
46  << is.tellg() << ", next char is " << CharToString(c);
47  }
48 }
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_PARANOID_ASSERT(cond)
Definition: kaldi-error.h:206
std::string CharToString(const char &c)
Definition: kaldi-utils.cc:36

◆ ReadBasicType< double >()

void ReadBasicType< double > ( std::istream &  is,
bool  binary,
double *  d 
)

Definition at line 98 of file io-funcs.cc.

References rnnlm::d, KALDI_ERR, KALDI_PARANOID_ASSERT, and kaldi::ReadBasicType().

98  {
99  KALDI_PARANOID_ASSERT(d != NULL);
100  if (binary) {
101  float f;
102  int c = is.peek();
103  if (c == sizeof(*d)) {
104  is.get();
105  is.read(reinterpret_cast<char*>(d), sizeof(*d));
106  } else if (c == sizeof(f)) {
107  ReadBasicType(is, binary, &f);
108  *d = f;
109  } else {
110  KALDI_ERR << "ReadBasicType: expected float, saw " << is.peek()
111  << ", at file position " << is.tellg();
112  }
113  } else {
114  is >> *d;
115  }
116  if (is.fail()) {
117  KALDI_ERR << "ReadBasicType: failed to read, at file position "
118  << is.tellg();
119  }
120 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_PARANOID_ASSERT(cond)
Definition: kaldi-error.h:206

◆ ReadBasicType< float >()

void ReadBasicType< float > ( std::istream &  is,
bool  binary,
float f 
)

Definition at line 73 of file io-funcs.cc.

References rnnlm::d, KALDI_ERR, KALDI_PARANOID_ASSERT, and kaldi::ReadBasicType().

73  {
74  KALDI_PARANOID_ASSERT(f != NULL);
75  if (binary) {
76  double d;
77  int c = is.peek();
78  if (c == sizeof(*f)) {
79  is.get();
80  is.read(reinterpret_cast<char*>(f), sizeof(*f));
81  } else if (c == sizeof(d)) {
82  ReadBasicType(is, binary, &d);
83  *f = d;
84  } else {
85  KALDI_ERR << "ReadBasicType: expected float, saw " << is.peek()
86  << ", at file position " << is.tellg();
87  }
88  } else {
89  is >> *f;
90  }
91  if (is.fail()) {
92  KALDI_ERR << "ReadBasicType: failed to read, at file position "
93  << is.tellg();
94  }
95 }
void ReadBasicType(std::istream &is, bool binary, T *t)
ReadBasicType is the name of the read function for bool, integer types, and floating-point types...
Definition: io-funcs-inl.h:55
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_PARANOID_ASSERT(cond)
Definition: kaldi-error.h:206

◆ ReadIntegerPairVector()

void ReadIntegerPairVector ( std::istream &  is,
bool  binary,
std::vector< std::pair< T, T > > *  v 
)
inline

Function for reading STL vector of pairs of integer types.

Definition at line 131 of file io-funcs-inl.h.

References KALDI_ASSERT, KALDI_ASSERT_IS_INTEGER_TYPE, and KALDI_ERR.

Referenced by DistributeComponentPrecomputedIndexes::Read(), ConvolutionModel::Read(), StatisticsExtractionComponentPrecomputedIndexes::Read(), StatisticsPoolingComponentPrecomputedIndexes::Read(), NnetComputation::Read(), kaldi::ReadBasicType(), and kaldi::UnitTestIo().

132  {
134  KALDI_ASSERT(v != NULL);
135  if (binary) {
136  int sz = is.peek();
137  if (sz == sizeof(T)) {
138  is.get();
139  } else { // this is currently just a check.
140  KALDI_ERR << "ReadIntegerPairVector: expected to see type of size "
141  << sizeof(T) << ", saw instead " << sz << ", at file position "
142  << is.tellg();
143  }
144  int32 vecsz;
145  is.read(reinterpret_cast<char *>(&vecsz), sizeof(vecsz));
146  if (is.fail() || vecsz < 0) goto bad;
147  v->resize(vecsz);
148  if (vecsz > 0) {
149  is.read(reinterpret_cast<char *>(&((*v)[0])), sizeof(T)*vecsz*2);
150  }
151  } else {
152  std::vector<std::pair<T, T> > tmp_v; // use temporary so v doesn't use extra memory
153  // due to resizing.
154  is >> std::ws;
155  if (is.peek() != static_cast<int>('[')) {
156  KALDI_ERR << "ReadIntegerPairVector: expected to see [, saw "
157  << is.peek() << ", at file position " << is.tellg();
158  }
159  is.get(); // consume the '['.
160  is >> std::ws; // consume whitespace.
161  while (is.peek() != static_cast<int>(']')) {
162  if (sizeof(T) == 1) { // read/write chars as numbers.
163  int16 next_t1, next_t2;
164  is >> next_t1;
165  if (is.fail()) goto bad;
166  if (is.peek() != static_cast<int>(','))
167  KALDI_ERR << "ReadIntegerPairVector: expected to see ',', saw "
168  << is.peek() << ", at file position " << is.tellg();
169  is.get(); // consume the ','.
170  is >> next_t2 >> std::ws;
171  if (is.fail()) goto bad;
172  else
173  tmp_v.push_back(std::make_pair<T, T>((T)next_t1, (T)next_t2));
174  } else {
175  T next_t1, next_t2;
176  is >> next_t1;
177  if (is.fail()) goto bad;
178  if (is.peek() != static_cast<int>(','))
179  KALDI_ERR << "ReadIntegerPairVector: expected to see ',', saw "
180  << is.peek() << ", at file position " << is.tellg();
181  is.get(); // consume the ','.
182  is >> next_t2 >> std::ws;
183  if (is.fail()) goto bad;
184  else
185  tmp_v.push_back(std::pair<T, T>(next_t1, next_t2));
186  }
187  }
188  is.get(); // get the final ']'.
189  *v = tmp_v; // could use std::swap to use less temporary memory, but this
190  // uses less permanent memory.
191  }
192  if (!is.fail()) return;
193  bad:
194  KALDI_ERR << "ReadIntegerPairVector: read failure at file position "
195  << is.tellg();
196 }
#define KALDI_ASSERT_IS_INTEGER_TYPE(I)
Definition: kaldi-utils.h:133
kaldi::int32 int32
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ ReadIntegerVector()

void ReadIntegerVector ( std::istream &  is,
bool  binary,
std::vector< T > *  v 
)
inline

Function for reading STL vector of integer types.

Definition at line 232 of file io-funcs-inl.h.

References KALDI_ASSERT, KALDI_ASSERT_IS_INTEGER_TYPE, and KALDI_ERR.

Referenced by kaldi::ConvertSpliceComponent(), FramePoolingComponent::InitData(), main(), NnetExample::Read(), RegressionTree::Read(), QuestionsForKey::Read(), RegtreeMllrDiagGmm::Read(), LogisticRegression::Read(), ConstIntegerSet< EventValueType >::Read(), RegtreeFmllrDiagGmm::Read(), DiscriminativeSupervision::Read(), HmmTopology::Read(), CuArray< Int32Pair >::Read(), DiscriminativeNnetExample::Read(), AmSgmm2::Read(), ConvolutionModel::Read(), StatisticsExtractionComponentPrecomputedIndexes::Read(), ConvolutionComputation::Read(), NnetComputation::Command::Read(), NnetComputation::Read(), TdnnComponent::Read(), TdnnComponent::PrecomputedIndexes::Read(), Sgmm2GauPost::Read(), SpecAugmentTimeMaskComponentPrecomputedIndexes::Read(), SpliceComponent::Read(), SpliceMaxComponent::Read(), SumGroupComponent::Read(), PermuteComponent::Read(), kaldi::ReadBasicType(), FramePoolingComponent::ReadData(), BlockSoftmax::ReadData(), fst::ReadILabelInfo(), kaldi::nnet3::ReadVectorAsChar(), kaldi::UnitTestIo(), kaldi::UnitTestIoNew(), and kaldi::UnitTestIoPipe().

234  {
236  KALDI_ASSERT(v != NULL);
237  if (binary) {
238  int sz = is.peek();
239  if (sz == sizeof(T)) {
240  is.get();
241  } else { // this is currently just a check.
242  KALDI_ERR << "ReadIntegerVector: expected to see type of size "
243  << sizeof(T) << ", saw instead " << sz << ", at file position "
244  << is.tellg();
245  }
246  int32 vecsz;
247  is.read(reinterpret_cast<char *>(&vecsz), sizeof(vecsz));
248  if (is.fail() || vecsz < 0) goto bad;
249  v->resize(vecsz);
250  if (vecsz > 0) {
251  is.read(reinterpret_cast<char *>(&((*v)[0])), sizeof(T)*vecsz);
252  }
253  } else {
254  std::vector<T> tmp_v; // use temporary so v doesn't use extra memory
255  // due to resizing.
256  is >> std::ws;
257  if (is.peek() != static_cast<int>('[')) {
258  KALDI_ERR << "ReadIntegerVector: expected to see [, saw "
259  << is.peek() << ", at file position " << is.tellg();
260  }
261  is.get(); // consume the '['.
262  is >> std::ws; // consume whitespace.
263  while (is.peek() != static_cast<int>(']')) {
264  if (sizeof(T) == 1) { // read/write chars as numbers.
265  int16 next_t;
266  is >> next_t >> std::ws;
267  if (is.fail()) goto bad;
268  else
269  tmp_v.push_back((T)next_t);
270  } else {
271  T next_t;
272  is >> next_t >> std::ws;
273  if (is.fail()) goto bad;
274  else
275  tmp_v.push_back(next_t);
276  }
277  }
278  is.get(); // get the final ']'.
279  *v = tmp_v; // could use std::swap to use less temporary memory, but this
280  // uses less permanent memory.
281  }
282  if (!is.fail()) return;
283  bad:
284  KALDI_ERR << "ReadIntegerVector: read failure at file position "
285  << is.tellg();
286 }
#define KALDI_ASSERT_IS_INTEGER_TYPE(I)
Definition: kaldi-utils.h:133
kaldi::int32 int32
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ ReadToken()

void ReadToken ( std::istream &  is,
bool  binary,
std::string *  token 
)

ReadToken gets the next token and puts it in str (exception on failure).

If PeekToken() had been previously called, it is possible that the stream had failed to unget the starting '<' character. In this case ReadToken() returns the token string without the leading '<'. You must be prepared to handle this case. ExpectToken() handles this internally, and is not affected.

Definition at line 154 of file io-funcs.cc.

References kaldi::CharToString(), KALDI_ASSERT, and KALDI_ERR.

Referenced by kaldi::nnet2::ExpectOneOrTwoTokens(), kaldi::ExpectOneOrTwoTokens(), Component::Init(), LinearTransform::InitData(), AffineTransform::InitData(), MultiBasisComponent::InitData(), ParallelComponent::InitData(), ParametricRelu::InitData(), Splice::InitData(), AveragePoolingComponent::InitData(), MaxPoolingComponent::InitData(), RecurrentComponent::InitData(), FramePoolingComponent::InitData(), SimpleSentenceAveragingComponent::InitData(), LstmProjected::InitData(), BlstmProjected::InitData(), ConvolutionalComponent::InitData(), Rbm::InitData(), BlockSoftmax::InitData(), CopyComponent::InitData(), SentenceAveragingComponent::InitData(), Dropout::InitData(), AddShift::InitData(), Rescale::InitData(), NnetExample::Read(), LogisticRegression::Read(), LinearVtln::Read(), NnetIo::Read(), ContextDependency::Read(), FmpeOptions::Read(), AccumFullGmm::Read(), LdaEstimate::Read(), NnetDiscriminativeSupervision::Read(), IoSpecification::Read(), Sgmm2FmllrGlobalParams::Read(), NormalizeComponent::Read(), NnetChainSupervision::Read(), AccumDiagGmm::Read(), DropoutComponent::Read(), Questions::Read(), MleAmSgmm2Accs::Read(), HmmTopology::Read(), TransitionModel::Read(), FullGmm::Read(), CompressedMatrix::Read(), PackedMatrix< float >::Read(), Component::Read(), ConvolutionComponent::Read(), DiagGmm::Read(), Nnet::Read(), AmSgmm2::Read(), NnetComputation::MatrixInfo::Read(), LstmNonlinearityComponent::Read(), OnlineIvectorEstimationStats::Read(), Vector< float >::Read(), NnetComputation::Read(), TdnnComponent::Read(), BackpropTruncationComponent::Read(), NonlinearComponent::Read(), ConstantComponent::Read(), NaturalGradientAffineComponent::Read(), Matrix< BaseFloat >::Read(), AffineComponent::Read(), AffineComponentPreconditioned::Read(), AffineComponentPreconditionedOnline::Read(), SpliceComponent::Read(), SpliceMaxComponent::Read(), NoOpComponent::Read(), SumGroupComponent::Read(), ClipGradientComponent::Read(), DctComponent::Read(), ConstantFunctionComponent::Read(), Convolutional1dComponent::Read(), NaturalGradientPerElementScaleComponent::Read(), CompositeComponent::Read(), kaldi::ReadBasicType(), ParametricRelu::ReadData(), AffineTransform::ReadData(), RecurrentComponent::ReadData(), LinearTransform::ReadData(), LstmProjected::ReadData(), BlstmProjected::ReadData(), MultiBasisComponent::ReadData(), Rbm::ReadData(), Dropout::ReadData(), ComponentPrecomputedIndexes::ReadNew(), Component::ReadNew(), UpdatableComponent::ReadUpdatableCommon(), kaldi::UnitTestIo(), kaldi::UnitTestIoNew(), and kaldi::UnitTestIoPipe().

154  {
155  KALDI_ASSERT(str != NULL);
156  if (!binary) is >> std::ws; // consume whitespace.
157  is >> *str;
158  if (is.fail()) {
159  KALDI_ERR << "ReadToken, failed to read token at file position "
160  << is.tellg();
161  }
162  if (!isspace(is.peek())) {
163  KALDI_ERR << "ReadToken, expected space after token, saw instead "
164  << CharToString(static_cast<char>(is.peek()))
165  << ", at file position " << is.tellg();
166  }
167  is.get(); // consume the space.
168 }
#define KALDI_ERR
Definition: kaldi-error.h:147
std::string CharToString(const char &c)
Definition: kaldi-utils.cc:36
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ WriteBasicType()

void WriteBasicType ( std::ostream &  os,
bool  binary,
t 
)

WriteBasicType is the name of the write function for bool, integer types, and floating-point types.

They all throw on error.

Definition at line 34 of file io-funcs-inl.h.

References KALDI_ASSERT_IS_INTEGER_TYPE, and KALDI_ERR.

Referenced by LstmNonlinearityComponent::ConsolidateMemory(), kaldi::TestBuildTree(), kaldi::TestGenRandStats(), kaldi::UnitTestIo(), kaldi::UnitTestIoNew(), kaldi::UnitTestIoPipe(), AccumAmDiagGmm::Write(), ScalarClusterable::Write(), AffineXformStats::Write(), CompressedAffineXformStats::Write(), MlltAccs::Write(), NnetExample::Write(), AmNnetSimple::Write(), RegressionTree::Write(), CuBlockMatrix< Real >::Write(), QuestionsForKey::Write(), RegtreeMllrDiagGmm::Write(), Index::Write(), LinearVtln::Write(), AmDiagGmm::Write(), GaussClusterable::Write(), FmpeOptions::Write(), DistributeComponent::Write(), BasisFmllrAccus::Write(), AccumFullGmm::Write(), RegtreeFmllrDiagGmm::Write(), LdaEstimate::Write(), ContextDependency::Write(), Sgmm2FmllrGlobalParams::Write(), IoSpecification::Write(), NormalizeComponent::Write(), SparseVector< Real >::Write(), GaussPostHolder::Write(), NnetDiscriminativeExample::Write(), AccumDiagGmm::Write(), BasisFmllrEstimate::Write(), NnetChainExample::Write(), Questions::Write(), RegtreeMllrDiagGmmAccs::Write(), FmllrSgmm2Accs::Write(), DiscriminativeSupervision::Write(), BasicHolder< BasicType >::Write(), HmmTopology::Write(), MleAmSgmm2Accs::Write(), RefineClustersOptions::Write(), TransitionModel::Write(), VectorClusterable::Write(), PackedMatrix< float >::Write(), NnetOptimizeOptions::Write(), RestrictedAttentionComponent::Write(), ComputationRequest::Write(), Component::Write(), GrammarFst::Write(), ConvolutionComponent::Write(), DiscriminativeNnetExample::Write(), RegtreeFmllrDiagGmmAccs::Write(), ElementwiseProductComponent::Write(), SparseMatrix< float >::Write(), ConstantEventMap::Write(), BatchNormComponent::Write(), BasicVectorHolder< BasicType >::Write(), Nnet::Write(), ConvolutionModel::Write(), AmSgmm2::Write(), StatisticsExtractionComponent::Write(), TimeHeightConvolutionComponent::Write(), IvectorExtractor::Write(), TableEventMap::Write(), ConstArpaLm::Write(), SplitEventMap::Write(), NnetComputation::MatrixInfo::Write(), NnetComputation::MatrixDebugInfo::Write(), NnetComputation::SubMatrixInfo::Write(), ConvolutionComputation::Write(), BasicVectorVectorHolder< BasicType >::Write(), LstmNonlinearityComponent::Write(), NnetComputation::Command::Write(), VectorBase< float >::Write(), NonlinearComponent::Write(), OnlineIvectorEstimationStats::Write(), StatisticsPoolingComponent::Write(), ConvolutionComputationIo::Write(), MaxoutComponent::Write(), NnetComputation::Write(), TdnnComponent::Write(), BasicPairVectorHolder< BasicType >::Write(), MaxpoolingComponent::Write(), IvectorExtractorStats::Write(), BackpropTruncationComponent::Write(), TdnnComponent::PrecomputedIndexes::Write(), PnormComponent::Write(), Sgmm2GauPost::Write(), BackpropTruncationComponentPrecomputedIndexes::Write(), RepeatedAffineComponent::Write(), ComputationCache::Write(), ConstantComponent::Write(), PowerComponent::Write(), ScaleComponent::Write(), DropoutMaskComponent::Write(), MatrixBase< float >::Write(), NaturalGradientAffineComponent::Write(), AffineComponent::Write(), GeneralDropoutComponent::Write(), LinearComponent::Write(), AffineComponentPreconditioned::Write(), GeneralDropoutComponentPrecomputedIndexes::Write(), AffineComponentPreconditionedOnline::Write(), SpecAugmentTimeMaskComponent::Write(), SpecAugmentTimeMaskComponentPrecomputedIndexes::Write(), SpliceComponent::Write(), SpliceMaxComponent::Write(), NoOpComponent::Write(), BlockAffineComponent::Write(), SumBlockComponent::Write(), BlockAffineComponentPreconditioned::Write(), ClipGradientComponent::Write(), DctComponent::Write(), DropoutComponent::Write(), PerElementOffsetComponent::Write(), AdditiveNoiseComponent::Write(), ConstantFunctionComponent::Write(), Convolutional1dComponent::Write(), NaturalGradientPerElementScaleComponent::Write(), ScaleAndOffsetComponent::Write(), CompositeComponent::Write(), Rbm::WriteAsNnet(), kaldi::WriteBuildTreeStats(), kaldi::nnet3::WriteCindexVector(), kaldi::nnet3::WriteCindexVectorElementBinary(), AveragePoolingComponent::WriteData(), MaxPoolingComponent::WriteData(), ParametricRelu::WriteData(), SimpleSentenceAveragingComponent::WriteData(), AffineTransform::WriteData(), RecurrentComponent::WriteData(), LinearTransform::WriteData(), ParallelComponent::WriteData(), LstmProjected::WriteData(), FramePoolingComponent::WriteData(), MultiBasisComponent::WriteData(), ConvolutionalComponent::WriteData(), BlstmProjected::WriteData(), Dropout::WriteData(), AddShift::WriteData(), Rescale::WriteData(), kaldi::WriteEventType(), fst::WriteILabelInfo(), kaldi::nnet3::WriteIndexVector(), kaldi::nnet3::WriteIndexVectorElementBinary(), kaldi::WritePosterior(), and UpdatableComponent::WriteUpdatableCommon().

35  {
36  // Compile time assertion that this is not called with a wrong type.
38  if (binary) {
39  char len_c = (std::numeric_limits<T>::is_signed ? 1 : -1)
40  * static_cast<char>(sizeof(t));
41  os.put(len_c);
42  os.write(reinterpret_cast<const char *>(&t), sizeof(t));
43  } else {
44  if (sizeof(t) == 1)
45  os << static_cast<int16>(t) << " ";
46  else
47  os << t << " ";
48  }
49  if (os.fail()) {
50  KALDI_ERR << "Write failure in WriteBasicType.";
51  }
52 }
#define KALDI_ASSERT_IS_INTEGER_TYPE(I)
Definition: kaldi-utils.h:133
#define KALDI_ERR
Definition: kaldi-error.h:147

◆ WriteBasicType< bool >()

void WriteBasicType< bool > ( std::ostream &  os,
bool  binary,
bool  b 
)

Definition at line 26 of file io-funcs.cc.

References KALDI_ERR.

26  {
27  os << (b ? "T":"F");
28  if (!binary) os << " ";
29  if (os.fail())
30  KALDI_ERR << "Write failure in WriteBasicType<bool>";
31 }
#define KALDI_ERR
Definition: kaldi-error.h:147

◆ WriteBasicType< double >()

void WriteBasicType< double > ( std::ostream &  os,
bool  binary,
double  f 
)

Definition at line 62 of file io-funcs.cc.

62  {
63  if (binary) {
64  char c = sizeof(f);
65  os.put(c);
66  os.write(reinterpret_cast<const char *>(&f), sizeof(f));
67  } else {
68  os << f << " ";
69  }
70 }

◆ WriteBasicType< float >()

void WriteBasicType< float > ( std::ostream &  os,
bool  binary,
float  f 
)

Definition at line 51 of file io-funcs.cc.

51  {
52  if (binary) {
53  char c = sizeof(f);
54  os.put(c);
55  os.write(reinterpret_cast<const char *>(&f), sizeof(f));
56  } else {
57  os << f << " ";
58  }
59 }

◆ WriteIntegerPairVector()

void WriteIntegerPairVector ( std::ostream &  os,
bool  binary,
const std::vector< std::pair< T, T > > &  v 
)
inline

Function for writing STL vectors of pairs of integer types.

Definition at line 93 of file io-funcs-inl.h.

References KALDI_ASSERT, KALDI_ASSERT_IS_INTEGER_TYPE, and KALDI_ERR.

Referenced by kaldi::ReadBasicType(), kaldi::UnitTestIo(), DistributeComponentPrecomputedIndexes::Write(), ConvolutionModel::Write(), StatisticsExtractionComponentPrecomputedIndexes::Write(), StatisticsPoolingComponentPrecomputedIndexes::Write(), and NnetComputation::Write().

94  {
95  // Compile time assertion that this is not called with a wrong type.
97  if (binary) {
98  char sz = sizeof(T); // this is currently just a check.
99  os.write(&sz, 1);
100  int32 vecsz = static_cast<int32>(v.size());
101  KALDI_ASSERT((size_t)vecsz == v.size());
102  os.write(reinterpret_cast<const char *>(&vecsz), sizeof(vecsz));
103  if (vecsz != 0) {
104  os.write(reinterpret_cast<const char *>(&(v[0])), sizeof(T) * vecsz * 2);
105  }
106  } else {
107  // focus here is on prettiness of text form rather than
108  // efficiency of reading-in.
109  // reading-in is dominated by low-level operations anyway:
110  // for efficiency use binary.
111  os << "[ ";
112  typename std::vector<std::pair<T, T> >::const_iterator iter = v.begin(),
113  end = v.end();
114  for (; iter != end; ++iter) {
115  if (sizeof(T) == 1)
116  os << static_cast<int16>(iter->first) << ','
117  << static_cast<int16>(iter->second) << ' ';
118  else
119  os << iter->first << ','
120  << iter->second << ' ';
121  }
122  os << "]\n";
123  }
124  if (os.fail()) {
125  KALDI_ERR << "Write failure in WriteIntegerPairVector.";
126  }
127 }
#define KALDI_ASSERT_IS_INTEGER_TYPE(I)
Definition: kaldi-utils.h:133
kaldi::int32 int32
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ WriteIntegerVector()

void WriteIntegerVector ( std::ostream &  os,
bool  binary,
const std::vector< T > &  v 
)
inline

Function for writing STL vectors of integer types.

Definition at line 198 of file io-funcs-inl.h.

References KALDI_ASSERT, KALDI_ASSERT_IS_INTEGER_TYPE, and KALDI_ERR.

Referenced by kaldi::ConvertAlignmentForPhone(), kaldi::GetIlabelMapping(), main(), kaldi::ProcessTopo(), kaldi::ReadBasicType(), kaldi::TestBuildTree(), kaldi::TestGenRandStats(), kaldi::UnitTestIo(), kaldi::UnitTestIoNew(), kaldi::UnitTestIoPipe(), NnetExample::Write(), RegressionTree::Write(), QuestionsForKey::Write(), RegtreeMllrDiagGmm::Write(), LogisticRegression::Write(), ConstIntegerSet< EventValueType >::Write(), RegtreeFmllrDiagGmm::Write(), DiscriminativeSupervision::Write(), HmmTopology::Write(), CuArray< Int32Pair >::Write(), DiscriminativeNnetExample::Write(), ConvolutionModel::Write(), AmSgmm2::Write(), StatisticsExtractionComponentPrecomputedIndexes::Write(), ConvolutionComputation::Write(), NnetComputation::Command::Write(), NnetComputation::Write(), TdnnComponent::Write(), TdnnComponent::PrecomputedIndexes::Write(), Sgmm2GauPost::Write(), SpecAugmentTimeMaskComponentPrecomputedIndexes::Write(), SpliceComponent::Write(), SpliceMaxComponent::Write(), SumGroupComponent::Write(), PermuteComponent::Write(), FramePoolingComponent::WriteData(), BlockSoftmax::WriteData(), fst::WriteILabelInfo(), and kaldi::nnet3::WriteVectorAsChar().

199  {
200  // Compile time assertion that this is not called with a wrong type.
202  if (binary) {
203  char sz = sizeof(T); // this is currently just a check.
204  os.write(&sz, 1);
205  int32 vecsz = static_cast<int32>(v.size());
206  KALDI_ASSERT((size_t)vecsz == v.size());
207  os.write(reinterpret_cast<const char *>(&vecsz), sizeof(vecsz));
208  if (vecsz != 0) {
209  os.write(reinterpret_cast<const char *>(&(v[0])), sizeof(T)*vecsz);
210  }
211  } else {
212  // focus here is on prettiness of text form rather than
213  // efficiency of reading-in.
214  // reading-in is dominated by low-level operations anyway:
215  // for efficiency use binary.
216  os << "[ ";
217  typename std::vector<T>::const_iterator iter = v.begin(), end = v.end();
218  for (; iter != end; ++iter) {
219  if (sizeof(T) == 1)
220  os << static_cast<int16>(*iter) << " ";
221  else
222  os << *iter << " ";
223  }
224  os << "]\n";
225  }
226  if (os.fail()) {
227  KALDI_ERR << "Write failure in WriteIntegerVector.";
228  }
229 }
#define KALDI_ASSERT_IS_INTEGER_TYPE(I)
Definition: kaldi-utils.h:133
kaldi::int32 int32
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ WriteToken() [1/2]

void WriteToken ( std::ostream &  os,
bool  binary,
const char *  token 
)

The WriteToken functions are for writing nonempty sequences of non-space characters.

They are not for general strings.

Definition at line 134 of file io-funcs.cc.

References kaldi::CheckToken(), KALDI_ASSERT, and KALDI_ERR.

Referenced by LstmNonlinearityComponent::ConsolidateMemory(), kaldi::ReadBasicType(), kaldi::UnitTestIo(), kaldi::UnitTestIoNew(), kaldi::UnitTestIoPipe(), AccumAmDiagGmm::Write(), ScalarClusterable::Write(), AffineXformStats::Write(), CompressedAffineXformStats::Write(), MlltAccs::Write(), NnetExample::Write(), AmNnetSimple::Write(), RegressionTree::Write(), QuestionsForKey::Write(), CuBlockMatrix< Real >::Write(), RegtreeMllrDiagGmm::Write(), LogisticRegression::Write(), LinearVtln::Write(), Index::Write(), AmDiagGmm::Write(), NnetIo::Write(), FmpeOptions::Write(), GaussClusterable::Write(), DistributeComponent::Write(), BasisFmllrAccus::Write(), RegtreeFmllrDiagGmm::Write(), AccumFullGmm::Write(), LdaEstimate::Write(), NnetDiscriminativeSupervision::Write(), Sgmm2FmllrGlobalParams::Write(), ContextDependency::Write(), NnetChainSupervision::Write(), IoSpecification::Write(), NormalizeComponent::Write(), SparseVector< Real >::Write(), NnetDiscriminativeExample::Write(), BasisFmllrEstimate::Write(), AccumDiagGmm::Write(), NnetChainExample::Write(), Questions::Write(), RegtreeMllrDiagGmmAccs::Write(), FmllrSgmm2Accs::Write(), DiscriminativeSupervision::Write(), FullGmm::Write(), MleAmSgmm2Accs::Write(), HmmTopology::Write(), RefineClustersOptions::Write(), VectorClusterable::Write(), TransitionModel::Write(), Plda::Write(), CompressedMatrix::Write(), Nnet::Write(), DistributeComponentPrecomputedIndexes::Write(), PackedMatrix< float >::Write(), NnetOptimizeOptions::Write(), RestrictedAttentionComponent::Write(), ComputationRequest::Write(), Component::Write(), EventMap::Write(), GrammarFst::Write(), DiagGmm::Write(), ConvolutionComponent::Write(), DiscriminativeNnetExample::Write(), RegtreeFmllrDiagGmmAccs::Write(), ElementwiseProductComponent::Write(), RestrictedAttentionComponent::PrecomputedIndexes::Write(), SparseMatrix< float >::Write(), ConstantEventMap::Write(), OnlineGmmAdaptationState::Write(), BatchNormComponent::Write(), ConvolutionModel::Write(), AmSgmm2::Write(), StatisticsExtractionComponent::Write(), OnlineIvectorExtractorAdaptationState::Write(), TimeHeightConvolutionComponent::Write(), IvectorExtractor::Write(), TableEventMap::Write(), ConstArpaLm::Write(), OnlineCmvnState::Write(), TimeHeightConvolutionComponent::PrecomputedIndexes::Write(), StatisticsExtractionComponentPrecomputedIndexes::Write(), SplitEventMap::Write(), NnetComputation::MatrixInfo::Write(), NnetComputation::MatrixDebugInfo::Write(), NnetComputation::SubMatrixInfo::Write(), ConvolutionComputation::Write(), LstmNonlinearityComponent::Write(), NnetComputation::Command::Write(), VectorBase< float >::Write(), NonlinearComponent::Write(), OnlineIvectorEstimationStats::Write(), StatisticsPoolingComponent::Write(), ConvolutionComputationIo::Write(), MaxoutComponent::Write(), StatisticsPoolingComponentPrecomputedIndexes::Write(), NnetComputation::Write(), TdnnComponent::Write(), MaxpoolingComponent::Write(), IvectorExtractorStats::Write(), BackpropTruncationComponent::Write(), TdnnComponent::PrecomputedIndexes::Write(), PnormComponent::Write(), Sgmm2GauPost::Write(), BackpropTruncationComponentPrecomputedIndexes::Write(), RepeatedAffineComponent::Write(), ComputationCache::Write(), ConstantComponent::Write(), PowerComponent::Write(), ScaleComponent::Write(), DropoutMaskComponent::Write(), MatrixBase< float >::Write(), NaturalGradientAffineComponent::Write(), AffineComponent::Write(), GeneralDropoutComponent::Write(), LinearComponent::Write(), AffineComponentPreconditioned::Write(), GeneralDropoutComponentPrecomputedIndexes::Write(), AffineComponentPreconditionedOnline::Write(), SpecAugmentTimeMaskComponent::Write(), SpecAugmentTimeMaskComponentPrecomputedIndexes::Write(), SpliceComponent::Write(), SpliceMaxComponent::Write(), NoOpComponent::Write(), BlockAffineComponent::Write(), SumBlockComponent::Write(), BlockAffineComponentPreconditioned::Write(), SumGroupComponent::Write(), PermuteComponent::Write(), ClipGradientComponent::Write(), DctComponent::Write(), FixedLinearComponent::Write(), FixedAffineComponent::Write(), FixedScaleComponent::Write(), PerElementScaleComponent::Write(), FixedBiasComponent::Write(), DropoutComponent::Write(), PerElementOffsetComponent::Write(), AdditiveNoiseComponent::Write(), ConstantFunctionComponent::Write(), Convolutional1dComponent::Write(), NaturalGradientPerElementScaleComponent::Write(), ScaleAndOffsetComponent::Write(), CompositeComponent::Write(), Rbm::WriteAsNnet(), kaldi::WriteBuildTreeStats(), kaldi::nnet3::WriteCindexVector(), AveragePoolingComponent::WriteData(), MaxPoolingComponent::WriteData(), ParametricRelu::WriteData(), SimpleSentenceAveragingComponent::WriteData(), AffineTransform::WriteData(), RecurrentComponent::WriteData(), LinearTransform::WriteData(), ParallelComponent::WriteData(), LstmProjected::WriteData(), FramePoolingComponent::WriteData(), MultiBasisComponent::WriteData(), ConvolutionalComponent::WriteData(), BlstmProjected::WriteData(), Rbm::WriteData(), Dropout::WriteData(), AddShift::WriteData(), Rescale::WriteData(), kaldi::WriteEventType(), kaldi::nnet3::WriteIndexVector(), kaldi::WriteToken(), and UpdatableComponent::WriteUpdatableCommon().

134  {
135  // binary mode is ignored;
136  // we use space as termination character in either case.
137  KALDI_ASSERT(token != NULL);
138  CheckToken(token); // make sure it's valid (can be read back)
139  os << token << " ";
140  if (os.fail()) {
141  KALDI_ERR << "Write failure in WriteToken.";
142  }
143 }
void CheckToken(const char *token)
Definition: io-funcs.cc:122
#define KALDI_ERR
Definition: kaldi-error.h:147
#define KALDI_ASSERT(cond)
Definition: kaldi-error.h:185

◆ WriteToken() [2/2]

void WriteToken ( std::ostream &  os,
bool  binary,
const std::string &  token 
)

Definition at line 150 of file io-funcs.cc.

References kaldi::WriteToken().

150  {
151  WriteToken(os, binary, token.c_str());
152 }
void WriteToken(std::ostream &os, bool binary, const std::string &token)
Definition: io-funcs.cc:150