All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
Versions of Kaldi

Versioning scheme

During its lifetime, Kaldi has has three different versioning methods. Originally Kaldi was a subversion (svn)-based project, and was hosted on Sourceforge. Then Kaldi was moved to github, and for some time the only version-number available was the git hash of the commit.

In January 2017 we introduced a version number scheme. The first version of Kaldi was 5.0.0, in recognition of the fact that the project had already existed for quite a long time. The basic scheme is major/minor/patch, but the "patch" version number may also encompass features (usually back-compatible ones). The "patch number" automatically increases whenever a commit to Kaldi is merged on github.

We only intend to change the major or minor version number when making relatively larger changes, or non-back compatible changes. Version 5.1 of Kaldi is currently being prepared. When that is finished (probably in early Feburary 2017), the latest version of 5.0.x will be backed up to a branch named '5.0', and 'master' will point to version 5.1.0. We may continue to update the 5.0 branch with fixes and the like, depending on demand.

We always plan to recommend that Kaldi users check out the latest version of 'master', since actively supporting multiple versions would increase our workload.

Versions (and changes)

This section lists the version numbers of Kaldi with the commit messages for each patch commit (by "patch commit" we mean a commit that does not increase the major or minor version number). Each time we add a new major/minor version number we will include a longer section explaining the changes involved.

Version 5.0

This is the first major/minor version number after introducing the versioning scheme. The latest revision of version 5.0 is saved as branch "5.0" on github.

Specific patches:

5.0.0 c160a98 2017-01-05 [build] Adding versioning mechanism to Kaldi. This is version 5.0.0 (#1306)

5.0.1 53e33dc 2017-01-06 [scripts] nnet3 scripts: minor bug fixes in error-handling code (#1321)

5.0.2 f15be63 2017-01-06 [scripts] minor updates of scripts in nnet1 (#1318)

5.0.3 735b2b1 2017-01-06 [egs] minor fix in fisher_swbd/s5/local/chain/ (#1320)

5.0.4 a4b2091 2017-01-07 added reverberation based augmentation recipe for TDNN+LSTM model on AMI; a fix to aspire recipe (#1314)

5.0.5 5cdbd18 2017-01-07 [build] Makefile changes: print time for tests; dash-compatibility change (#1322)

5.0.6 e09e92f 2017-01-08 [egs] minor fixes for mandarin recipes (#1325)

5.0.7 722df9a 2017-01-09 [build] Fix dash problems [set shell to /bin/bash in Makefile]

5.0.8 f646952 2017-01-09 [build] src/base/ change regexp syntax to support older bash versions. (#1327)

5.0.9 5f15d25 2017-01-10 [egs] cosmetic; small typo fix in sre08 i-vector DNN extraction (#1330)

5.0.10 69b616d 2017-01-10 [src,egs,scripts] Remove sinusoid detection code and old SGMM code/scripts (#1329)

5.0.11 cd06802 2017-01-10 [src] nnet3: fixed bug (LstmNonlinearityComponent::ZeroStats() was not implemented) (#1302)

5.0.12 748a4a9 2017-01-11 [src] Restore to sgmm2bin (was deleted when sgmmbin deleted)

5.0.13 95f0fec 2017-01-11 [src] nnet1: replacing 'bc' by 'awk' in steps/nnet/ (#1333)

5.0.14 728b303 2017-01-11 [scrips] fix bug in when using G.carpa (#1334)

5.0.15 42291a6 2017-01-12 [doc,egs] Fix path from lm to lmbin in kaldi for dummies and various egs/*/*/

5.0.16 421485e 2017-01-12 [src] nnet3: Remove deprecated options which are no longer used (#1335)

5.0.17 847035e 2017-01-13 [egs] Remove some unused scripts

5.0.18 ea25438 2017-01-13 [build] Fix return code of tools/extras/ (#1337)

5.0.19 e1e7bbf 2017-01-13 [scripts] nnet3: fix to xconfig parsing to enable e.g. "input@-1" in xconfig descriptors (#1338)

5.0.20 1dabfa5 2017-01-14 [src] cosmetic changes to rnnlm-related code (#1283)

5.0.21 4d2c78e 2017-01-16 Some bug fixes to I/O code for nnet3

5.0.22 6c0a012 2017-01-16 Remove checking code which would fail if code updated in middle of training run.

5.0.23 f7b2fe7 2017-01-16 Bug fix (resolve failing test)

5.0.24 390aff4 2017-01-18 [egs] Fix typo in egs/csj/s5/local/ (#1351)

5.0.25 e8e5928 2017-01-19 [src] cudamatrix: added {Cu,}VectorBase::SetRandUniform(), + optimize/fix cu-rand code (#1352)

5.0.26 d117c95 2017-01-20 [src] Some style-related fixes (and fix compiler warnings) in src/lm/

5.0.27 b9c1e8b 2017-01-20 [src] Modify some tests that were slow on travis, to make them faster. (#1347)

5.0.28 df730e8 2017-01-21 [src] Implement CuMatrixBase<Real>::CopyColsFromVec() (#1361)

5.0.29 0440417 2017-01-23 [src,scripts]: Adding dropout schedule option to nnet3 (#1248)

5.0.30 61510ca 2017-01-22 nnet3/report : Modified directory specification options in (#1368)

5.0.31 4507183 2017-01-24 [build] fixing patch for OpenFst's compat.h for Windows build (#1373)

5.0.32 99b7d96 2017-01-24 [scripts] lexicon learning: update missing defaults and help message; other fixes (#1360)

5.0.33 5c205ea 2017-01-24 [scripts] nnet3/report : Added support for parsing latest compute_{train,valid}*.log (#1371)

5.0.34 f42b3ce 2017-01-24 [scripts] nnet1,make_denlats: prevent crash when utt-ids contain slashes (#1374)

5.0.35 82167f9 2017-01-24 [scripts] remove import statement that was creating problems in nnet3 scripts.

5.0.36 9208165 2017-01-25 [src] nnet3: Add the "per-frame" option to DropoutComponent (#1324)

5.0.37 e227eda 2017-01-25 [doc] Documentation changes; add scripts to automatically add documentation of patch versions.

5.0.38 0c1517c 2017-01-25 [egs] swbd/s5c, chain recipes: fix hardcoded directory name (#1377)

5.0.39 c96a0aa 2017-01-26 [build]: use BSD-compatible mktemp in get_version*.sh; thanks; @andrely.

5.0.40 6ec2eb0 2017-01-26 [build]: fix to previous commit regarding usage of mktemp.

5.0.41 3b7fd1f 2017-01-26 [scripts] Set path in utils/ just like in or to avoid path problems (#1379)

5.0.42 193bb92 2017-01-26 [build]: minor bug fix in maintenance/documentation script.

5.0.43 5af1983 2017-01-27 [scripts] Fix nnet3 training scripts for bug introduced in #1371

5.0.44 2f8c33d 2017-01-27 [scripts,egs] make steps/dict/ more robust RE empty g2p prons (#1378)

5.0.45 9d53e05 2017-01-29 [scripts] nnet3 training: fix for bug introduced when dropout scripting support was merged.

5.0.46 da59179 2017-01-30 [scripts] steps/dict/ minor fixes (#1385)

5.0.47 01d96bf 2017-02-01 [scripts] Fix bug in dropout code found by Tanel Alumae

5.0.48 67cabd0 2017-02-01 [build]: resolving OpenFst compilation issue with gcc-6.x (#1392)

5.0.49 f1d7891 2017-02-01 [egs] Add new graphemic system for Gale Arabic, with newer nnet scripts (#1298)

5.0.50 21cfe99 2017-02-02 [build] Windows build: generate missing base/version.h; cosmetic changes (#1397)

5.0.51 cd97bd2 2017-02-02 [build]: Enable cross compilation, including to android. (#726)

5.0.52 c747ed5 2017-02-06 [build] fixing issue introduced in the previous win commit (#1399)

5.0.53 21c8031 2017-02-07 [egs] Fix to HKUST nnet2/3 scripts. (#1401)

5.0.54 dc454cc 2017-02-08 [egs,scripts,src] Add BABEL s5d recipe; various associated fixes (#1356)

5.0.55 9a1244f 2017-02-08 [egs] removing empty files in BABEL recipe (#1406)

5.0.56 2fd8da9 2017-02-09 [scripts] Give prepare_lang the option to generate more phone disambig symbols (#1408)

5.0.57 bcc71b6 2017-02-09 [egs] sprakbanken recipe: add nnet3 examples.

5.0.58 bd73932 2017-02-10 [src] Get rid of clang 3.9 warnings in table-reading code (#1414)

5.0.59 cc1d677 2017-02-10 [egs] egs/fisher_swbd/s5/local/online/, change default stage to 0 (#1416)

5.0.60 9226202 2017-02-11 [egs] Add example scripts for Frisian-Dutch language (FAME! corpus)

5.0.61 37b5352 2017-02-11 [src,doc] Documentation updates; fixes to comments.

5.0.62 68cee21 2017-02-17 [src] Make various tests faster, especially without GPU (#1428)

Version 5.1

Version 5.1 is the current master branch of Kaldi. Some of the major changes introduced in version 5.1 are:

  • Kaldi now requires C++11 to compile, and we support only the latest version of OpenFst (1.6.0). (This simplifies Kaldi's code, and will later enable the threading code to be rewritten to use C++11's better and more portable mechanisms).
  • The way chunk size and feature context is handled in nnet3 is changed to allow variable chunk size and shorter context at utterance boundaries. See Context and chunk-size in the "nnet3" setup for more information.
  • A new decoding mechanism, Looped decoding, is introduced in nnet3; this allows faster and more-easily-online decoding for recurrent setups (but only unidirectionally-recurrent ones, like LSTMs but not BLSTMs).
  • Online decoding with nnet3 models is now rewritten; it's faster and it supports models like LSTMs.
  • The sequence-training scripts in nnet3 are refactored and are now simpler and use less disk space.
  • There are scripts for segmentation of long transcribed audio files.
5.1.0 68cee21 2017-02-17 [src] Make various tests faster, especially without GPU (#1428)

5.1.1 2145519 2017-02-17 [build] Updating version file-- this commit marks version 5.1.0

5.1.2 9b9b561 2017-02-17 Kaldi 5.1.0 (#1342)

5.1.3 6477dec 2017-02-17 [build] Change travis build to use clang; remove -rdynamic from CXXFLAGS (it's in LDFLAGS)

5.1.4 23338bf 2017-02-19 [scripts] fix syntax error in [thanks: daniel galvez]

5.1.5 7d538e2 2017-02-19 [build] Increase OpenFst version 1.6.0->1.6.1. (#1434)

5.1.6 8458587 2017-02-20 [build] Use github not sourceforge for IRSTLM [sourceforge repo dead?] (#1435)

5.1.7 d218412 2017-02-20 [egs] clean up the HKUST scripts and add scoring filters (#1436)

5.1.8 7b862c7 2017-02-20 [src,doc] Update version documentation with version 5.1; makefile fix; add details to info string for nnet3 component.

5.1.9 483192e 2017-02-20 [build] update .gitignore: ignore openfst-1.6.1 (#1439)

5.1.10 8c77d2c 2017-02-21 [src,egs,scripts] Support frame-subsampling with non-chain nnet3 models (#1238)

5.1.11 2c3df88 2017-02-21 [build] cloning IRSTLM, use https to access github, not ssh (#1441)

5.1.12 9866a4c 2017-02-22 [build] Update Travis configuration to get builds to complete on time. (#1443)

5.1.13 351da28 2017-02-22 [egs] Iban recipe: MacOSX compatibility fixes (#1448)

5.1.14 ae4a5ef 2017-02-23 [scripts] Make it so i-vector ID is not required for steps/nnet3/

5.1.15 70f5360 2017-02-23 [build] Stop from crashing when set -u is active (#1451)

5.1.16 dff2a3f 2017-02-24 upgrade the sox calls to use more compatible options (#1453)

5.1.17 37bebd1 2017-02-25 [egs] fix typo in egs/swbd/s5c/local/nnet3/ (#1452)

5.1.18 4cda604 2017-02-25 [scripts] xconfig: make scripts work when LDA-like preconditioning layer is not used (#1447)

5.1.19 089e596 2017-02-25 [build] Update OpenFst minimum version check to 1.6 in tools/Makefile. (#1455)

5.1.20 344e1ad 2017-02-25 [src] Fix bug in (prevent crash in nnet3 online decoding).

5.1.21 25b1299 2017-02-25 [egs] fix MacOSX incompatibilities in calls of paste (#1457)

5.1.22 d60e3cc 2017-02-27 [egs,scripts] Misc script fixes; refactor wsj/s5 examples; update tedlium/s5_r2 (#1456)

5.1.23 3f18012 2017-02-27 [scripts,egs] Replace SGE-specific options to (etc.) with generic options (#1461)

5.1.24 1db4bba 2017-02-27 [scripts] Fix to nnet2 script (won't change results) (#1462)

5.1.25 ee45757 2017-02-28 [scripts] nnet3: remove import from tdnn in the xconfig script since it no longer exists (#1464)

5.1.26 0c4441e 2017-02-28 [egs] Add online-decoding example in some Swbd chain scripts (#1466)

5.1.27 4a58ab9 2017-02-28 [scripts], prevent function returning false in case reco2file_and_channel does not exist (#1467)

5.1.28 f6b011f 2017-03-01 [src] kwsbin/, fix bug introduced in the last babel recipe commit (#1470)

5.1.29 d5a9d5c 2017-03-02 [src] add CUDA kernel for backprop of NormalizeLayer (#1458)

5.1.30 ba77d97 2017-03-03 [src] fix kws pipeline after upgrading to openfst-1.6 (#1471)

5.1.31 828f0b9 2017-03-03 [egs] Add discriminative training script for WSJ/TDNN-LSTM (#1468)

5.1.32 5bad9b2 2017-03-06 [src] bug-fix in gst plugin code (issue on g++ 5.4.0) (#1479)

5.1.33 34128c5 2017-03-06 [scripts,egs] sMBR on LFR xent system with shifted feats (#1477)

5.1.34 5bfa9e5 2017-03-08 [egs] Minor cosmetic changes in voxforge example script (#1483)

5.1.35 bd23a10 2017-03-10 [egs] Fixes to URLs in vystadial example script.

5.1.36 1a4dbf6 2017-03-13 [src] nnet1: fixing issue in multi-task training (#1491)

5.1.37 eba49a0 2017-03-14 [build] Bump OpenFst version to v1.6.2 (#1492)

5.1.38 5c98096 2017-03-16 [egs] swbd/chain : added blstm script using fast-LSTM; added BLSTM+TDNN script. (#1497)

5.1.39 3fec956 2017-03-17 [egs] update fisher_swbd recipe (fixes to how things are installed). (#1498)

5.1.40 eac0baa 2017-03-19 [src] sort cuda kernel function declarations to make searching easier. (#1501)

5.1.41 25ca8e4 2017-03-19 [build] Android compilation, bug-fixes (#1502)

5.1.42 ed33b1f 2017-03-19 [doc] Add a note to about Android cross compilation (#1503)

5.1.43 965cc49 2017-03-20 [egs] ami : Added tdnn_lstm recipe with fast-lstmp layer. Added tdnn_lstm recipe with -1 delay at lowest lstm layer (#1505)

5.1.44 39ad410 2017-03-22 [scripts] prevent failure when doesn't exist (#1508)

5.1.45 ef415a7 2017-03-22 [src] Fix exit code of (#1510)

5.1.46 9a6feea 2017-03-23 [egs] fixes to babel pipeline; thanks to Fred Richardson (#1509)

5.1.47 0d0316b 2017-03-23 [src,scripts]: Several unrelated cosmetic changes

5.1.48 24bc222 2017-03-27 [misc] remove eXecute permissions where not needed (#1515)

5.1.49 244251f 2017-03-28 [egs] Fix to egs/wsj/s5/ (unset variable) (#1517)

5.1.50 8b458ae 2017-03-29 [src] Adding noexcept to hashing function objects (#1519)

5.1.51 13d300f 2017-03-29 [src,doc] Fix several unrelated minor problems. Thanks: gaoxinglong

5.1.52 e9d7993 2017-03-31 [src] (minor) Added missing SetZero() to NaturalGradientAffineComponent::Scale() if scale==0.0 (#1522)

5.1.53 e8ca375 2017-04-01 [scripts,egs] Adding options for using PCA instead of LDA+MLLT for ivectors used in ASR. Results are reported in the default TDNN recipe in AMI. Updating steps/online/nnet2/{,} so that they now backup the contents of their destination directory if it already exists. (#1514)

5.1.54 60f9327 2017-04-02 [build,src,doc] Modify to deal better with whitespace (avoid space in version); minor fixes (#1526)

5.1.55 3c94401 2017-04-04 [build]: remove openfst check (#1531)

5.1.56 d8b34d4 2017-04-04 [src] cudamatrix: speed up AddColSumMat with transfrom reduce kernel template (#1530)

5.1.57 e5b1419 2017-04-04 [src] Cosmetic change: remove 'train.tra' from usage messages (#1529)

5.1.58 0157686 2017-04-05 [src] nnet1: improving the GPU diagnostics, (#1532)

5.1.59 d9efe6e 2017-04-11 [src] Fix copy-feats for using the --write-num-frames and --compress true flags at the same time (#1541)

5.1.60 5b70933 2017-04-12 [scripts] fix to (thanks: David Pye)

5.1.61 b2d5d36 2017-04-12 [scripts] nnet3 : fix issue where LDA estimation failed for LSTMs with label delay (#1540)

5.1.62 31a528b 2017-04-13 [src] Change ConvertStringToReal to be locale-independent (i.e. always-US). Fixes android issue. (#1513)

5.1.63 7fcc311 2017-04-13 [egs,scripts,src] SID and LID tools and scripts: cosmetic improvements, better error-handling, and various minor fixes; results unchanged. (#1543)

5.1.64 179d9b6 2017-04-14 [src] add template instantiations for ConvertStringToReal, address issue #1544

5.1.65 8891750 2017-04-14 [egs] AMI TDNN Results Update (#1545)

5.1.66 9e06caa 2017-04-15 [srcipts] steps/nnet3/report/ plot 5,50,95th percentile of value and derivative instead of mean+-stddev (#1472)

5.1.67 81346fc 2017-04-16 Update travis.yml so PRs to kaldi_52 are built

5.1.68 46f3285 2017-04-19 [egs] Added check for kaldi_lm being installed in fisher_swbd recipe. (#1558)

5.1.69 7223462 2017-04-20 [doc] Small edit to hmm.dox, clarifying something

5.1.70 d82bbd7 2017-04-20 [src] nnet3 online silence weighting - adding frame subsampling factor (#1559)

5.1.71 d8be99a 2017-04-20 [src,scripts,egs] Add dropout for nnet3 LSTMs, with recipes. (#1537)

5.1.72 015c96f 2017-04-20 [src] Add documentation for dropout function.

5.1.73 73489ae 2017-04-21 [src] Fix to testing code, change threshold to resolve failure (#1565)

5.1.74 87d95c5 2017-04-21 [egs] Add mini-librispeech example scripts [intended as a sanity-checker/tutorial setup] (#1566)

5.1.75 c68a576 2017-04-21 [egs] adding results and cleanup in mini-librispeech

5.1.76 1f51ef5 2017-04-23 [egs] Add soft link for mini-librispeech setup

5.1.77 06047be 2017-04-25 [src] Minor bug-fixes in compute-wer-bootci and WSJ Thanks: @osadjadi

5.1.78 8b9e874 2017-04-25 [egs] bug-fix in egs/ami/s5/ (#1577)

5.1.79 a39e580 2017-04-28 [build] Remove warning about dash (I believe we have no problem with it now)

5.1.80 fc20b76 2017-05-01 [egs] fix small bug in wsj [thanks: Hang Lv]

5.1.81 e527572 2017-05-01 [sripts] cosmetic fix, correct example usage. (#1600)

5.1.82 1cd6d2a 2017-05-02 [egs] mini-librispeech: download data from openslr if it's not present (#1598)

5.1.83 3906ae9 2017-05-04 [src] bug-fix for memory bug in nnet computation cache code

5.1.84 ba13848 2017-05-04 [src] Fix bug that can cause crash for for kernel-sizes=n*65536.

5.1.85 ad6491a 2017-05-05 [src] Add --write-per-frame-acoustic-loglikes options to alignment programs (#1601)

5.1.86 7c04d62 2017-05-06 [scripts] nnet1: minor update i-vector and mpe scripts (#1607)

5.1.87 578c176 2017-05-07 [src] cosmetic change to const-arpa-lm-building code; remove too-general template. (#1610)

5.1.88 6eebc04 2017-05-09 [src,scripts,egs] Segmenting long erroneous recordings (#1167)

5.1.89 ab5c96c 2017-05-11 [egs] thchs30 cmd and stage bug fix (#1619)

5.1.90 b1e8601 2017-05-13 [src] Change to GPU synchronization, for speed (disables GPU stats by default) (#1617)

5.1.91 8cbb74c 2017-05-13 [src] Fix template instantiation bug causing failure if DOUBLEPRECISION=1

5.1.92 6666682 2017-05-15 [egs,scripts] Updates to BUT-specific settings (affects only Brno team); changes RE verbose level in nnet1 scripts.

5.1.93 a7e6183 2017-05-15 [src] fix a small bug: logging cuda elapsed time (#1623)

5.1.94 7af2128 2017-05-17 [src,scripts,egs] Add capability for multilingual training with nnet3; babel_multilang example.

5.1.95 9638950 2017-05-18 [doc] small fix RE queue configuration.

5.1.96 ae45ee2 2017-05-19 [scripts] Fix to long-utterance segmentation script (#1631)

5.1.97 45ee31f 2017-05-20 [src] Make parsing error-msg more informative (thanks: Stefan-Adrian Toma)

5.1.98 b13f56b 2017-05-21 [src] fix regarding first/last chunk's right-context in chain models (#1632)

5.1.99 86766bd 2017-05-21 [scripts, egs]: fix to egs/lre07/v2 (test was trained on); other updates to LRE scripts.

5.1.100 dc32e0b 2017-05-22 [scripts] Fix steps/cleanup/ to actually add the top-n-words into the lms (#1637)

5.1.101 d6cf1bd 2017-05-23 [scripts,egs] Fixes to long-recording segmentation (#1639)

5.1.102 b5eac09 2017-05-24 [scripts] fix bugs in [thanks: Filip Jurcicek]

5.1.103 0f5e16e 2017-05-25 [build] Check python version is 2.7*, not just 2.*.

5.1.104 31683a3 2017-05-25 [build] Change to not look for yum if apt-get present.

5.1.105 6739d60 2017-05-25 [scripts] Make more informative error in when prints something

5.1.106 6cc8e3a 2017-05-26 [src] Use STL thread support library instead of pthread. (#1350)

5.1.107 c210b44 2017-05-26 [scripts] Fix bugs in automatic report generation for nnet3 training

5.1.108 76c8c77 2017-05-26 [egs] Ported Fisher spanish recipe to use new LDC dir structure. Other small fixes (#1647)

5.1.109 6b906a3 2017-05-26 [egs] updated the LDC web address for wsj0-train-spkrinfo.txt (#1648)

5.1.110 0b0dbb0 2017-05-26 [scripts,egs] minor script fix; fixes in various recipes (#1649)

5.1.111 b6c34db 2017-05-26 [scripts] Lexicon expansion script -- fix for LM-probs, make it work for non-ASCII langs or langs w. large grapheme set (#1650)

5.1.112 e9646f0 2017-05-26 [src] keyword search: fix invalid assumption about the end states (#1651)

5.1.113 93c069c 2017-05-27 [src] Fix bug in newly refactored threading code

5.1.114 7267281 2017-05-27 [scripts] Getting egs, limit max open filehandles to 512 (thanks: gaoxinglong9999)

Version 5.2

Version 5.2 is the current master branch of Kaldi. Some of the changes introduced in version 5.2 are:

  • Upgrades to nnet3 to support batch-norm and convolutional components; recipes for certain image tasks (like CIFAR).
  • nnet3 training script simplifications and refactoring.
  • Some of the recipes are upgraded to include dropout and the –proportional-shrink option (which approximates l2 regularization); this improves results.
5.2.0 7267281 2017-05-27 [scripts] Getting egs, limit max open filehandles to 512 (thanks: gaoxinglong9999)

5.2.1 393ef73 2017-05-29 [build] Upgrade .version (this is official start of kaldi 5.2)

5.2.2 cb7d176 2017-05-29 Merge pull request #1547 from kaldi-asr/kaldi_52

5.2.3 a57eab6 2017-05-29 [scripts] Fix bug in PR #1646 (#1658)

5.2.4 972efd7 2017-05-31 [scripts] nnet3 script cleanups; add --proportional-shrink in more places. (#1659)

5.2.5 3e57783 2017-05-31 [src] minor bugfix in convolutional component (doesn't affect experiments)