Repository navigation
Docum 9CE9 ent backwards compatibility guarantees #898
Description
Does Grain have any sort of backwards compatibility guarantee that a given piece of code will always produce the same shuffle of a dataset on all versions of Grain (within a major version)? It would be great if you could document what guarantees you aim to provide in the README and/or on Read the Docs.
Apologies if these guarantees are already documented somewhere. Thanks for providing this great library :)
Activity
iindyk commentedon Jun 10, 2025
in theory we do not guarantee this, in practice, our index shuffle implementation doesn't change often, so most likely it will not change across versions. Another piece of the puzzle is random transformations (RandomMap): they use numpy RNGs and numpy itself does not guarantee backwards compatibility for RNGs. So, even if we make an effort to make sure that shuffle is backwards compatible, it will not be enough for e2e user pipeline. re: documenting -- agree, we should document this.