All the code is available at the git repository https://gitlab.inria.fr/tmathieu/guix-rl/ In the tests directory are the scripts I use to test reproducibility with a readme containing the expected results and in the guix-rl directory are the package definitions.
I use python, and the most complicated libraries in the lot are the physics simulator mujoco (this is the simulator I was talking about, compiled mainly from C++) and pytorch (neural network library). For now, I was only testing evaluation with a fixed policy (i.e. I don't train the neural network, it is pre-trained and saved in a zip in the tests folder). I tried from another pwd it does not seem to change, and I now print the initial state of the simulator (which depends on the seed) and I checked that they are both the same on the two computers, so the reproducibility problem does not come from random initialization (although there could be randomness in the evaluation process itself, but I can't think of where there could be additional randomness). Best, Timothée ----- Mail original ----- > De: "Rutherther" <ruthert...@ditigal.xyz> > À: "Timothee Mathieu" <timothee.math...@inria.fr> > Cc: "help-guix" <help-guix@gnu.org> > Envoyé: Samedi 3 Mai 2025 21:53:37 > Objet: Re: Reproducibility of guix shell container across different host OS > Timothee Mathieu <timothee.math...@inria.fr> writes: > >>>> So could you clarify on the point you made earlier? You've ran guix pack >>>> on Arch, sent it to Ubuntu, unpacked, ran, and the result you got was >>>> the same as on Arch? Or was the result the same as earlier on Ubuntu? >> >> The result was the same as earlier on Ubuntu, the pack results are the same >> as >> the guix shell results, I don't get reproducibility across the two computers. >> >>>> What if you make the pack on Ubuntu, unpack and run it there, and on >>>> Arch as well? >> >> Nothing changes, still Ubuntu laptop and Arch laptop give different results, >> even though the pack are the same. > > Okay, good, that is more what I would expect then. > > I will give it more thought and get back to you, I currently don't > really have a specific thing in mind that would cause this. However, it > would help if you shared more details on the language and libraries > involved, how you set the random seed and where do random values come in > the process (the best would be to see the code that actually uses > the random) > > And as an initial test you could try generating a few random numbers > manually to see if they are the same on Arch and Ubuntu. > > But maybe this whole thing is actually something else than randomness. > What about stuff like the pwd or files under pwd, are they the same in > the container? As an absurd example, the program could be checking if > file `hello_world` exists and based on that change the results... of > course in reality it's going to be something more subtle. > > Regards > Rutherther
