Thanks,
For checking that the environments were the same (point 1) from before), I
tried both methods. Both gave me that the two environments (i.e. all the
packages and dependencies) are exactly the same: the diff of the two guix graph
was empty and the guix pack had exactly the same hash, so I did not even need
diffoscope: everything is the same.
Then, I don't know where is the problem and I have no idea how to debug it. It
should be something that is not handled by guix shell, but as I don't
understand enough what could possibly not be fixed by guix shell --container, I
do not know where to look.
Best,
Timothée
----- Mail original -----
> De: "Rutherther" <ruthert...@ditigal.xyz>
> À: "Timothee Mathieu" <timothee.math...@inria.fr>
> Envoyé: Samedi 3 Mai 2025 13:21:39
> Objet: Re: Reproducibility of guix shell container across different host OS
> Timothee Mathieu <timothee.math...@inria.fr> writes:
>
>> Reading your answer, I guess my "Guix problems" are these two:
>>
>> 1) How do you check and diff two guix shell environments? I guess this is
>> what
>> you are talking about when speaking about store closure, I was not aware of
>> that concept and did not find how to in the documentation, how do you check
>> the
>> diff of the store closure of both shells?
>
> There are couple of ways, one way could be with guix pack you did
> earlier, and check the contents in there. Guix will put the shell
> closure inside of the pack, so you can check its contents.
> Tools like diffoscope[1] can then help with the diff.
>
> A way without guix pack indirection would be to utilize the references
> stored in the guix database. There is guix gc --references that will
> show you references of a specific path, but it's not recursive by
> itself. There is also guix graph that is recursive, and can operate on
> references of store paths.
>
> There are a couple of ways to get profile path that is produced by a
> shell, probably the easiest is `guix shell ... -- sh -c 'echo
> $GUIX_ENVIRONMENT'`, note that sh is required here, so the shell has
> to contain it or it cannot be pure/container (and that doesn't matter as
> the path produced by guix doesn't depend on whether container or pure is
> used)
>
> Then you pass it to guix graph --type=references, this produces graphviz
> graph. Not sure what the easiest way to export nodes is, but one is to
> use dot -T plain and then grep and awk it.
>
> One liner could be:
> `guix graph --type=references "$(guix shell ... -- sh -c 'echo
> $GUIX_ENVIRONMENT')" | dot -Tplain | grep node | awk '{print $2}'`
>
> First important is to check the paths here. They would differ for
> example because of the locales that I pointed out earlier. But it could
> be other things.
>
> Then it's important to check all the actual contents and their
> difference, if the paths won't match, it's expected the contents don't
> have to match either. Howerver, if the paths do match and there is still
> difference in contents, that would definitely be Guix reproducibility
> issue.
>
> [1] https://diffoscope.org/
>
>> 2) How can I make sure that two guix shell --container are both the same
>> i.e. do
>> I only need to compare the guix describe and check that the packages are the
>> same?
>
> No, that is not sufficient to check reproducibility issues. Checking the
> full contents is important as well. If there was a reproducibility
> issue, it could mean that `guix build XYZ` on one machine will produce a
> different output on machine A and on machine B, but their paths will be
> the same as the inputs are the same.
>
> Reproducibility is something Guix is trying to achieve and it usually
> does, but there can be some issues, it's not so easy to reach 100 %
> reproducibility. There are even tools to check reproducibility issues,
> guix challenge and guix build --check, but those are useful when you're
> testing one particular package, since you have a shell with a few
> packages that transitively probably do have many dependencies, they
> won't be so useful here.
>
>>
>> My guess was that the problem was not in the randomness because I fixed the
>> seed
>> and I managed to reproduce the results on other computers but basically with
>> the computers giving either the first or the second result and if the
>> randomness was involved then the results would be more disparate but of
>> course
>> I may be wrong about this. And remark that recently I managed to reproduce
>> the
>> Arch result on an Ubuntu computer with guix pack (but not on the initial
>> Ubuntu
>> computer which still gives the same old different result) so it does not seem
>> to be an Arch versus Ubuntu result.
>>
>
> The reason I mentioned it was that the libraries could hypothetically
> choose different algorithms based on what was available, at least with
> default config or something.
>
>> Thanks all of you for all your help,
>> Timothée
>>
>> ----- Mail original -----
>>> De: "Rutherther" <ruthert...@ditigal.xyz>
>>> À: "Timothee Mathieu" <timothee.math...@inria.fr>
>>> Cc: "help-guix" <help-guix@gnu.org>
>>> Envoyé: Samedi 3 Mai 2025 12:14:45
>>> Objet: Re: Reproducibility of guix shell container across different host OS
>>
>>> Timothee Mathieu <timothee.math...@inria.fr> writes:
>>>
>>>> Hello,
>>>>
>>>> Sorry I did not believe that the precise values were relevant but here
>>>> they are.
>>>> The average cumulative reward (score) of the agent for exactly the same
>>>> script
>>>> and using Guix for the environment is 1658.3733235021457 on Arch Laptop and
>>>> 1820.325441905902 on the Ubuntu one. But I think due to the feedback loop
>>>> of
>>>> such simulator (if there is a small difference in the action at time t, it
>>>> can
>>>> imply a large difference at the end of the process) this could be due to a
>>>> small difference in the computations.
>>>
>>> Hi Timothee,
>>>
>>> this is not really what I had in mind. Reproducibility of Guix is simply
>>> that the produced outputs written to the gnu store are the same given
>>> same inputs. That an arbitrary script ran produces different value is
>>> not something that can be mapped to that easily. Hence I would be
>>> interested to see a diff between the store closures of both shells,
>>> starting with difference in hashes of output paths, and ending with diff
>>> of their contents. One common issue can be with different locale, some
>>> of the outputs will depend on the locale. Especially if guix was once
>>> installed with the guix install script and once with the distro package
>>> manager, there could be a discrepancy. I don't know how library you're
>>> using is determining what algorithm to choose for randomness, so I
>>> cannot say if differences in the outputs that only change because of
>>> locale can matter.
>>>
>>> If there is no difference in store closure of the shell (meaning all
>>> paths referenced by the shell), you're likely not debugging an issue of
>>> Guix, but of something else. Ie. the library you're using could be using
>>> different random implementations based on something that is available on
>>> one distro, but not on the other.
>>>
>>> Regards
> >> Rutherther
