wrt to outdating files. What I would actually expect from looking at the
code is for it to fail to produce a valid $handle and fallback to
non-precomp loading.
my $handle = (
self.may-precomp and (
my $loaded = self.load($id, :since($source.modified),
:@precomp-stores) # already precompiled?
or self.precompile($source, $id,
:source-name($dependency.source-name), :force($loaded ~~ Failure))
and self.load($id, :@precomp-stores) # if not do it now
)
);
1. File changes at timestamp X
2. p6 is run at timestamp X
3. p6 detects file has changed in the first self.load() (so it returns a
fail "Outdated precompiled $unit")
4. so it does a :force recompile
5. then tries to self.load() a second time -- but because we are still at
time X the second load still returns failure
6. $precompiled gets set to False (which means a fallback non-precomp load
should be used)
After a few "note"s around the place the above seems to be what is
happening but I could be wrong.
I'm not sure how we end up with the current failure mode (symbols missing
from GLOBAL). The only thing that stands out to me is that it's
nqp::exit(0) ie exit success, when precomp fails. I don't have a picture in
my mind of the whole process yet so exit(0) might make sense. This
precompilation is complicated stuff!
On Mon, May 23, 2016 at 10:13 AM Lloyd Fournier <lloyd.fo...@gmail.com>
wrote:
> I've done some more investigation. There are two seperate issues:
>
> 1. The main issue wrt this ticket. It hits you will you are editing code.
> It can be reproduced (on any platform) with:
>
> P6="perl6"
> lib="templib"
> test -e $lib && rm -r $lib;
> mkdir $lib
> sleep="sleep 2";
> echo 'class Top1 { }; need Needed;' > $lib/Top1.pm6
> echo 'class Needed { };' > $lib/Needed.pm6;
> echo 'class Top2 { }; need Needed;' > $lib/Top2.pm6;
>
> while true; do
> printf '#' >> $lib/Needed.pm6 # touch isn't enough here
> $sleep;
> $P6 -I$lib -e 'need Top1; need Top2; say GLOBAL::.keys;'
> done
>
> ====
> which will fail on the second iteration of the loop:
>
> (Top2 Needed Top1)
> ===SORRY!===
> Missing or wrong version of dependency '/root/templib3/Needed.pm6
> (Needed)' (from '/root/templib3/Top2.pm6 (Top2)')
>
> This one is different because the content of the file has to actually
> change rather than just the mtime to trigger it.
>
> 2. A smaller issue which is what I was investigating in the prior
> response. If you load/recompile a compunit in the same second as the source
> is modified it will fail. (your processor and disk has to be fast enough to
> pull this off which I *think* is why nine couldn't reproduce it rather than
> this being a mac specific thing).
>
> This race condition essentially happens in this line in
> !load-dependencies:
>
> return False if not $srcIO.e or $modified <= $srcIO.modified;
>
> because it's <= it opens the possibility that a precomp is marked as out
> of date a second time even after it has just been re-precompiled in the
> same process. The consequence seems to be a bad serialisation. (of course
> I'm not suggesting it shouldn't be <= but I guess the fact that something
> has just been re-precompiled should override this check).
>
>
> On Sun, May 22, 2016 at 7:12 AM Lloyd Fournier <lloyd.fo...@gmail.com>
> wrote:
>
>> nine++ has attempted to fix this issue:
>>
>>
>> https://github.com/rakudo/rakudo/commit/c59e4dc44772cb09edeb8aa7f0ce0385f951cf5d
>>
>> Though it seems to have made the failure more random.
>>
>> Try the following:
>> while true; do
>> test -e lib && rm -r lib; # this is needed to trigger it for some
>> reason
>> mkdir lib;
>> echo 'class One { }; need Two;' > lib/One.pm6
>> echo 'class Two { }; need Three;' > lib/Two.pm6
>> echo '' > lib/Three.pm6
>> perl6 -Ilib -e 'need One;'
>> touch lib/Three.pm6
>> perl6 -Ilib -e 'need One; say GLOBAL::.keys;'
>> done
>>
>> gives me:
>>
>> (Two One)
>> ()
>> (Two One)
>> ()
>> (Two One)
>> ()
>> (Two One)
>> (Two One)
>> (Two One)
>> (Two One)
>> (Two One)
>> ()
>>
>> I wrote a test and push it to roast.
>>
>>
>> https://github.com/perl6/roast/commit/4e8b2e02e77e607593907380d30898ff5a926bd7
>>
>> It seems to have the same issue.
>>
>> On Sat, May 21, 2016 at 1:45 AM Lloyd Fournier <lloyd.fo...@gmail.com>
>> wrote:
>>
>>> It seems to have been introduced by one of nine's commits on the 13th.
>>> This or one of the commits just before it I think:
>>>
>>>
>>> https://github.com/rakudo/rakudo/commit/4fb3f94fcd39699f69e9d175315f9f1357e8faf3
>>>
>>>
>>> On Fri, May 20, 2016 at 11:26 PM Lloyd Fournier <lloyd.fo...@gmail.com>
>>> wrote:
>>>
>>>> err leme try that again
>>>>
>>>> Precomp is broken for dependency chains 3 or longer when one of the
>>>> dependencies down the chain is *changed*.
>>>>
>>>> To expand: Sometimes it gives you a problem where you get "Missing or
>>>> wrong version of dependency" and sometimes symbols that should have been
>>>> merged from GLOBALish just aren't there which is the case in the example
>>>> above.
>>>>
>>>> On Fri, May 20, 2016 at 11:18 PM Lloyd Fournier <lloyd.fo...@gmail.com>
>>>> wrote:
>>>>
>>>>> I've been getting something similar and I've managed to golf it.
>>>>>
>>>>> mkdir lib;
>>>>> echo 'class One { }; need Two;' > lib/One.pm6
>>>>> echo 'class Two { }; need Three;' > lib/Two.pm6
>>>>> echo '' > lib/Three.pm6
>>>>> perl6 -Ilib -e 'need One; say (GLOBAL::<One>:exists ?? "OK" !! "NOT
>>>>> OK")'
>>>>> echo "changing file" && touch lib/Two.pm6 # Three.pm6 will also break
>>>>> perl6 -Ilib -e 'need One; say (GLOBAL::<One>:exists ?? "OK" !! "NOT
>>>>> OK")'
>>>>>
>>>>> summary: Precomp is broken for dependency chains 3 or longer when
>>>>> one of the dependencies down the chain is broken.
>>>>>
>>>>> Do we have a way of writing tests for precomp?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, May 17, 2016 at 4:23 AM mt1957 <perl6-bugs-follo...@perl.org>
>>>>> wrote:
>>>>>
>>>>>> # New Ticket Created by mt1957
>>>>>> # Please include the string: [perl #128156]
>>>>>> # in the subject line of all future correspondence about this issue.
>>>>>> # <URL: https://rt.perl.org/Ticket/Display.html?id=128156 >
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I get the following error more often than before after some edits in
>>>>>> my
>>>>>> source files.
>>>>>>
>>>>>> Missing or wrong version of dependency
>>>>>>
>>>>>> '/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
>>>>>> (MongoDB::Wire)' (from
>>>>>>
>>>>>> '/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
>>>>>> (MongoDB::Cursor)'
>>>>>>
>>>>>> A remedy to this is to touch all source files in the library I work on
>>>>>> and than retry. I think that this should not happen, at least not that
>>>>>> often (I've seen it several times in about a week I think, twice this
>>>>>> evening, to give an idea).
>>>>>>
>>>>>> This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
>>>>>> 2016.04-134-g9879233
>>>>>> implementing Perl 6.c.
>>>>>>
>>>>>> Greetings,
>>>>>> Marcel
>>>>>>
>>>>>
