On 16 September 2014 21:06, Niko Tyni <nt...@debian.org> wrote:
> (Cc'ing Yves instead of the scary github address as he
> explicitly asked for that in the upstream ticket.)
>
>
Thanks. :-)
> On Wed, Jul 16, 2014 at 05:06:29PM +0200, gregor herrmann wrote:
> > On Tue, 15 Jul 2014 13:03:56 -0700, Yves Orton wrote:
>
> > > This is so frustrating. Is there anyone in the Debian community that
> knows
> > > these platforms that can help?
> >
> > Hm, probably but I'm not sure who knows about all of them.
> > Cc'ing the Debian bug report.
>
> Hi, I looked at the Sereal-Encoder problems briefly. The current state
> with Debian libsereal-encoder-perl 3.002-1 (unpatched 3.002) is that it
> only fails on 64-bit big-endian architectures (s390x and powerpc64).
>
> The failures are all in t/700_roundtrip/ with the snappy tests, and
> involve array refs and long strings. I was able to reduce at least one
> of them to this:
>
> #!/usr/bin/perl -w
> use blib;
> use Sereal::Decoder;
> use Sereal::Encoder;
> my $opt = { snappy => 1, use_protocol_v1 => 1};
>
> my $len = shift || 509;
>
> my $s="A"x$len;
> my $e = Sereal::Encoder->new($opt)->encode([$s, $s]);
> my $d = Sereal::Decoder->new($opt)->decode($e);
>
> # print $e;
> print "" . ($d->[0] eq $s ? "" : "not ") . "ok 1 first element\n";
> print "" . ($d->[1] eq $s ? "" : "not ") . "ok 2 second element\n";
>
> which gives "not ok 2" on s390x when the length is 509 or greater.
>
> The problem is with the encoder: amd64 and s390x encoded contents differ,
> and feeding the amd64 encoded output to an s390x decoder works fine.
> The second element of the decoded array is just the string "3" in the
> incorrect case.
>
>
Yes, indeed. Snappy is getting something wrong.
IIRC, prior to this patch:
commit 910842fc8aa3b9666926e2a40ad8f7c6d5e1c6d5
Author: Yves Orton <yves.or...@booking.com>
Date: Tue Jul 15 13:32:14 2014 +0200
rework alignedness checks in Sereal code so that we do unaligned only
on x86
Snappy did not work at all.
https://github.com/Sereal/Sereal/commit/910842fc8aa3b9666926e2a40ad8f7c6d5e1c6d5
> I see the length of the encoded output shortens dramatically at the 508 ->
> 509 step, looks like some compression kicks in.
>
> I'm attaching the binary output from both s390x (incorrect) and amd64
> (correct) for reference for length==509. Hope they make it through,
> the md5sums should be
>
> 8787f3dd9a3bfa23bd91eb08c40f3716 509.amd64
> 276cbe59ce957915cc92b52fb0b7b51c 509.s390x
>
> All this makes me think that the problem is in snappy/csnappy_compress.c,
> but I haven't really delved into it yet.
>
> Hope this helps. Yves, please let me know if I can help in any way.
>
Well I am wondering if you can look the patch I referenced above and see if
you can work out what you need to do to get s390x to do the right thing? As
you have identified yourself this is clearly a problem with the snappy
support. But we pass test on other big-endian 64 bit platform as far as I
understand (although I might be wrong).
I have CC'ed Jarkko as he did some of the work getting us to build properly
on platforms with strict alignment and big-endian encodings.
Yves
--
perl -Mre=debug -e "/just|another|perl|hacker/"
