Bug - Strange issue with mod_perl 2.0.10 / Apache 2.4 corrupting nfreeze data

2016-04-03 Thread Alex Masidlover
-8<-- Start Bug Report 8<--
1. Problem Description:


I'm running Perl 5.20.2 on a Gentoo virtual machine. We've had some
code in place that uses a mod_perl handler to send complex hashref
based structures between two servers using nfreeze / thaw; the code
needed to be high performance therefore we used nfreeze/thaw rather
than XML or similar. This has been working correctly on a variety of
architectures since July 2014.

The mod_perl server does:

print nfreeze( $message_hash )

The client connects to the server use LWP::UserAgent and does:

my $message = thaw( $response->decoded_content() );

The message from client to server is similarly encoded with nfreeze /
thaw in a cgi parameter in multi-part-form-data.

This has all worked perfectly up until I upgraded to Apache 2.4 /
mod_perl 2.0.10 - I've even downgraded just those two packages to
mod_perl 2.0.8 / Apache 2.2.31 and it starts working again. The message
from the client gets to the server correctly and the server logs and
processes the message correctly, but it then fails with an 'Out of
memory!' error from the client when the response is received (due to
this bug in Storable.pm) - however, regardless of that bug, something
is corrupting the data in transit...

I've picked apart what's going on and found there are an additional two
bytes being inserted somehow. I've recorded the data in a /tmp/ file
before the print statement in the mod_perl handler and I've used perl
-d to inspect what's received:

  DB<3> say "0x$_" for unpack "(H2)*", $response->content()
0x05
0x0a
0x03
0x00
0x00
0x00
0x01
0x08
0xc2
0x81
0x00
0x00
0x00
0x02
0x6f
0x6b

However, the server sent:

0x05
0x0a
0x03
0x00
0x00
0x00
0x01
0x08
0x00
0x00
0x00
0x02
0x6f
0x6b

So somehow 2 bytes have appeared in the middle of the message 0x08 and
0xc2.

I've also tried adding Content-Type => 'x-application/binary' and
Content-Length headers to see if that fixes the issue.

Thanks!

2. Used Components and their Configuration:

*** mod_perl version 2.10

*** using /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-
multi/Apache2/BuildConfig.pm

*** Makefile.PL options:
  MP_APR_CONFIG  => /usr/bin/apr-1-config
  MP_APR_LIB => aprext
  MP_APXS=> /usr/sbin/apxs2
  MP_COMPAT_1X   => 1
  MP_DEBUG   => 0
  MP_GENERATE_XS => 1
  MP_LIBNAME => mod_perl
  MP_TRACE   => 0
  MP_USE_DSO => 1


*** The httpd binary was not found


*** (apr|apu)-config linking info

 -laprutil-1 -lldap -llber -ldb-4.8 -lgdbm  -lexpat
-L/var/tmp/portage/dev-libs/apr-util-1.5.4/temp
 -lapr-1 -luuid -lrt -lcrypt  -lpthread -ldl 



*** /usr/bin/perl -V
Summary of my perl5 (revision 5 version 20 subversion 2) configuration:
   
  Platform:
osname=linux, osvers=2.6.32-openvz-feoktistov.1, archname=x86_64-
linux-thread-multi
uname='linux alex.zednax.com 2.6.32-openvz-feoktistov.1 #2 smp wed
apr 6 14:38:37 bst 2011 x86_64 intel(r) xeon(r) cpu l5630 @ 2.13ghz
genuineintel gnulinux '
config_args='-des -Duseshrplib -Darchname=x86_64-linux-thread
-Dcc=x86_64-pc-linux-gnu-gcc -Doptimize=-O -pipe -Dldflags=-Wl,-O1
-Wl,--as-needed -Dprefix=/usr -Dinstallprefix=/usr
-Dsiteprefix=/usr/local -Dvendorprefix=/usr -Dscriptdir=/usr/bin
-Dprivlib=/usr/lib64/perl5/5.20.2
-Darchlib=/usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi
-Dsitelib=/usr/local/lib64/perl5/5.20.2
-Dsitearch=/usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi
-Dvendorlib=/usr/lib64/perl5/vendor_perl/5.20.2
-Dvendorarch=/usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-
multi -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3
-Dvendorman1dir=/usr/share/man/man1 -Dvendorman3dir=/usr/share/man/man3
-Dman1ext=1 -Dman3ext=3pm -Dlibperl=libperl.so.5.20.2
-Dlocincpth=/usr/include  -Dglibpth=/lib64 /usr/lib64  -Duselargefiles
-Dd_semctl_semun -Dcf_by=Gentoo -Dmyhostname=localhost -Dperladmin=root
@localhost -Dinstallusrbinperl=n -Ud_csh -Uusenm -Di_ndbm -Di_gdbm
-Di_db -Dusethreads -DDEBUGGING=none -Dinc_version_list=5.20.0/x86_64-
linux-thread-multi 5.20.0 5.20.1/x86_64-linux-thread-multi 5.20.1  -
Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Dnoextensions=ODBM_File'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
  Compiler:
cc='x86_64-pc-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
-fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O -pipe',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing
-pipe'
ccversion='', gccversion='4.7.3', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
a

Re: Bug - Strange issue with mod_perl 2.0.10 / Apache 2.4 corrupting nfreeze data

2016-04-03 Thread Vincent Veyron
On Sun, 03 Apr 2016 14:11:23 +0100
Alex Masidlover  wrote:

> 
> This has all worked perfectly up until I upgraded to Apache 2.4 /
> mod_perl 2.0.10 - 

After upgrading to Apache 2.4 and mod_perl 2.0.9, I had to make those two 
changes to my application :

In a PerlOutputFilterHandler, change '$content .= $buffer' to '$content .= 
decode_utf8($buffer)'

And in response handlers, change '$args{$_} = $req->param($_)' to '$args{$_} = 
decode_utf8($req->param($_))'

Not sure it applies to your case, but something changed in Apache 2.4 
concerning UTF-8 data.

If I understood correctly, anything that goes through APR::Table is considered 
UTF-8, however the SvUTF8 flag is not set, so you get double encoding when 
processing your data.

-- 
Bien à vous, Vincent Veyron

https://marica.fr/
Gestion des contentieux, des dossiers de sinistres assurance et des contrats 
pour le service juridique