Thanks George.
For anyone interested in debugging this problem, debian has an extensive
collection of build logs over many architectures
https://buildd.debian.org/status/logs.php?pkg=hdf5 (going back 12 years!)
As far as I know, the corruption is limited to the H5T_NATIVE_LDOUBLE type.
You can check your particular build with the following test
#include <hdf5.h>
int main() {
return !(H5Tget_order(H5T_NATIVE_LDOUBLE) == H5Tget_order(
H5T_NATIVE_DOUBLE));
}
It exits with code 1 if the long double has different byte ordering than
double (which is technically possible, but highly suspicious).
Otherwise the patch I sent earlier in this thread seems to do the trick,
although what exactly is going wrong is still beyond my understanding.
Third option: you can define an equivalent of H5T_NATIVE_LDOUBLE yourself.
The following creates a data type representing a long double as implemented
by gcc on x86 architectures (see
http://en.wikipedia.org/wiki/Long_double#Implementations for details)
hid_t ldouble_datatype = H5Tcopy(H5T_NATIVE_DOUBLE);
H5Tset_size(ldouble_datatype, sizeof(long double));
H5Tset_precision(ldouble_datatype, 80);
H5Tset_fields (ldouble_datatype, 79, 64, 15, 0, 64);
H5Tset_pad(ldouble_datatype, H5T_PAD_ZERO, H5T_PAD_ZERO);
H5Tset_inpad(ldouble_datatype, H5T_PAD_ZERO);
H5Tset_ebias(ldouble_datatype, 16383);
H5Tset_norm(ldouble_datatype, H5T_NORM_NONE);
Best wishes,
Andrea
On 4 September 2013 22:58, George N. White III <[email protected]> wrote:
> Another historical reference to the obscurity of this code is: <
> https://bugs.gentoo.org/show_bug.cgi?id=118777>.
>
> I've been building HDF5 libraries for use with NASA SeaDAS, and recently
> have started using HDF5 with R and GDAL. The SeaDAS builds are static, and
> I don't find the "unable to calculate alignment for long double" message in
> my SeaDAS build logs on linux and OS X. For R and GDAL, however, I need
> dynamic libraries and those build logs do have the "unable to calculate
> alignment for long double" message on both linux and OS X.
>
>
> On Tue, Sep 3, 2013 at 9:50 PM, Andrea Bedini <[email protected]>wrote:
>
>> Hi,
>>
>> I found something else (I know, I should stop :)). I am not entirely sure
>> but it seems that when H5detect fails it writes "unable to calculate
>> alignment for long double" on stderr so this message should be observable
>> on build logs (although buried by other warnings). The packages on debian
>> sid and testing for both i386 and x86-64 seem to be affected:
>>
>>
>> https://buildd.debian.org/status/fetch.php?pkg=hdf5&arch=amd64&ver=1.8.11-3%2Bb1&stamp=1377024563
>>
>> https://buildd.debian.org/status/fetch.php?pkg=hdf5&arch=i386&ver=1.8.11-3%2Bb1&stamp=1377025110
>>
>> But here's the exciting part: look what I found
>>
>>
>> http://www.unidata.ucar.edu/mailing_lists/archives/gembud/2010/msg00052.html
>>
>> It's a build log from 2010 for HDF5 v1.6.5 and gcc-4.4.3 that says "unable
>> to calculate alignment for long double".
>>
>> If my understanding is correct, nor 1.8.11 or gcc 4.8.0 would be the
>> problem and it would be that piece of code just doesn't work properly.
>>
>> Best wishes,
>> Andrea
>>
>>
>>
>> On 4 September 2013 08:00, Andrea Bedini <[email protected]> wrote:
>>
>>> Hi Ray,
>>>
>>> thanks for giving it a look. Antonio made me notice that something else
>>> might be at work since the macro DETECT_F already zeroes the structure
>>> right before anything else:
>>>
>>> memset(&INFO, 0, sizeof(INFO)); #L299
>>>
>>> so I don't understand how the perm fields need to be zeroed again around
>>> line #L308. This still considering the "Byte Order" loop as a black box.
>>>
>>> As a side question: isn't there a more portable way of doing this? I am
>>> pretty sure H5detect.c might invoke a bunch of undefined behaviours given
>>> the amount of warning the compiler generates and of bit trickery.
>>>
>>> Best wishes,
>>> Andrea
>>>
>>>
>>>
>>>
>>> On 4 September 2013 05:43, Raymond Lu <[email protected]> wrote:
>>>
>>>> Andrea,
>>>>
>>>> We've verified that your solution is correct. We're putting your fix
>>>> into the library. Thanks for helping us.
>>>>
>>>> Ray
>>>>
>>>> On Sep 3, 2013, at 3:32 AM, Andrea Bedini wrote:
>>>>
>>>> Hi there,
>>>>
>>>> I think I have found the problem. The issue is in H5detect.c.
>>>> Macros DETECT_F and DETECT_I do not initialize properly the perm field in
>>>> the detected_t struct. As a result the routine fix_order is passed some
>>>> uninitialized memory which makes it fail. I have a small patch against
>>>> H5detect.c which fixes the problem by simply initializing the perm field
>>>> with zeros. Valgrind's tool memcheck would have exposed the problem.
>>>>
>>>> Best wishes,
>>>> Andrea
>>>>
>>>>
>>>>
>>>> On 3 September 2013 15:30, Andrea Bedini <[email protected]>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am experiencing the following issue with hdf5 and gcc 4.8.0
>>>>>
>>>>> Consider this very simple test
>>>>>
>>>>> #include <hdf5.h>
>>>>>
>>>>> int main() {
>>>>> switch (H5Tget_order(H5T_NATIVE_LDOUBLE)) {
>>>>> case H5T_ORDER_LE:
>>>>> printf("H5Tget_order(H5T_NATIVE_LDOUBLE) = H5T_ORDER_LE\n");
>>>>> break;
>>>>> case H5T_ORDER_BE:
>>>>> printf("H5Tget_order(H5T_NATIVE_LDOUBLE) = H5T_ORDER_BE\n");
>>>>> break;
>>>>> case H5T_ORDER_VAX:
>>>>> printf("H5Tget_order(H5T_NATIVE_LDOUBLE) = H5T_ORDER_VAX\n");
>>>>> break;
>>>>> case H5T_ORDER_MIXED:
>>>>> printf("H5Tget_order(H5T_NATIVE_LDOUBLE) = H5T_ORDER_MIXED\n");
>>>>> break;
>>>>> case H5T_ORDER_NONE:
>>>>> printf("H5Tget_order(H5T_NATIVE_LDOUBLE) = H5T_ORDER_NONE\n");
>>>>> break;
>>>>> default:
>>>>> printf("here are dragons\n");
>>>>> }
>>>>> return 0;
>>>>> }
>>>>>
>>>>> on the same x86_64 GNU/Linux machine I get
>>>>>
>>>>> $ hdf5-1.8.11-gcc-4.7.0/my_test # compiled with gcc 4.7.0
>>>>> H5Tget_order(H5T_NATIVE_LDOUBLE) = H5T_ORDER_LE
>>>>>
>>>>> $ hdf5-1.8.11-gcc-4.8.0/my_test # compiled with gcc 4.8.0
>>>>> H5Tget_order(H5T_NATIVE_LDOUBLE) = H5T_ORDER_VAX
>>>>>
>>>>> So H5T_NATIVE_LDOUBLE is mis-detected. I tried to dig deeper and
>>>>> basically the fault must be in src/H5detect.c which is used to generate
>>>>> the
>>>>> definitions in src/H5Tinit.c
>>>>> I could not figure out what H5detect.c does wrong (it is not very
>>>>> readable, given its extensive use of macros) but the compiler does emit a
>>>>> lot of warnings (see https://gist.github.com/andreabedini/6419975).
>>>>>
>>>>> I think this must be related to the failure of dt_arith long double
>>>>> test observed recently.
>>>>>
>>>>> Any suggestion on how to fix this ?
>>>>>
>>>>> Best wishes,
>>>>> Andrea
>>>>>
>>>>>
>>>>> --
>>>>> Andrea Bedini <[email protected]>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Andrea Bedini <[email protected]>
>>>> <hdf5_uninitialized.patch>
>>>> _______________________________________________
>>>> Hdf-forum is for HDF software users discussion.
>>>> [email protected]
>>>>
>>>> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Hdf-forum is for HDF software users discussion.
>>>> [email protected]
>>>>
>>>> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Andrea Bedini <[email protected]>
>>>
>>
>>
>>
>> --
>> Andrea Bedini <[email protected]>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [email protected]
>>
>> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>
>>
>
>
> --
> George N. White III <[email protected]>
> Head of St. Margarets Bay, Nova Scotia
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
>
> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>
>
--
Andrea Bedini <[email protected]>
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
