Hi Weren't the CCP4 base-level routines re-written from FORTRAN to C sometime in the late 1990's? Very occasionally I used to find bugs that had been introduced in this process (or possibly not corrected...) so it's possible that Eleanor's file might be readable with a really old code base.
BTW, I have recently had cause to search out a VMS system and have found someone with a whole farm of running VAXes (or VAXen if you're geeky). Harry On 14 Nov 2018, at 09:46, Ian Tickle wrote: > The CCP4 routines for MTZ and map files are written in C and thus do not use > a Fortran unformatted OPEN statement, they use C-style block read & write. > > Cheers > > -- Ian > > > On Wed, 14 Nov 2018 at 08:59, Kay Diederichs <kay.diederi...@uni-konstanz.de> > wrote: > It is not necessary to do error-prone conversions manually: the ifort > Compiler understands the convert='VAXD' Option in its OPEN statement - see > https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier > > Thus one could just write a tiny read-write loop. > > HTH > Kay > > On Wed, 14 Nov 2018 00:51:02 +0000, Zhijie Li <zhijie...@utoronto.ca> wrote: > > >It's also said here, at the end of file : > > > >https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf > > > >"add 1 to the left, with the binary point" > > > >0.10000..... > > > > > > > >________________________________ > >From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Zhijie Li > ><zhijie...@utoronto.ca> > >Sent: Tuesday, November 13, 2018 7:43 PM > >To: CCP4BB@JISCMAIL.AC.UK > >Subject: Re: [ccp4bb] VERY old mtz file.. > > > > > >Hi all, > > > > > >I think I know why it is a division of 4 instead of 2 is involved in > >conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits > >(bias of 128 instead of 127, visible), another 2 is hidden in the scientific > >notation. > > > > > >I found this explanation+example on VAX F-float: > > > >http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html > > > > > >So for IEEE754 float32, if we want to represent a same 12.75 (1100.11) in > >the above example, we would first conceptually write it in scientific > >notation as 1.10011 x 1000 in binary. Then the mantissa part is the part > >after the dot filled with zero to 23 bits: '10011000000000000000000', the > >exponent part is 3+127=130 (dec)=10000010(bin). Then the binary IEEE754 > >float32 number is 0[10000010][10011000000000000000000]. (You can check it > >here: https://www.h-schmidt.net/FloatConverter/IEEE754.html) > > > > > >Now compare this with the VAX 12.75 in the linked example, we can find that > >besides the bias becoming 128, the conceptual binary scientific notation is > >actually 0.110011 x 10000, instead of 1.10011 x 1000. So the exponent > >needed is 4 instead of 3. Then the exponent bits are 4+128=132=10000100 and > >the VAX float32 becomes > > > >0[10000100][10011000000000000000000] ---if we write in a IEEE-style order. > >Note that the mantissa appears to be same as the ieee mantissa, and the > >exponent to be applied is 132-128=4. If this number is interpreted as > >IEEE754, then it will be 1.10011 x 2exp(132-127)=1.10011 x 100000, four > >times of what it should be. > > > > > >So, for normalised values, rearranging the VAX F-float bytes, reading as > >IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment in > >the ccp4 lib is neat.) > > > > > >In this link describing VAX floats, it is unfortunate that it only states > >that the bias for F-float is 128, but not that the mantissa starts from 0.01 > >instead of 0.1. Therefore the confusion. > > > >https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm > > > > > > > >Thanks to all responded! > > > > > >Zhijie > > > > > >________________________________ > >From: Ian Tickle <ianj...@gmail.com> > >Sent: Tuesday, November 13, 2018 4:54 PM > >To: Zhijie Li > >Cc: CCP4BB@JISCMAIL.AC.UK > >Subject: Re: [ccp4bb] VERY old mtz file.. > > > > > >Hi Zhijie > > > >It's definitely a factor 4. The code is in subroutine QTIEEE in the Fortran > >source I mentioned previously at this line: > > > >See line: > > > > A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2 > > > >If you prefer it in C code it's in function vaxF2ieeeF in: > > > >ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c > > > >See line: > > > >out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */ > > > >i.e. subtract 2 from exponent -> division by 4. > > > >Cheers > > > >-- Ian > > > > > >On Tue, 13 Nov 2018 at 19:52, Zhijie Li > ><zhijie...@utoronto.ca<mailto:zhijie...@utoronto.ca>> wrote: > >If somebody is going to send these files by email, please send one to me > >too. Thanks in advance. I actually prefer to get a MTZ file because the > >miller indices would serve as good clues for understanding the encodings. > >Even the first 1024 bytes of an MTZ would do (data array starts at byte 80 > >in MTZ). > > > >In my life I had only seen ieee754. According to what I can find, VAX has > >an exponent bias of 128 (ieee754 uses 127). Then it seems to me that when > >converting from vax to ieee a division of 2 is involved. However all > >procedures I have seen use a division of 4, which is quite puzzling to me. A > >real data file containing meaningful numbers (eg., HKL indices) would be > >very helpful. Thanks in advance. > > > >Zhijie > > > >> On Nov 13, 2018, at 2:21 PM, Johan Hattne > >> <hat...@ucla.edu<mailto:hat...@ucla.edu>> wrote: > >> > >> Related by not exactly on topic: would anybody on the list be able to > >> share old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX > >> reals/strings? I�d be interested to see what those files actually > >> look(ed) like. > >> > >> // Best wishes; Johan > >> > >>> On Nov 9, 2018, at 18:38, Zhijie Li > >>> <zhijie...@utoronto.ca<mailto:zhijie...@utoronto.ca>> wrote: > >>> > >>> Hi all, > >>> > >>> On linux there are a few good GUI HEX editors. Here I�d like to recommend > >>> BLESS, which conveniently displays all possible numerical interpretations > >>> of the four bytes under cursor. It also allows the user to switch between > >>> big endian or little endian through a checkbox. Unfortunately all floats > >>> are assumed to be IEEE754, therefore VAX floats won�t be interpreted > >>> correctly. ( The simplest way to convert vax to ieee float would be to > >>> write a little program to do some bit operations. I�d be happy to take > >>> that as my weekend project) > >>> > >>> > >>> BTW, along the line of space efficiency, I can�t help noticing that the > >>> miller indices are saved as float32 in mtz, as all other numbers in mtz. > >>> This certainly have made mtz format a beautiful homogeneous data format > >>> ;). In this particular case, if we have doubts about the reliability of > >>> the machine stamp, trying to restore the miller indices would be a good > >>> way to test hypotheses. > >>> > >>> Zhijie > >>> > >>>> On Nov 9, 2018, at 9:04 PM, James Holton > >>>> <0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk<mailto:0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk>> > >>>> wrote: > >>>> > >>>> As a beamline scientist I must say I am glad that diffraction image data > >>>> is not usually stored as ASCII text. In fact, I am slowly warming to > >>>> the idea of storing it as not just binary, but compressed formats. > >>>> Problem, I'm sure will be that it won't be long before we forget how to > >>>> decompress them, as most of the algorithms we are using aren't all that > >>>> widespread. Probably around the same time future generations will curse > >>>> us for using ASCII instead of unicode, which is a 16-bit standard. I'm > >>>> sure we will be reviled for limiting ourselves so, just to save a factor > >>>> of two in disk space. > >>>> In situations like this I always use the unix "od" command. It makes > >>>> everything "human readable" by converting the bytes into strings you can > >>>> read. Then it is just a matter of figuring out what the bytes are. > >>>> Unfortunately, "od" only decodes floats on the native platform, so if > >>>> the mtz is from another platform (Windows vs Linux, for example), then > >>>> you might need to do some swapping. Thus far, I have encountered files > >>>> that require one of a few swapping strategies in order to make them work: > >>>> > >>>> 1 2 3 4 - no swapping > >>>> > >>>> 4 3 2 1 - reverse all bytes > >>>> > >>>> 3 4 1 2 - swap words and swap bytes within the words > >>>> 2 1 4 3 - reverse of previous > >>>> > >>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2 before > >>>> swapping > >>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2 before > >>>> swapping > >>>> I'm sure there are other combinations, but the oldest MTZ I have is only > >>>> from 1996. > >>>> > >>>> -James Holton > >>>> MAD Scientist > >>>> > >>>> > >>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote: > >>>>> Anyone any idea what to do about this?? Created in 1992!! > >>>>> Seems unreadable.. > >>>>> > >>>>> No CTYP lines input for file: 1 > >>>>> Indices output even if all data items flagged "missing" > >>>>> Warning, NOT all LABOUT data lines given > >>>>> Warning: Machine stamp corrupted? Assuming native format. > >>>>>>>>>>> CCP4 library signal library_file:End of File (Error) > >>>>> > >>>>> > >>>>> To unsubscribe from the CCP4BB list, click the following link: > >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > >>>>> > >>>> > >>>> > >>>> To unsubscribe from the CCP4BB list, click the following link: > >>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > >>>> > >>> > >>> To unsubscribe from the CCP4BB list, click the following link: > >>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > >>> > >> > >> Research Specialist @ Gonen Lab > >> ____________________________________________________ > >> UCLA * 615 Charles E. Young Drive South > >> BSRB #347 * Los Angeles, CA 90095 > >> > >> ######################################################################## > >> > >> To unsubscribe from the CCP4BB list, click the following link: > >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > >######################################################################## > > > >To unsubscribe from the CCP4BB list, click the following link: > >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > >________________________________ > > > >To unsubscribe from the CCP4BB list, click the following link: > >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > >######################################################################## > > > >To unsubscribe from the CCP4BB list, click the following link: > >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
