The CCP4 routines for MTZ and map files are written in C and thus do not use a Fortran unformatted OPEN statement, they use C-style block read & write.
Cheers -- Ian On Wed, 14 Nov 2018 at 08:59, Kay Diederichs <kay.diederi...@uni-konstanz.de> wrote: > It is not necessary to do error-prone conversions manually: the ifort > Compiler understands the convert='VAXD' Option in its OPEN statement - see > > https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier > > Thus one could just write a tiny read-write loop. > > HTH > Kay > > On Wed, 14 Nov 2018 00:51:02 +0000, Zhijie Li <zhijie...@utoronto.ca> > wrote: > > >It's also said here, at the end of file : > > > >https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf > > > >"add 1 to the left, with the binary point" > > > >0.10000..... > > > > > > > >________________________________ > >From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Zhijie Li > <zhijie...@utoronto.ca> > >Sent: Tuesday, November 13, 2018 7:43 PM > >To: CCP4BB@JISCMAIL.AC.UK > >Subject: Re: [ccp4bb] VERY old mtz file.. > > > > > >Hi all, > > > > > >I think I know why it is a division of 4 instead of 2 is involved in > conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits > (bias of 128 instead of 127, visible), another 2 is hidden in the > scientific notation. > > > > > >I found this explanation+example on VAX F-float: > > > >http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html > > > > > >So for IEEE754 float32, if we want to represent a same 12.75 (1100.11) in > the above example, we would first conceptually write it in scientific > notation as 1.10011 x 1000 in binary. Then the mantissa part is the part > after the dot filled with zero to 23 bits: '10011000000000000000000', the > exponent part is 3+127=130 (dec)=10000010(bin). Then the binary IEEE754 > float32 number is 0[10000010][10011000000000000000000]. (You can check it > here: https://www.h-schmidt.net/FloatConverter/IEEE754.html) > > > > > >Now compare this with the VAX 12.75 in the linked example, we can find > that besides the bias becoming 128, the conceptual binary scientific > notation is actually 0.110011 x 10000, instead of 1.10011 x 1000. So the > exponent needed is 4 instead of 3. Then the exponent bits are > 4+128=132=10000100 and the VAX float32 becomes > > > >0[10000100][10011000000000000000000] ---if we write in a IEEE-style > order. Note that the mantissa appears to be same as the ieee mantissa, and > the exponent to be applied is 132-128=4. If this number is interpreted as > IEEE754, then it will be 1.10011 x 2exp(132-127)=1.10011 x 100000, four > times of what it should be. > > > > > >So, for normalised values, rearranging the VAX F-float bytes, reading as > IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment in > the ccp4 lib is neat.) > > > > > >In this link describing VAX floats, it is unfortunate that it only states > that the bias for F-float is 128, but not that the mantissa starts from > 0.01 instead of 0.1. Therefore the confusion. > > > >https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm > > > > > > > >Thanks to all responded! > > > > > >Zhijie > > > > > >________________________________ > >From: Ian Tickle <ianj...@gmail.com> > >Sent: Tuesday, November 13, 2018 4:54 PM > >To: Zhijie Li > >Cc: CCP4BB@JISCMAIL.AC.UK > >Subject: Re: [ccp4bb] VERY old mtz file.. > > > > > >Hi Zhijie > > > >It's definitely a factor 4. The code is in subroutine QTIEEE in the > Fortran source I mentioned previously at this line: > > > >See line: > > > > A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2 > > > >If you prefer it in C code it's in function vaxF2ieeeF in: > > > >ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c > > > >See line: > > > >out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */ > > > >i.e. subtract 2 from exponent -> division by 4. > > > >Cheers > > > >-- Ian > > > > > >On Tue, 13 Nov 2018 at 19:52, Zhijie Li <zhijie...@utoronto.ca<mailto: > zhijie...@utoronto.ca>> wrote: > >If somebody is going to send these files by email, please send one to me > too. Thanks in advance. I actually prefer to get a MTZ file because the > miller indices would serve as good clues for understanding the encodings. > Even the first 1024 bytes of an MTZ would do (data array starts at byte 80 > in MTZ). > > > >In my life I had only seen ieee754. According to what I can find, VAX > has an exponent bias of 128 (ieee754 uses 127). Then it seems to me that > when converting from vax to ieee a division of 2 is involved. However all > procedures I have seen use a division of 4, which is quite puzzling to me. > A real data file containing meaningful numbers (eg., HKL indices) would be > very helpful. Thanks in advance. > > > >Zhijie > > > >> On Nov 13, 2018, at 2:21 PM, Johan Hattne <hat...@ucla.edu<mailto: > hat...@ucla.edu>> wrote: > >> > >> Related by not exactly on topic: would anybody on the list be able to > share old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX > reals/strings? I�d be interested to see what those files actually look(ed) > like. > >> > >> // Best wishes; Johan > >> > >>> On Nov 9, 2018, at 18:38, Zhijie Li <zhijie...@utoronto.ca<mailto: > zhijie...@utoronto.ca>> wrote: > >>> > >>> Hi all, > >>> > >>> On linux there are a few good GUI HEX editors. Here I�d like to > recommend BLESS, which conveniently displays all possible numerical > interpretations of the four bytes under cursor. It also allows the user to > switch between big endian or little endian through a checkbox. > Unfortunately all floats are assumed to be IEEE754, therefore VAX floats > won�t be interpreted correctly. ( The simplest way to convert vax to ieee > float would be to write a little program to do some bit operations. I�d be > happy to take that as my weekend project) > >>> > >>> > >>> BTW, along the line of space efficiency, I can�t help noticing that > the miller indices are saved as float32 in mtz, as all other numbers in > mtz. This certainly have made mtz format a beautiful homogeneous data > format ;). In this particular case, if we have doubts about the > reliability of the machine stamp, trying to restore the miller indices > would be a good way to test hypotheses. > >>> > >>> Zhijie > >>> > >>>> On Nov 9, 2018, at 9:04 PM, James Holton < > 0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk<mailto: > 0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk>> wrote: > >>>> > >>>> As a beamline scientist I must say I am glad that diffraction image > data is not usually stored as ASCII text. In fact, I am slowly warming to > the idea of storing it as not just binary, but compressed formats. > Problem, I'm sure will be that it won't be long before we forget how to > decompress them, as most of the algorithms we are using aren't all that > widespread. Probably around the same time future generations will curse us > for using ASCII instead of unicode, which is a 16-bit standard. I'm sure we > will be reviled for limiting ourselves so, just to save a factor of two in > disk space. > >>>> In situations like this I always use the unix "od" command. It makes > everything "human readable" by converting the bytes into strings you can > read. Then it is just a matter of figuring out what the bytes are. > >>>> Unfortunately, "od" only decodes floats on the native platform, so if > the mtz is from another platform (Windows vs Linux, for example), then you > might need to do some swapping. Thus far, I have encountered files that > require one of a few swapping strategies in order to make them work: > >>>> > >>>> 1 2 3 4 - no swapping > >>>> > >>>> 4 3 2 1 - reverse all bytes > >>>> > >>>> 3 4 1 2 - swap words and swap bytes within the words > >>>> 2 1 4 3 - reverse of previous > >>>> > >>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2 > before swapping > >>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2 > before swapping > >>>> I'm sure there are other combinations, but the oldest MTZ I have is > only from 1996. > >>>> > >>>> -James Holton > >>>> MAD Scientist > >>>> > >>>> > >>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote: > >>>>> Anyone any idea what to do about this?? Created in 1992!! > >>>>> Seems unreadable.. > >>>>> > >>>>> No CTYP lines input for file: 1 > >>>>> Indices output even if all data items flagged "missing" > >>>>> Warning, NOT all LABOUT data lines given > >>>>> Warning: Machine stamp corrupted? Assuming native format. > >>>>>>>>>>> CCP4 library signal library_file:End of File (Error) > >>>>> > >>>>> > >>>>> To unsubscribe from the CCP4BB list, click the following link: > >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > >>>>> > >>>> > >>>> > >>>> To unsubscribe from the CCP4BB list, click the following link: > >>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > >>>> > >>> > >>> To unsubscribe from the CCP4BB list, click the following link: > >>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > >>> > >> > >> Research Specialist @ Gonen Lab > >> ____________________________________________________ > >> UCLA * 615 Charles E. Young Drive South > >> BSRB #347 * Los Angeles, CA 90095 > >> > >> ######################################################################## > >> > >> To unsubscribe from the CCP4BB list, click the following link: > >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > >######################################################################## > > > >To unsubscribe from the CCP4BB list, click the following link: > >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > >________________________________ > > > >To unsubscribe from the CCP4BB list, click the following link: > >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > >######################################################################## > > > >To unsubscribe from the CCP4BB list, click the following link: > >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
