The CCP4 library routines are supposed to read MTZ files in any recognised format, provided the correct machine stamp is present, so I’m puzzled that it doesn’t work in this case. In the days when multiple float formats were around, this certainly did work. The files were always written in the native format, and converted if required on reading. As far as I know the code is all still in the libraries, so it should still work. Phil
> On 14 Nov 2018, at 08:58, Kay Diederichs <kay.diederi...@uni-konstanz.de> > wrote: > > It is not necessary to do error-prone conversions manually: the ifort > Compiler understands the convert='VAXD' Option in its OPEN statement - see > https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier > > Thus one could just write a tiny read-write loop. > > HTH > Kay > > On Wed, 14 Nov 2018 00:51:02 +0000, Zhijie Li <zhijie...@utoronto.ca> wrote: > >> It's also said here, at the end of file : >> >> https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf >> >> "add 1 to the left, with the binary point" >> >> 0.10000..... >> >> >> >> ________________________________ >> From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Zhijie Li >> <zhijie...@utoronto.ca> >> Sent: Tuesday, November 13, 2018 7:43 PM >> To: CCP4BB@JISCMAIL.AC.UK >> Subject: Re: [ccp4bb] VERY old mtz file.. >> >> >> Hi all, >> >> >> I think I know why it is a division of 4 instead of 2 is involved in >> conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits >> (bias of 128 instead of 127, visible), another 2 is hidden in the scientific >> notation. >> >> >> I found this explanation+example on VAX F-float: >> >> http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html >> >> >> So for IEEE754 float32, if we want to represent a same 12.75 (1100.11) in >> the above example, we would first conceptually write it in scientific >> notation as 1.10011 x 1000 in binary. Then the mantissa part is the part >> after the dot filled with zero to 23 bits: '10011000000000000000000', the >> exponent part is 3+127=130 (dec)=10000010(bin). Then the binary IEEE754 >> float32 number is 0[10000010][10011000000000000000000]. (You can check it >> here: https://www.h-schmidt.net/FloatConverter/IEEE754.html) >> >> >> Now compare this with the VAX 12.75 in the linked example, we can find that >> besides the bias becoming 128, the conceptual binary scientific notation is >> actually 0.110011 x 10000, instead of 1.10011 x 1000. So the exponent >> needed is 4 instead of 3. Then the exponent bits are 4+128=132=10000100 and >> the VAX float32 becomes >> >> 0[10000100][10011000000000000000000] ---if we write in a IEEE-style order. >> Note that the mantissa appears to be same as the ieee mantissa, and the >> exponent to be applied is 132-128=4. If this number is interpreted as >> IEEE754, then it will be 1.10011 x 2exp(132-127)=1.10011 x 100000, four >> times of what it should be. >> >> >> So, for normalised values, rearranging the VAX F-float bytes, reading as >> IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment in >> the ccp4 lib is neat.) >> >> >> In this link describing VAX floats, it is unfortunate that it only states >> that the bias for F-float is 128, but not that the mantissa starts from 0.01 >> instead of 0.1. Therefore the confusion. >> >> https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm >> >> >> >> Thanks to all responded! >> >> >> Zhijie >> >> >> ________________________________ >> From: Ian Tickle <ianj...@gmail.com> >> Sent: Tuesday, November 13, 2018 4:54 PM >> To: Zhijie Li >> Cc: CCP4BB@JISCMAIL.AC.UK >> Subject: Re: [ccp4bb] VERY old mtz file.. >> >> >> Hi Zhijie >> >> It's definitely a factor 4. The code is in subroutine QTIEEE in the Fortran >> source I mentioned previously at this line: >> >> See line: >> >> A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2 >> >> If you prefer it in C code it's in function vaxF2ieeeF in: >> >> ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c >> >> See line: >> >> out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */ >> >> i.e. subtract 2 from exponent -> division by 4. >> >> Cheers >> >> -- Ian >> >> >> On Tue, 13 Nov 2018 at 19:52, Zhijie Li >> <zhijie...@utoronto.ca<mailto:zhijie...@utoronto.ca>> wrote: >> If somebody is going to send these files by email, please send one to me >> too. Thanks in advance. I actually prefer to get a MTZ file because the >> miller indices would serve as good clues for understanding the encodings. >> Even the first 1024 bytes of an MTZ would do (data array starts at byte 80 >> in MTZ). >> >> In my life I had only seen ieee754. According to what I can find, VAX has >> an exponent bias of 128 (ieee754 uses 127). Then it seems to me that when >> converting from vax to ieee a division of 2 is involved. However all >> procedures I have seen use a division of 4, which is quite puzzling to me. A >> real data file containing meaningful numbers (eg., HKL indices) would be >> very helpful. Thanks in advance. >> >> Zhijie >> >>> On Nov 13, 2018, at 2:21 PM, Johan Hattne >>> <hat...@ucla.edu<mailto:hat...@ucla.edu>> wrote: >>> >>> Related by not exactly on topic: would anybody on the list be able to share >>> old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX reals/strings? >>> I�d be interested to see what those files actually look(ed) like. >>> >>> // Best wishes; Johan >>> >>>> On Nov 9, 2018, at 18:38, Zhijie Li >>>> <zhijie...@utoronto.ca<mailto:zhijie...@utoronto.ca>> wrote: >>>> >>>> Hi all, >>>> >>>> On linux there are a few good GUI HEX editors. Here I�d like to recommend >>>> BLESS, which conveniently displays all possible numerical interpretations >>>> of the four bytes under cursor. It also allows the user to switch between >>>> big endian or little endian through a checkbox. Unfortunately all floats >>>> are assumed to be IEEE754, therefore VAX floats won�t be interpreted >>>> correctly. ( The simplest way to convert vax to ieee float would be to >>>> write a little program to do some bit operations. I�d be happy to take >>>> that as my weekend project) >>>> >>>> >>>> BTW, along the line of space efficiency, I can�t help noticing that the >>>> miller indices are saved as float32 in mtz, as all other numbers in mtz. >>>> This certainly have made mtz format a beautiful homogeneous data format >>>> ;). In this particular case, if we have doubts about the reliability of >>>> the machine stamp, trying to restore the miller indices would be a good >>>> way to test hypotheses. >>>> >>>> Zhijie >>>> >>>>> On Nov 9, 2018, at 9:04 PM, James Holton >>>>> <0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk<mailto:0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk>> >>>>> wrote: >>>>> >>>>> As a beamline scientist I must say I am glad that diffraction image data >>>>> is not usually stored as ASCII text. In fact, I am slowly warming to the >>>>> idea of storing it as not just binary, but compressed formats. Problem, >>>>> I'm sure will be that it won't be long before we forget how to >>>>> decompress them, as most of the algorithms we are using aren't all that >>>>> widespread. Probably around the same time future generations will curse >>>>> us for using ASCII instead of unicode, which is a 16-bit standard. I'm >>>>> sure we will be reviled for limiting ourselves so, just to save a factor >>>>> of two in disk space. >>>>> In situations like this I always use the unix "od" command. It makes >>>>> everything "human readable" by converting the bytes into strings you can >>>>> read. Then it is just a matter of figuring out what the bytes are. >>>>> Unfortunately, "od" only decodes floats on the native platform, so if the >>>>> mtz is from another platform (Windows vs Linux, for example), then you >>>>> might need to do some swapping. Thus far, I have encountered files that >>>>> require one of a few swapping strategies in order to make them work: >>>>> >>>>> 1 2 3 4 - no swapping >>>>> >>>>> 4 3 2 1 - reverse all bytes >>>>> >>>>> 3 4 1 2 - swap words and swap bytes within the words >>>>> 2 1 4 3 - reverse of previous >>>>> >>>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2 before >>>>> swapping >>>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2 before >>>>> swapping >>>>> I'm sure there are other combinations, but the oldest MTZ I have is only >>>>> from 1996. >>>>> >>>>> -James Holton >>>>> MAD Scientist >>>>> >>>>> >>>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote: >>>>>> Anyone any idea what to do about this?? Created in 1992!! >>>>>> Seems unreadable.. >>>>>> >>>>>> No CTYP lines input for file: 1 >>>>>> Indices output even if all data items flagged "missing" >>>>>> Warning, NOT all LABOUT data lines given >>>>>> Warning: Machine stamp corrupted? Assuming native format. >>>>>>>>>>>> CCP4 library signal library_file:End of File (Error) >>>>>> >>>>>> >>>>>> To unsubscribe from the CCP4BB list, click the following link: >>>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>>>> >>>>> >>>>> >>>>> To unsubscribe from the CCP4BB list, click the following link: >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>>> >>>> >>>> To unsubscribe from the CCP4BB list, click the following link: >>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>> >>> >>> Research Specialist @ Gonen Lab >>> ____________________________________________________ >>> UCLA * 615 Charles E. Young Drive South >>> BSRB #347 * Los Angeles, CA 90095 >>> >>> ######################################################################## >>> >>> To unsubscribe from the CCP4BB list, click the following link: >>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >> ######################################################################## >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >> ________________________________ >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >> ######################################################################## >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
