Hi Okay, I can read that (sort of) with errors using a copy of mtzdmp from ccp4 v 5.0, dated 23/01/04; I can tell that Ben Luisi's name is in there (hello Ben!) and that the file was "converted to MTZ file mark 1 on 15/ 5/92", inter alia. I can even tell that there are five columns and 17218 reflections...
Unfortunately, I get an error in reading the reflections themselves - > LIST OF REFLECTIONS > =================== > > >>>>>> CCP4 library signal library_file:Bad mode (Error) > raised in ccp4_file_readfloat <<<<<< > >>>>>> System signal 22:Invalid argument (Error) > raised in ccp4_file_read <<<<<< > <B><FONT COLOR="#FF0000"><!--SUMMARY_BEGIN--> > MTZDUMP: Normal termination of mtzdump > Times: User: 0.0s System: 0.0s Elapsed: 0:00 My feeling is that all the discussion about VAXes etc may be irrelevant - in my view it's more likely that the programs for reading and writing the MTZ format changed (and the MTZ format itself changed as well) and are no longer capable of dealing with "MTZ file mark 1"... If I can actually get the VAXes to which I referred to read an old DAT tape, I may be able to build a copy of CCP4 from 1993... but that's a problem for another day (anyone got a SCSI DDS drive I could plug into a VAX?). On 14 Nov 2018, at 10:25, Eleanor Dodson wrote: > Here is the file I was trying to read - please feel free to play with it!! > Eleanor > > On Wed, 14 Nov 2018 at 10:17, Harry Powell > <0000193323b1e616-dmarc-requ...@jiscmail.ac.uk> wrote: > Hi > > Weren't the CCP4 base-level routines re-written from FORTRAN to C sometime in > the late 1990's? Very occasionally I used to find bugs that had been > introduced in this process (or possibly not corrected...) so it's possible > that Eleanor's file might be readable with a really old code base. > > BTW, I have recently had cause to search out a VMS system and have found > someone with a whole farm of running VAXes (or VAXen if you're geeky). > > Harry > > On 14 Nov 2018, at 09:46, Ian Tickle wrote: > >> The CCP4 routines for MTZ and map files are written in C and thus do not use >> a Fortran unformatted OPEN statement, they use C-style block read & write. >> >> Cheers >> >> -- Ian >> >> >> On Wed, 14 Nov 2018 at 08:59, Kay Diederichs >> <kay.diederi...@uni-konstanz.de> wrote: >> It is not necessary to do error-prone conversions manually: the ifort >> Compiler understands the convert='VAXD' Option in its OPEN statement - see >> https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier >> >> Thus one could just write a tiny read-write loop. >> >> HTH >> Kay >> >> On Wed, 14 Nov 2018 00:51:02 +0000, Zhijie Li <zhijie...@utoronto.ca> wrote: >> >> >It's also said here, at the end of file : >> > >> >https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf >> > >> >"add 1 to the left, with the binary point" >> > >> >0.10000..... >> > >> > >> > >> >________________________________ >> >From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Zhijie Li >> ><zhijie...@utoronto.ca> >> >Sent: Tuesday, November 13, 2018 7:43 PM >> >To: CCP4BB@JISCMAIL.AC.UK >> >Subject: Re: [ccp4bb] VERY old mtz file.. >> > >> > >> >Hi all, >> > >> > >> >I think I know why it is a division of 4 instead of 2 is involved in >> >conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits >> >(bias of 128 instead of 127, visible), another 2 is hidden in the >> >scientific notation. >> > >> > >> >I found this explanation+example on VAX F-float: >> > >> >http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html >> > >> > >> >So for IEEE754 float32, if we want to represent a same 12.75 (1100.11) in >> >the above example, we would first conceptually write it in scientific >> >notation as 1.10011 x 1000 in binary. Then the mantissa part is the part >> >after the dot filled with zero to 23 bits: '10011000000000000000000', the >> >exponent part is 3+127=130 (dec)=10000010(bin). Then the binary IEEE754 >> >float32 number is 0[10000010][10011000000000000000000]. (You can check it >> >here: https://www.h-schmidt.net/FloatConverter/IEEE754.html) >> > >> > >> >Now compare this with the VAX 12.75 in the linked example, we can find that >> >besides the bias becoming 128, the conceptual binary scientific notation is >> >actually 0.110011 x 10000, instead of 1.10011 x 1000. So the exponent >> >needed is 4 instead of 3. Then the exponent bits are 4+128=132=10000100 and >> >the VAX float32 becomes >> > >> >0[10000100][10011000000000000000000] ---if we write in a IEEE-style order. >> >Note that the mantissa appears to be same as the ieee mantissa, and the >> >exponent to be applied is 132-128=4. If this number is interpreted as >> >IEEE754, then it will be 1.10011 x 2exp(132-127)=1.10011 x 100000, four >> >times of what it should be. >> > >> > >> >So, for normalised values, rearranging the VAX F-float bytes, reading as >> >IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment in >> >the ccp4 lib is neat.) >> > >> > >> >In this link describing VAX floats, it is unfortunate that it only states >> >that the bias for F-float is 128, but not that the mantissa starts from >> >0.01 instead of 0.1. Therefore the confusion. >> > >> >https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm >> > >> > >> > >> >Thanks to all responded! >> > >> > >> >Zhijie >> > >> > >> >________________________________ >> >From: Ian Tickle <ianj...@gmail.com> >> >Sent: Tuesday, November 13, 2018 4:54 PM >> >To: Zhijie Li >> >Cc: CCP4BB@JISCMAIL.AC.UK >> >Subject: Re: [ccp4bb] VERY old mtz file.. >> > >> > >> >Hi Zhijie >> > >> >It's definitely a factor 4. The code is in subroutine QTIEEE in the >> >Fortran source I mentioned previously at this line: >> > >> >See line: >> > >> > A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2 >> > >> >If you prefer it in C code it's in function vaxF2ieeeF in: >> > >> >ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c >> > >> >See line: >> > >> >out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */ >> > >> >i.e. subtract 2 from exponent -> division by 4. >> > >> >Cheers >> > >> >-- Ian >> > >> > >> >On Tue, 13 Nov 2018 at 19:52, Zhijie Li >> ><zhijie...@utoronto.ca<mailto:zhijie...@utoronto.ca>> wrote: >> >If somebody is going to send these files by email, please send one to me >> >too. Thanks in advance. I actually prefer to get a MTZ file because the >> >miller indices would serve as good clues for understanding the encodings. >> >Even the first 1024 bytes of an MTZ would do (data array starts at byte 80 >> >in MTZ). >> > >> >In my life I had only seen ieee754. According to what I can find, VAX has >> >an exponent bias of 128 (ieee754 uses 127). Then it seems to me that when >> >converting from vax to ieee a division of 2 is involved. However all >> >procedures I have seen use a division of 4, which is quite puzzling to me. >> >A real data file containing meaningful numbers (eg., HKL indices) would be >> >very helpful. Thanks in advance. >> > >> >Zhijie >> > >> >> On Nov 13, 2018, at 2:21 PM, Johan Hattne >> >> <hat...@ucla.edu<mailto:hat...@ucla.edu>> wrote: >> >> >> >> Related by not exactly on topic: would anybody on the list be able to >> >> share old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX >> >> reals/strings? I�d be interested to see what those files actually >> >> look(ed) like. >> >> >> >> // Best wishes; Johan >> >> >> >>> On Nov 9, 2018, at 18:38, Zhijie Li >> >>> <zhijie...@utoronto.ca<mailto:zhijie...@utoronto.ca>> wrote: >> >>> >> >>> Hi all, >> >>> >> >>> On linux there are a few good GUI HEX editors. Here I�d like to >> >>> recommend BLESS, which conveniently displays all possible numerical >> >>> interpretations of the four bytes under cursor. It also allows the user >> >>> to switch between big endian or little endian through a checkbox. >> >>> Unfortunately all floats are assumed to be IEEE754, therefore VAX floats >> >>> won�t be interpreted correctly. ( The simplest way to convert vax to >> >>> ieee float would be to write a little program to do some bit operations. >> >>> I�d be happy to take that as my weekend project) >> >>> >> >>> >> >>> BTW, along the line of space efficiency, I can�t help noticing that the >> >>> miller indices are saved as float32 in mtz, as all other numbers in mtz. >> >>> This certainly have made mtz format a beautiful homogeneous data format >> >>> ;). In this particular case, if we have doubts about the reliability of >> >>> the machine stamp, trying to restore the miller indices would be a good >> >>> way to test hypotheses. >> >>> >> >>> Zhijie >> >>> >> >>>> On Nov 9, 2018, at 9:04 PM, James Holton >> >>>> <0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk<mailto:0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk>> >> >>>> wrote: >> >>>> >> >>>> As a beamline scientist I must say I am glad that diffraction image >> >>>> data is not usually stored as ASCII text. In fact, I am slowly warming >> >>>> to the idea of storing it as not just binary, but compressed formats. >> >>>> Problem, I'm sure will be that it won't be long before we forget how >> >>>> to decompress them, as most of the algorithms we are using aren't all >> >>>> that widespread. Probably around the same time future generations will >> >>>> curse us for using ASCII instead of unicode, which is a 16-bit >> >>>> standard. I'm sure we will be reviled for limiting ourselves so, just >> >>>> to save a factor of two in disk space. >> >>>> In situations like this I always use the unix "od" command. It makes >> >>>> everything "human readable" by converting the bytes into strings you >> >>>> can read. Then it is just a matter of figuring out what the bytes are. >> >>>> Unfortunately, "od" only decodes floats on the native platform, so if >> >>>> the mtz is from another platform (Windows vs Linux, for example), then >> >>>> you might need to do some swapping. Thus far, I have encountered files >> >>>> that require one of a few swapping strategies in order to make them >> >>>> work: >> >>>> >> >>>> 1 2 3 4 - no swapping >> >>>> >> >>>> 4 3 2 1 - reverse all bytes >> >>>> >> >>>> 3 4 1 2 - swap words and swap bytes within the words >> >>>> 2 1 4 3 - reverse of previous >> >>>> >> >>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2 before >> >>>> swapping >> >>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2 before >> >>>> swapping >> >>>> I'm sure there are other combinations, but the oldest MTZ I have is >> >>>> only from 1996. >> >>>> >> >>>> -James Holton >> >>>> MAD Scientist >> >>>> >> >>>> >> >>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote: >> >>>>> Anyone any idea what to do about this?? Created in 1992!! >> >>>>> Seems unreadable.. >> >>>>> >> >>>>> No CTYP lines input for file: 1 >> >>>>> Indices output even if all data items flagged "missing" >> >>>>> Warning, NOT all LABOUT data lines given >> >>>>> Warning: Machine stamp corrupted? Assuming native format. >> >>>>>>>>>>> CCP4 library signal library_file:End of File (Error) >> >>>>> >> >>>>> >> >>>>> To unsubscribe from the CCP4BB list, click the following link: >> >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >>>>> >> >>>> >> >>>> >> >>>> To unsubscribe from the CCP4BB list, click the following link: >> >>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >>>> >> >>> >> >>> To unsubscribe from the CCP4BB list, click the following link: >> >>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >>> >> >> >> >> Research Specialist @ Gonen Lab >> >> ____________________________________________________ >> >> UCLA * 615 Charles E. Young Drive South >> >> BSRB #347 * Los Angeles, CA 90095 >> >> >> >> ######################################################################## >> >> >> >> To unsubscribe from the CCP4BB list, click the following link: >> >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> > >> >######################################################################## >> > >> >To unsubscribe from the CCP4BB list, click the following link: >> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> > >> >________________________________ >> > >> >To unsubscribe from the CCP4BB list, click the following link: >> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> > >> >######################################################################## >> > >> >To unsubscribe from the CCP4BB list, click the following link: >> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> > >> >> ######################################################################## >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> > > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > > <semioxyhb_tstate_BetaCobalt.mtz> ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
