On Fri, 14 Feb 2014 17:25:36 +0000, Derek Logan <derek.lo...@biochemistry.lu.se> wrote:
>Hi Tim, > >> I would actually recommend using pointless (or xprep) instead of >> xdsconv. It is much easier to use and maybe even less error prone. > >I take the point about pointless but XPREP is commercial software sold by >Bruker and costs �700 (as I remember), so is not really an option for everyone. pointless is a good program, and I also use it, but to call it a replacement for XDSCONV would be wrong. But yes, it can be used to convert XDS_ASCII.HKL to a multi-record MTZ file (this is just a format conversion!) which is then fed into CCP4 truncate (which calculates amplitudes from intensities). At that point, you have replaced one implementation of French&Wilson 1978 by a different one. Which one is better is a possible question (and you _can_ find meaningful information about this in the CCP4BB archives), but the differences are probably not very significant for real data. I didn't even know that xprep calculates amplitudes from intensities, but I wonder why it should do it any better than other well-tested programs. Concerning usage of programs, everybody has his/her preferences, but what could be simpler than a 2-liner XDSCONV.INP like INPUT_FILE=XDS_ASCII.HKL OUTPUT_FILE=temp.hkl CCP4 ! or CCP4_F or CCP4_I or SHELX or CNS and then running XDSCONV by running "xdsconv"? At least there's not much room for mistakes. Within XDSGUI (http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/XDSGUI) , running XDSCONV is a matter of two mouse clicks, nothing else. > >> All your quotes from the output are perfectly consistent. The first >> table tells you there are 190093 unmerged reflections in total, but >> only 44059 unique reflections. Hence if you ask xdsconv to merge the >> output (MERGE=TRUE), it will do so and only write 44047 (a few less >> than 44059 because it rejects those 19 unmerged reflections with >> I<-3sigma). > >I beg to disagree, and this was the point of my question. The output of >XDSCONV literally says that 190093 reflections are read, and [of those, my >interpretation] 44047 are accepted. I may be a pedant but I can't read that >output in any other way. To me it looks like it is reading only the >reflections that already fall into the asymmetric unit and is ignoring all the >others. So if XDSCONV is really doing what it is supposed to, I would suggest >rephrasing that output line. good point about the rephrasing. I'll see to making the wording consistent between XDSCONV and XDS. But irrespective of the wording, it does take all observations into account when calculating the intensity (and amplitude) of the unique reflections (as it should - ignoring reflections would not make sense, and would produce significantly worse data). However, it does so not by calculating the geometric mean (which you seem to assume), but by calculating the weighted mean. Weighting is done with the variances, and here it also does not differ from (c)truncate or other programs. > >> The documentation tells you: >> "MERGE=TRUE means that the weighted mean of symmetry equivalent >> reflection intensities appearing in the input file will be determined >> and used in the output file." > >Yes, I paraphrased this in my original mail. I like to believe that the manual >describes what the program is doing, but the output isn't consistent with this >in my view. > >> xscale does not scale data in a crystallographic sense since scaling >> is already done in the CORRECT step of XDS. It put the data on a >> common scale (in a sophisticated manner), i.e. if you only have one >> data set there is no need to run xscale except for the thinner shells >> for the statistics table compared to CORRECT. > >No need indeed, except that including XSCALE doesn't leave me sleepless about >the data having been merged. And maybe correction for radiation damage when >you have sufficient multiplicity? > >But yes, pedantry aside, I will start using pointless in my csh pipelines. please report back whether that changes (or even improves) your results! It is always very good when people compare programs in a meaningful way, but from my own experience I can say that meaningful comparisons are sometimes not entirely straightforward to get right. (for the German-speaking: "wer misst, misst Mist!") good luck, Kay > >Best wishes >Derek > >> On 02/14/2014 03:57 PM, Derek Logan wrote: >>> Hi, >>> >>> I am a long time user of XDS (20 years this year) but all the same >>> I find that I have constant angst about losing observations because >>> I don't understand what goes in in the conversion steps to get to >>> CCP4 format. I used to believe that XSCALE was always necessary, >>> and I always use it in my workflow even if there is only one >>> dataset (after all, there's nothing to lose), but my Ph.D. students >>> pointed out to me that XDSCONV could take output directly from >>> CORRECT, and they often do it this way. The XDS wiki and XDSCONV >>> docs seems to confirm this: using MERGE=TRUE in XDSCONV should >>> output the geometrical mean of the observations. However I am >>> worried by what the XDSCONV output says. For today's example >>> CORRECT gives me this: >>> >>> NUMBER OF REFLECTIONS IN SELECTED SUBSET OF IMAGES 190093 NUMBER >>> OF REJECTED MISFITS 3842 NUMBER OF >>> SYSTEMATIC ABSENT REFLECTIONS 34 NUMBER OF ACCEPTED >>> OBSERVATIONS 186217 NUMBER OF UNIQUE ACCEPTED >>> REFLECTIONS 44059 >>> >>> XDSCONV says the following: >>> >>> FRIEDEL'S_LAW=TRUE MERGE=TRUE NUMBER OF REFLECTION RECORDS ON INPUT >>> FILE 190093 NUMBER OF IGNORED REFLECTIONS (I< -3.0*SIGMA) >>> 19 NUMBER OF REFLECTIONS ACCEPTED FROM INPUT FILE 44047 >>> >>> To my literal mind this says it is throwing away most of the >>> observations. Now if I merge the reflections in XSCALE first it >>> says this: >>> >>> FRIEDEL'S_LAW=TRUE MERGE=TRUE NUMBER OF REFLECTION RECORDS ON INPUT >>> FILE 44046 NUMBER OF IGNORED REFLECTIONS (I< -3.0*SIGMA) >>> 0 NUMBER OF REFLECTIONS ACCEPTED FROM INPUT FILE 44046 >>> >>> which makes more sense. If I compare the first few reflections of >>> the output file with structure factor amplitudes from XDSCONV for >>> each scenario they are different, but I believe that is because >>> XSCALE has put the intensities on an absolute scale and CORRECT has >>> not. >>> >>> Basically all I want to know is that the output from XDSCONV is >>> misleadingly worded, i.e. that even if it appears to say that only >>> the asymmetric unit has been accepted, actually all observations >>> have gone in and the geometric mean is indeed output. That would >>> put my mind to rest! >>> >>> /Derek >>> ________________________________________________________________________ >>> >>> >> Derek Logan tel: +46 46 222 1443 >>> Associate Professor mob: +46 76 >>> 8585 707 Dept. of Biochemistry and Structural Biology >>> www.cmps.lu.se<http://www.cmps.lu.se> Centre for Molecular Protein >>> Science >>> www.maxlab.lu.se/node/307<http://www.maxlab.lu.se/node/307> Lund >>> University, Box 124, 221 00 Lund, Sweden >>> >>> >>> >>> >>> >>> >> >> - -- >> - -- >> Dr Tim Gruene >> Institut fuer anorganische Chemie >> Tammannstr. 4 >> D-37077 Goettingen >> >> GPG Key ID = A46BEE1A >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.12 (GNU/Linux) >> Comment: Using GnuPG with Icedove - http://www.enigmail.net/ >> >> iD8DBQFS/jWBUxlJ7aRr7hoRAk0NAJ4nlKwfBMUdnbkpXzNz92+XtX1FXACfQTwJ >> S92/lRMo5Jn9YHUJqLVi9mA= >> =UiLX >> -----END PGP SIGNATURE-----
