On Thu, 16 May 2013 18:25:35 +0100, Frank von Delft <frank.vonde...@sgc.ox.ac.uk> wrote:
>Dear Gerard - thanks, very informative! Two questions: > >1. >Do I understand correctly, that you say XDS will throw together for >integration counts from many images even if they're spaced widely >throughout the dataset, i.e. through the various passes? > No >i.e. if I set up my data collection as 5 complete revolutions with low >beam transmission - will XDS know to combine image 15 and 15+(360deg) >and 15+(720deg) etc? By default? Or do I have tell it to do this >explicitly, in which case, how? > You cannot > >2. >If I have my 5x360 degrees of images, what metric / criterion do I use >to decide whether to use only data up to 512deg or 839deg or 1469deg? > Look at cc1/2 as a function of frame number, for the high-resolution shell. Cut before the curve starts to fall. You should change some parameter after every 360 degrees, e.g. distance, detector shift up/down and left/right, or wavelength to avoid having the same systematic errors. Expose such that you have not less than 1 count/pixel, i.e. do not go overboard with low exposure. HTH, Kay > >Cheers >Frank > > > > > >On 16/05/2013 18:03, Gerard Bricogne wrote: >> Dear James, >> >> A week ago I wrote what I thought was a perhaps excessively long and >> overly dense message in reply to Theresa's initial query, then I thought I >> should sleep on it before sending it, and got distracted by other things. >> >> I guess you may well have used that whole week composing yours ;-) and >> reading it just now makes the temptation of sending mine irresistible. I am >> largely in agreement with you about the need to change mental habits in this >> field, and hope that the emphasis on various matters in my message below is >> sufficiently different from yours to make a distinct contribution to this >> very important discussion. Your analysis of pile-up effects goes well beyond >> anything I have ever looked at. However, in line with Theresa's initial >> question, I would say that, while I agree with you that the best strategy >> for collecting "native data" is no strategy at all, this isn't the case when >> collecting data for phasing. In that case one needs to go back and consider >> how to measure accurate differences of intensities, not just accurate >> intensities on their own. That is another subject, on which I was going to >> follow up so as to fully answer Theresa's message - but perhaps that should >> come in another installment! >> >> >> With best wishes, >> >> Gerard. >> >> -- >> On Tue, May 07, 2013 at 12:04:33AM +0100, Theresa Hsu wrote: >>> Dear crystallographers >>> >>> Is there a good source/review/software to obtain tips for good data >> collection strategy using PILATUS detectors at synchrotron? Do we need to >> collect sweeps of high and low resolution data separately? For anomalous >> phasing (MAD), does the order of wavelengths used affect structure solution >> or limit radiation damage? >>> >>> Thank you. >>> >>> Theresa >> -- >> >> Dear Theresa, >> >> You have had several excellent replies to your question. Perhaps I >> could venture to add a few more comments, remarks and suggestions, which can >> be summarised as follows: with a Pilatus, (1) use fine slicing, (2) use >> strategies combining low exposure with high multiplicity, and (3) use XDS! >> >> As the use of Pilatus detectors has spread widely, it has been rather >> puzzling to come across so many instances when these detectors are misused, >> sometimes on the basis of explicit expert advice that is simply misguided. A >> typical example will be to see images collected on a Pilatus 6M with an >> image width of 1 degree and an exposure time of 1 second. When you see this, >> you know that there is some erroneous thinking (or habit) behind it. >> >> When talking to various users who have ended up with such datasets, and >> with people who advocate this kind of strategy, it seems clear that a number >> of irrational concerns about fine-slicing and low-exposure+high-multiplicity >> strategies have tended to override published rational arguments in favour of >> those strategies: there is a fear that if the images being collected do not >> show spots discernible by the naked eye to the resolution limit that is >> being aimed for, the integration software will then somehow not be able to >> find those spots in order to integrate them, and the final data resolution >> will be lower than expected. Perhaps this may be of concern in relation with >> the use of some integration programs, but if you use XDS, which implements a >> full 3D approach to image integration, this is simply not the case: XDS will >> collect all the counts belonging to a given reflection, whether those counts >> are all from a spot on a single 1-degree image exposed for 1 second, or from >> 10 consecutive images of 0.1 degree width exposed for 0.1 second each, or >> from 100 images obtained by grouping together the same 10 images as >> previously collected in 10 successive passes with a 10-fold attenuated beam. >> The hallmark of the Pilatus detector is to lead to equivalent signal/noise >> ratios for the last two ways of measuring that reflection, because it is a >> photon counter and has zero readout noise: therefore the combination >> Pilatus+XDS is a powerful one. >> >> What is different between these three strategies, however, is the >> quality of the overall dataset they will produce. There is nothing new in >> what I am describing below: it is all in the references that Bob Sweet gave >> you in his reply, or is an obvious consequence of what is found in these >> references. >> >> In case 1 (1-degree, 1 second - "coarse slicing") you would presumably >> also be (mis-)advised to use a strategy aiming at collecting a complete >> dataset in the minimum number of images. These strategies used to make sense >> in the days of films, of image plates, and even of CCDs because of the image >> readout noise, but they have no place any longer in the context of Pilatus >> detectors. First of all, using 1-degree image widths can only degrade the >> precision with which 2D spots on images are lifted to 3D reciprocal space >> for indexing, and hence worsen the quality of that indexing and therefore >> the accuracy with which the spot locations will be predicted (unless you >> carefully "post-refine") - then the integration step perhaps does need to >> "hunt" for those spots locally, and needs them to be somewhat visible. >> Secondly, 1 degree is usually greater than the angular width of a typical >> reflection: the integration process will therefore pick up more background >> noise (variance) than it would have done with a smaller image width. >> Thirdly, by collecting only enough images to reach completeness you will >> have substantial radiation damage in your late images compared to the early >> ones (if you don't, it means you have under-exposed your crystal) and will >> therefore end up with internal inconsistencies in your dataset, as well as >> perhaps some extra, spurious anisotropy of diffraction limits as a result of >> having to impose increasingly stringent resolution cut-offs in the later >> images. This will affect the internal scaling of that dataset and the final >> quality of the merged data. >> >> In case 2 (0.1 degree, 0.1 second - "fine slicing") you will have a >> more precise sampling of the 3D shape of each spot, hence more accurate >> indexing and prediction of spot positions if you use a genuinely 3D >> integration program like XDS. Thanks to that increased precision, spots can >> be integrated "blind", even if they are not terribly visible in the images, >> and the same number of photons will be collected with no penalty in terms of >> noise level, thanks to the photon-counting noiseless-readout nature of the >> Pilatus detector. An improvement will be that the finely sampled 3D shape of >> the spots will be used by XDS to minimise the impact of background variance >> on the integrated intensities. On the other hand, the differential radiation >> damage between early and late images will still be the same as in case 1 if >> you have chosen one of those old-style strategies (and associated beam >> intensity setting) that aim at just about exhausting the useful lifetime of >> the crystal by the time you reach completeness. >> >> In case 3 (like case 2, but collecting n times more images with an >> n-fold attenuated beam once you have collected a few "characterisation >> images" without that attenuation to carry out the initial indexing) you >> still have the two advantages of case 2 (the same total number of photons >> will be picked up by XDS, even if the individual images are now so weak that >> you can't see anything) but you are spreading the radiation damage so thinly >> over multiple successive complete datasets that you can choose to later >> apply a cut-off on image number at the processing stage, when the statistics >> tell you that diffraction quality has become degraded beyond some critical >> level. This is much preferable to having to apply different resolution >> cut-offs to different images towards the end of a barely complete dataset, >> as in cases 1 and 2. The impact of radiation damage will be quite smoothly >> and uniformly distributed across the final unique reflections, and your >> scaling problems (as well as any spurious anisotropy in your diffraction >> limits) will be minimised. >> >> >> This is becoming quite a long message: you can see why I included a >> summary of it at the beginning! Returning to it for a conclusion: Pilatus >> detectors, fine-slicing with low-exposure and high-multiplicity strategies, >> and XDS are a unique winning combination. If fears that another integration >> program may not perform as well as XDS on fine-sliced data make you feel >> tempted to revert to old-fashioned strategies (case 1) because it supposedly >> makes no difference: resist the temptation! Switch to those Pilatus-adapted >> strategies and to XDS, and enjoy the very real difference in the results! >> >> >> With best wishes, >> >> Gerard >> >> and colleagues at Global Phasing. >> >> -- >> On Tue, May 07, 2013 at 12:04:33AM +0100, Theresa Hsu wrote: >>> Dear crystallographers >>> >>> Is there a good source/review/software to obtain tips for good data >> collection strategy using PILATUS detectors at synchrotron? Do we need to >> collect sweeps of high and low resolution data separately? For anomalous >> phasing (MAD), does the order of wavelengths used affect structure solution >> or limit radiation damage? >>> Thank you. >>> >>> Theresa
