Re: [ccp4bb] Merging data to increase multiplicity

2011-01-30 Thread Zbyszek Otwinowski
I concur with Kay, particularly with point d) and its consequences. Sometimes it is obvious which result is better, often it is not. For example, one of the hkl users was testing new options and found that all the statistics (including refinement r-free) were worse, but the experimental and refine

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-30 Thread Bosch, Juergen
I should also add to Kay's list the following: The tests we performed assumed that the default values of the programs (d*Trek, Mosflm 6.13?, XDS) in ~2006 were reasonable, so with tweaking you might get a better milage, but that was not the point when we had to deal with lots of data and no tim

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-30 Thread Kay Diederichs
Am 20:59, schrieb Van Den Berg, Bert: I have heard this before. I’m wondering though, does anybody know of a systematic study where different data processing programs are compared with real-life, non-lysozyme data? Bert Bert, some time ago I tried to start something to this effect - take a lo

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Johan Turkenburg
Hi Dirk, This example compares integration software in combination with the scaling program, which is what usually happens. Obviously, the scaling program does more than just scaling, it also handles rejections. It is possibly this procedure that makes most of the difference. For example, the defa

[ccp4bb] FW: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Colin Nave
CP4BB@JISCMAIL.AC.UK] On Behalf Of herman.schreu...@sanofi-aventis.com Sent: 28 January 2011 15:05 To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Merging data to increase multiplicity For me, it means a reflecting-range (as defined by XDS) of 5-10 or more degrees and spots being visible o

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Poul Nissen
Not sure if you ever got an answer to the original question (but I have had some mail mishabs today, so perhaps some messages are missing in my thread) Anyway: for sure NO - you cannot process the data with different parameter settings and then merge the different integrations as a way to gain in

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Edward A. Berry
I see two questions here: -Can we assume an unrealistically low mosaicity in order to reduce overlaps. -Is there any benefit in merging data from the same frames integrated with different strategy? As for cheating on the mosaicity, which I euphemistically call "peak sampling", I think it can gi

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Pete Meyer
A few things that might be worth looking at: 1. How is your beam divergence varying as you fix mosaicity at different levels? Does it look relatively stable at a realistic value for the beamline? If I'm remembering correctly, mosaicity and beam divergence are highly correlated within mosflm.

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Bosch, Juergen
ng the internet. From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Anastassis Perrakis Sent: Friday, January 28, 2011 8:11 AM To: CCP4BB@JISCMAIL.AC.UK<mailto:CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] Merging data to increase multiplicity

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Jim Pflugrath
: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Merging data to increase multiplicity ... but, back to the main point, my advice would be to only limit the mosaicity, to get better completeness by avoiding overlaps. Its not ideal, in the sense that you would be over-estimating the partial fraction of

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Bryan Lepore
On Fri, Jan 28, 2011 at 9:35 AM, wrote: > [...] due to the 3-dimensional profile fitting. what is the specific difference between 3-d profile fitting and using a sliding window of more than one image? -Bryan

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Herman . Schreuder
Colin Nave Sent: Friday, January 28, 2011 3:50 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Merging data to increase multiplicity Can people say how "high mosaicity" is defined. High relative to what? Is it high relative to th

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Colin Nave
4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of herman.schreu...@sanofi-aventis.com Sent: 28 January 2011 14:36 To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Merging data to increase multiplicity My experience (unpublished) is that XDS works very well for high-mosaicity crystals du

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Herman . Schreuder
: [ccp4bb] Merging data to increase multiplicity I have heard this before. I'm wondering though, does anybody know of a systematic study where different data processing programs are compared with real-life, non-lysozyme data? Bert

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Bryan Lepore
On Jan 28, 2011, at 8:45, "Bosch, Juergen" wrote: > Mark Robien and I did a "systematic" study on about 30 data sets while we > were at SGPP. can you name the detector(s)? -Bryan

[ccp4bb] AW: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Hovestreydt, Eric
21) 50997-5311 / +31 (15) 2152-501 Mobile: +49 (173) 7000-615 Von: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] Im Auftrag von Van Den Berg, Bert Gesendet: Freitag, 28. Januar 2011 14:38 An: CCP4BB@JISCMAIL.AC.UK Betreff: Re: [ccp4bb] Merging da

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Anastassis Perrakis
... but, back to the main point, my advice would be to only limit the mosaicity, to get better completeness by avoiding overlaps. Its not ideal, in the sense that you would be over-estimating the partial fraction of most partial reflections, and thus systematically underestimating intensities

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Van Den Berg, Bert
Interesting! But when will it be published? :-) On 1/28/11 8:45 AM, "Bosch, Juergen" wrote: Yes. But we have not published this. Mark Robien and I did a "systematic" study on about 30 data sets while we were at SGPP. The easy cases can be processed with anything the difficult cases worked

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Dirk Kostrewa
Hi Bert, here is one anecdotal evidence: a couple of years ago, I had one real in-house 3 A data set from a crystal after a quick iodide soak and processed the images with denzo/scalepack, mosflm/scala and xds/xscale. I got lower Rsym, higher I/sig(I) and better anomalous signal with xds. Mor

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Bosch, Juergen
Yes. But we have not published this. Mark Robien and I did a "systematic" study on about 30 data sets while we were at SGPP. The easy cases can be processed with anything the difficult cases worked only with XDS. This was mostly SeMet data or HA data, so de novo phasing no MR stuff. If you compar

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Van Den Berg, Bert
I have heard this before. I'm wondering though, does anybody know of a systematic study where different data processing programs are compared with real-life, non-lysozyme data? Bert On 1/28/11 7:58 AM, "Bosch, Juergen" wrote: I was a bit reductive with my statement (iPhone) The equation

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Bosch, Juergen
I was a bit reductive with my statement (iPhone) The equation below is suppose to read: If you have bad data, then you need to process with XDS in order to get the maximum out of your data. Thanks Tim, Jürgen - Jürgen Bosch Johns Hopkins Bloomberg School of Public Health Department of Bioch

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Tim Gruene
Dear Jürgen, is this an assignment operator or an equal sign? For if it's the latter it could read that the result of processing data with XDS are bad data, which is rather rude and probably not what you meant. Tim On Fri, Jan 28, 2011 at 06:55:43AM -0500, Jürgen Bosch wrote: > Bad data = proces

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread José Trincão
Ah, yes, I was missing that. The statistics will be wrong. But in principle I will get an mtz with better data, because I am integrating more observations which would have been rejected by being missed at low resolution if the mosaicity was set too low or being rejected by overlaps at high resol

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Ian Tickle
Jose - you're missing the fact that the same dataset processed in different ways are not statistically independent datasets! Increasing the multiplicity for independent data reduces the uncertainty because the calculation of the SU assumes statistical independence. Cheers -- Ian On Fri, Jan 28,

Re: [ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread Jürgen Bosch
Bad data = processing with XDS Jürgen .. Jürgen Bosch Johns Hopkins Bloomberg School of Public Health Department of Biochemistry & Molecular Biology Johns Hopkins Malaria Research Institute 615 North Wolfe Street, W8708 Baltimore, MD 21205 Phone: +1-410-614-4742 Lab: +1-

[ccp4bb] Merging data to increase multiplicity

2011-01-28 Thread José Trincão
Hello all, I have been trying to squeeze the most out of a bad data set (P1, anisotropic, crystals not reproducible). I had very incomplete data due to high mosaicity and lots of overlaps. The completeness was about 80% overall to ~3A. Yesterday I noticed that I could process the data much bette