Hi Phoebe, I believe the usefulness and general correctness of the French & Wilson protocol (as implemented in TRUNCATE) has been well-validated over the years. There was some debate about 20 years ago on whether or not to use it, but I think pretty much everyone has converged on using it for the past many years.
Even with modern detectors, weak reflections (generally at high resolution) will be of the same order of magnitude as background, i.e. positive reflections (they MUST all be positive) end up being negative, after background subtraction, in part due to counting statistics variation or slight mis-estimation of background or a poor profile fit. The whole point of TRUNCATE is to fix this experimental artifact as best as possible, using in part the prior Bayesian knowledge that all measured reflections (that is, their intensities) must be positive. I think some other assumptions go into TRUNCATE as well (approximate Wilson statistics), but I leave that to the true experts (many of whom are enjoying dinner at this hour!). I'm sure the experts can give a more cogent answer, but I think the bottom line, to get you going, is: Use TRUNCATE. Dave David Borhani, Ph.D. D. E. Shaw Research, LLC 120 West Forty-Fifth Street, 39th Floor New York, NY 10036 [EMAIL PROTECTED] 212-478-0698 http://www.deshawresearch.com > -----Original Message----- > From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On > Behalf Of Phoebe Rice > Sent: Monday, September 08, 2008 3:30 PM > To: CCP4BB@JISCMAIL.AC.UK > Subject: [ccp4bb] truncate ignorance > > Dear Experts, > > At the risk of exposing excess ignorance, truncate makes me > very nervous because I don't quite get exactly what it is > doing with my data and what its assumptions are. > > From the documentation: > ======================================================== > ... the "truncate" procedure (keyword TRUNCATE YES, the > default) calculates a best estimate of F from I, sd(I), and > the distribution of intensities in resolution shells (see > below). This has the effect of forcing all negative > observations to be positive, and inflating the weakest > reflections (less than about 3 sd), because an observation > significantly smaller than the average intensity is likely > to be underestimated. > ========================================================= > > But is it really true, with data from nice modern detectors, > that the weaklings are underestimated? > Do I really want to inflate them? > Exactly what assumptions is it making about the expected > distributions? > How compatible are those assumptions with serious anisotropy > and the wierd Wilson plots that nucleic acids give? > > Note the original 1978 French and Wilson paper says: > "It is nevertheless important to validate this agreement for > each set of data independently, as the presence of atoms in > special positions or the existence of noncrystallographic > elements of symmetry (or pseudosymmetry) may abrogate the > application of these prior beliefs for some crystal > structures." > > Please help truncate my ignorance ... > > Phoebe > > ========================================================== > Phoebe A. Rice > Assoc. Prof., Dept. of Biochemistry & Molecular Biology > The University of Chicago > phone 773 834 1723 > http://bmb.bsd.uchicago.edu/Faculty_and_Research/01_Faculty/01 > _Faculty_Alphabetically.php?faculty_id=123 > > RNA is really nifty > DNA is over fifty > We have put them > both in one book > Please do take a > really good look > http://www.rsc.org/shop/books/2008/9780854042722.asp >
