Dear Ian, That was in fact one of my reasons for only calculating the free R at the end of a SHELXL refinement run (the other reason, now less important, was to save some CPU time). I have to add that I am no longer completely convinced that I made the right decision all those years ago. A stable refinement in which R decreases but Rfree goes through a minimum and then starts to rise might be a useful indication of overfitting?!
Best wishes, George Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-22582 On Mon, 16 Feb 2009, Ian Tickle wrote: > Clemens, I know we've had this discussion several times before, but I'd > like to take you up on the point you made that reducing Rfree-R is > necessarily always a 'good thing'. Suppose the refinement had started > from a point where Rfree was biased, e.g. the test set in use had > previously been part of the working set, so that Rfree-R was too small. > In that case one would hope and indeed expect that Rfree-R would > increase on further refinement now excluding the test set. Shouldn't > the criterion be that Rfree-R should attain its expected value > (dependent of course on the observation/parameter ratio and the > weighting parameters), so a high value of |(Rfree-R) - <Rfree-R>| is > bad, i.e. any significant deviations of (Rfree-R) from its expectation > are bad? > > I would go further than that and say that anyway Rfree is meaningless > unless the refinement has converged, i.e. reached its maximum (local or > global) total likelihood (i.e. data+restraints). So one simply cannot > compare the Rfree (or Rfree-R) values at the beginning and end of a run. > The purpose of Rfree (or better free likelihood) is surely to compare > the *results* of *different* runs where convergence has been attained > and where the *refinement protocol* (i.e. selection of parameters to > vary and weighting parameters) has been varied, and then to choose as > the optimal protocol (and therefore optimal result) the one that gave > the lowest Rfree (or highest free likelihood). > > Rfree-R is then used as a subsidiary test to verify that it has attained > its expected value, if not then something is wrong, i.e. either the > refinement didn't converge (Rfree-R lower than <Rfree-R>) or there are > non-random errors (Rfree-R higher than <Rfree-R>), or a combination of > factors. > > Cheers > > -- Ian > > > -----Original Message----- > > From: owner-ccp...@jiscmail.ac.uk [mailto:owner-ccp...@jiscmail.ac.uk] > On > > Behalf Of Clemens Vonrhein > > Sent: 13 February 2009 17:15 > > To: CCP4BB@JISCMAIL.AC.UK > > Subject: Re: [ccp4bb] unstable refinement > > > > * you don't mention if the R and Rfree move up identically - or if you > > have a faster increase in R than in Rfree, which would mean that > > your R-factors are increasing (bad I guess) but your Rfree-R gap is > > closing down (good). > > > > So moving from R/Rfree=0.20/0.35 to R/Rfree=0.32/37 is different > > than moving from R/Rfree=0.20/0.25 to R/Rfree=0.23/0.28. > > > Disclaimer > This communication is confidential and may contain privileged information > intended solely for the named addressee(s). It may not be used or disclosed > except for the purpose for which it has been sent. If you are not the > intended recipient you must not review, use, disclose, copy, distribute or > take any action in reliance upon it. If you have received this communication > in error, please notify Astex Therapeutics Ltd by emailing > i.tic...@astex-therapeutics.com and destroy all copies of the message and any > attached documents. > Astex Therapeutics Ltd monitors, controls and protects all its messaging > traffic in compliance with its corporate email policy. The Company accepts no > liability or responsibility for any onward transmission or use of emails and > attachments having left the Astex Therapeutics domain. Unless expressly > stated, opinions in this message are those of the individual sender and not > of Astex Therapeutics Ltd. The recipient should check this email and any > attachments for the presence of computer viruses. Astex Therapeutics Ltd > accepts no liability for damage caused by any virus transmitted by this > email. E-mail is susceptible to data corruption, interception, unauthorized > amendment, and tampering, Astex Therapeutics Ltd only send and receive > e-mails on the basis that the Company is not liable for any such alteration > or any consequences thereof. > Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science Park, > Cambridge CB4 0QA under number 3751674 >
