We have used this work a lot over the years in PDB-REDO and while the model for the Rfree/R ratio does not seem to work anymore for model refinement tools, the model for sigmaRfree (equal to R-free divided by the square root of the number of test set reflections) still holds. Also if you do k-fold cross validation (which we do for small test sets) the standard deviation of R-free ends up pretty much the same as the predicted value.
Tim Gruene did work on Rcomplete (https://www.pnas.org/doi/10.1073/pnas.1502136112) which combine the results form the k refinements in cross validation and Rcomplete comes out to be very close to the average Rfree from all k refinements. In CCP4, k-fold cross validation is easy because your reflections are divided in (roughly) equal groups. You just need to set the test set number from 0 to all the other values. Cheers, Robbie > -----Original Message----- > From: CCP4 bulletin board <[email protected]> On Behalf Of Gerard > Bricogne > Sent: Monday, June 23, 2025 17:38 > To: [email protected] > Subject: Re: [ccp4bb] free R in shells > > Dear Randy, > > Might it be these? > > https://doi.org/10.1107/S0907444997013875 > > https://doi.org/10.1107/S0907444999016868 > > > Best wishes, > > Gerard > > -- > On Mon, Jun 23, 2025 at 03:33:36PM +0000, Randy John Read wrote: > > Hi, > > > > I think it was Ian Tickle who worked out the statistical error in R-free > > that > arises from taking different random samples, though I can’t track down the > publication quickly. Anyway, the take-home from this is that you don’t expect > all free sets to give the same R-free, and you shouldn’t over-interpret small > differences in R-free. > > > > Does anyone have the citation for this? > > > > Best wishes, > > > > Randy > > > > > On 23 Jun 2025, at 15:55, Oganesyan, Vaheh > <[email protected]> wrote: > > > > > > <!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; > > > panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 > 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Aptos;} /* Style Definitions */ > p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; font-size:12.0pt; > font-family:"Aptos",sans-serif;} a:link, span.MsoHyperlink {mso-style- > priority:99; color:blue; text-decoration:underline;} span.EmailStyle19 {mso- > style-type:personal-reply; font-family:"Aptos",sans-serif; color:windowtext;} > .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt; mso- > ligatures:none;} @page WordSection1 {size:8.5in 11.0in; margin:1.0in 1.0in > 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} --> Hi all, I’ve spent > some time going through different flags for free reflections for (somewhat > stupid) reason to get lower Rfree. I made sure all different flags were used > only for each of sets of refinement. I wanted to satisfy my inner belief that > it > doesn’t matter which flag is used (between 0 and 19). To be short: I failed. > > > Free reflections have been chosen “randomly”, no suspicion there. > However, there was clear difference at the end. The difference was about 2- > 3% difference in Rfree value. This tells me that the randomness has some > sort of rule, which makes “random” choice not so random. Having found this > I also tried to see “a rule” that breaks this randomness, like even numbers, > odds, etc. I did not. Because this was only on few cases I won’t even try to > connect it to # of molecules per AU, or SG. For each structure (I tried this > for > 2-3 structures about 10 years ago) it was a different flag. > > > I’m not sure if DCC also is looking through different flags, but at the > > > end it > finds the best, making these exercises unnecessary. > > > Sorry, cannot present a case. Was too long ago. > > > Vaheh Oganesyan, Ph.D.<image001.png>R&D | Biologics Engineering One > > > Medimmune Way, Gaithersburg, MD 20878 > > > T: 301-398-5851 > > > [email protected] > > > From: CCP4 bulletin board <[email protected]> On Behalf Of > > > Randy John Read > > > Sent: Monday, June 23, 2025 10:29 AM > > > To: [email protected] > > > Subject: Re: [ccp4bb] free R in shells Hi Ben, > > > > > > I would be very interested if you have a case where it makes a difference > to do this. At one point I was convinced that it had been important when we > were working on the structure of a Shiga-like toxin bound to trisaccharide > (1bos), with four pentamers in the asymmetric unit. However, Pavel Afonine > challenged me to show that the free set was less biased when chosen in > shells than when chosen randomly, and even in that relatively extreme case I > couldn’t see evidence of it. So it’s probably not worth the bother. Also, if > you > select the free set randomly, it’s distributed over the same resolutions as > the > working data, which arguably is important when you’re using it to calibrate > the sigma(A) estimates for likelihood targets. > > > > > > Best wishes, > > > > > > Randy > > > > > > > On 23 Jun 2025, at 12:35, Ben Bax <[email protected]> wrote: > > > > > > > > Hi, > > > > How do you select R-free in shells with CCP4? > > > > Thanks, Ben Bax > > > > > > > > > ############################################################### > ### > > > > ###### > > > > > > > > To unsubscribe from the CCP4BB list, click the following link: > > > > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > > > > > This message was issued to members of > > > > http://www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by > > > > http://www.jiscmail.ac.uk/, terms & conditions are available at > > > > https://www.jiscmail.ac.uk/policyandsecurity/ > > > > > > ----- > > > Randy J. Read > > > Department of Haematology, University of Cambridge Cambridge > > > Institute for Medical Research Tel: +44 1223 336500 The Keith Peters > > > Building Hills Road E-mail: [email protected] Cambridge CB2 0XY, U.K. > > > www-structmed.cimr.cam.ac.uk > > > > > > > > > > ############################################################### > ##### > > > #### > > > > > > To unsubscribe from the CCP4BB list, click the following link: > > > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > > > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > > > available at https://www.jiscmail.ac.uk/policyandsecurity/ > > > > > > Confidentiality Notice: This message is private and may contain > confidential and proprietary information. If you have received this message in > error, please notify us and remove it from your system and note that you > must not copy, distribute or take any action in reliance on it. Any > unauthorized use or disclosure of the contents of this message is not > permitted and may be unlawful. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ----- > > Randy J. Read > > Department of Haematology, University of Cambridge > > Cambridge Institute for Medical Research Tel: +44 1223 336500 > > The Keith Peters Building > > Hills Road E-mail: > > [email protected] > > Cambridge CB2 0XY, U.K. > > www-structmed.cimr.cam.ac.uk > > > > > > > ############################################################### > ####### > > ## > > > > To unsubscribe from the CCP4BB list, click the following link: > > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > > available at https://www.jiscmail.ac.uk/policyandsecurity/ > > ############################################################### > ######### > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at > https://www.jiscmail.ac.uk/policyandsecurity/ ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
