Re: [ccp4bb] Does NCS bias a randomly-chosen test set (even if not enforced)? [ccp4bb] an over refined structure

2008-02-21 Thread Edward Berry
Dale Tronrud wrote: In summary, this argument depends on two assertions that you can argue with me about: 1) When a parameter is being used to fit the signal it was designed for, the resulting model develops predictive power and can lower both the working and free R. When a signal is per

Re: [ccp4bb] an over refined structure

2008-02-12 Thread Edward Berry
Dale Tronrud wrote: In summary, this argument depends on two assertions that you can argue with me about: 1) When a parameter is being used to fit the signal it was designed for, the resulting model develops predictive power and can lower both the working and free R. When a signal

Re: [ccp4bb] an over refined structure

2008-02-12 Thread Dale Tronrud
Edward Berry wrote: Dirk Kostrewa wrote: Dear Dean and others, Peter Zwart gave me a similar reply. This is very interesting discussion, and I would like to have a somewhat closer look to this to maybe make things a little bit clearer (please, excuse the general explanations - this might be

Re: [ccp4bb] an over refined structure

2008-02-11 Thread Bart Hazes
Dale Tronrud wrote: Bart Hazes wrote: Dale Tronrud wrote: [EMAIL PROTECTED] wrote: > Rotational near-crystallographic ncs is easy to handle this way, but > what about translational pseudo-symmetry (or should that be > pseudo-translational symmetry)? In such cases one whole set of spots is

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Dale Tronrud
Bart Hazes wrote: Dale Tronrud wrote: [EMAIL PROTECTED] wrote: > Rotational near-crystallographic ncs is easy to handle this way, but > what about translational pseudo-symmetry (or should that be > pseudo-translational symmetry)? In such cases one whole set of spots is > systematically wea

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Bart Hazes
Dale Tronrud wrote: [EMAIL PROTECTED] wrote: > Rotational near-crystallographic ncs is easy to handle this way, but > what about translational pseudo-symmetry (or should that be > pseudo-translational symmetry)? In such cases one whole set of spots is > systematically weaker than the other se

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Dale Tronrud
[EMAIL PROTECTED] wrote: > Rotational near-crystallographic ncs is easy to handle this way, but > what about translational pseudo-symmetry (or should that be > pseudo-translational symmetry)? In such cases one whole set of spots is > systematically weaker than the other set. Then what is the > "t

Re: [ccp4bb] an over refined structure

2008-02-08 Thread price
Rotational near-crystallographic ncs is easy to handle this way, but what about translational pseudo-symmetry (or should that be pseudo-translational symmetry)? In such cases one whole set of spots is systematically weaker than the other set. Then what is the "theoretically correct" way to cal

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Axel Brunger
In such cases, we always define the test set first in the high-symmetry space group choice. Then, if it is warranted to lower the crystallographic symmetry and replace with NCS symmetry, we expand the test set to the lower symmetry space group. In other words, the test set itself will be inva

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Ronald E Stenkamp
I've looked at about 10 cases where structures have been refined in lower symmetry space groups. When you make the NCS operators into crystallographic operators, you don't change the refinement much, at least in terms of structural changes. That's the case whether NCS restraints have been appl

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Sue Roberts
Back in the old days, when I worked on crystal structures with 15 or 20 atoms or so, the symptoms of missed crystallographic symmetry included instability of the refinement, high correlations between parameters, and (relatively) large deviations between equivalent bond distances and bond an

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Edward Berry
Dirk Kostrewa wrote: Dear Dean and others, Peter Zwart gave me a similar reply. This is very interesting discussion, and I would like to have a somewhat closer look to this to maybe make things a little bit clearer (please, excuse the general explanations - this might be interesting for begin

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Ian Tickle
s), and thus miss potentially valuable information. -- Ian > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dirk Kostrewa > Sent: 08 February 2008 10:10 > To: CCP4BB > Subject: Re: [ccp4bb] an over refined structure > > Dear Da

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Dirk Kostrewa
ECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] an over refined structure I agree that the difference in Rwork to Rfree is quite acceptable at your resolution. You cannot/ should not

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Dale Tronrud
andom selection or Rfree + NCS (especially 4x or higher) will drive Rfree down unfairly. Doug Ohlendorf -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.

Re: [ccp4bb] an over refined structure

2008-02-08 Thread Dirk Kostrewa
Ohlendorf -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] an over refined structure I agree that the difference

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Phil Jeffrey
If you think about it, there is an analogy to relaxing geometrical constraints, which also allows the refinement to put atoms into "density". The reason it usually doesn't help Rfree is that the density is spurious. At least some of the incorrect structure determinations of the early 90's (that

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Dean Madden
Hi Phil, Here I will disagree. R-free rewards you for putting in atom in density which an atom belongs in. It doesn't necessarily reward you for putting the *right* atom in that density, but it does become difficult to do that under normal circumstances unless you have approximately the righ

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Edward Berry
Dean Madden wrote: Hi Ed, This is an intriguing argument, but I know (having caught such a case as a reviewer) that even in cases of low NCS symmetry, Rfree can be significantly biased. I think the reason is that the discrepancy between pairs of NCS-related reflections (i.e. Fo-Fo') is genera

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Edward Berry
Agreed, and this is even more true if you consider R-merge is calculated on I's and Rfree on F's, Rmerge of 5% should contribute 2.5% to Rfree; and furthermore errors add vectorially so it would be more like ,025/sqrt(2). I guess I have to take all those other errors that have to do with the inab

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Dean Madden
Hi Ed, This is an intriguing argument, but I know (having caught such a case as a reviewer) that even in cases of low NCS symmetry, Rfree can be significantly biased. I think the reason is that the discrepancy between pairs of NCS-related reflections (i.e. Fo-Fo') is generally significantly s

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Phil Jeffrey
al Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] an over refined structure I agree that the difference in Rwork to Rfree

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Jon Wright
Dear Ed, I don't see how you "decouple" symmetry mates in the case of a wrong space group. Symmetry mates should agree with each other typically within "R_sym" or "R_merge" percent, eg; about 2-5% . Observed and calculated reflections agree within "R_Factor" of each other, so about 20-30%. Th

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Dean Madden
mix Rwork and Rfree sets. Normal random selection or Rfree + NCS (especially 4x or higher) will drive Rfree down unfairly. Doug Ohlendorf -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4B

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Edward Berry
Actually the bottom lines below were my argument in the case that you DO apply strict NCS (although the argument runs into some questionable points if you follow it out). In the case that you DO NOT apply NCS, there is a second decoupling mechanism: Not only the error in Fo may be opposite for th

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Phil Jeffrey
sage- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] an over refined structure I agree that the difference in Rwork to Rfree is quite ac

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Edward Berry
Dean Madden wrote: Hi Dirk, I disagree with your final sentence. Even if you don't apply NCS restraints/constraints during refinement, there is a serious risk of NCS "contaminating" your Rfree. Consider the limiting case in which the "NCS" is produced simply by working in an artificially low

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Axel Brunger
---Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] an over refined structure I agree that the difference in Rwork to Rf

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Anastassis Perrakis
Bottom line: thin shells are not a perfect solution, but if NCS is present, choosing the free set randomly is *never* a better choice, and almost always significantly worse. hmmm ... I wonder if that is true. For low order NCS (two- three- fold, even five-fold) I don't believe that thin she

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Dean Madden
Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] an over refined structure I agree that the difference in Rwork to Rfree is quite acceptable at your resolution. You cannot/ should not use Rfactors as a criteria for structure correctn

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Dirk Kostrewa
m selection or Rfree + NCS (especially 4x or higher) will drive Rfree down unfairly. Doug Ohlendorf -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] an ov

Re: [ccp4bb] an over refined structure

2008-02-07 Thread Doug Ohlendorf
-Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Eleanor Dodson Sent: Tuesday, February 05, 2008 3:38 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] an over refined structure I agree that the difference in Rwork to Rfree is quite acceptable at your resolution

Re: [ccp4bb] an over refined structure

2008-02-05 Thread Ian Tickle
have opinions on how many waters you expect to find at various resolutions. HTH -- Ian > -Original Message- > From: Sun Tang [mailto:[EMAIL PROTECTED] > Sent: 04 February 2008 22:32 > To: Ian Tickle > Cc: CCP4BB@JISCMAIL.AC.UK > Subject: RE: [ccp4bb] an over refined s

Re: [ccp4bb] an over refined structure

2008-02-05 Thread Eleanor Dodson
I agree that the difference in Rwork to Rfree is quite acceptable at your resolution. You cannot/ should not use Rfactors as a criteria for structure correctness. As Ian points out - choosing a different Rfree set of reflections can change Rfree a good deal. certain NCS operators can relate refl

Re: [ccp4bb] an over refined structure

2008-02-05 Thread Clemens Vonrhein
Hi Sun, On Mon, Feb 04, 2008 at 02:15:05PM -0800, Sun Tang wrote: > I used NCS before rigid body refinement. After that I did not put > NCS restraints in the restrained refinement and TLS+restrained > refinement because it raised the R/Rfree quite a lot. Use NCS. Really! There is never a reason

Re: [ccp4bb] an over refined structure

2008-02-04 Thread Sun Tang
porting I think the latter is very unlikely, possibly you just need to tweak the X-ray and/or restraint weights. HTH Cheers -- Ian > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Sun Tang > Sent: 04 February 2008 16:56 > To: Boaz Shaana

Re: [ccp4bb] an over refined structure

2008-02-04 Thread Sun Tang
Hi Anastassis, Thank you very much for your suggestions. I answered the questions as follows. I used NCS before rigid body refinement. After that I did not put NCS restraints in the restrained refinement and TLS+restrained refinement because it raised the R/Rfree quite a lot. The resolution is

[ccp4bb] an over refined structure

2008-02-04 Thread George M. Sheldrick
Dear Sun, If we take Ian's formula for the ratio of R(free) to R(work) from his paper Acta D56 (2000) 442-450 and make some reasonable approximations, we can reformulate it as: R(free)/R(work) = sqrt[(1+Q)/(1-Q)] with Q = 0.025pd^3(1-s) where s is the fractional solvent content, d is the reso

Re: [ccp4bb] an over refined structure

2008-02-04 Thread Ian Tickle
you are reporting I think the latter is very unlikely, possibly you just need to tweak the X-ray and/or restraint weights. HTH Cheers -- Ian > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Sun Tang > Sent: 04 February 2008 16

Re: [ccp4bb] an over refined structure

2008-02-04 Thread price
What is the resolution? Are you using ncs restraints (you probably should, on the coordinates but not on the Bs)? How does your Ramachandran plot look? You might tighten the geometry even more. Aside from all theoretical arguments about how big the rmsd's should be, if they're rather loose, t

Re: [ccp4bb] an over refined structure

2008-02-04 Thread Anastassis Perrakis
Hi - I don't think there is something necessarily wrong with the values you report. A few questions to see *if* something is wrong are: - as you wrote to Tim you have NCS: do you use NCS restraints ? - what is the resolution / B factor of the data ? - have the data been checked for twining ?

Re: [ccp4bb] an over refined structure

2008-02-04 Thread Sun Tang
Hi Tim, Thank you for your and information and suggestions. There are two indepdent molecules in the asymmetric unit and one molecule does not have very good density, especially in the N-terminus. Do you think that I should remove the region in the refinement? Best, Sun Tim Gruene <[EMAIL P

Re: [ccp4bb] an over refined structure

2008-02-04 Thread Tim Gruene
I would agree that the difference is suspiciously high. I. Tickle and others have published analytical expressions for how to estimate the ratio between R and Rfree, just google for "tickle rfree" to find the references. You easily achieve a large difference by adding too many waters which jus

Re: [ccp4bb] an over refined structure

2008-02-04 Thread Sun Tang
amachandran outliers, etc., map) are OK you should just be happy with your structure. Cheers, Boaz - Original Message - From: Sun Tang <[EMAIL PROTECTED]> Date: Monday, February 4, 2008 18:41 Subject: [ccp4bb] an over refined structure To: CCP4BB@JISCMAIL.A

[ccp4bb] an over refined structure

2008-02-04 Thread Sun Tang
Hello All, I refined a structure with Refmac in CCP4i and the R/Rfree is 0.215/0.277. The difference between R and Rfree is too much even though I used 0.01 for weighting term in the refinement (the default value is 0.3). The RMSD for bond length and bond angle is 0.016 A and 1.7 degree. What