Dear all, I agree with Ian that the term resolution has a useful definition in crystallography, largely referring to the set up of the diffraction instrument. I also agree with James' point that the diffraction limit is not the same as the error bar on position but I didn't know anyone used it for this and certainly not the astronomers (one could look at the requirements for LIGO if one wants to see extremely small changes in distances, but that refers to an instrument again). I am not confident that one can get the community to use the term diffraction limit for how far useful information extends in reciprocal space. It is great to see an information based approach being used for some of the metrics used to measure this quantity whatever it is called but, as Randy says, there can be information beyond any limit given by a single threshold.
My main concern is whether adopting some of these correlation coefficient based terms for other fields, as Marin suggested, would be particularly useful. Even in crystallography they could be misleading if one wants to identify small changes, or low contrast objects, in the electron density. For example, a special measure was developed to handle the low contrast given by small changes in anomalous scattering (CCanom). This approach would not always be possible for cases such as low occupancy of a bound small molecule. Remaining within structural biology, there is increasing interest to use techniques such as electron tomography to identify cellular structure, including the positions of particular proteins with respect to each other and to organelles within the cell. A large variety of different contrasts is likely to be present and the distribution of contrasts could change during the life cycle (e.g. condensed versus uncondensed chromatin). If one doesn't know the density distribution as a function of spatial frequency one could be mislead by some of these metrics. However, I presume the FSC based metrics could be adopted for in in situ structure determination of proteins via electron tomography. Normally, in statistics, a single number on its own is not particularly useful. Colin -----Original Message----- From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> On Behalf Of Randy John Read Sent: Thursday, October 10, 2024 11:02 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Review: Linearity and Resolution in X-Ray Crystallography and Electron Microscopy [You don't often get email from 0000eafc0efa8263-dmarc-requ...@jiscmail.ac.uk. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Dear Charles, Thanks for the kind words, and thanks for the links to your papers! I hadn’t run across them before — they look very interesting and relevant, but it will take a bit of time to digest them! Randy > On 9 Oct 2024, at 21:36, Sindelar, Charles <charles.sinde...@yale.edu> wrote: > > <!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; > panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Aptos; panose-1:2 11 0 > 4 2 2 2 2 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, > div.MsoNormal {margin:0in; font-size:12.0pt; font-family:"Aptos",sans-serif;} > a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; > text-decoration:underline;} p.MsoListParagraph, li.MsoListParagraph, > div.MsoListParagraph {mso-style-priority:34; margin-top:0in; > margin-right:0in; margin-bottom:0in; margin-left:.5in; font-size:12.0pt; > font-family:"Aptos",sans-serif;} .MsoChpDefault {mso-style-type:export-only; > font-size:10.0pt; mso-ligatures:none;} @page WordSection1 {size:8.5in 11.0in; > margin:1.0in 1.0in 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} /* > List Definitions */ @list l0 {mso-list-id:254442374; > mso-list-template-ids:513585478;} @list l1 {mso-list-id:1398825241; > mso-list-template-ids:-1688579456;} @list l1:level1 {mso-level-start-at:2; > mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in;} > @list l2 {mso-list-id:2038388914; mso-list-type:hybrid; > mso-list-template-ids:-4959192 531246552 67698713 67698715 67698703 67698713 > 67698715 67698703 67698713 67698715;} @list l2:level1 > {mso-level-text:"\(%1\)"; mso-level-tab-stop:none; > mso-level-number-position:left; text-indent:-.25in;} @list l2:level2 > {mso-level-number-format:alpha-lower; mso-level-tab-stop:none; > mso-level-number-position:left; text-indent:-.25in;} @list l2:level3 > {mso-level-number-format:roman-lower; mso-level-tab-stop:none; > mso-level-number-position:right; text-indent:-9.0pt;} @list l2:level4 > {mso-level-tab-stop:none; mso-level-number-position:left; > text-indent:-.25in;} @list l2:level5 {mso-level-number-format:alpha-lower; > mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in;} > @list l2:level6 {mso-level-number-format:roman-lower; > mso-level-tab-stop:none; mso-level-number-position:right; > text-indent:-9.0pt;} @list l2:level7 {mso-level-tab-stop:none; > mso-level-number-position:left; text-indent:-.25in;} @list l2:level8 > {mso-level-number-format:alpha-lower; mso-level-tab-stop:none; > mso-level-number-position:left; text-indent:-.25in;} @list l2:level9 > {mso-level-number-format:roman-lower; mso-level-tab-stop:none; > mso-level-number-position:right; text-indent:-9.0pt;} ol {margin-bottom:0in;} > ul {margin-bottom:0in;} --> Randy, what a beautiful paper! I love it. I’m > also a huge fan of the you guys’ resolve_cryo_em density modification paper, > which had a major impact on my group’s ability to interpret several > challenging structures. > Many moons ago when I was a postdoc with Niko Grigorieff, I got interested > in this problem of Fourier over-sampling and we published a couple papers > working out the correction factor (particle/volume ratio) and its > relationship to FSC’s and the Wiener filter. > https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/22613568/ > https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/21757012/ > Two of our results I thought were especially fun, and maybe not > commonly appreciated even now, were > > • one can estimate the particle/volume ratio just by doing signal > analysis on a small interior region of the 3D map > > • the ‘ad hoc’ input parameter T of Relion, still required as user > input in the current version for 3D classification, corresponds to the > inverse of this correction factor Best wishes, Chuck > From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of > Randy John Read <0000eafc0efa8263-dmarc-requ...@jiscmail.ac.uk> > Date: Tuesday, October 8, 2024 at 4:07 AM > To: CCP4BB@JISCMAIL.AC.UK <CCP4BB@JISCMAIL.AC.UK> > Subject: Re: [ccp4bb] Review: Linearity and Resolution in X-Ray > Crystallography and Electron Microscopy Dear Marin, > > In crystallography we do have the information gain measure (based on > Kullback-Leibler divergence) that my group put forward and implemented in our > Phaser program (https://doi.org/10.1107/s2059798320001588). Signal and noise > aren’t isotropic, so information gain isn’t isotropic either. However, we’ve > observed that the resolution at which the average information gain is about > 1/2 bit per reflection corresponds roughly to the resolution limits suggested > by other techniques. Given the interpretation of information gain as the > maximum log-likelihood-gain that one could achieve from an observation with a > perfect model, it’s a very natural measure to use for the useful resolution. > I don’t think this measure has gained much traction in the crystallographic > community yet, but it’s becoming more widely available in some data analysis > tools. > > We’ve used the same KL-divergence approach to estimate the information gain > from a Fourier term in a cryo-EM reconstruction > (https://doi.org/10.1107/s2059798323001596). In the implementation of this in > our EM-placement docking software, we have anisotropic estimates of signal > and noise, so again the information gain is anisotropic. Somewhat to my > surprise (given the differences in the derivations), our information gain > measure turns out to be equivalent to yours > (https://doi.org/10.48550/arXiv.2009.03223) if we assume that the signal and > noise are isotropic. As you point out there, for cryo-EM reconstructions it’s > essential to consider the effect of over-sampling of the Fourier transform > and the corresponding lack of independence of the Fourier terms, so this has > an over-sampling correction factor. > > Best wishes, > > Randy Read > > > On 8 Oct 2024, at 00:02, Marin van Heel <marin.vanh...@gmail.com> wrote: > > > > Dear Marius Schmidt > > > > In my (our) original FRC/FSC papers (1982; 1986 ; 2000; 2004; 2017; 2020; > > 2024) the linearity of these correlation functions/metrics have been > > extensively discussed. Historically, EM started at a low resolution > > "blobology" level whereas X-ray crystallography (XRC) at that time, already > > had reached atomic resolution. This led to the belief that the XRC > > resolution metrics ( like phase residuals and R-factors) were also > > appropriate as resolution metrics for EM. However, in XRC the measurables > > are diffraction patterns for which amplitudes corresponding phases had to > > be derived iteratively. In EM and in imagining in general, the measurables > > are the images themselves, that contain both the amplitude information and > > the phase information. To revert to the then already established XRC > > resolution metrics like phase residuals or R-factors, implied discarding > > the most important part of the available information (see the Why-O-Why ). > > (https://www.linkedin.com/posts/marin-van-heel-5845b422b_whyowhyarchive-activity-7149738255154946048-Oc93/?utm_source=share&utm_medium=member_desktop). > > That problem was realized soon and the mentioned FRC and FSC metrics were > > thus suggested which exploit all the available information. Thus, the XRC > > atomic resolution technique of the 1980s came with a low-quality resolution > > metric whereas the Cryo-EM low-resolution blobology approach of the 1980s > > came with a high-quality resolution metric. > > Thus, in summary, all resolution criteria in XRC are ad-hoc > > non-linear metrics that have no general validity outside of XRC. Looking at > > only the amplitudes of a diffraction pattern is like finding the highest > > resolution spot in a diffraction pattern, where, even if the spot is > > clearly visible, that does not mean one would be able to find its phase. We > > need a more comprehensive metric that has a wide range of applicability. > > In other words, where a CC1-2 metric cannot be applied to assess the 3D > > brain scan of a brain-tumor patient, the FRC / FSC, and the newest FRI / > > FSI metrics can be applied in all cases where 2D and 3D data are dealt with! > > Hope this helps, > > > > Marin van Heel > > > > On Mon, Oct 7, 2024 at 3:04 PM Marius Schmidt <smar...@uwm.edu> wrote: > > I think this is taken care of: > > The CC1/2 and the CC1/2* are appropriate metrics for the resolution limit. > > They are all spit out by newer data processing software. > > The CC1/2 is directly comparable to the FSC. Many people use CC1/2 = > > 1/e as the resolution limit. > > In many cases of data the CC1/2 = 1/e is equivalent to I/sigI of 1, > > which is used sometimes as a metric for the resolution limit (some > > use I/sigI = 2), and in more cases the CC1/2 corresponds to Rmerge in the > > range of 40%. > > For serial crystallography, the R-split goes through the roof at > > CC1/2 = 1/e, so the CC1/2 is the better metric. > > > > Best > > Marius > > > > > > > > > > > > Marius Schmidt, Dr. rer. Nat. (habil.) Professor University of > > Wisconsin-Milwaukee Kenwood Interdisciplinary Research Complex > > Physics Department, Room 3087 > > 3135 North Maryland Avenue > > Milwaukee, Wi 53211 > > phone (office): 1-414-229-4338 > > phone (lab): 414-229-3946 > > email: smar...@uwm.edu > > https://uw/ > > m.edu%2Fphysics%2Fpeople%2Fschmidt-marius%2F&data=05%7C02%7Ccolin.na > > ve%40DIAMOND.AC.UK%7Cff13b539bc6847e073ea08dce91318b5%7C9d27ba740100 > > 4d0d81ff1d728dae8df6%7C0%7C0%7C638641516630075137%7CUnknown%7CTWFpbG > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn > > 0%3D%7C60000%7C%7C%7C&sdata=yYSSFAmrsZa7W1yLs3PaWb5KogW9OD48ewdSxhxz > > mgI%3D&reserved=0 > > https://si/ > > tes.uwm.edu%2Fsmarius%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7 > > Cff13b539bc6847e073ea08dce91318b5%7C9d27ba7401004d0d81ff1d728dae8df6 > > %7C0%7C0%7C638641516630091636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj > > AwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7 > > C&sdata=bZVKWy7bWOyCJUgwuEV5Zf7AP1nObYb%2BDPROwJ5TCls%3D&reserved=0 > > https://ww/ > > w.bioxfel.org%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cff13b53 > > 9bc6847e073ea08dce91318b5%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0 > > %7C638641516630105084%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC > > JQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata= > > yRISJ3NYO2TnvO%2BG1gZISrDz4qw9ShYLi%2BEQVDM02Fo%3D&reserved=0 > > Nature News and Views: > > https://ww/ > > w.nature.com%2Farticles%2Fd41586-023-00504-4&data=05%7C02%7Ccolin.na > > ve%40DIAMOND.AC.UK%7Cff13b539bc6847e073ea08dce91318b5%7C9d27ba740100 > > 4d0d81ff1d728dae8df6%7C0%7C0%7C638641516630119767%7CUnknown%7CTWFpbG > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn > > 0%3D%7C60000%7C%7C%7C&sdata=3FTCMPklwOm3Low%2ButTsPY2LEhp%2Bx7uAw7zR > > js9kVZ4%3D&reserved=0 > > > > From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Marin > > van Heel <marin.vanh...@gmail.com> > > Sent: Monday, October 7, 2024 11:24 AM > > To: CCP4BB@JISCMAIL.AC.UK <CCP4BB@JISCMAIL.AC.UK> > > Subject: [ccp4bb] Review: Linearity and Resolution in X-Ray > > Crystallography and Electron Microscopy Dear All, > > > > Sayan Bhakta and I have recently posted the preprint of a review on > > resolution and linearity which will appear in a book to be launched on the > > 16th of October 2024. > > ( https://doi.org/10.1201/9781003326106 ). It is the first Cryo-EM review > > that I have been involved in for 25 years. > > In our preparation, I was quite amazed about what other authors wrote (or > > did not write) in their many reviews on these matters. > > For example, I missed any serious discussion about resolution metrics in > > X-ray crystallography, which technique is fundamentally non-linear. > > Linearity is a prerequisite for defining the resolution of any instrument. > > The iterative refinements applied in X-ray crystallography (and sometimes > > Cryo-EM) makes that all Phase-residuals and R-factors or fixed threshold > > values cannot be used to compare the results of independently conducted > > experiments. What is an obvious consequence of the lack of universality of > > such metrics like phase-residuals and R-factors, is that they cannot be > > used outside of the immediate context in which they were defined, like > > X-ray crystallography or structural biology. In contrast, the > > Fourier-Ring-Correlation (FRC); Fourier-Shell-Correlation (FSC) and their > > recent successors: the Fourier-Ring-Information (FRI) and the > > Fourier-Shell-Information (FSI), plus their integrated versions, are > > universal metrics that are applicable to all fields of science where 2D and > > 3D data are dealt with! > > > > https://do/ > > i.org%2F10.31219%2Fosf.io%2F5empt&data=05%7C02%7Ccolin.nave%40DIAMON > > D.AC.UK%7Cff13b539bc6847e073ea08dce91318b5%7C9d27ba7401004d0d81ff1d7 > > 28dae8df6%7C0%7C0%7C638641516630144995%7CUnknown%7CTWFpbGZsb3d8eyJWI > > joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C6000 > > 0%7C%7C%7C&sdata=kYOAHs8jpJCh3yr7%2BcA%2Bi%2BpJqp6nW1MPGdTYyiRxBJA%3 > > D&reserved=0 > > > > Have fun reading it! > > > > Marin > > > > > > > > > > > > To unsubscribe from the CCP4BB list, click the following link: > > https://ww/ > > w.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DCCP4BB%26A%3D1&d > > ata=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cff13b539bc6847e073ea08dce > > 91318b5%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638641516630159 > > 954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB > > TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata=nCsRT806CHKFWLchth > > XiCfUWNg%2B2vT1vNK4rSRibqYE%3D&reserved=0 > > To unsubscribe from the CCP4BB list, click the following link: > > https://ww/ > > w.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DCCP4BB%26A%3D1&d > > ata=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cff13b539bc6847e073ea08dce > > 91318b5%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638641516630173 > > 414%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB > > TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata=iRKp%2FLpCWk1ZsnBO > > j9YqsGv7qQCqJRw4z3mnWrNNPAY%3D&reserved=0 > > ----- > Randy J. Read > Department of Haematology, University of Cambridge > Cambridge Institute for Medical Research Tel: +44 1223 336500 > The Keith Peters Building > Hills Road E-mail: > rj...@cam.ac.uk > Cambridge CB2 0XY, U.K. > www-structmed.cimr.cam.ac.uk > > > ###################################################################### > ## > > To unsubscribe from the CCP4BB list, click the following link: > https://www/. > jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DCCP4BB%26A%3D1&data= > 05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cff13b539bc6847e073ea08dce91318b > 5%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638641516630186690%7CUn > known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW > wiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata=ZY2T6EYgs%2B8lwBWpjUNNGnv7LSfO > FE3LX1WBqUGirvk%3D&reserved=0 > > This message was issued to members of > http://www.j/ > iscmail.ac.uk%2FCCP4BB&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cff1 > 3b539bc6847e073ea08dce91318b5%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7 > C0%7C638641516630200397%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC > JQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata=GG > x9T4jpaRZ8CJV3PXjMnMgN5ikgyP6QOMrVIIVRk94%3D&reserved=0, a mailing > list hosted by > http://www.j/ > iscmail.ac.uk%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cff13b539b > c6847e073ea08dce91318b5%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C6 > 38641516630212868%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata=bf4a5W3d > z7Zi65vjq1L3SQJobnidw5iZ21IGC%2FtmOsk%3D&reserved=0, terms & > conditions are available at > https://www/. > jiscmail.ac.uk%2Fpolicyandsecurity%2F&data=05%7C02%7Ccolin.nave%40DIAM > OND.AC.UK%7Cff13b539bc6847e073ea08dce91318b5%7C9d27ba7401004d0d81ff1d7 > 28dae8df6%7C0%7C0%7C638641516630224812%7CUnknown%7CTWFpbGZsb3d8eyJWIjo > iMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C > %7C%7C&sdata=WW7O3WFjwE2iE6jt7iMj%2FlruzLyo4c%2BsZYKrqJhXpe4%3D&reserv > ed=0 ----- Randy J. Read Department of Haematology, University of Cambridge Cambridge Institute for Medical Research Tel: +44 1223 336500 The Keith Peters Building Hills Road E-mail: rj...@cam.ac.uk Cambridge CB2 0XY, U.K. www-structmed.cimr.cam.ac.uk ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of http://www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by http://www.jiscmail.ac.uk/, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom. ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
