All, the first hurdle will of course be whether the AlphaFold model works as a MR model, even with the 100% completeness and sequence identity of a bespoke model. The question is what B factors to use or which disordered bits to leave out, as that can strongly influence the result (perhaps use info from a similar structure). If it doesn't work in MR that's a pretty good indication that it's too far from reality to be useful for looking at detailed interactions.
Does anyone know of a systematic investigation of the success rate of AlphaFold models in MR ? That would be useful ammunition ! Cheers -- Ian On Sun, 2 Apr 2023 at 11:51, Gerard Bricogne <[email protected]> wrote: > Dear all, > > I think that quoting general viewpoints and statements, however > knowledgeable and respected their authors may be, will only exacerbate the > climate of clashing prejudices between two camps and is bound to sustain a > war of opinions rather than lead to a rational acceptance that something > has > changed. The frustration is that one camp (the AlphaFold believers) can be > viewed as in effect preventing experiments that could prove it wrong. > > One way to deal with this obstruction would be to provide, in each > particular case, evidence that the AlphaFold results "do not cut it" as the > sole provider of 3D information within the project at hand. This means that > every grant proposal requesting resources towards a crystallographic > structure solution should document the fact that AlphaFold predictions have > been performed (or, often, looked up in a database of pre-cooked results) > but do not provide the accuracy required for the proposed investigation. If > this step of writing up the "Background" section of the grant actually > delivers a useful result, then everyone will be happy; and if it doesn't, > then the case for the need to allocate resources to solving the structure > by > crystallography will be unassailable. In this way, AlphaFold will be a game > changer (we have known that since July 2021) but not a game killer. Savvas > alluded to a similar approach, but it could be made a formal requirement > acceptable to both proposers and reviewers, who would then both be dealing > with the situation in a scientific rather than dogmatic manner. > > > With best wishes, > > Gerard. > > -- > On Sat, Apr 01, 2023 at 06:54:33PM +0000, Goldman, Adrian wrote: > > I think this is all true - and I’ve been putting things like this into > my (failing) grants - but I get the dispiriting sense that the medics think > (to borrow a line from hamlet) “the applicant doth protest too much > methinks”. > > > > Well if as per James H today ;), we deposit coordinates to 1sf, > alphafold will be just fine. > > > > Of course the coordinates won’t be of any use to anybody, but the > pictures will be nice. > > > > Adrian > > > > Sent from my iPhone > > > > > On 1 Apr 2023, at 21:39, Randy John Read <[email protected]> wrote: > > > > > > There’s also this preprint with Tom Terwilliger as lead author: > https://www.biorxiv.org/content/10.1101/2022.11.21.517405v1. The title is > “AlphaFold predictions: great hypotheses but no match for experiment”. > > > > > > Best wishes, > > > > > > Randy > > > > > >> On 1 Apr 2023, at 18:18, Savvas Savvides < > [email protected]> wrote: > > >> > > >> Dear Rams, > > >> > > >> I salute you for sharing this. > > >> > > >> Just a week ago, I also received a remark along these lines on a > declined grant application. The remark was the only unfavourable point, > which suggested that it must have weighed disproportionally towards the > negative outcome. This was a two-stage evaluation process and the grant was > cut in stage-1 where it was evaluated by a small group of evaluators, none > of whom was a structural biologist/biochemist. Stage-2 would have involved > peer review by international experts. > > >> > > >> Despite my initial disbelief about what this remark might have caused > and upon reflection, I realized that it might be time to become proactive > in future applications in anticipation of the apparent growing trend > towards such remarks and perceptions. > > >> > > >> I think that a generalized form of preemptive text might not serve > the purpose well, but perhaps well-articulated statements specific to the > proposed biological problem at hand (perhaps aided by illustrations > demonstrating the inability of structure prediction to address the problem > at hand) might be the better way to go. Even though many of us who teach > courses in experimental structural biology and structural bioinformatics at > undergraduate and graduate levels are already actively addressing many of > these issues, there is a much bigger and far more senior scientific > population out there that makes important decisions on science > policy/funding/infrastructures/evaluations/recruitment/etc that are not > getting such educational exposure. > > >> > > >> The following resources provide good material and starting points to > reflect and elaborate upon. > > >> > > >> The article by Perrakis and Sixma in EMBO Reports > https://www.embopress.org/doi/full/10.15252/embr.202154046 > > >> > > >> The recent comment paper in Nature Methods by Thomas Jane > > >> https://doi.org/10.1038/s41592-022-01760-4 > > >> > > >> A correspondence in Science by Moore, Hendrickson, Henderson and > Brunger > > >> > https://www.science.org/doi/10.1126/science.abn9422?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%20%200pubmed > > >> > > >> > > >> Best wishes > > >> Savvas > > >> > > >> > > >> ---- > > >> Savvas Savvides > > >> VIB Center for Inflammation Research > > >> Ghent University, Dept. of Biochemistry & Microbiology > > >> Technologiepark 71, 9052 Ghent, Belgium > > >> > > >> Email: [email protected] ; [email protected] > > >> Phone: +32 (0)472 928 519 (mobile) ; +32 (0)9 331 36 60 (office) > > >> Web: https://savvideslab.sites.vib.be/en#/ > > >> > > >>>> On 1 Apr 2023, at 16:57, Subramanian, Ramaswamy < > [email protected]> wrote: > > >>> > > >>> Ian, > > >>> > > >>> Thank you. This is not an April fools.. > > >>> Rams > > >>> [email protected] > > >>> > > >>> > > >>> > > >>>> On Apr 1, 2023, at 10:46 AM, Ian Tickle <[email protected]> wrote: > > >>>> > > >>>> ---- External Email: Use caution with attachments, links, or > sharing data ---- > > >>>> > > >>>> > > >>>> Hi Ramaswamy > > >>>> > > >>>> I assume this is an April Fool's but it's still a serious question > because many reviewers who are not crystallographers or electron > microscopists may not fully appreciate the difference currently between the > precision of structures obtained by experimental and predictive methods, > though the latter are certainly catching up. The answer of course lies in > the mean co-ordinate precision, related to the map resolution. > > >>>> > > >>>> Quoting https://people.cryst.bbk.ac.uk/~ubcg05m/precgrant.html : > > >>>> > > >>>> "The accuracy and precision required of an experimentally > determined model of a macromolecule depends on the biological questions > being asked of the structure. Questions involving the overall fold of a > protein, or its topological similarity to other proteins, can be answered > by structures of fairly low precision such as those obtained from very low > resolution X-ray crystal diffraction data [or AlphaFold]. Questions > involving reaction mechanisms require much greater accuracy and precision > as obtained from well-refined, high-resolution X-ray structures, including > proper statistical analyses of the standard uncertainties (s.u.'s) of > atomic positions and bond lengths.". > > >>>> > > >>>> According to https://www.nature.com/articles/s41586-021-03819-2 : > > >>>> > > >>>> The accuracy of AlphaFold structures at the time of writing (2021) > was around 1.0 Ang. RMSD for main-chain and 1.5 Ang. RMSD for side-chain > atoms and probably hasn't changed much since. This is described as "highly > accurate"; however this only means that AlphaFold's accuracy is much higher > in comparison with other prediction methods, not in comparison with > experimental methods. Also note that AlphaFold's accuracy is estimated by > comparison with the X-ray structure which remains the "gold standard"; > there's no way (AFAIK) of independently assessing AlphaFold's accuracy or > precision. > > >>>> > > >>>> Quoting https://scripts.iucr.org/cgi-bin/paper?S0907444998012645 : > > >>>> > > >>>> "Data of 0.94 A resolution for the 237-residue protein concanavalin > A are used in unrestrained and restrained full-matrix inversions to provide > standard uncertainties sigma(r) for positions and sigma(l) for bond > lengths. sigma(r) is as small as 0.01 A for atoms with low Debye B values > but increases strongly with B." > > >>>> > > >>>> There's a yawning gap between 1.0 - 1.5 Ang. and 0.01 Ang.! > Perhaps AlphaFold structures should be deposited using James Holton's new > PDB format (now that is an April Fool's !). > > >>>> > > >>>> One final suggestion for a reference in your grant application: > https://www.biorxiv.org/content/10.1101/2022.03.08.483439v2 . > > >>>> > > >>>> Cheers > > >>>> > > >>>> -- Ian > > >>>> > > >>>> > > >>>> On Sat, 1 Apr 2023 at 13:06, Subramanian, Ramaswamy < > [email protected]> wrote: > > >>>> Dear All, > > >>>> > > >>>> I am unsure if all other groups will get it - but I am sure this > group will understand the frustration. > > >>>> > > >>>> My NIH grant did not get funded. A few genuine comments - they > make excellent sense. We will fix that. > > >>>> > > >>>> One major comment is, “Structures can be predicted by alpfafold and > other software accurately, so the effort put on the grant to get structures > by X-ray crystallography/cryo-EM is not justified.” > > >>>> > > >>>> The problem is when a company with billions of $$s develops a > method and blasts it everywhere - the message is so pervasive… > > >>>> > > >>>> Question: Is there a canned consensus paragraph that one can add > with references to grants with structural biology (especially if the review > group is not a structural biology group) to say why the most modern > structure prediction programs are not a substitute for structural work? > > >>>> > > >>>> Thanks. > > >>>> > > >>>> > > >>>> Rams > > >>>> [email protected] > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> To unsubscribe from the CCP4BB list, click the following link: > > >>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > >>>> > > >>> > > >>> > > >>> To unsubscribe from the CCP4BB list, click the following link: > > >>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > >>> > > >> > > >> > > >> To unsubscribe from the CCP4BB list, click the following link: > > >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > >> > > > > > > ----- > > > Randy J. Read > > > Department of Haematology, University of Cambridge > > > Cambridge Institute for Medical Research Tel: +44 1223 336500 > > > The Keith Peters Building > > > Hills Road > E-mail: [email protected] > > > Cambridge CB2 0XY, U.K. > www-structmed.cimr.cam.ac.uk > > > > > > > > > > ######################################################################## > > > > > > To unsubscribe from the CCP4BB list, click the following link: > > > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > available at https://www.jiscmail.ac.uk/policyandsecurity/ > > > > ######################################################################## > > > > To unsubscribe from the CCP4BB list, click the following link: > > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > available at https://www.jiscmail.ac.uk/policyandsecurity/ > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > available at https://www.jiscmail.ac.uk/policyandsecurity/ > ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
