I absolutely agree with Guenter. This non experts speaking wrong things is increasingly happening nowadays. Thanks Guenter, for pointing this out. A theory unless it is given experimental data derived evidence it is still a prediction (just like weather prediction, unless it is realised in the physical world it will still be a very good knowledge based guess, which we all revere and appreciate, even if the predicted rainfall does not occur). Ignorance with use of Alpha-fold models by non experimental structural biologists and non acceptable remarks that come out of these non experimental structural biologists including the top experts in the field and also unrelated fields, is something not at all right happening. This email chain also emphasises this aspect. This needs to be corrected and addressed very seriously at a large level. Perhaps at the same level at which the alpha-fold success itself was propagated. Few articles have been published in recent journals towards this... but it has not yet reached the mass. *It has to be at an even larger level, perhaps a commentary in Nature, Science and Cell journals by the top experimental structural biologists including Nobel Laureates in the field? It will have more impact on the non-experts if there are Nobel Laureates included in this task.*
Best regards, Natesh -- ---------------------------------------------------------- "Live Simply and do Serious Things .. " - Dorothy Mary Crowfoot Hodgkin OM, FRS "In Science truth always wins" - Max Ferdinand Perutz OM FRS ---------------------------------------------------------- Dr. Ramanathan Natesh Associate Professor, School of Biology and Center for High-Performance Computing (CHPC), Founding and Current President of Cryo Electron Microscopy and 3 Dimensional Image Processing Society of India (CEM3DIPSI), Indian Institute of Science Education and Research Thiruvananthapuram (IISER-TVM), Maruthamala P.O., Vithura, Thiruvananthapuram, 695551, Kerala, India nat...@iisertvm.ac.in http://faculty.iisertvm.ac.in/natesh *Researcher ID*: http://www.researcherid.com/rid/C-4488-2008 *ORCID*: http://orcid.org/0000-0002-1145-5962 Vidwan-ID : 94134: http://iisertvm.irins.org/profile/94134 *PUBLONS*: https://publons.com/author/1520837/ramanathan-natesh#profile Office Ph. 0091- 471-2778087 On Fri, 5 May 2023 at 17:57, Guenter Fritz < guenter.fritz.phenix.c...@gmail.com> wrote: > Dear all, > > taking AlphaFold models for " true" experimental structures seems to > become a serious problem. > I am just returning from a meeting (not a structural biology meeting) and > saw one model after other. And the non-structural biologist used terms like > "we calculated a structure" or "a AlphaFold crystal structure" or "the > structure was accurate, is was all blue". > Alphafold models were used to predict ion channels, electron transfer > pathways, enzyme mechanisms, and yes, not tested by experiments. Wide > ranging conclusions were drawn on these pure models, which I would not dare > to draw on limited experimental data. > There is something going severely wrong. > And don't get me wrong, I think AlphaFold and other prediction software is > great to create testable models (like MR models) or try to figure out how > proteins might assemble and so on. > But I get the impression that too many of our colleagues got the > impression that pressing a button replaces experiments. > Seeing that grants are rejected by such arguments is alarming and we > should do something. > > Best wishes, > Guenter > > Very sorry to hear about your grant. I've been there. It is crushing to be > rejected, and frustrating when the reason given is ... wrong. > > My journalist friends wonder why scientists don't like talking to > journalists. This is why. I remember when the first results from XFELs > were published, and it was immediately declared that there was no longer a > need for NASA, whose sole purpose (apparently) was to grow bigger and > better crystals in space. (?!) I find the idea that AlphaFold has > eliminated the need to solve any more structures equally ludicrous. > > I think the best analogy for what has happened in structural biology is > the same impact a Star Trek style "transporter device" would have on your > daily commute. Except this "transporter" is only accurate enough to get you > within a mile or two of your house. Most of the time. Don't worry, its not > going to beam you inside a rock or into the sky, as it was trained on data > with good Clashscores (we think). But, you are on your own getting the rest > of the way home. This "Last Mile" of transportation networks is actually > the most challenging, and expensive, but also the most critical. In > structural biology, the "Last Angstrom" between prediction and actuality is > equally important, but also fraught with difficulty. It may seem like a > short distance, until you have to walk it. So, despite amazing progress, it > is still premature to dismantle infrastructure, and definitely a bad idea > to nail your front door shut. > > Personally, I see this structure prediction revolution as nothing more nor > less than the fruition of Structural Genomics. It started in the final days > of the 20th century. I was there! The stated goal of that worldwide > initiative was to create the data set that would be needed by some future > (at the time) homology modelling technology to do exactly what AlphaFold > does: get us "close enough". And then Greg Petsko asked: what is "close > enough"? He called it "The Grail problem". By what metric do you declare > victory? He made an excellent suggestion: > > "But there is an obvious method of evaluation that will allow any > structure prediction method to be assessed. It is simply to demand that the > method produce a model that can be used to solve the corresponding protein > crystal structure by the method of molecular replacement." > -Greg Petsko - June 9, 2000 > https://doi.org/10.1186/gb-2000-1-1-comment002 > > This is the thing that just changed. Structure prediction has finally > crossed the "G-P threshold". Not 100% of the time, but impressively often > now, the predictions can be used for MR. This is a massively useful tool! > Not the end of the field, but rather the beginning of an exciting new era > where success rates skyrocket. > > Scores like the GDT used in CASP were developed with this Grail Problem > criterion in mind, and I think that is what John Moult and others meant > when they said things that got quoted like this: > "Scores above 90 on the 100-point scale are considered on par with > experimental methods, Moult says." > https://www.science.org/doi/full/10.1126/science.370.6521.1144 > > Meaning that the predicted models work as search models for MR about as > often as search models derived from homologous (and yes, "experimentally > determined") structures. A GDT of 100 does NOT mean the model is better > than the data. That is not even how it works. > > But, unfortunately, this seems to have gotten paraphrased and > sensationalized: > > "generally considered to be competitive with the same results obtained via > experimental methods" > > https://www.sciencealert.com/ai-solves-50-year-old-biology-grand-challenge-decades-before-experts-predicted > > "software predictions finally match structures calculated from > experimental data" > https://www.science.org/doi/full/10.1126/science.370.6521.1144 > > "comparable in quality to experimental structures" > https://www.nature.com/articles/d41586-020-03348-4 > > "accuracy comparable to laboratory experiments" > https://www.bbc.com/news/science-environment-55133972 > > <sigh> > > The only kind of diffraction where prediction is better than experiment is > that of monoatomic gasses. These curves can be derived very accurately and > completely from fundamental constants of physics. This is where those > tables of atomic scattering factors used by refinement programs come from. > For a while, the experimentally measured curves were used, but once > Hartree, Fock, Slater, Cromer, Mann and others worked out how to do the > self-consistent field calculations accurately, by the late 1960s the > calculated form factors supplanted the measured ones. > > You might also say that for "small molecule" crystals the models are > better than the data. Indeed, the CSD did not require experimental data to > be deposited until fairly recently. The coordinates were considered more > accurate than the intensities because publication requirements for chemical > crystallography R factors are low enough to be dominated by experimental > noise only. Nevertheless, despite the phase problem being cracked by > direct methods in the 1980s, your local chemistry department has yet to > shut down their diffractometer. Why? Because they need it. And for > macromolecular structures, the systematic errors between refined > coordinates and their corresponding data are about 4-5x larger than > experimental error. So, don't delete your image data! Not for a while yet. > > -James Holton > MAD Scientist > > > On 4/1/2023 7:57 AM, Subramanian, Ramaswamy wrote: > > Ian, > > Thank you. This is not an April fools.. > Rams > subra...@purdue.edu > > > > On Apr 1, 2023, at 10:46 AM, Ian Tickle <ianj...@gmail.com> > <ianj...@gmail.com> wrote: > > ---- *External Email*: Use caution with attachments, links, or sharing > data ---- > > Hi Ramaswamy > > I assume this is an April Fool's but it's still a serious question because > many reviewers who are not crystallographers or electron microscopists may > not fully appreciate the difference currently between the precision of > structures obtained by experimental and predictive methods, though the > latter are certainly catching up. The answer of course lies in the > mean co-ordinate precision, related to the map resolution. > > Quoting https://people.cryst.bbk.ac.uk/~ubcg05m/precgrant.html : > > "The accuracy and precision required of an experimentally determined > model of a macromolecule depends on the biological questions being asked of > the structure. Questions involving the overall fold of a protein, or its > topological similarity to other proteins, can be answered by structures of > fairly low precision such as those obtained from very low resolution X-ray > crystal diffraction data [or AlphaFold]. Questions involving reaction > mechanisms require much greater accuracy and precision as obtained from > well-refined, high-resolution X-ray structures, including proper > statistical analyses of the standard uncertainties (*s.u.'s*) of atomic > positions and bond lengths.". > > According to https://www.nature.com/articles/s41586-021-03819-2 : > > The accuracy of AlphaFold structures at the time of writing (2021) was > around 1.0 Ang. RMSD for main-chain and 1.5 Ang. RMSD for side-chain atoms > and probably hasn't changed much since. This is described as "highly > accurate"; however this only means that AlphaFold's accuracy is much higher > in comparison with other prediction methods, not in comparison with > experimental methods. Also note that AlphaFold's accuracy is estimated by > comparison with the X-ray structure which remains the "gold standard"; > there's no way (AFAIK) of independently assessing AlphaFold's accuracy or > precision. > > Quoting https://scripts.iucr.org/cgi-bin/paper?S0907444998012645 : > > "Data of 0.94 A resolution for the 237-residue protein concanavalin A are > used in unrestrained and restrained full-matrix inversions to provide > standard uncertainties sigma(r) for positions and sigma(l) for bond > lengths. sigma(r) is as small as 0.01 A for atoms with low Debye B values > but increases strongly with B." > > There's a yawning gap between 1.0 - 1.5 Ang. and 0.01 Ang.! Perhaps > AlphaFold structures should be deposited using James Holton's new PDB > format (now that is an April Fool's !). > > One final suggestion for a reference in your grant application: > https://www.biorxiv.org/content/10.1101/2022.03.08.483439v2 . > > Cheers > > -- Ian > > > On Sat, 1 Apr 2023 at 13:06, Subramanian, Ramaswamy <subra...@purdue.edu> > wrote: > >> Dear All, >> >> I am unsure if all other groups will get it - but I am sure this group >> will understand the frustration. >> >> My NIH grant did not get funded. A few genuine comments - they make >> excellent sense. We will fix that. >> >> One major comment is, “Structures can be predicted by alpfafold and other >> software accurately, so the effort put on the grant to get structures by >> X-ray crystallography/cryo-EM is not justified.” >> >> The problem is when a company with billions of $$s develops a method and >> blasts it everywhere - the message is so pervasive… >> >> *Question: I*s there a canned consensus paragraph that one can add with >> references to grants with structural biology (especially if the review >> group is not a structural biology group) to say why the most modern >> structure prediction programs are not a substitute for structural work? >> >> Thanks. >> >> >> Rams >> subra...@purdue.edu >> >> >> >> >> ------------------------------ >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 >> > > > ------------------------------ > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > ------------------------------ > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > ------------------------------ > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
