Several people have mentioned lack of peer review as a reason to doubt the significance of the AlphaFold2 results. There are different routes to peer review and, while the results have not been published in a peer review journal, I would have to say (as someone who has been an assessor for two CASPs, as well as having editorial responsibilities for a peer-reviewed journal), the peer review at CASP is much more rigorous than the peer review that most papers undergo. The targets are selected from structures that have recently been solved but not published or disseminated, and even just tweeting a C-alpha trace is probably enough to get a target cancelled. In some cases (as we’ve heard here) the people determining the structure are overly optimistic about when their structure solution will be finished, so even they may not know the structure at the time it is predicted. The assessors are blinded to the identities of the predictors, and they carry out months of calculations and inspections of the models, computing ranking scores before they find out who made the predictions. Most assessors try to bring something new to the assessment, because the criteria should get more stringent as the predictions get better, and they have new ideas of what to look for, but there’s always some overlap with “traditional” measures such as GDT-TS, GDT-HA (more stringent high-accuracy version of GDT) and lDDT.
Of course we’d all like to know the details of how AlphaFold2 works, and the DeepMind people could have been (and should be) much more forthcoming, but their results are real. They didn’t have any way of cheating, being selective about what they reported, or gaming the system in any other way that the other groups couldn’t do. (And yes, when we learned that DeepMind was behind the exceptionally good results two years ago at CASP13, we made the same half-jokes about whether Gmail had been in the database they were mining!) Best wishes, Randy Read > On 9 Dec 2020, at 10:36, Hughes, Jonathan > <jon.hug...@bot3.bio.uni-giessen.de> wrote: > > i think the answer to all these doubts and questions is quite simple: the > AlphaFold2 people must make all details of their methods public (source code) > and, as would probably be necessary, open their system for inspection and use > by independent experts. isn't that what peer review and reproducibility are > all about? those rules date from the time before every tom, dick and > henriette could publicize anything they like inside their own zuckerberg > bubble. my opinion is that this is a virtual infectious disease that will > cause humanity far bigger problems than corona ever will – i just hope i'm > wrong! > best > jon > > Von: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> Im Auftrag von Mark J van > Raaij > Gesendet: Mittwoch, 9. Dezember 2020 11:14 > An: CCP4BB@JISCMAIL.AC.UK > Betreff: Re: [ccp4bb] External: Re: [ccp4bb] AlphaFold: more thinking and > less pipetting (?) > > on the day the news came out, I did wonder if the AlphaFold2 team somehow had > access to all the preliminary PDB files sent around via Gmail (which belongs > to the same company), but more as a joke/conspirational thought. > "our" target T1052, was also predicted very well by domains and as a monomer. > It will be interesting to see how well future iterations of the method can > assemble the complete protein chain and the complete protein chains into the > correct heteromer. > > Mark J van Raaij > Dpto de Estructura de Macromoleculas > Centro Nacional de Biotecnologia - CSIC > calle Darwin 3 > E-28049 Madrid, Spain > tel. (+34) 91 585 4616 > Section Editor Acta Crystallographica F > https://journals.iucr.org/f/ > > > On 9 Dec 2020, at 10:37, Cedric Govaerts <cedric.govae...@ulb.ac.be> wrote: > > Dear All > > After about 10 (!) years of (very) hard work we solved the structures of our > dearest membrane transporter. Dataset at 2.9 And resolution, fairly > anisotropic, experimental phasing, and many looooong nights with Coot and > Buster to achieve model refinement. > > The experimental structure had a well defined ligand nicely coordinated but > also a lipid embedded inside the binding cavity (a complete surprise but > biologically relevant) and two detergent molecules well defined > (experimental/crystallisation artefact). > > As our paper was accepted basically when CASP organisers were calling for > targets I offered my baby to the computing Gods. However we only provided the > sequence to CASP, no info regarding any ligand or lipid. > > Less than a month after, the CASP team contacted us and send us the best > model. In fact it was 2 half models as the transporter is a pseudo dimer, > with the N-lobe and C-lobe moving relative to each other during transport > cycle, thus divided as two domains in CASP. > > The results were breathtaking. 0.7 And RSMD on one half, 0.6 on the other. > And yes, group 427 was the superpower (did not know at the time that it was > AlphaFold). > > We had long discussions with the CASP team, as -for us- this almost exact > modelling was dream-like (or science fiction) and -at some point- we were > even suspecting fraud, as our coordinates had travelled over the internet a > few times around when interacting with colleagues. The organisers reassured > us that we were not the only target that had been “nailed” so no reason to > suspect any wrongdoing. > > To this day I am still baffled and I would be happy to hear from the > community, maybe from some of the CASP participants. > > The target is T024, the “perfect" models are domain-split version (T024-D1 > and T024-D2), as AlphaFold2 did not perform so well on the complete assembly. > Deposited PDB is 6T1Z > > Cedric > > PS: I should also note that many other groups performed very well, much > better than I would have dreamed, including on the full protein but just not > as crazy-good. > — > Prof. Cedric Govaerts, Ph.D. > Universite Libre de Bruxelles > Campus Plaine. Phone :+32 2 650 53 77 > Building BC, Room 1C4 203 > Boulevard du Triomphe, Acces 2 > 1050 Brussels > Belgium > http://govaertslab.ulb.ac.be/ > > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > ----- Randy J. Read Department of Haematology, University of Cambridge Cambridge Institute for Medical Research Tel: +44 1223 336500 The Keith Peters Building Fax: +44 1223 336827 Hills Road E-mail: rj...@cam.ac.uk Cambridge CB2 0XY, U.K. www-structmed.cimr.cam.ac.uk ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
