Tim, Harry,
Do you get these numbers for "excluding citations from special issues dedicated to databases and web servers"? I meant to contrast the 25.9 I got in this section to Kay's "excluding citations from special issues, is estimated to be around 5.6 to 5.8". These are very different numbers that two instances of the same chatbot claim are the same thing! Getting 19.1, 20.1, even 25.9 as the same thing, I would be inclined to file as "rounding errors or something with a pretty large standard deviation". But 5.8 versus 25.9? Clearly the chatbot has no notion of what these numbers mean (and how could it?). Happy solstice vacation! Guillaume ________________________________ From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Nave, Colin (DLSLtd,RAL,LSCI) <000064fdcfc6624b-dmarc-requ...@jiscmail.ac.uk> Sent: Thursday, December 19, 2024 11:11:24 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] NAR impact factor This is all a good illustration of Goodhart's Law Colin Sent from Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Harry Powell <0000193323b1e616-dmarc-requ...@jiscmail.ac.uk> Sent: Thursday, December 19, 2024 5:56:48 PM To: CCP4BB@JISCMAIL.AC.UK <CCP4BB@JISCMAIL.AC.UK> Subject: Re: [ccp4bb] NAR impact factor Hi It seems that East London (the one in the UK, not South Africa) is more similar to Germany in this respect than Austria is - I get identical figures to Kay’s (or does the LLM know that we’ve both developed integration programs, and bias accordingly?) Harry > On 19 Dec 2024, at 15:00, Tim Gruene <tim.gru...@univie.ac.at> wrote: > > On Thu, 19 Dec 2024 10:19:58 +0000 > Guillaume Gaullier <guillaume.gaull...@kemi.uu.se> wrote: > >> Hello, >> >> I clicked this link, and got this conclusion: >> >> "The adjusted impact factor of Nucleic Acids Research, excluding >> citations from special issues dedicated to databases and web servers, >> is approximately 25.9. This indicates a significant impact when >> focusing solely on the regular articles published in the journal." >> >> Does this experiment tell us more about the NAR impact factor (should >> we even care about this in the first place?) or about the reliability >> and reproducibility of chatbots powered by large language models? >> >> Food for thought in any case. >> Cheers, >> >> Guillaume > Hi Guillaume, > > you IP maybe connects more often to the US, and the bias to make > everything greater is enlarged. From Austria, the first answer is 20.1, > which is still bigger than Kay's 19.1 - make sense to me ;-) > Cheers, > Tim > > -- > -- > Tim Gruene > Head of the Core Facility Crystal Structure Analysis > Faculty of Chemistry > University of Vienna > > Phone: +43-1-4277-70202 > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fccsa.univie.ac.at%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358192329%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ZKy48NCQ%2BUB4INsNC%2BqJL1UZXosofe%2F5QBbD%2FyUAk7k%3D&reserved=0<https://ccsa.univie.ac.at/> > > GPG Key ID = A46BEE1A > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DCCP4BB%26A%3D1&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358219508%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=SheDnpPjAPZ%2BQnC0IYNtX88yDfsZcirvQvq%2FEV5%2FJKo%3D&reserved=0<https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1> > > This message was issued to members of > https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2FCCP4BB&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358233857%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=b%2FOTgO8aKxfMTtAkILQa4zAWS4VNeA0AW4kvCEGyMsI%3D&reserved=0<http://www.jiscmail.ac.uk/CCP4BB>, > a mailing list hosted by > https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358246342%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ZcSNzDoOqqF5UGHK%2FVQeVsBC89B7vwe55Ah3%2BVvFseY%3D&reserved=0<http://www.jiscmail.ac.uk/>, > terms & conditions are available at > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fpolicyandsecurity%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358258555%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Q112layXiApzvZgtMmK8bAOsQtdeiw6ElJRvKqFBhLk%3D&reserved=0<https://www.jiscmail.ac.uk/policyandsecurity/> ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DCCP4BB%26A%3D1&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358270138%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ZHrpmAwaiYRfZeanZDwtlQ%2FlAih0tlkFbQ7zVg5rQp4%3D&reserved=0<https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1> This message was issued to members of https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2FCCP4BB&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358284595%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=DzXcJA9FCWGpBEG4tWwCeaJzjVt4LY7g8s8n4g5I0f4%3D&reserved=0<http://www.jiscmail.ac.uk/CCP4BB>, a mailing list hosted by https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358296613%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=PGVPDajAS7i1PUB1VDVuonpb2S6Sm43j07QDqd6xJK0%3D&reserved=0<http://www.jiscmail.ac.uk/>, terms & conditions are available at https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fpolicyandsecurity%2F&data=05%7C02%7Ccolin.nave%40DIAMOND.AC.UK%7Cd63b9c37ea91416a6a4b08dd205687be%7C9d27ba7401004d0d81ff1d728dae8df6%7C0%7C0%7C638702278358308614%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=PO38GF7q3GM0SOhGOyRln2raJ43huNlE%2BD2LfDtMXQU%3D&reserved=0<https://www.jiscmail.ac.uk/policyandsecurity/> This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom. ________________________________ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 VARNING: Klicka inte på länkar och öppna inte bilagor om du inte känner igen avsändaren och vet att innehållet är säkert. CAUTION: Do not click on links or open attachments unless you recognise the sender and know the content is safe. När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/ E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: http://www.uu.se/en/about-uu/data-protection-policy ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
