Ninovolador created this task. Ninovolador added projects: Pywikibot, ProofreadPage. Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTION **Feature summary** Currently, when using `pywikibot.proofreadpage.IndexPage` 's function `page_gen()` with options `only_existing` and `filter_ql` set to anything causes the generator to actually load every page to get the quality level or check if the page exists, which is very slow (I use a 200 Mbps connection and it took around 3 minutes to loop through a 600-pages index). I noticed that in the function `_get_page_mappings` we already get the info of the quality level and the existence of every page at once, when we parse the Index's HTML. I concoted a very crude non-conformant implementation that gets that info from there, and it loop through the index in around 10 seconds, which is considerably faster. I pasted the modified version of the script here <https://pastebin.com/jKsMGeEg>. Relevant changes in lines 1147-1305 **Use case(s)**: from pywikibot.proofreadpage import IndexPage from pywikibot import Site index = IndexPage(Site('es', 'wikisource'), 'Índice:Vocabulario en lengua castellana y mexicana Alonso de Molina 1571.pdf') print(index.num_pages) for page in index.page_gen(only_existing=True): print(page.title(), 'exists') **Benefits** (why should this be implemented?): I think it's just better TASK DETAIL https://phabricator.wikimedia.org/T409023 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
_______________________________________________ pywikibot-bugs mailing list -- [email protected] To unsubscribe send an email to [email protected]
