Hi, The purpose of captcha is to keep out automated scraping. By defeating that purpose, we trigger a nuclear arms race which will end in one of these outcomes: - The institution will (at great cost) achieve a way to effectively make scraping impossible. - The institution shall shut down its public information portal and declare that please use offline methods. - The institution shall push for government to hunt down and prosecute those who scrape it and will "make an example" by destroying some peoples' lives.
Just sharing the logical conclusions for your kind consideration. In one scraping activity in the past I had followed a hybrid approach : Through automated keyboard strokes, I could automate everything upto the captcha stage, then I could type in the captcha manually, then start off another set of keystrokes. It sped up the process by a lot. I was using "AutoHotKey" in windows years ago; there have been a lot of innovations since then in this kind of automating. This approach is also subject to its own arms race of course, but less chance of going nuclear, because with this you're not crashing the institution's servers. Regards Nikhil VJ On Tuesday, November 10, 2020 at 9:49:39 PM UTC+5:30 [email protected] wrote: > Hi Lovish, > > So the link you are sharing can be worked around with Selenium I believe. > One complex challenge you'd face would be to decode the captcha everytime > and send it back correctly. > > The way I solved it for one of my work projects was to download captcha > image everytime and pre-process image and OCR it to get a result. Got to a > level where 9 out of 10 requests worked perfectly. But another option we > tried was to go the way of Deep Learning and train a model to recognize and > classify text letters. Which took longer but still worked good enough. > > P.S - Form can easily be filled with Selenium. Captcha and the next steps > might be another challenge. > > Thanks & Regards- > Rahul Gupta > > On Tue, 10 Nov, 2020, 9:43 pm Lovish Sciences, < > [email protected]> wrote: > >> Hi, >> >> Would it be possible to scrap data by using this link >> <https://services.ecourts.gov.in/ecourtindia_v4_bilingual/cases/fir1.php?state=D&state_cd=1&dist_cd=25>. >> >> Instead of scrapping through Court Complex option, can we do it from Court >> Establishment option ? >> >> On Mon, 9 Nov 2020 at 12:04, Nikhil VJ <[email protected]> wrote: >> >>> Hi Lovish, >>> >>> Is there any link on this site where you are able to navigate through >>> different data items? I only see pages where you have to input certain case >>> code etc. >>> Please share more details with directions and examples - links or >>> screenshots - of what you need. >>> Then maybe somebody might be able to help. >>> >>> -- >>> Cheers, >>> Nikhil VJ >>> https://nikhilvj.co.in >>> >>> >>> On Mon, Nov 9, 2020 at 8:35 AM Lovish Sharma < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> I am working as an associate for an NGO working in the field of crimes >>>> against women. Currently I am doing research on crimes against women in >>>> prominent cities. For that, I need to scrap the data from e-courts >>>> website, *https://districts.ecourts.gov.in/ >>>> <https://districts.ecourts.gov.in/>* . >>>> >>>> Kindly help me with that. >>>> >>>> -- >>>> Datameet is a community of Data Science enthusiasts in India. Know more >>>> about us by visiting http://datameet.org >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "datameet" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/datameet/4c1b2005-8594-4afe-ba88-a8e7147a5798n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/datameet/4c1b2005-8594-4afe-ba88-a8e7147a5798n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> Datameet is a community of Data Science enthusiasts in India. Know more >>> about us by visiting http://datameet.org >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "datameet" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/datameet/CAH7jeuOpmAiHPc_T63qHD%2BzGu3U9_N-MaQ6GhSvt612X6UDMBg%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/datameet/CAH7jeuOpmAiHPc_T63qHD%2BzGu3U9_N-MaQ6GhSvt612X6UDMBg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Thanks & Regards, >> Lovish >> >> -- >> Datameet is a community of Data Science enthusiasts in India. Know more >> about us by visiting http://datameet.org >> --- >> You received this message because you are subscribed to the Google Groups >> "datameet" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/datameet/CAGcYNcENfcJQwSwq_DiATr3tbFOGqVJvKJbJQeECw%3DJ8ZiDb8g%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/datameet/CAGcYNcENfcJQwSwq_DiATr3tbFOGqVJvKJbJQeECw%3DJ8ZiDb8g%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups "datameet" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/5a519594-021d-4bfb-a623-ffb3a71ea32dn%40googlegroups.com.
