On 05/01/16 00:51, Ian Kelly wrote: > On Mon, Jan 4, 2016 at 4:38 PM, Steven D'Aprano <st...@pearwood.info> wrote: >> On Tue, 5 Jan 2016 07:50 am, livems...@gmail.com wrote: >> >>> So what is the fastest way to make 400 HTTP requests using "requests" >>> library and also using tor proxy? >> >> >> Since this will be I/O bound, not CPU bound, probably use separate threads. >> >> Push the 400 requests into a queue, then create N threads, where N will need >> to be determined by experiment, but will probably be something like 4 or 8, >> and let each thread pop a request from the queue as needed. >> >> Are you experienced with threads? Do you need further information about >> using threads and queues? > > Also see the concurrent.futures module in the standard library, which > makes this sort of setup very simple to implement. > Why would someone want to make 400 HTTP requests in a short time?
-- Tony van der Hoff | mailto:t...@vanderhoff.org Ariège, France | -- https://mail.python.org/mailman/listinfo/python-list
