Duncan,
I couldn't make out much from the code.
Instead this is what I did.
threads = []
nloops = range(len(lRawData))
for i in nloops:
(sUrl, sFile, download_size, checksum) =
stripper(lRawData[i])
t = threading.Thread(target=download_from_web, args=(sUrl,
sFile, sSourceDir, None))
# = pypt_thread(download_from_web, i,
stripper(lRawData[i]))
threads.append(t)
i = 0
join_i = 0
while i < nloops:
counter = 0
while counter < 3:
threads[i].start()
counter += 1
i += 1
counter = 0
join_i = i - 3
while counter < 3:
threads[join_i].join()
counter += 1
join_i += 1
Is this correct ? Comments!!
Ritesh
Duncan Booth wrote:
> Ritesh Raj Sarraf wrote:
>
> > I'm planning to implement threads in my application so that multiple
> > items can be downloaded concurrently. I want the thread option to be
> > user-defined.
> >
> > Looking at the documentation of threads (Core Python Programming), I've
> > noticed that all threads are executed a once. Depending upon what they
> > are doing, some finish early and some later.
> >
> > But I want to implement something like:
> >
> > for item in list_items:
> > for num in thread_args:
> > thread[num].start()
> > thread[num].start()
> >
> > Is this the correct way of threading applications ?
> > This is the first time I'd be doing threading. So was looking for
> > comments and suggestions.
> >
>
> What you want is to use a pool of threads so that you can configure how
> many requests are issued at a time (you don't want to try to issue 100
> requests all in parallel). You can communicate with the threads through a
> Queue.
>
> So if the code for a thread looks like:
>
> def run(request, response):
> while 1:
> item = request.get()
> if item is None:
> break
> response.put(download_from_web(item))
>
> # your main loop can be something like:
>
> requestQueue = Queue()
> responseQueue = Queue()
> thread_pool = [
> Thread(target=run, args=(requestQueue, responseQueue)
> for i in range(numthreads)]
> for t in thread_pool: t.start()
>
> for item in list_items:
> requestQueue.put(item)
>
> for i in range(len(list_items)):
> response = responseQueue.get()
> handle_response(response)
>
> # and then to shut down the threads when you've finished:
> for t in thread_pool:
> requestQueue.put(None)
> for t in thread_pool:
> t.join()
--
http://mail.python.org/mailman/listinfo/python-list
