Re: Multi-threaded FTP Question
There are n instances of ftplib.FTP.
Funny thing is that I tried to run the code twice, concurrently, in two
separate IDLE instances. Each instance had four threads/ftp calls.
Again, only five ftp connections were allowed on my computer. The code
errored out on the sixth attempt.
I wonder if there' s some underlying software being called that has a
limit on the number of ftp connections allowed at any point in time.
I'm tempted to try this on cygwin to see if the behavior is different.
Derek
import ftplib, zipfile, datetime, os
from threading import Thread, Lock
class ftpDownload(Thread):
def __init__(self, year, quarter):
Thread.__init__(self)
self.quarter = str(quarter)
self.year = str(year)
self.filename = ""
self.ftp = ftplib.FTP('xxx')
self.ftp.login()
self.lock = Lock()
def run(self):
filehandle = open(self.filename,'wb')
self.ftp.retrbinary('RETR /x/xxx/' + self.year + '/QTR'
+ self.quarter + '/xxx.zip', filehandle.write)
filehandle.close()
self.ftp.close()
ftplist = []
for year in range(1990,2000):
for quarter in range(1,5):
current = ftpDownload(year, quarter)
ftplist.append(current)
current.start()
for job in ftplist:
job.join()
Jeremy Jones wrote:
> [EMAIL PROTECTED] wrote:
> > I'm trying to use ftp in python in a multi-threaded way on a windows
> > box - python version 2.4.3. Problem is that it appears that it's only
> > possible to have five instances/threads at one point in time. Errors
> > look like:
> >
> >File "C:\Python24\lib\ftplib.py", line 107, in __init__
> > self.connect(host)
> > File "C:\Python24\lib\ftplib.py", line 132, in connect
> > self.welcome = self.getresp()
> > File "C:\Python24\lib\ftplib.py", line 208, in getresp
> > resp = self.getmultiline()
> > File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
> > line = self.getline()
> > File "C:\Python24\lib\ftplib.py", line 184, in getline
> > if not line: raise EOFError
> > EOFError
> >
> > Is it possible to have more than five simultaneous ftp connections?
> >
> > Thanks.
> >
> > Derek
>
> I replied to this about 4 hours ago from my gmail email account (not my
> google groups account associated with the same email addres), but
> haven't seen it show up, so I apologize if this is a dupe.
>
> Would you mind posting your code? Are you trying to pass the same FTP
> connection object to all 5 threads?
>
> - Jeremy M. Jones
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-threaded FTP Question
Thanks so much for your help on this. The server that I'm connecting to is the culprit. They only allow five connections at a time. I assumed that it was a code issue. I think that we're conditioned to expect that the problem is on the software side of things. -Derek [EMAIL PROTECTED] wrote: > [EMAIL PROTECTED] wrote: > > I'm trying to use ftp in python in a multi-threaded way on a windows > > box - python version 2.4.3. Problem is that it appears that it's only > > possible to have five instances/threads at one point in time. Errors > > look like: > > > >File "C:\Python24\lib\ftplib.py", line 107, in __init__ > > self.connect(host) > > File "C:\Python24\lib\ftplib.py", line 132, in connect > > self.welcome = self.getresp() > > File "C:\Python24\lib\ftplib.py", line 208, in getresp > > resp = self.getmultiline() > > File "C:\Python24\lib\ftplib.py", line 194, in getmultiline > > line = self.getline() > > File "C:\Python24\lib\ftplib.py", line 184, in getline > > if not line: raise EOFError > > EOFError > > > > Is it possible to have more than five simultaneous ftp connections? > > > > Thanks. > > > > Derek > > It might be XP SP2's worm protection as well: > > http://www.speedguide.net/read_articles.php?id=1497 -- http://mail.python.org/mailman/listinfo/python-list
Re: How to start more than one process at the same time?
Have you considered a multi-threaded solution?
The following websites offer reasonable examples:
http://en.wikibooks.org/wiki/Programming:Python/Threading
http://www.wellho.net/solutions/python-python-threads-a-first-example.html
-Derek
Dirk Hagemann wrote:
> Hi!
>
> How can I start several jobs at the same time with python? I want to
> collect data from some servers and I don't want to wait until the first
> server is finished. These jobs should run parallel to save time.
> What I tried is this:
> os.popen('regdmp -m server1
> "HKEY_LOCAL_MACHINE\\Software\\Intel\\LanDesk\\VirusProtect6\\CurrentVersion\\Clients"
> >> av_server1.txt')
> os.popen('regdmp -m server2
> "HKEY_LOCAL_MACHINE\\Software\\Intel\\LanDesk\\VirusProtect6\\CurrentVersion\\Clients"
> >> av_server2.txt')
> os.popen('regdmp -m server3
> "HKEY_LOCAL_MACHINE\\Software\\Intel\\LanDesk\\VirusProtect6\\CurrentVersion\\Clients"
> >> av_server3.txt')
>
> I didn't get an error-message, but I also didn't get a result. No
> text-file was created.
> I also tried to do the same thing via batch-files, but that didn't
> change anything.
>
> Is it just not possible or what am I doing wrong?
>
> Thanks for any help!
> Dirk Hagemann
--
http://mail.python.org/mailman/listinfo/python-list
Multi-threaded FTP Question
I'm trying to use ftp in python in a multi-threaded way on a windows box - python version 2.4.3. Problem is that it appears that it's only possible to have five instances/threads at one point in time. Errors look like: File "C:\Python24\lib\ftplib.py", line 107, in __init__ self.connect(host) File "C:\Python24\lib\ftplib.py", line 132, in connect self.welcome = self.getresp() File "C:\Python24\lib\ftplib.py", line 208, in getresp resp = self.getmultiline() File "C:\Python24\lib\ftplib.py", line 194, in getmultiline line = self.getline() File "C:\Python24\lib\ftplib.py", line 184, in getline if not line: raise EOFError EOFError Is it possible to have more than five simultaneous ftp connections? Thanks. Derek -- http://mail.python.org/mailman/listinfo/python-list
