socketServer questions
I have written a python socketServer program and I have a few questions that I hope the group can answer... here is a simple version of the server: class tr_handler(SocketServer.StreamRequestHandler): def handle(self): data = self.rfile.readline(300) data = str.strip(data) bytes = str(len(data)) public_ip = self.client_address[0] serv_date = time.strftime('%Y-%m-%d', time.localtime()) serv_time = time.strftime('%H:%M:%S', time.localtime()) # Note that 'data; comes from the client. fp = file('/home/rbt/Desktop/tr_report.txt', 'a') fp.write(data+"\t"+serv_date+"\t"+serv_time+"\t"+public_ip+"\t"+bytes+"\n") fp.close() if __name__=='__main__': server = SocketServer.TCPServer( ('', 55503), tr_handler) server.serve_forever() --- 1. Do I need to use threads to handle requests, if so, how would I incorporate them? The clients are light and fast never sending more than 270 bytes of data and never connecting for more than 10 seconds at a time. There are currently 500 clients and potentially there could be a few thousand... how high does the current version scale? 2. What's the proper way to handle server exceptions (server stops, fails to run at boot, etc.)? 3. How do I keep people from tampering with the server? The clients send strings of data to the server. All the strings start with x and end with y and have z in the middle. Is requiring x at the front and y at the back and z someplace in the middle enough to keep people out? I'm open to suggestions. Thanks! rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: socketServer questions
On Fri, 2005-10-07 at 09:17 -0700, Paul Rubinhttp: wrote: > > 3. How do I keep people from tampering with the server? The clients > > send strings of data to the server. All the strings start with x and > > end with y and have z in the middle. Is requiring x at the front and > > y at the back and z someplace in the middle enough to keep people > > out? I'm open to suggestions. > > It only keeps them out if they don't know to use that x..y..z pattern > and maybe not even then. Get a copy of "Security Engineering" by > Ross Anderson to have an idea of what you're dealing with, especially > if your server controls something valuable. The server just logs data, nothing else. It's not private or important data... just sys admin type stuff (ip, mac addy, etc.). I just don't want some script kiddie discovering it and trying to 'hack' it. By doing so, they'd fill the log up with crap. So, If the data doesn't contain x, y, and z and if the data is too big or too small, I record it to a 'tamper' log and tell the leet hacker to 'go away'. -- http://mail.python.org/mailman/listinfo/python-list
Re: socketServer questions
On Fri, 2005-10-07 at 15:07 -0700, Paul Rubinhttp: wrote: > rbt <[EMAIL PROTECTED]> writes: > > The server just logs data, nothing else. It's not private or important > > data... just sys admin type stuff (ip, mac addy, etc.). I just don't > > want some script kiddie discovering it and trying to 'hack' it. By doing > > so, they'd fill the log up with crap. So, If the data doesn't contain x, > > y, and z and if the data is too big or too small, I record it to a > > 'tamper' log and tell the leet hacker to 'go away'. > > Well, rather than this x,y,z stuff, it's best to do it properly and > authenticate the records with the hmac module. Off-topic here, but you've caused me to have a thought... Can hmac be used on untrusted clients? Clients that may fall into the wrong hands? How would one handle message verification when one cannot trust the client? What is there besides hmac? Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: socketServer questions
On Sat, 2005-10-08 at 14:09 -0700, Paul Rubinhttp: wrote: > rbt <[EMAIL PROTECTED]> writes: > > Off-topic here, but you've caused me to have a thought... Can hmac be > > used on untrusted clients? Clients that may fall into the wrong hands? > > How would one handle message verification when one cannot trust the > > client? What is there besides hmac? Thanks, rbt > > I don't understand the question. HMAC requires that both ends share a > secret key; does that help? That's what I don't get. If both sides have the key... how can it be 'secret'? All one would have to do is look at the code on any of the clients and they'd then know everything, right? > What do you mean by verification? I'm trying to keep script kiddies from tampering with a socket server. I want the server to only load a valid or verified string into its log database and to discard everything else. Strings could come to the socket server from anywhere on the Net from any machine. This is outside my control. What is there to prevent a knowledgeable person from finding the py code on a client computer, understanding it and then being able to forge a string that the server will accept? Does that make sense? -- http://mail.python.org/mailman/listinfo/python-list
One last thing about SocketServer
I've read more about sockets and now, I have a better understanding of them. However, I still have a few SocketServer module questions: When used with SocketServer how exactly does socket.setdefaulttimeout() work? Does it timeout the initial connect request to the socket server or does it timeout the session between the connecting client socket and the client socket the server generated to handle the incoming request? Also, since the *only* thing a 'socket server' does is to create 'client sockets' to handle requests, how do I use socket object features on these generated clients to manage and/or monitor them? The SocketServer module is great, but it seems to hide too many details of what it's up to! Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: socketServer questions
On Mon, 2005-10-10 at 05:54 -0700, Paul Rubinhttp: wrote: > rbt <[EMAIL PROTECTED]> writes: > > > I don't understand the question. HMAC requires that both ends share a > > > secret key; does that help? > > > > That's what I don't get. If both sides have the key... how can it be > > 'secret'? All one would have to do is look at the code on any of the > > clients and they'd then know everything, right? > > Yes, clients have to keep the key secure. > > > > What do you mean by verification? > > > > I'm trying to keep script kiddies from tampering with a socket server. I > > want the server to only load a valid or verified string into its log > > database and to discard everything else. > > If the clients can keep a secret key secure, then use hmac. Note that > if there's lots of clients, they shouldn't all use the same secret key. > Instead, for client #i, let that client's key be something like > hmac(your_big_secret, str(i)).digest() > and the client would send #i as part of the string. How is this different from sending a pre-defined string from the client that the server knows the md5 hash of? The clients know the string, the server knows the hash of that string. Also, could this not be done both ways? So that, if an attacker figures out the string he's supposed to send from a client to the server (which he could easily do). He could not easily figure out the string the server should send back as all he would have is the hash of that string. So, before the actual data is sent from the client to the server. The client would send it's secret string that the server would verify and then if that worked, the server would send its own secret string that the client must verify. We'd have two secret strings instead of one. > You'd use > #i to recompute the client's key and then use that derived key to > verify the string. This is called "key derivation" or "key > diversification". If an attacker gets hold of that client's key and > starts hosing you, you can disable that key without affecting the > other ones. (The client is issued only the derived key and never sees > the big secret). This is interesting. I didn't know that was possible. > > > Strings could come to the socket server from anywhere on the Net from > > any machine. This is outside my control. What is there to prevent a > > knowledgeable person from finding the py code on a client computer, > > understanding it and then being able to forge a string that the server > > will accept? > > Yes, if you're concerned about insecure clients, you have a much more > complex problem. But your x..z..y scheme is far worse than hmac. > Once the attacker figures that out, there's no security at all. I dropped the x,y,z scheme after your first response ;) > > What is the actual application, if you can say? Depending on the > environment and constraints, various approaches are possible. Nothing important. It just logs network data. It's an anti-theft program for laptops that phones home data like this: public and private IP(s), MAC addy, date, time, etc. Maybe I'm putting too much thought into it. Python encourages good design and I try to follow that encouragement when coding... even for trivial things such as this. -- http://mail.python.org/mailman/listinfo/python-list
Re: socketServer questions
On Mon, 2005-10-10 at 07:46 -0700, Paul Rubinhttp: wrote: > rbt <[EMAIL PROTECTED]> writes: > > > Instead, for client #i, let that client's key be something like > > > hmac(your_big_secret, str(i)).digest() > > > and the client would send #i as part of the string. > > > > How is this different from sending a pre-defined string from the client > > that the server knows the md5 hash of? The clients know the string, the > > server knows the hash of that string. > > I'm confused, I don't understand what that md5 whatever would do for you. > I'm assuming the server is secure and the clients are less secure. > > > Also, could this not be done both ways? So that, if an attacker figures > > out the string he's supposed to send from a client to the server (which > > he could easily do). He could not easily figure out the string the > > server should send back as all he would have is the hash of that string. > > I'm still confused OK, we'll leave it at that and just accept that we're from different planets ;) Thanks for the help. -- http://mail.python.org/mailman/listinfo/python-list
speeding up Python when using wmi
Here's a quick and dirty version of winver.exe written in Python: http://filebox.vt.edu/users/rtilley/public/winver/winver.html It uses wmi to get OS information from Windows... it works well, but it's slow... too slow. Is there any way to speed up wmi? In the past, I used the platform and sys modules to do some of what winver.exe does and they were rather fast. However, since switching to wmi (for a more accurate representation) thinngs have gotten slow... real slow. I suppose this could be a wmi only issue not related at all to Python. Any tips or ideas? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: speeding up Python when using wmi
Tim Golden wrote: > [rbt] > >> Here's a quick and dirty version of winver.exe written in Python: > > [.. snip ..] > >> It uses wmi to get OS information from Windows... it works well, but >> it's slow... too slow. Is there any way to speed up wmi? > >> In the past, I used the platform and sys modules to do some of what >> winver.exe does and they were rather fast. However, since switching to > >> wmi (for a more accurate representation) thinngs have gotten slow... >> real slow. > >> I suppose this could be a wmi only issue not related at all to Python. > > In short, I recommend replacing the wmi module by the underlying > calls which it hides, and replacing Tkinter by a win32gui MessageBox. > The wmi module does some magicish things which are useful for > interactive > browsing, but will only slow you down if you know exactly what you need. > As you don't need anything more than a native message box, don't > bother with GUI loops etc. Windows will do that for you in a Modal > Dialog (here message box). > > This was going to be a longer post comparing versions, but in short > running this code: > > > import win32gui > import win32com.client > > for os in win32com.client.GetObject ("winmgmts:").InstancesOf > ("Win32_OperatingSystem"): > win32gui.MessageBox ( > 0, > os.Properties_ ("Caption").Value + "\n" + \ > os.Properties_ ("TotalVisibleMemorySize").Value + "\n" + \ > os.Properties_ ("Version").Value + "\n" + \ > os.Properties_ ("CSDVersion").Value, > "Platform Info", > 0 > ) > Wow... thanks. I didn't expect someone to completely rewrite it like that. I'll use your example and name it PyWinver and attribute it to you. Hope you don't mind. Great learning experience. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: How to execute an EXE via os.system() with spaces in the directory name?
[EMAIL PROTECTED] wrote: > This comes up from time to time. The brain damage is all Windows', not > Python's. Here's one thread which seems to suggest a bizarre doubling > of the initial quote of the commandline. > > http://groups.google.com/group/comp.lang.python/browse_frm/thread/89d94656ea393d5b/ef40a65017848671 I do this: # remove spaces from ends of filenames. for root, dirs, files in os.walk('programs'): for fname in files: new_fname = fname.strip() if new_fname != fname: new_path = os.path.join(root,new_fname) old_path = os.path.join(root,fname) os.renames(old_path,new_path) # remove spaces from middle of filenames. for root, dirs, files in os.walk('programs'): for f in files: new_f = string.replace(f, ' ' , '-') new_path = os.path.join(root,new_f) old_path = os.path.join(root,f) os.renames(old_path,new_path) # install files. for root, dirs, files in os.walk('programs'): installable = ['.exe', '.msi', '.EXE', '.MSI'] for f in files: ext = os.path.splitext(f) if ext[1] in installable: print f install = os.system(os.path.join(root,f)) -- http://mail.python.org/mailman/listinfo/python-list
extract python install info from registry
On windows xp, is there an easy way to extract the information that Python added to the registry as it was installed? -- http://mail.python.org/mailman/listinfo/python-list
Re: extract python install info from registry
Laszlo Zsolt Nagy wrote: > rbt wrote: > >> On windows xp, is there an easy way to extract the information that >> Python added to the registry as it was installed? >> >> > Using regedit.exe, look at the registry keys and values under > > HKEY_LOCAL_MACHINE\Software\Python > > If you need to know how to read the registry from Python: please install > the python win32 extensions (or use ActivePython). > > Les > There's more to it than that... isn't there? I've used _winreg and the win32 extensions in the past when working with the registry. I thought perhaps someone had already scripted something to extract this info. I'm creating a Python plugin for Bartpe (Windows Pre-Install Environment) and it works OK, but to make it work _exactly_ like it does on XP (.py and .pyw associate with python and pythonw), I need to extract the reg entries so I can recreate them in the WinPE environment. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: extract python install info from registry
gene tani wrote: >> There's more to it than that... isn't there? I've used _winreg and the >> win32 extensions in the past when working with the registry. I thought >> perhaps someone had already scripted something to extract this info. >> > > Yes, a small firm named Microsoft has done this (but not tested w/2.4): > > http://www.microsoft.com/technet/scriptcenter/scripts/python/os/registry/osrgpy01.mspx > That tells me this: Caption: Registry Current Size: 2 Description: Registry Install Date: 20051125152108.00-300 Maximum Size: 54 Name: Microsoft Windows XP Professional|C:\WINDOWS|\Device\Harddisk0\Partition1 Proposed Size: 54 Status: OK -- http://mail.python.org/mailman/listinfo/python-list
arbitrary number of arguments in a function declaration
How do I set up a function so that it can take an arbitrary number of arguments? For example, I have a bunch of expenses which may grow or shrink depending on the client's circumstance and a function that sums them up... hard coding them is tedious. How might I make this dynamic so that it can handle any amount of expenses? def tot_expenses(self, e0, e1, e2, e3): pass -- http://mail.python.org/mailman/listinfo/python-list
Re: arbitrary number of arguments in a function declaration
Nick Coghlan wrote: rbt wrote: How do I set up a function so that it can take an arbitrary number of arguments? For example, I have a bunch of expenses which may grow or shrink depending on the client's circumstance and a function that sums them up... hard coding them is tedious. How might I make this dynamic so that it can handle any amount of expenses? def tot_expenses(self, e0, e1, e2, e3): pass The Python Tutorial is a wonderful thing. . . But so is this list ;) Anyway, you can either set up your function to take a proper list, and then discover that the sum function already exists to add up the contents of a list: def tot_expenses(self, expenses): self.total_expenses = sum(expenses) Or, have the function take a variable number of arguments, and do the same thing: def tot_expenses(self, *args): self.total_expenses = sum(args) Cheers, Nick. Many thanks! -- http://mail.python.org/mailman/listinfo/python-list
ftplib with unknown file names
How can I use ftplib to retrieve files when I do not know their names? I can do this to get a listing of the directory's contents: ftp_server.retrlines('LIST') The output from this goes to the console and I can't figure out how to turn that into something I can use to actually get the files (like a list of file names). I read a bit about the callback function that can be passed to retrlines but I couldn't figure out how to use it. Any help is appreciated. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: ftplib with unknown file names
Jeremy Jones wrote: rbt wrote: How can I use ftplib to retrieve files when I do not know their names? I can do this to get a listing of the directory's contents: ftp_server.retrlines('LIST') The output from this goes to the console and I can't figure out how to turn that into something I can use to actually get the files (like a list of file names). I read a bit about the callback function that can be passed to retrlines but I couldn't figure out how to use it. Any help is appreciated. Thanks! .nlst(argument) will return a list of file names. Here are the docs for the nlst command: http://www.python.org/doc/current/lib/ftp-objects.html HTH, Jeremy Jones Very good Jeremy! Thank you for pointing that out. It works great. -- http://mail.python.org/mailman/listinfo/python-list
exceptions and items in a list
If I have a Python list that I'm iterating over and one of the objects in the list raises an exception and I have code like this: try: do something to object in list except Exception: pass Does the code just skip the bad object and continue with the other objects in the list, or does it stop? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: exceptions and items in a list
Andrey Tatarinov wrote: rbt wrote: If I have a Python list that I'm iterating over and one of the objects in the list raises an exception and I have code like this: try: do something to object in list except Exception: pass Does the code just skip the bad object and continue with the other objects in the list, or does it stop? # skip bad object and continue with others for object in objects: try: #do something to object except Exception: pass # stop at first bad object try: for object in objects: #do something to object except Exception: pass Thanks Andrey. That's a great example of how to do it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Debian says "Warning! you are running an untested version of Python." on 2.3
Nick Craig-Wood wrote: > Alex Stapleton <[EMAIL PROTECTED]> wrote: >> Whenever I run python I get >> >> "Warning! you are running an untested version of Python." >> >> prepended to the start of any output on stdout. >> >> This is with Debian and python 2.3 (running the debian 2.1 and 2.2 >> binaries doesn't have this effect) > > What version of a) Debian and b) python are you running? > > I don't have that problem here (I'm running testing/sarge) Same here... Debian testing with Python2.3 no problem. Perhaps he's running Debian unstable? Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: dynamic data types
Charlie wrote: Hi, The description of Python always mentions "very high level dynamic data types". Now, I can't seem to find any examples of these (nothing described with this term anyway). Is this simply refering to built-in dynamic data structures such as lists and dictionaries, with a great deal of operators defined on? Or is there something else meant by "dynamic data types" in Python? Regards, Charlie I've always thought of it like this... in C, we have to do something like this when declaring a variable: int x = 0; We had to specifically tell the language compiler that x is an integer. In Python, all we have to do is: x = 0 The interpretor knows that x is an integer. We can also change the type like this: str(x) float(x) long(x) etc... To me, this is why Python types are called dynamic. They are easy to setup and easy to modify when compared to older, more static languages. Bye -- http://mail.python.org/mailman/listinfo/python-list
Re: Install Python 2.4 on Fedora 3 Core
Bill wrote: I have less than a week experience on linux, so I am a new newbie. Python 2.3 came preinstalled. I installed version 2.4. All seemed to go well except it installed to usr/local? 1. Was it wrong to install when logged in as 'root'? Does it make a difference? 2. I looked in the package editor and there was no way to uninstall 2.3? Should I? If so, how can I? If not,what are the problems, if any, of having both. Thank you for your help. Bill, /usr/local is the path on Unix systems where add-on software is traditionally installed. RH used to use Python heavily for many aspects of their RHL distribution. I suspect they still do this with Fedora. This is probably why the version of Python that came with Fedore cannot be removed (at least not easily). It's OK to have the RH version and the latest Python.org version installed. You'll just have to specify the path to the version you wish to use when running your scripts like this: /usr/local/python There are some more advanced things you could do and I suspect others will give you better advice. But all in all, you're OK. Have fun. -- http://mail.python.org/mailman/listinfo/python-list
Memory Usage
Would a Python process consume more memory on a PC with lots of memory? For example, say I have the same Python script running on two WinXP computers that both have Python 2.4.0. One computer has 256 MB of Ram while the other has 2 GB of Ram. On the machine with less Ram, the process takes about 1 MB of Ram. On the machine with more Ram, it uses 9 MB of Ram. Is this normal and expected behavior? Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory Usage
Peter Hansen wrote: rbt wrote: Would a Python process consume more memory on a PC with lots of memory? For example, say I have the same Python script running on two WinXP computers that both have Python 2.4.0. One computer has 256 MB of Ram while the other has 2 GB of Ram. On the machine with less Ram, the process takes about 1 MB of Ram. On the machine with more Ram, it uses 9 MB of Ram. Is this normal and expected behavior? It's probably not normal if this is *really* the memory usage, but I would expect to see such behaviour, given how difficult it is to measure *actual* memory usage. How are you measuring it? Just by looking at the Mem Usage column in the Task Manager? -Peter That's right. I look at that column. Should I measue mem usage in some other way? -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 and proxy question
Fuzzyman wrote: urllib2 (under windows) will auto-detect your proxy settings and use those. Normally that's a good thing (I guess), except when it's not ! How do I switch off this behaviour ? I'm behind a censoring proxy and wanting to test things *locally*. IE is set to not use the proxy when fetching local adresses, but urllib2 ignores that part of the setting and uses the proxy for everything. The only way I can test are changing my IE settings back and forth every time. Most annoying. I can see how to *add* a new proxy to urllib2, but not how to force it to not use a proxy. I may well be missing something obvious though. Anyone able to help ? Regards, Fuzzy http://www.voidspace.org.uk/python/index.shtml "Alternatively, the optional proxies argument may be used to explicitly specify proxies. It must be a dictionary mapping scheme names to proxy URLs, where an empty dictionary causes no proxies to be used" # Don't use any proxies filehandle = urllib.urlopen(some_url, proxies={}) -- http://mail.python.org/mailman/listinfo/python-list
exclude binary files from os.walk
Is there an easy way to exclude binary files (I'm working on Windows XP) from the file list returned by os.walk()? Also, when reading files and you're unsure as to whether or not they are ascii or binary, I've always thought it safer to 'rb' on the read, is this correct... and if so, what's the reasoning behind this? Again all of this pertains to files on Windows XP and Python 2.4 Many thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: exclude binary files from os.walk
Grant Edwards wrote: On 2005-01-26, rbt <[EMAIL PROTECTED]> wrote: Is there an easy way to exclude binary files (I'm working on Windows XP) from the file list returned by os.walk()? Sure, assuming you can provide a rigorous definition of 'binary files'. :) non-ascii -- http://mail.python.org/mailman/listinfo/python-list
Re: MSI Difficulties
brolewis wrote: I am trying to deploy Python onto a number of laptops and have been trying to take advantage of Python 2.4's MSI installer. I have tried using the following commands to install, but to no avail: msiexec /i python-2.4.msi /qb ALLUSERS=1 -- and -- msiexec /i python-2.4.msi /qb ALLUSERS=1 ADDLOCAL=ALL However both of these commands fail to update the Windows path to include C:\Python24 and as such creates problems for me when trying to actually use Python. How can I rectify this situation? Is there a command I am missing? Do I need to manually update the Path, and if so, is there a programatic way I can do this? Thanks in advance I've always manually updated the path... but I suppose there may be a way to do this automatically during install or programatically (bat file or py script afterwards). Perhaps someone more knowledgeable can answer these questions? Best of luck! -- http://mail.python.org/mailman/listinfo/python-list
Re: The next Xah-lee post contest
Steve wrote: Hi All, For sometime now, I have just been a passive lurker on this list. Of late I saw an increase in the number of posts by Xah Lee, and I have to admit, what he lacks in understanding of the various programming languages he talks about, he makes up for in creativity. So, I was wondering, how would it be to be Mr Lee. That got me thinking of his next post. Well, I know through my days of lurking around, a lot of people here love creative challenges ...so here's one for you. Write up the next Xah Lee post... Unix donkey! You not elegant. You have poor design. Sloppy Perl monkey! You be lazy! You code very very bad. Xah know all! -- http://mail.python.org/mailman/listinfo/python-list
Description Field in WinXP Services
How does one associate a "Description" with a Windows service written in Python? I've just started experimenting with Python services. Here's my code... copied straight from Mr. Hammond's "Python Programming on Win32": import win32serviceutil import win32service import win32event class test_py_service(win32serviceutil.ServiceFramework): _svc_name_ = "test_py_service" ## Tried the following to no avail. _svc_description_ = "Test written by Brad" _svc_display_name_ = "Test Python Service" def __init__(self, args): win32serviceutil.ServiceFramework.__init__(self, args) self.hWaitStop = win32event.CreateEvent(None, 0, 0, None) def SvcStop(self): self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING) win32event.SetEvent(self.hWaitStop) def SvcDoRun(self): win32event.WaitForSingleObject(self.hWaitStop, win32event.INFINITE) if __name__ == '__main__': win32serviceutil.HandleCommandLine(test_py_service) -- http://mail.python.org/mailman/listinfo/python-list
Re: Description Field in WinXP Services
Roger Upole wrote: ChangeServiceConfig2 is the api functions that sets the description, but it's not in the win32service module (yet). Roger OK, I can use _winreg to add the 'Description' field under the appropriate registry key. -- http://mail.python.org/mailman/listinfo/python-list
Re: Description Field in WinXP Services
rbt wrote: Roger Upole wrote: ChangeServiceConfig2 is the api functions that sets the description, but it's not in the win32service module (yet). Roger OK, I can use _winreg to add the 'Description' field under the appropriate registry key. Here's an example of it... kludgey but it works ;) from _winreg import * import time def Svc_Description(): try: key_location = r"SYSTEM\CurrentControlSet\Services\test_py_service" svc_key = OpenKey(HKEY_LOCAL_MACHINE, key_location, 0, KEY_ALL_ACCESS) SetValueEx(svc_key,'Description',0,REG_SZ,u"Brad's Test Python Service") CloseKey(svc_key) except Exception, e: print e Svc_Description() time.sleep(10) -- http://mail.python.org/mailman/listinfo/python-list
string issue
Either I'm crazy and I'm missing the obvious here or there is something wrong with this code. Element 5 of this list says it doesn't contain the string 255, when that's *ALL* it contains... why would it think that??? import time ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98', '127.0.0.1', '255.0.0.0', '255', '128.173.255.34'] for ip in ips: if '255' in ip: try: print "Removing", ip ips.remove(ip) except Exception, e: print e print ips time.sleep(5) Someone tell me I'm going crazy ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: string issue
Thanks guys... list comprehension it is! Bill Mill wrote: On Fri, 04 Feb 2005 14:23:36 -0500, rbt <[EMAIL PROTECTED]> wrote: Either I'm crazy and I'm missing the obvious here or there is something wrong with this code. Element 5 of this list says it doesn't contain the string 255, when that's *ALL* it contains... why would it think that??? import time ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98', '127.0.0.1', '255.0.0.0', '255', '128.173.255.34'] for ip in ips: if '255' in ip: try: print "Removing", ip ips.remove(ip) except Exception, e: print e print ips time.sleep(5) You're gong crazy: ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98', ... '127.0.0.1', '255.0.0.0', '255', '128.173.255.34'] for ip in ips: ... if '255' in ip: print ip ... 255.255.255.255 255.0.0.0 255 128.173.255.34 The problem is that you're operating in-place on an array while it's being iterated over. Since the iterator is only created once, you're can't change the array while you're iterating over it. Instead, try a list comprehension: ips = [ip for ip in ips if '255' not in ip] ips ['128.173.120.79', '198.82.247.98', '127.0.0.1'] Peace Bill Mill bill.mill at gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: string issue
Steve Holden wrote: rbt wrote: Either I'm crazy and I'm missing the obvious here or there is something wrong with this code. Element 5 of this list says it doesn't contain the string 255, when that's *ALL* it contains... why would it think that??? import time ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98', '127.0.0.1', '255.0.0.0', '255', '128.173.255.34'] for ip in ips: if '255' in ip: try: print "Removing", ip ips.remove(ip) except Exception, e: print e print ips time.sleep(5) Someone tell me I'm going crazy ;) You are modifying the list as you iterate over it. Instead, iterate over a copy by using: for ip in ips[:]: ... regards Steve Very neat. That's a trick that everyone should know about. I vote it goes in Dr. Dobbs newsletter. -- http://mail.python.org/mailman/listinfo/python-list
Re: string issue
Alan McIntyre wrote: I think it's because you're modifying the list as you're iterating over In this case then, shouldn't my 'except Exception' raise an error or warning like: "Hey, stupid, you can't iterate and object and change it at the same time!" Or, perhaps something similar? -- http://mail.python.org/mailman/listinfo/python-list
Re: string issue
Alan McIntyre wrote: I think it's because you're modifying the list as you're iterating over it. One last clarification on this. It's OK to modify the elements of a list, but not the list itself while iterating over it... is that the correct way to think about this? -- http://mail.python.org/mailman/listinfo/python-list
Re: string issue
John J. Lee wrote: Steve Holden <[EMAIL PROTECTED]> writes: [...] You are modifying the list as you iterate over it. Instead, iterate over a copy by using: for ip in ips[:]: ... Just to help popularise the alternative idiom, which IMO is significantly less cryptic (sane constructors of mutable objects almost always make a copy, and list is no exception: it's guaranteed to do so): for ip in list(ips): ... Works back to at least Python 1.5.2. John I don't know that that approach is less cryptic. ips is already a list... it looks cryptic to make it a list again, doesn't it? IMO, the two are equally cryptic. The epitome of clarity would be copy(ips)... now *that* makes sense, of course, ips[:] or list(ips) work equally well to the programmer who has learned them. -- http://mail.python.org/mailman/listinfo/python-list
Re: string issue
Bill Mill wrote: On Fri, 04 Feb 2005 15:25:04 -0500, rbt <[EMAIL PROTECTED]> wrote: John J. Lee wrote: Steve Holden <[EMAIL PROTECTED]> writes: [...] You are modifying the list as you iterate over it. Instead, iterate over a copy by using: for ip in ips[:]: ... Just to help popularise the alternative idiom, which IMO is significantly less cryptic (sane constructors of mutable objects almost always make a copy, and list is no exception: it's guaranteed to do so): for ip in list(ips): ... Works back to at least Python 1.5.2. John I don't know that that approach is less cryptic. ips is already a list... it looks cryptic to make it a list again, doesn't it? IMO, the two are equally cryptic. The epitome of clarity would be copy(ips)... now *that* makes sense, of course, ips[:] or list(ips) work equally well to the programmer who has learned them. Howsabout: from copy import copy ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98', ... '127.0.0.1', '255.0.0.0', '255', '128.173.255.34'] for ip in copy(ips): ... if '255' in ip: ... ips.remove(ip) ... ips ['128.173.120.79', '198.82.247.98', '127.0.0.1'] But I still think that the list comprehension is the best. Peace Bill Mill bill.mill at gmail.com Wow, I did not know that a copy module existed. I made all that up about copy being the perfect example here. Great minds think alike ;) I fell Guidoish. -- http://mail.python.org/mailman/listinfo/python-list
breaking out of nested loop
What is the appropriate way to break out of this while loop if the for loop finds a match? while 1: for x in xrange(len(group)): try: mix = random.sample(group, x) make_string = ''.join(mix) n = md5.new(make_string) match = n.hexdigest() if match == target: print "Collision!!!" print make_string Stop = time.strftime("%H:%M:%S-%m-%d-%y", time.localtime()) print "Stop", Stop break else: continue except Exception, e: print e -- http://mail.python.org/mailman/listinfo/python-list
Re: breaking out of nested loop
Thanks guys... that works great. Now I understand why sometimes logic such as 'while not true' is used ;) On Tue, 2005-07-12 at 10:51 -0400, Peter Hansen wrote: > rbt wrote: > > What is the appropriate way to break out of this while loop if the for > > loop finds a match? > > Define a flag first: > > keepGoing = True > > > while 1: > while keepGoing: > > > for x in xrange(len(group)): > > try: > ... > > if match == target: > > print "Collision!!!" > > print make_string > > Set the flag here, then do the break: >keepGoing = False > > > break > > Tada... > > -Peter -- http://mail.python.org/mailman/listinfo/python-list
all possible combinations
Say I have a list that has 3 letters in it: ['a', 'b', 'c'] I want to print all the possible 4 digit combinations of those 3 letters: 4^3 = 64 abaa aaba aaab acaa aaca aaac ... What is the most efficient way to do this? -- http://mail.python.org/mailman/listinfo/python-list
Re: all possible combinations
On Thu, 2005-07-14 at 00:47 +1000, Steven D'Aprano wrote: > On Wed, 13 Jul 2005 10:21:19 -0400, rbt wrote: > > > Say I have a list that has 3 letters in it: > > > > ['a', 'b', 'c'] > > > > I want to print all the possible 4 digit combinations of those 3 > > letters: > > > > 4^3 = 64 > > > > > > abaa > > aaba > > aaab > > acaa > > aaca > > aaac > > ... > > > > What is the most efficient way to do this? > > Efficient for who? The user? The programmer? The computer? Efficient use > of speed or memory or development time? The CPU > > If you want the fastest runtime efficiency, a lookup table of > pre-calculated values. That is an O(1) operation, and you don't get any > faster than that. > > If you expect to extend the program to arbitrary lists, pre-calculation > isn't practical, so you need an algorithm to calculate permutations (order > matters) or combinations (order doesn't matter). My list is not arbitrary. I'm looking for all 'combinations' as I originally posted. Order does not matter to me... just all possibilities. -- http://mail.python.org/mailman/listinfo/python-list
Re: all possible combinations
On Wed, 2005-07-13 at 10:21 -0400, rbt wrote: > Say I have a list that has 3 letters in it: > > ['a', 'b', 'c'] > > I want to print all the possible 4 digit combinations of those 3 > letters: > > 4^3 = 64 > > > abaa > aaba > aaab > acaa > aaca > aaac > ... > > What is the most efficient way to do this? Expanding this to 4^4 (256) to test the random.sample function produces interesting results. It never finds more than 24 combinations out of the possible 256. This leads to the question... how 'random' is sample ;) Try it for yourselves: test = list('1234') combinations = [] while 1: combo = random.sample(test, 4) possibility = ''.join(combo) if possibility not in combinations: print possibility combinations.append(possibility) continue else: continue -- http://mail.python.org/mailman/listinfo/python-list
Re: all possible combinations
On Wed, 2005-07-13 at 11:09 -0400, rbt wrote: > On Wed, 2005-07-13 at 10:21 -0400, rbt wrote: > > Say I have a list that has 3 letters in it: > > > > ['a', 'b', 'c'] > > > > I want to print all the possible 4 digit combinations of those 3 > > letters: > > > > 4^3 = 64 > > > > > > abaa > > aaba > > aaab > > acaa > > aaca > > aaac > > ... > > > > What is the most efficient way to do this? > > Expanding this to 4^4 (256) to test the random.sample function produces > interesting results. It never finds more than 24 combinations out of the > possible 256. This leads to the question... how 'random' is sample ;) > > Try it for yourselves: > > test = list('1234') > > combinations = [] > while 1: > combo = random.sample(test, 4) > possibility = ''.join(combo) > if possibility not in combinations: > print possibility > combinations.append(possibility) > continue > else: > continue > Someone pointed out off-list that this is doing permutation, not combination. Is there a way to make random.sample to do combinations? -- http://mail.python.org/mailman/listinfo/python-list
Re: all possible combinations
Thanks to all who were helpful... some of you guys are too harsh and cynical. Here's what I came up with. I believe it's a proper combination, but I'm sure someone will point out that I'm wrong ;) groups = [list('abc'),list('abc'),list('abc'),list('abc')] already = [] while 1: LIST = [] for g in groups: sample = random.sample(g, 1) LIST.append(sample[0]) STRING = ''.join(LIST) if STRING not in already: print STRING already.append(STRING) if len(already) == 81: break On Thu, 2005-07-14 at 23:18 +1000, John Machin wrote: > Steven D'Aprano wrote: > > On Thu, 14 Jul 2005 08:49:05 +1000, John Machin wrote: > > > > > >>"You keep using that word. I do not think it means what you think it means." > >> > >>Both of you please google("define: combination") > > > > > > Combination: "a coordinated sequence of chess moves". > > > > "An option position that is effected by either a purchase of two long > > positions or two short positions. The investor purchases a call and a put > > (or sells a call and a put) with different expiration dates and/or > > different strike prices." > > > > Or perhaps "in Scheme, a function call, consisting of a function name and > > arguments written within parentheses." > > > > Yes, mathematically the definition of combination includes that order does > > not matter. But that certainly isn't the case in common English. Now, > > John, given the tone of the posts you are complaining about, > > Wrong -- no complaint. Another quote: "It's a joke, Joyce!" > > > do you think > > I was using combination in the precise mathematical sense, or the common > > English sense? > > As in "Please don't get your combinations in a twist?"? > > > > > (Hint: the very first definition Google finds is "a collection of things > > that have been combined; an assemblage of separate parts or qualities ". > > Not a word there about order mattering or not.) > -- http://mail.python.org/mailman/listinfo/python-list
Re: all possible combinations
Wow. That's neat. I'm going to use it. Thanks! On Thu, 2005-07-14 at 19:52 -0400, Peter Hansen wrote: > Bengt Richter wrote: > > On Thu, 14 Jul 2005 17:10:37 -0400, William Park <[EMAIL PROTECTED]> wrote: > > It's a one liner in Python too ;-) > > > > >>> print ' '.join([x+y+z+q for s in ['abc'] for x in s for y in s for z > > in s for q in s]) > > Or for the cost of an import and a lambda, you can keep it looking real > obscure and generalize it to any size of sequence ('abcdef' or whatever) > and a result length of up to 52 elements: > > >>> from string import letters as L > >>> cartesian = lambda seq, num: eval("list(%s for __ in [seq] > %s)" % ('+'.join(L[:num]), 'for %s in __ ' * num % tuple(L[:num]))) > # (there are spaces at any line breaks above) > > >>> cartesian('abcde', 6) > ['aa', 'ab', 'ac', 'ad', 'ae', 'ba', > ... > 'ec', 'ed', 'ee'] > >>> len(_) > 15625 > > > > -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Python scripts wont run - HELP
On Mon, 2005-07-18 at 17:22 +0100, John Abel wrote: > windozbloz wrote: > > >Bye Bye Billy Bob... > > > >Hello All, > >I'm a fairly literate windoz amateur programmer mostly in visual basic. I > >have switched to SuSE 9.2 Pro and am trying to quickly come up to speed > >with Python 2.3.4. I can run three or four line scripts from the command > >line but have not been able to execute a script from a file. > > > >I have used EMACS and JEDIT to create small test routines. I would right > >click the file and set properties to executable. I would then click the > >icon, the bouncy ball would do its thing then a dialog box would flash on > >the screen for a fraction of a second. I could tell it had a progress bar > >on it but could not catch anything else on it. Then nothing else would > >happen. > > > >If I could execute a script the world would once again be my playground... > >PLEASE HELP. > > > > > > > > > > > You will need to include > > #!/usr/bin/python > > At the top of your script. > > HTH > > J Or, better yet: #!/usr/bin/env python ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: goto
On Mon, 2005-07-18 at 12:27 -0600, Steven Bethard wrote: > Hayri ERDENER wrote: > > what is the equivalent of C languages' goto statement in python? > > Download the goto module: > http://www.entrian.com/goto/ > And you can use goto to your heart's content. And to the horror of all > your friends/coworkers. ;) > > STeVe Shouldn't that be "to the horror of all your goto-snob friends." IMO, most of the people who deride goto do so because they heard or read where someone else did. Many of the world's most profitable software companies (MS for example) have thousands of goto statements in their code... oh the horror of it all. Why aren't these enlightened-by-the-gods know-it-alls as profitable as these obviously ignorant companies? -- http://mail.python.org/mailman/listinfo/python-list
Re: goto
10 PRINT "YOU'RE NOT RIGHT IN THE HEAD." 20 GOTO 10 On Tue, 2005-07-19 at 02:33 +, Leif K-Brooks wrote: > rbt wrote: > > IMO, most of the people who deride goto do so because they heard or read > > where someone else did. > > 1 GOTO 17 > 2 mean,GOTO 5 > 3 couldGOTO 6 > 4 with GOTO 7 > 5 what GOTO 3 > 6 possibly GOTO 24 > 7 you! GOTO 21 > 8 that GOTO 18 > 9 really, GOTO 23 > 10 understandable? > 11 neat.GOTO 16 > 12 and GOTO 25 > 13 are GOTO 9 > 14 IGOTO 26 > 15 wrongGOTO 20 > 16 IGOTO 2 > 17 Yes, GOTO 14 > 18 simple GOTO 12 > 19 agreeGOTO 4 > 20 with GOTO 22 > 21 GotosGOTO 13 > 22 somethingGOTO 8 > 23 really GOTO 11 > 24 be GOTO 15 > 25 easily GOTO 10 > 26 totally GOTO 19 -- http://mail.python.org/mailman/listinfo/python-list
Re: goto
On Tue, 2005-07-19 at 10:02 -0400, George Sakkis wrote: > "rbt" <[EMAIL PROTECTED]> wrote: > > > On Mon, 2005-07-18 at 12:27 -0600, Steven Bethard wrote: > > > Hayri ERDENER wrote: > > > > what is the equivalent of C languages' goto statement in python? > > > > > > Download the goto module: > > > http://www.entrian.com/goto/ > > > And you can use goto to your heart's content. And to the horror of all > > > your friends/coworkers. ;) > > > > > > STeVe > > > > Shouldn't that be "to the horror of all your goto-snob friends." > > > > IMO, most of the people who deride goto do so because they heard or read > > where someone else did. > > > > Many of the world's most profitable software companies (MS for example) > > have thousands of goto statements in their code... oh the horror of it > > all. Why aren't these enlightened-by-the-gods know-it-alls as profitable > > as these obviously ignorant companies? > > > It should not really come as a shock that the same fellow who came up with a > brilliant efficient way > to generate all permutations (http://tinyurl.com/dnazs) is also in favor of > goto. > > Coming next from rbt: "Pointer arithmetic in python ?". > > George > > I have moments of brilliance and moments of ignorance. You must admit though, that was a unique way of generating permutations... how many other people would have thought of that approach? It did solve my problem ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: goto
On Wed, 2005-07-20 at 03:43 +1000, Steven D'Aprano wrote: > On Tue, 19 Jul 2005 11:29:58 -0400, rbt wrote: > > >> It should not really come as a shock that the same fellow who came up with > >> a brilliant efficient way > >> to generate all permutations (http://tinyurl.com/dnazs) is also in favor > >> of goto. > >> > >> Coming next from rbt: "Pointer arithmetic in python ?". > >> > >> George > >> > >> > > > > I have moments of brilliance and moments of ignorance. You must admit > > though, that was a unique way of generating permutations... how many > > other people would have thought of that approach? It did solve my > > problem ;) > > Sorry rbt, but your algorithm isn't unique, nor was it clever, and in fact > your implementation wasn't very good even by the undemanding requirements > of the algorithm. It is just a minor modification of bogosort (also known > as "bozo-sort") algorithm: > > http://en.wikipedia.org/wiki/Bogosort > > I quote: > > "...bogosort is 'the archetypal perversely awful algorithm', one example > of which is attempting to sort a deck of cards by repeatedly throwing the > deck in the air, picking the cards up at random, and then testing whether > the cards are in sorted order." > > Bogosort is nothing to be proud of, except as a joke. It *was* a joke. -- http://mail.python.org/mailman/listinfo/python-list
broken links
How can I find broken links (links that point to files that do not exist) in a directory and remove them using Python? I'm working on RHEL4 Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: broken links
I found it: os.path.exists(path) On Fri, 2005-07-22 at 09:22 -0400, rbt wrote: > How can I find broken links (links that point to files that do not > exist) in a directory and remove them using Python? I'm working on RHEL4 > > Thanks, > rbt -- http://mail.python.org/mailman/listinfo/python-list
pretty windows installer for py scripts
Any recommendations on a windows packager/installer that's free? I need it to allow non-tech users to install some python scripts... you know, "Click Next"... "Click Next"... "Click Finish"... "You're Done!" and everything just magically works ;) -- http://mail.python.org/mailman/listinfo/python-list
Get Mac OSX Version
Is there a similar function to sys.getwindowsversion() for Macs? Many thanks! -- http://mail.python.org/mailman/listinfo/python-list
appended crontab entries with py script
How can I safely append a crontab entry to a crontab file progammatically with Python? I need to handle crontabs that currently have entries and crontabs that are empty. Also, I'd like this to work across Linux and BSD systems. Any pointers? -- http://mail.python.org/mailman/listinfo/python-list
Re: appended crontab entries with py script
On Tue, 2005-09-13 at 23:18 -0400, Mike Meyer wrote: > rbt <[EMAIL PROTECTED]> writes: > > > How can I safely append a crontab entry to a crontab file > > progammatically with Python? > > Well, one way would be to invoke the system crontab utility and use an > "editor" that passes the file to your program, and reads the results > back. > > > I need to handle crontabs that currently have entries and crontabs that > > are empty. Also, I'd like this to work across Linux and BSD systems. > > > > Any pointers? > > I think most Free Unix systems use the Vixie cron, and the non-free > ones have a "crontab" command (do some of them call it cron?) with the > same API. So you're pretty safe using that. > > If you want to assume that you're going to have the vixie cron, you > could dig into it's guts to see what it does for locking, and do that > by hand. > > current_crontab.txt') cur_cron.read() cur_cron.close() fp = file('current_crontab.txt', 'a') print >> fp, "0 * * * * %s/.theft_recovery.py" %home fp.close() load = os.popen('crontab current_crontab.txt') load.read() load.close() -- http://mail.python.org/mailman/listinfo/python-list
win32 service and time.sleep()
I have a win32 service written in Python. It works well. It sends a report of the status of the machine via email periodically. The one problem I have is this... while trying to send an email, the script loops until a send happens and then it breaks. Should it be unable to send, it sleeps for 10 minutes with time.sleep(600) and then wakes and tries again. This is when the problem occurs. I can't stop the service while the program is sleeping. When I try, it just hangs until a reboot. Can some suggest how to fix this? Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
win32 service and time.sleep()
I have a win32 service written in Python. It works well. It sends a report of the status of the machine via email periodically. The one problem I have is this... while trying to send an email, the script loops until a send happens and then it breaks. Should it be unable to send, it sleeps for 10 minutes with time.sleep(600) and then wakes and tries again. This is when the problem occurs. I can't stop the service while the program is sleeping. When I try, it just hangs until a reboot. Can some suggest how to fix this? Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: Finding where to store application data portably
On Tue, 2005-09-20 at 23:03 +0100, Tony Houghton wrote: > I'm using pygame to write a game called Bombz which needs to save some > data in a directory associated with it. In Unix/Linux I'd probably use > "~/.bombz", in Windows something like > "C:\Documents And Settings\\Applicacation Data\Bombz". > > There are plenty of messages in the archives for this group about how to > find the correct location in Windows, but what about Mac OS? ~/.bombz works equally well on OSX. -- http://mail.python.org/mailman/listinfo/python-list
safest way to open files on all platforms
I believe that this is the safest way to open files on Windows, Linux, Mac and Unix, but I wanted to ask here just to be sure: fp = file('filename', 'rb') The 'b' on the end being the most important ingredient (especially on Windows as a simple 'r' on a binary file might cause some sort of corruption). Anyway, am I right in saying this? That 'rb' is the safest way to open files for reading and that it should work well on *all* Python supported platforms? Many thanks, RBT -- http://mail.python.org/mailman/listinfo/python-list
Re: safest way to open files on all platforms
Fredrik Lundh wrote: "rbt" wrote: I believe that this is the safest way to open files on Windows, Linux, Mac and Unix, but I wanted to ask here just to be sure: fp = file('filename', 'rb') The 'b' on the end being the most important ingredient (especially on Windows as a simple 'r' on a binary file might cause some sort of corruption). Anyway, am I right in saying this? That 'rb' is the safest way to open files for reading and that it should work well on *all* Python supported platforms? "rb" works on all platforms, yes. but it doesn't work well if you're reading a text file. (when reading text files, the "U" option may also be useful. see doc for details) I'm using 'rb' in a situation where all files on the drive are opened. I'm not checking how the file is encoded before opening it (text, unicode, jpeg, etc.) That's why I though 'rb' would be safest. Can 'U' be used with 'rb'? Should it be? From what I read, 'U' handles the different ways in which the OS handles the 'end of line' on text files, but other than that, I don't think it's useful for me. -- http://mail.python.org/mailman/listinfo/python-list
os.walk() usage
I'm trying to write very small, modular code as functions to break up a big monolithic script that does a file system search for particular strings. The script works well, but it's not easy to maintain or add features to. I'd like to have a function that builds a list of files with os.walk() and then have other functions accept that list as a parameter and modify it as needed. For example, if the user has specified that certain files and folders be excluded from the walk, I'd like to have functions like this: def build_list(path): fs_list = os.walk(path) return fs_list def skip_files(fs_list): remove files return fs_list def skip_dirs(fs_list): remove dirs return fs_list def search(fs_list): pass The problem I'm encountering is passing the list to other functions. It's almost as if each function needs to build the list itself (walk the filesystem)... which gets in the way of what I was asked to do (break this thing up into modular, maintainable pieces). Any tips on this? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: os.walk() usage
[EMAIL PROTECTED] wrote: every object in os.walk() returns a 3-tuple, like below, it seems your code assumes it returns only a list of files. for d in os.walk('c:\\temp'): (dirpath, dirnames, filenames) = d print dirpath print dirnames print filenames Thank you, this fixed it. I didn't read the docs well enough ;) -- http://mail.python.org/mailman/listinfo/python-list
more os.walk() issues... probably user error
This function is intended to remove unwanted files and dirs from os.walk(). It will return correctly *IF* I leave the 'for fs in fs_objects' statement out (basically leave out the entire purpose of the function). It's odd, when the program goes into that statment... even when only a 'pass', and nothing else is present, nothing is returned. Why is that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on either platform. def build_clean_list(self, path): file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files'] fs_objects = os.walk(path, topdown=True) ## for fs in fs_objects: ## ##for f in fs[2]: ##if f in file_skip_list: ##print f ##fs[2].remove(f) ## ##for d in fs[1]: ##if d in dir_skip_list: ##print d ##fs[1].remove(d) return fs_objects -- http://mail.python.org/mailman/listinfo/python-list
Re: more os.walk() issues... probably user error
rbt wrote: This function is intended to remove unwanted files and dirs from os.walk(). It will return correctly *IF* I leave the 'for fs in fs_objects' statement out (basically leave out the entire purpose of the function). It's odd, when the program goes into that statment... even when only a 'pass', and nothing else is present, nothing is returned. Why is that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on either platform. def build_clean_list(self, path): file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files'] fs_objects = os.walk(path, topdown=True) ## for fs in fs_objects: ## ##for f in fs[2]: ##if f in file_skip_list: ##print f ##fs[2].remove(f) ## ##for d in fs[1]: ##if d in dir_skip_list: ##print d ##fs[1].remove(d) return fs_objects Just to clarify, it's wrong of me to say that 'nothing is returned'... in either case, this is what is returned: Here's what was returned and its type: But, I can't iterate over the returned object when I descend into the for statement I mentioned above. -- http://mail.python.org/mailman/listinfo/python-list
Re: more os.walk() issues... probably user error
Kent Johnson wrote: rbt wrote: rbt wrote: This function is intended to remove unwanted files and dirs from os.walk(). It will return correctly *IF* I leave the 'for fs in fs_objects' statement out (basically leave out the entire purpose of the function). It's odd, when the program goes into that statment... even when only a 'pass', and nothing else is present, nothing is returned. Why is that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on either platform. def build_clean_list(self, path): file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files'] fs_objects = os.walk(path, topdown=True) fs_objects is a generator, not a list. This loop is exhausting fs_objects, so when you return fs_objects is at the end of iteration, there is nothing left. That makes sense. Thanks for the explanation. I've never used generators before. ## for fs in fs_objects: ## ##for f in fs[2]: ##if f in file_skip_list: ##print f ##fs[2].remove(f) ## ##for d in fs[1]: ##if d in dir_skip_list: ##print d ##fs[1].remove(d) Add this here: yield fs and take out the return. This turns build_clean_list() into a generator function and you will be able to iterate the result. I'll try this. Will the changes I made (file and dir removals from os.walk()) be reflected in the generator object? Is it safe to remove objects this way and pass the results in a generator on to another function? Sorry for all the questions, I just like to fully understand something before I start doing it with confidence. rbt Kent return fs_objects Just to clarify, it's wrong of me to say that 'nothing is returned'... in either case, this is what is returned: Here's what was returned and its type: But, I can't iterate over the returned object when I descend into the for statement I mentioned above. -- http://mail.python.org/mailman/listinfo/python-list
os.walk()
Could someone demonstrate the correct/proper way to use os.walk() to skip certain files and folders while walking a specified path? I've read the module docs and googled to no avail and posted here about other os.walk issues, but I think I need to back up to the basics or find another tool as this isn't going anywhere fast... I've tried this: for root, dirs, files in os.walk(path, topdown=True): file_skip_list = ['file1', 'file2'] dir_skip_list = ['dir1', 'dir2'] for f in files: if f in file_skip_list files.remove(f) for d in dirs: if d in dir_skip_list: dirs.remove(d) NOW, ANALYZE THE FILES And This: files = [f for f in files if f not in file_skip_list] dirs = [d for d in dirs if dir not in dir_skip_list] NOW, ANAYLZE THE FILES The problem I run into is that some of the files and dirs are not removed while others are. I can be more specific and give exact examples if needed. On WinXP, 'pagefile.sys' is always removed, while 'UsrClass.dat' is *never* removed, etc. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.walk()
Roel Schroeven wrote: rbt wrote: The problem I run into is that some of the files and dirs are not removed while others are. I can be more specific and give exact examples if needed. On WinXP, 'pagefile.sys' is always removed, while 'UsrClass.dat' is *never* removed, etc. Keep in mind that the comparisons are done case sensitive; are you sure that there's no problem regarding uppercase/lowercase? I've noticed that. I've tried most all combinations possible with the same results. -- http://mail.python.org/mailman/listinfo/python-list
re.compile and very specific searches
Is it possible to use re.compile to exclude certain numbers? For example, this will find IP addresses: ip = re.compile('\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}') But it will also find 999.999.999.999 (something which could not possibly be an IPv4 address). Can re.compile be configured to filter results like this out? -- http://mail.python.org/mailman/listinfo/python-list
Re: re.compile and very specific searches
John Machin wrote: Diez B. Roggisch wrote: So I'd suggest you dump re and do it like this: address = "192.168.1.1" def validate_ip4(address): digits = address.split(".") if len(digits) == 4: for d in digits: if int(d) < 0 or int(d) > 255: return False return True The OP wanted to "find" IP addresses -- unclear whether re.search or re.match is required. Your solution doesn't address the search case. For the match case, it needs some augmentation. It will fall apart if presented with something like "..." or "comp.lang.python.announce". AND while I'm at it ... in the event of a valid string of digits, it will evaluate int(d) twice, rather unnecessarily & uglily. So: match case: ! for s in strings_possibly_containing_digits: ! # if not(s.isdigit() and 0 <= int(s) <= 255): # prettier, but test on zero is now redundant ! if not s.isdigit() or int(s) > 255: and the search case: DON'T dump re; it can find highly probable candidates (using a regexp like the OP's original or yours) a damn sight faster than anything else this side of C or Pyrex. Then you validate the result, with a cut-down validator that relies on the fact that there are 4 segments and they contain only digits: This is what I ended up doing... re.compile and then findall(data) does an excellent job finding all strings that look like ipv4 addys, then the split works just as well in weeding out strings that are not actual ipv4 addys. Thanks to all for the advice! -- http://mail.python.org/mailman/listinfo/python-list
Re: How to wrap a class's methods?
Jeff Shannon wrote: You could probably also do this as a factory function, rather than as a class (also untested!): def Wrapper(func): def wrapped(self, *args, **kwargs): s, r = func(self, *args, **kwargs) if s != 'OK': raise NotOK((s,r)) return r return wrapped I believe that this will be semantically almost equivalent, but conceptually slightly simpler. Jeff Shannon This is a nice example. I have used sub-functions (functions within functions) recently with some code, but I've wondered how proper it is to do this. Is this type of thing frowned upon? -- http://mail.python.org/mailman/listinfo/python-list
searching pdf files for certain info
Not really a Python question... but here goes: Is there a way to read the content of a PDF file and decode it with Python? I'd like to read PDF's, decode them, and then search the data for certain strings. Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: searching pdf files for certain info
Andreas Lobinger wrote: Aloha, rbt wrote: Not really a Python question... but here goes: Is there a way to read the content of a PDF file and decode it with Python? I'd like to read PDF's, decode them, and then search the data for certain strings. First of all, http://groups.google.de/groups?selm=400CF2E3.29506EAE%40netsurf.de&output=gplain still applies here. If you can deal with a very basic implementation of a pdf-lib you might be interested in http://sourceforge.net/projects/pdfplayground In the CVS (or the current snapshot) you can find in ppg/Doc/text_extract.txt an example for text extraction. >>> import pdffile >>> import pages >>> import zlib >>> pf = pdffile.pdffile('../pdf-testset1/a.pdf') >>> pp = pages.pages(pf) >>> c = zlib.decompress(pf[pp.pagelist[0]['/Contents']].stream) >>> op = pdftool.parse_content(c) >>> sop = [x[1] for x in op if x[0] in ["'", "Tj"]] >>> for a in sop: print a[0] Wishing a happy day LOBI Thanks guys... what if I convert it to PS via printing it to a file or something? Would that make it easier to work with? -- http://mail.python.org/mailman/listinfo/python-list
Re: searching pdf files for certain info
Andreas Lobinger wrote: Aloha, rbt wrote: Thanks guys... what if I convert it to PS via printing it to a file or something? Would that make it easier to work with? Not really... The classical PS Drivers (f.e. Acroread4-Unix print-> ps) simply define the pdf graphics and text operators as PS commands and copy the pdf content directly. Wishing a happy day LOBI I downloaded ghostscript for Win32 and added it to my PATH (C:\gs\gs8.15\lib AND C:\gs\gs8.15\bin). I found that ps2ascii works well on PDF files and it's entirely free. Usage: ps2ascii PDF_file.pdf > ASCII_file.txt However, bundling a 9+ MB package with a 5K script and convincing users to install it is another matter altogether. -- http://mail.python.org/mailman/listinfo/python-list
Re: searching pdf files for certain info
Tom Willis wrote: I tried that for something not python related and I was getting sporadic spaces everywhere. I am assuming this is not the case in your experience? On Tue, 22 Feb 2005 10:45:09 -0500, rbt <[EMAIL PROTECTED]> wrote: Andreas Lobinger wrote: Aloha, rbt wrote: Thanks guys... what if I convert it to PS via printing it to a file or something? Would that make it easier to work with? Not really... The classical PS Drivers (f.e. Acroread4-Unix print-> ps) simply define the pdf graphics and text operators as PS commands and copy the pdf content directly. Wishing a happy day LOBI I downloaded ghostscript for Win32 and added it to my PATH (C:\gs\gs8.15\lib AND C:\gs\gs8.15\bin). I found that ps2ascii works well on PDF files and it's entirely free. Usage: ps2ascii PDF_file.pdf > ASCII_file.txt However, bundling a 9+ MB package with a 5K script and convincing users to install it is another matter altogether. -- http://mail.python.org/mailman/listinfo/python-list For my purpose, it works fine. I'm searching for certain strings that might be in the document... all I need is a readable file. Layout, fonts and/or presentation is unimportant to me. -- http://mail.python.org/mailman/listinfo/python-list
Re: searching pdf files for certain info
Tom Willis wrote: Well sporadic spaces in strings would cause problems would it not? an example The String: "Patient Face Sheet"--->pdftotext--->"P a tie n t Face Sheet" I'm just curious if you see anything like that, since I really have no clue about ps or pdf etc...but I have a strong desire to replace a really flaky commercial tool. And if I can do it with free stuff, all the better my boss will love me. No, I do not see that type of behavior. I'm looking for strings that resemble SS numbers. So my strings look like this: nnn-nn-. The ps2ascii util in ghostscript reproduces strings in the format that I expect. BTW, I'm not using pdftotext. I'm using *ps2ascii*. -- http://mail.python.org/mailman/listinfo/python-list
hidden attribute on Windows files
How do I enable the hidden attribute when creating files on Windows computers? I'd *really* prefer to do from the standard Python installer (no win32 extensions). Any tips? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: autoexecution in Windows
Earl Eiland wrote: How does one make a Python program auto-execute in Windows? Earl No program (python or other) can just arbitrarily execute. A user has to click it or a cron-like utility (Task Scheduler) has to execute it at a set time. registry entries (such as run) can execute programs too. Also, proper Windows services can be configured to start at boot. Again, nothing can just arbitrarily execute. If it could, viruses would be a *nightmare* -- http://mail.python.org/mailman/listinfo/python-list
Re: autoexecution in Windows
Earl Eiland wrote: In Linux, if I make the first line #!/path/to/Python, all I have to do to execute the program is type ./FileName (assuming my pwd is the same as FileName). what's the Windows equivalent? Earl On Mon, 2005-03-07 at 13:36, F. Petitjean wrote: Le Mon, 07 Mar 2005 13:25:35 -0700, Earl Eiland a Ãcrit : How does one make a Python program auto-execute in Windows? Earl write a virus ? :-) What do you mean by  auto-execute  ? Regards Look at the 'pathext' variable under the system's environmental variables. Add .py and .pyw to that and you're good to go. -- http://mail.python.org/mailman/listinfo/python-list
os.walk(entire filesystem)
More of an OS question than a Python question, but it is Python related so here goes: When I do os.walk('/') on a Linux computer, the entire file system is walked. On windows, however, I can only walk one drive at a time (C:\, D:\, etc.). Is there a way to make os.walk() behave on Windows as it behaves on Linux? I'd like to walk the entire file system at once... not one drive at a time. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Reading a HP Printer Web Interface
Hello there, Depending on the firmware version of the HP printer and the model type, one will encounter a myriad of combinations of the following strings while reading the index page: hp HP color Color Printer Printer Status Status: Device: Device Status laserjet LaserJet How can I go about determining if a site is indeed the Web interface to a HP printer? The goal is to remove all HP printers from a list of publicly available Web sites... I've tried this approach, but it gets messy quickly when I attempt to account for all possible combinations that HP uses: f = urllib2.urlopen("http://%s"; %host) data = f.read() f.close() if 'hp' or 'HP' and 'color' or 'Color' and 'Printer' or 'Printer Status' in data: DISREGARD THE IP I'm sure there's a more graceful way to go about this while maintaining a high degree of accuracy and as few false positives as possible. Any tips or pointers? Thanks in advance! -- http://mail.python.org/mailman/listinfo/python-list
Re: spaces in re.compile()
AndrewN wrote: d = re.compile(' \d{3}\.\d{3}\.\d{3} ') d.findall(' 123.345.678 ') [' 123.345.678 '] Works for me. Yes, you're correct. That works if there is a space at the front and back. However, place '123.345.678' in a file by itself and it doesn't work. What I'm trying to avoid is something like this '1234.345.6789' Notice the 4 chars in the first and last part? findall gets '234.345.678' and returns positive... I thought that by requiring spaces I could avoid matches such as this, but I was wrong. How can I get what I'm looking for w/o getting the other stuff as well? -- http://mail.python.org/mailman/listinfo/python-list
spaces in re.compile()
Is it possible to use spaces in a re.compile()? For example, I want to make sure one space exists right before this string and right after it: re.compile ('\d{3,3}\.\d{3,3}\.\d{3,3}\.\d{3,3}') I've tried this, but it didn't work: re.compile (' \d{3,3}\.\d{3,3}\.\d{3,3}\.\d{3,3} ') Any ideas? -- http://mail.python.org/mailman/listinfo/python-list
Re: spaces in re.compile()
Jeff Epler wrote: Maybe you want r'\b'. From 'pydoc sre': \b Matches the empty string, but only at the start or end of a word. import re r = re.compile( r'\btest\b' ) print r.findall("testy") print r.findall(" testy ") print r.findall(" test ") print r.findall("test") That works great. Thanks for the tip! -- http://mail.python.org/mailman/listinfo/python-list
os.stat access and modify time same on WinXP
I'm using the standard NTFS file system. The only time the access time is updated is when the file is modified or saved (with no changes). What's up with that? Shouldn't a read/view update the access time? -- http://mail.python.org/mailman/listinfo/python-list
Re: os.stat access and modify time same on WinXP
Peter Hansen wrote: rbt wrote: I'm using the standard NTFS file system. The only time the access time is updated is when the file is modified or saved (with no changes). What's up with that? Shouldn't a read/view update the access time? See http://www.microsoft.com/resources/documentation/Windows/XP/all/reskit/en-us/Default.asp?url=/resources/documentation/Windows/XP/all/reskit/en-us/prkc_fil_punq.asp Quoting: ''' The Last Access Time on disk is not always current because NTFS looks for a one-hour interval before forcing the Last Access Time updates to disk. NTFS also delays writing the Last Access Time to disk when users or programs perform read-only operations on a file or folder, such as listing the folder's contents or reading (but not changing) a file in the folder. If the Last Access Time is kept current on disk for read operations, all read operations become write operations, which impacts NTFS performance. Note * File-based queries of Last Access Time are accurate even if all on-disk values are not current. NTFS returns the correct value on queries because the accurate value is stored in memory. ''' Does that help? (Just went through this myself the other day.) -Peter Yes, it does. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
truncating a file from the top down
Hi guys, I need to truncate a file from the top down. I imagine doing something like this: if os.stat says the file is too big: read the file trim = only keep the last 2008 bytes (This is where I get stuck) write trim back out to the original file Would someone demonstrate the *best* most efficient way of doing this? Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: truncating a file from the top down
Mike Rovner wrote: Right. Thanks for the correction. Fredrik Lundh wrote: Mike Rovner wrote: if os.stat says the_file is too big: fh = open(the_file, 'rb') fh.seek(2008, 2) should be fh.seek(-2008, 2) right? data = fh.read() fh.close() assert len(data)==2008 # you may want some error processing here fh = open(the_file, 'wb') fh.write(data) fh.close() or if os.path.getsize(the_file) > TOO_BIG: fh = open(the_file, 'rb+') fh.seek(-2008, 2) data = fh.read() fh.seek(0) # rewind fh.write(data) fh.truncate() fh.close() Thanks for the info guys! -- http://mail.python.org/mailman/listinfo/python-list
Re: How to execute a cmd line program without invoking console window?
Tian wrote: In Windows, I have been simply using os.system() to run command line program in python. but there will be a black console window. How can I run the program without invoking that window? i guess there are some function with which I can redirect the output? name your scripts with .pyw extensions instead of .py extensions -- http://mail.python.org/mailman/listinfo/python-list
Re: Write an hexadecimal file
Larry Bates wrote: There is not such thing as a hexadecimal file. Right, 300 is 300 whether you choose to represent it in decimal, binary, hex, etc... it's still only 300 of something ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: Change between Python 2.3 and 2.4 under WinXP
Martin v. Löwis wrote: Of course, it is not all that clear what the OP actually wanted. For all we know, he wanted to "alternate quickly (with batch file or similary) between python23 and python24"... Maybe off-topic for this thread, but I noticed that when installing 2.4.1 that 2.4.0 is automatically removed. Does 2.4 do the same thing to 2.3 versions? -- http://mail.python.org/mailman/listinfo/python-list
check interpreter version before running script
Is there a recommended or 'Best Practices' way of checking the version of python before running scripts? I have scripts that use the os.walk() feature (introduced in 2.3) and users running 2.2 who get errors. Instead of telling them, 'Upgrade you Python Install, I'd like to use sys.version or some other way of checking before running. Whatever I do, I need it to work on Linux, Mac and Windows. I thought of sys.version... but getting info out of it seems awkward to me. First 5 chars are '2.4.1' in string format have to split it up and convert it to ints to do proper checking, etc. It doesn't seem that sys.version was built with this type of usage in mind. So, what is the *best* most correct way to go about this? Thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: Add System Path?!?
[EMAIL PROTECTED] wrote: Hello NG, I have a GUI (written in wxPython) that calls some external exe files. Some of them requires that I add to the PATH variable 1 directory. Basically, the exe are located in: /MyApp/Solvers/FirstExe /MyApp/Solvers/SecondExe And so on, while the dll needed by these exe are located in: /MyApp/MyDll These exe files do not work if I don't set the PATH variable also to that adress. I know I can do it by hand (and also my users can), but I wonder if there is a way to do it in Python... Thank you for all suggestions/pointers. Andrea. Check out the 'Path' value under this registry key: "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment" You can import _winreg to edit it as you like. I think it's a string... just append your path(s) to them. rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: check interpreter version before running script
Peter Otten wrote: rbt wrote: Is there a recommended or 'Best Practices' way of checking the version of python before running scripts? I have scripts that use the os.walk() feature (introduced in 2.3) and users running 2.2 who get errors. Instead of telling them, 'Upgrade you Python Install, I'd like to use sys.version or some other way of checking before running. I like import os try: os.walk except AttributeError: # implement fallback No need to remember in which version the desired feature came to be. Peter Thanks for all the tips. I found this tip from Peter the best for my situation. -- http://mail.python.org/mailman/listinfo/python-list
shebang in cross platform scripts
Haven't tested this on Windows yet... thought I'd ask here: Does the line below have any negative impact on Windows machines? I develop and test mostly on Unix, but my scripts are often used on Win systems too. #!/usr/bin/env python Many thanks, rbt -- http://mail.python.org/mailman/listinfo/python-list
Re: Distributing Python Apps and MySQL
dcrespo wrote: Hi there... I want to distribute my python apps and the MySQL Database in the easiest way possible. I mean a user just run the installation file and all is automaticly installed. Any suggestions? My knowledge: I know, as many of you, that there's py2exe for compiling python apps for running under Windows. But what about the database structure and data? I think it could be reached through a .qry run in the MySQL database from an installation instruction. But (one more time) what about the automated installation of the MySQL database without user intervention? Daniel Crespo I don't think one can distribute mysql within a software package w/o buying a commercial license to do so. Check out their licensing on their website here: "When your application is not licensed under either the GPL-compatible Free Software License as defined by the Free Software Foundation or approved by OSI, and you intend to or you may distribute MySQL software, you must first obtain a commercial license to the MySQL product." http://www.mysql.com/company/legal/licensing/commercial-license.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Interpreter problem
Steve Holden wrote: Greg Lindstrom wrote: I am using python 2.3.5 on a Linux system and have an odd problem dealing with the 'sha-bang' line. I have a file, driver.py which starts with #!/usr/bin/python and works fine (that is, when I type in ./driver.py at the command prompt the file runs as expected). I have another file, myotherfile.py which starts with the exact same line (#!/usr/bin/python) but I get : bad interpreter: No such file or directory There's almost certainly a carriage return as well as a newline in the shebang line. [...] regards Steve Not so. I get the same result with python 2.3 and 2.4 on Debian Linux Testing. Nothing odd at all in the shebang. -- http://mail.python.org/mailman/listinfo/python-list
Re: Interpreter problem
Steve Holden wrote: rbt wrote: Steve Holden wrote: Greg Lindstrom wrote: I am using python 2.3.5 on a Linux system and have an odd problem dealing with the 'sha-bang' line. I have a file, driver.py which starts with #!/usr/bin/python and works fine (that is, when I type in ./driver.py at the command prompt the file runs as expected). I have another file, myotherfile.py which starts with the exact same line (#!/usr/bin/python) but I get : bad interpreter: No such file or directory There's almost certainly a carriage return as well as a newline in the shebang line. [...] regards Steve Not so. I get the same result with python 2.3 and 2.4 on Debian Linux Testing. Nothing odd at all in the shebang. What, you are telling me you've checked the file with a command like head driver.py | od -bc and verified the absence of any extraneous characters? regards Steve You're right: [EMAIL PROTECTED]:~$ cd /usr/local/bin [EMAIL PROTECTED]:/usr/local/bin$ head web* | od -bc 000 043 041 057 165 163 162 057 142 151 156 057 160 171 164 150 157 # ! / u s r / b i n / p y t h o 020 156 015 012 144 145 146 040 167 145 142 137 142 141 143 153 165 n \r \n How can this be fixed? vim doesn't see it. -- http://mail.python.org/mailman/listinfo/python-list