Command-line tool able to take multiple commands at one time?
Per subject - I realize I can copy/paste a line at a time into an interactive session when I'm trying to debug, but was wondering if there is any tool out there that allows me to copy sections of working Python scripts to paste into my interactive console and let those run so I don't have to copy line-by-line. Not sure if iPython would meet that criteria or not or even if such a beast exists. It would be helpful for debugging some of my simpler, yet still lengthy at times, scripts. Thanks in advance. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Re: Command-line tool able to take multiple commands at one time?
I'll give it a try. I've been using PythonWin and SPE recently and hadn't really messed with IDLE too much as I liked the somewhat more advanced features of the other IDEs. I'll check it out again. Also thanks to all for the IP hints - I may check those out as well. -Pete "Devan L" <[EMAIL PROTECTED]> wrote: > > Peter A. Schott wrote: > > Per subject - I realize I can copy/paste a line at a time into an > > interactive > > session when I'm trying to debug, but was wondering if there is any tool out > > there that allows me to copy sections of working Python scripts to paste > > into my > > interactive console and let those run so I don't have to copy line-by-line. > > > > Not sure if iPython would meet that criteria or not or even if such a beast > > exists. It would be helpful for debugging some of my simpler, yet still > > lengthy > > at times, scripts. > > > > Thanks in advance. > > > > -Pete Schott > > It's called IDLE (At least, the interactive session with it). Some > people dislike IDLE, though. -- http://mail.python.org/mailman/listinfo/python-list
Re: Command-line tool able to take multiple commands at one time?
OK - I justed tested and may be doing something wrong, but it didn't work when I just tried it. I have something like this: X = "Value1" Y = "Value2" Z = "Value3" etc at the top of my script. When I copy/paste those three lines all at once into IDLE's interactive window, X is defined, Y and Z are not. That's more the behaviour I was hoping for - the ability to run parts of my code at a time in order to work through issues without too much trouble in mostly-working code. TIA, -Pete Schott Mike Meyer <[EMAIL PROTECTED]> wrote: > Peter A. Schott <[EMAIL PROTECTED]> writes: > > Per subject - I realize I can copy/paste a line at a time into an > > interactive > > session when I'm trying to debug, but was wondering if there is any tool out > > there that allows me to copy sections of working Python scripts to paste > > into my > > interactive console and let those run so I don't have to copy line-by-line. > > Um, just curious - doesn't copying multiple lines at a time work now? > That works fine for me on both OS X and Unix. > > http://mail.python.org/mailman/listinfo/python-list
Catching error text like that shown in console?
I know there's got to be an easy way to do this - I want a way to catch the error text that would normally be shown in an interactive session and put that value into a string I can use later. I've tried just using a catch statement and trying to convert the output to string, but this doesn't always work. I really don't have programs complex enough to do a lot of specific catching at this point - I just want to know: 1. something failed 2. here's the error output/traceback Anyone know how I can do this or if it's possible? Didn't find anything doing a search so I figured I'd hit up the experts. Thanks. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Best way to trap errors in ftplib?
Using ftplib.FTP object for a project we have here to upload/download files. I
know that I can wrap everything in try/except blocks, but I'm having trouble
getting the exact error messages out of the Exceptions.
I'd like to either send an e-mail or log the failure to a database. It would
also be nice to see what the message is and perhaps add some additional logic
for delay/retry or something similar.
Has anyone done any of the above? Willing to share some samples?
Here's a little of my code - maybe you can see what I'm doing/not doing.
def myuploadbinaryfunction(args here):
objFTP = ftplib.FTP(HostName, UserName, Password)
objFTP.cwd(RemotePath)
try:
objFTP.storbinary("STOR " + RemoteFile,
file(os.path.join(LocalPath,
LocalFile), "rb"))
SQLLogging.LogFTP(LocalFile, Company, 1)
except Exception, errObj:
SQLLogging.LogFTP(LocalFile, Company, 0)
reportError("Failed to upload " + LocalFile, str(errObj))
print Exception
print errObj
In the above, reportError will send an e-mail to alert. I realize that the
errObj in this case may not have the ability to convert to string, but am not
sure how to get the output otherwise. Not sure how to handle a retry of a
failure, either.
Thank you for your help!
-Peter Schott
--
http://mail.python.org/mailman/listinfo/python-list
Re: python to mssql
Brane, What are you wanting to know specifically? What is your client machine running? I've started using the Win32 modules to connect using ADODB and that works for me on Windows XP (hopefully 2003 as well, but not tested yet). I pulled those samples from someplace on the net using Google and adapted a couple of times until they worked the way I needed them to. I looked at pymssql and it didn't seem to be updated recently where I felt comfortable. Just my opinion, it may be the best out there. I guess knowing more about what you're looking for would help. -Pete Schott Brane <[EMAIL PROTECTED]> wrote: > can someone please give me some info regarding subject > please advice > regards > brane -- http://mail.python.org/mailman/listinfo/python-list
FTPLIB - retry files?
Is there any way to retry sending files with some delay up to a set number on failure? Sometimes we encounter a locked file on our server or the destination server and we want to retry that file in X seconds. Not exactly sure how to go about that right now short of moving the files elsewhere and then queuing them up again. Anyone done this? I'm still kind of new to this so am open to suggestions on how this might be possible as well. Thanks for your help. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
FTPLIB & FTPS or SFTP?
Does the ftplib support SFTP or FTPS? Is that part of a different module? We have a handful of partners who use FTPS or SFTP and I need to pull/push files to/from them. Thank you for all of your help. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Re: FTPLIB & FTPS or SFTP?
Can't seem to hit the site right now. I'll have to try back later. From what I can tell, there aren't any mirrors set up, either. Two quick recommendations is promising, though. Thanks for the recommendations. -Pete "Roger Binns" <[EMAIL PROTECTED]> wrote: > > "Peter A. Schott" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > > have a handful of partners who use FTPS or SFTP and I need to pull/push > > files > > to/from them. > > For SFTP, run don't walk, over to http://www.lag.net/paramiko/ > > Paramiko is a pure Python(*) implementation of SSH and all the > sub-protocols such as SFTP. Works on all platforms, very responsive > author, works reliably and LGPL license. > > (*) It does require PyCrypto which is a native code package available > for all platforms. > > Roger > -- http://mail.python.org/mailman/listinfo/python-list
ftplib help - delete from server after download results in 0-byte file
Got a strange scenario going on here in that I could have sworn this worked
yesterday. I am issuing binary retrieval calls to an FTP server, writing to a
file, close the file, then removing the file from the remote site. When I do
this, I end up with 0 byte files. I was hoping to avoid parsing a list of
remote and local files and matching them up that way because it would be easier
to remove on successful retrieve. I'm including some sample code and really
hoping someone can point out where I'm messing up.
Thanks.
import os, ftplib
DPI = "C:/DestinationFolder/"
#Actually named different in code, but shorter for reading here.
#Note - objFTP is already opened - this is not the issue.
objFTP.cwd("/ksdata_in")
#I'm open to better ways to do this. Can't use nlst if empty.
TestDir = objFTP.dir()
if TestDir <> None:
FTPRemoteList = objFTP.nlst()
#If the remote file is type SEM or PGP, do this.
for filename in [filename for filename in FTPRemoteList if \
(os.path.splitext(filename)[1] in [".sem",".SEM",".PGP",".pgp"])]:
try:
DestinationFile = open(os.path.join(DPI, filename), "wb")
print DestinationFile
objFTP.retrbinary("RETR " + filename, DestinationFile.write)
SQLLogging.LogFTP(filename, CompanyInput, 1, 'ToDrive')
DestinationFile.close()
#This is the culprit. When this is active, it writes a 0 byte file locally
# and deletes the file on the server.
#objFTP.delete(filename)
except:
SQLLogging.LogFTP(filename, CompanyInput, 0, 'ToUs')
print Exception
#This is my workaround in the meantime. If I've downloaded the files, get
#another remote listing and delete and files from remote that are downloaded
#locally and larger than 1Kbish for PGP files. SEM files are 0 bytes.
TestDir = objFTP.dir()
if TestDir <> None:
FTPRemoteList = objFTP.nlist()
for filename in FTPRemoteList:
if os.path.isfile(os.path.join(DownloadedPathInput, filename))
and \
os.path.splitext(filename)[1] in [".pgp",".PGP"] and \
os.stat(os.path.join(DownloadedPathInput, filename))[6] > 1000:
objFTP.delete(filename)
if os.path.isfile(os.path.join(DownloadedPathInput, filename))
and \
os.path.splitext(filename)[1] in [".SEM",".sem"]:
objFTP.delete(filename)
--
http://mail.python.org/mailman/listinfo/python-list
Re: httplib/HTTPS Post Problem
Have you tried using pycurl? That may be an easier way to port over your CURL
code directly. Relatively easy to use, too.
-Pete
[EMAIL PROTECTED] wrote:
> Hi,
>
> Sorry to post what might seem like a trivial problem here, but its
> driving me mad!
>
> I have a simple https client that uses httplib to post data to a web
> server.
>
> When I post over http & https using curl the data is recieved by the
> web server with no problems.
>
> When I post using my python client the headers get there, but the body
> of the message does not.
>
> My code is pretty standard and has the format:
>
>
> httplib.HTTPSConnection.debuglevel = 1
> connection = httplib.HTTPSConnection(host_name, key_file = key,
> cert_file = cert)
> connection.putrequest("POST", path)
> connection.putheader("Content-Length", str(len(body)))
>
> ...(some more headers)...
>
> connection.endheaders()
> connection.send(body)
>
> response = connection.getresponse()
> connection.close()
>
> (some code has been removed for clarity)..
>
> I can see in the debug messages the body getting sent, but nothing
> arrives at
> the server...
>
> I think I would understand whats going on better if I knew how Python
> uses the
> underlying socket - does it
>
> a) open the socket, send the header & body together or
> b) send the header, wait, then send the body?
>
> I think the answer to this question solve my problem - can anyone help?
>
> Thanks.
>
> p.s. I'm using Python 2.3.3 [GCC 3.3.3 (SuseLinx)]
--
http://mail.python.org/mailman/listinfo/python-list
Help sorting a list by file extension
Trying to operate on a list of files similar to this: test.1 test.2 test.3 test.4 test.10 test.15 test.20 etc. I want to sort them in numeric order instead of string order. I'm starting with this code: import os for filename in [filename for filename in os.listdir(os.getcwd())]: print filename #Write to file, but with filenames sorted by extension Desired result is a file containing something like: C:\MyFolder\test.1,test.001 C:\MyFolder\test.2,test.002 C:\MyFolder\test.3,test.003 C:\MyFolder\test.4,test.004 C:\MyFolder\test.10,test.010 C:\MyFolder\test.15,test.015 C:\MyFolder\test.20,test.020 I need to order by that extension for the file output. I know I've got to be missing something pretty simple, but am not sure what. Does anyone have any ideas on what I'm missing? Thanks. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Re: Help sorting a list by file extension
OK - I actually got something working last night with a list that is then converted into a dictionary (dealing with small sets of data - < 200 files per run). However, I like the sorted list option - I didn't realize that was even an option within the definition and wasn't quite sure how to get there. I realized I could use os.path.splitext and cast that to an int, but was having trouble with the sort. My files only have a single "." in them so this will work well for me. (from Tom's code) [name for dec,name in sorted((int(os.path.splitext(nm)[1][1:]),nm) for nm in namelist)] I'll give that a try - it would eliminate the dictionary part of my code and be a little more efficient. Thanks to all for the quick responses. -Pete Peter A. Schott <[EMAIL PROTECTED]> wrote: > Trying to operate on a list of files similar to this: > > test.1 > test.2 > test.3 > test.4 > test.10 > test.15 > test.20 > > etc. > > I want to sort them in numeric order instead of string order. I'm starting > with > this code: > > import os > > for filename in [filename for filename in os.listdir(os.getcwd())]: > print filename > #Write to file, but with filenames sorted by extension > > > Desired result is a file containing something like: > C:\MyFolder\test.1,test.001 > C:\MyFolder\test.2,test.002 > C:\MyFolder\test.3,test.003 > C:\MyFolder\test.4,test.004 > C:\MyFolder\test.10,test.010 > C:\MyFolder\test.15,test.015 > C:\MyFolder\test.20,test.020 > > I need to order by that extension for the file output. > > I know I've got to be missing something pretty simple, but am not sure what. > Does anyone have any ideas on what I'm missing? > > Thanks. > > -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Re: Text/IDE Python Editor?
Also, it depends on what you're trying to do. GUI, Web, Service-type apps, etc. I have started using Stan's Python Editor (www.stani.be) because I like the general features it offers. It does require wxpython to run, though. I've also used Boa Constructor and DrPython - both pretty good environments. I believe that they are both dependent on wxpython as well. Some people like Eric - now available for XP using PyQt under the Qt GPL'd version. I had a little trouble with it running correctly - probably due to a config issue on my part. I will admit to not spending a lot of time with it, though. If you don't need the GUI IDE, there are a whole bunch of options available. More than I'm able to delve into properly. Best answer is to play around a little with the different IDE's and find the one that works for your coding style. Also, check the archives for this group. This topic has come up quite a bit recently. -Pete Simon Brunning <[EMAIL PROTECTED]> wrote: > On 8/12/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Any recommendations on a editior/IDE for programming in python? > > http://wiki.python.org/moin/PythonEditors -- http://mail.python.org/mailman/listinfo/python-list
Python & SQL Server DTS?
Got a strange issue here. I know that MS allows other scripting languages to run in its ActiveX tasks in DTS. I can change one of the tasks to run Python code, create a Main function, and run it manually. However, once I try to run the entire package, the package hangs at this step. It doesn't appear to even start to execute. I've added code in my function to raise a message box using the win32 extensions. They work when I run the task manually, but none appear when I run it as part of the package. Anyone experienced this? Anyone worked with Python & DTS before? Any suggestions? Thanks in advance. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
ftplib - issues with some files not completing?
Any tips/tricks on how to ensure that I've got the complete file before closing the local file? I think my code is basically correct and works more than 99% of the time, but it still has an occasional issue with an encrypted file where it doesn't receive the whole file and I can't decrypt it. Should I checksum the files somehow or is there a standard trick to ensure I get the complete file? Thanks in advance. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Re: High Level FTP library
I've used the ftputil package before with some success. It's not bad, but it really depends on what you're trying to do. If you write a couple of custom functions to get you started, you should be able to put those in a module and re-use as needed. I know that I had to code my own stuff to handle some non-standard FTP flavors for some of our partners. Knowing how to work ftplib was pretty helpful at that point as ftputil didn't support those versions of FTP. It still did a great job for regular stuff, but not for that. Hope that helps some. -Pete "Paulo Pinto" <[EMAIL PROTECTED]> wrote: > Hello, > > Is there any Python library similar to NET::FTP from Perl? > ftplib seems too lowlevel. > > I already found a few, but would like to get one that is > endorsed by the community. > > Thanks, > Paulo > -- http://mail.python.org/mailman/listinfo/python-list
Sorta noob question - file vs. open?
Been reading the docs saying that file should replace open in our code, but this
doesn't seem to work:
# Open file for writing, write something, close file
MyFile = file("MyFile.txt", "w")
MyFile.write("This is a test.")
MyFile.close()
However, using:
MyFile = open("MyFile.txt", "w")
MyFile.write("This is a test.")
MyFile.close()
I have no problems.
I'm sure that I'm missing something here about using open vs file, but am not
sure what. Probably just mis-read something. If anyone can point me to what I
didn't quite get, I'd appreciate the information.
Thanks.
-Pete
--
http://mail.python.org/mailman/listinfo/python-list
Re: Sorta noob question - file vs. open?
I'll have to try this again. I obviously did something wrong in my code. I was
getting errors about not being able to write a string because it wasn't
supported. It was driving me nuts for a while until I just gave up and went
back to open(). I'll do some more playing and if I continue to get errors, I'll
post the code I'm using at the time along with any errors I got.
I apologize for not including the details. A thread I was reading reminded me
about that odd behaviour from a couple of weeks ago and I wanted to see if I was
way off base as I could consistently reproduce it then on Python 2.4.1.
Thanks to all who replied. If open is still preferred, I will stick with that.
:-)
-Pete
Scott David Daniels <[EMAIL PROTECTED]> wrote:
> Peter A. Schott wrote:
> > Been reading the docs saying that file should replace open in our code, but
> > this
> > doesn't seem to work:
> This was really a misstatement; open is still preferred. file is now
> a built in class, and its constructor is the same as open. I think the
> current docs have been fixed.
>
> > # Open file for writing, write something, close file
> > MyFile = file("MyFile.txt", "w")
> > MyFile.write("This is a test.")
> > MyFile.close()
> But you've given us no idea what went wrong.
>
> > However, using:
> > MyFile = open("MyFile.txt", "w")
> > MyFile.write("This is a test.")
> > MyFile.close()
> I cannot help you if you don't tell me what went wrong.
>
> Both bits of code seem to do the same thing for me.
>
>
> --Scott David Daniels
> [EMAIL PROTECTED]
--
http://mail.python.org/mailman/listinfo/python-list
Re: FTP Error: Windows AS/400
I've used DOS FTP to hit AS/400 systems before and usually have to completely
qualify the file name or ensure that I'm in the correct directory. When you use
command-line FTP to get the file, do these commands mimic your commands?
What version of python are you using?
If you issue PWD on the FTP site, what is returned when you're in the correct
folder? Can you use the sendcmd() function to mimic the functionality of cwd()
for this purpose?
I don't have easy access to an AS/400 with which to test, but maybe there's an
idea or two here that's worth trying.
-Pete
"Tim G." <[EMAIL PROTECTED]> wrote:
> I am trying to use Win Python to ftp files from an AS/400 IFS directory
> down to my Windows machine.
>
> I seem to get stuck when I am trying to send a command to the AS/400 to
> switch file systems from native to IFS and then to issue a cd to my
> folder. I get the error below.
>
> If anyone has had experience trying to ftp from a 400, I would greatly
> appreciate any info on the topic.
>
>
> Code:
>
> import ftplib, os
>
> filename=''
> path = "jde7333"
> os.chdir('c:\\ftp_jde400')
>
> ftp = ftplib.FTP('test400')
> ftp.login('oneworld', 'onew0r1d')
> #ftp.sendcmd('quote site namefmt 1')
> ftp.sendcmd('site namefmt 1')
> ftp.cwd(path)
> #remotefiles = ftp.nlst()
> #ftp.retrlines('LIST')
> #for filename in remotefiles:
> #file = open(filename, 'wb')
> #ftp.retrbinary('RETR ' + filename, file.write, 1024)
> #file.close()
>
> ftp.quit()
>
>
> Error:
>
> Traceback (most recent call last):
> File "C:\Python24\Tools\scripts\ftp400.py", line 10, in ?
> ftp.cwd(path)
> File "C:\Python24\lib\ftplib.py", line 494, in cwd
> return self.voidcmd(cmd)
> File "C:\Python24\lib\ftplib.py", line 246, in voidcmd
> return self.voidresp()
> File "C:\Python24\lib\ftplib.py", line 221, in voidresp
> resp = self.getresp()
> File "C:\Python24\lib\ftplib.py", line 214, in getresp
> raise error_perm, resp
> ftplib.error_perm: 501 Unknown extension in database file name.
--
http://mail.python.org/mailman/listinfo/python-list
ftplib - file.write, file.close() - verify finished writing locally before delete source?
If I want to verify that a file has finished writing before deleting the remote
file, what would be the best method?
Current code on Python 2.4:
#filename - remote FTP server File Name
#NewFile - local file copy of the remote file
#objFTP - standard ftplib.FTP object
NewFile = open(os.path.join(InputPath, RemoteFileName), "wb")
objFTP.retrbinary("RETR " + filename, NewFile.write)
NewFile.close()
#Here is my problem - when I call this directly after NewFile.close(), I end up
#with 0-byte local file and successfully delete the remote file.
objFTP.delete(filename)
Any ideas on how to keep the remote file from being deleted if the local file
isn't finished writing? Any ideas on why I might be retrieving a 0-byte file
locally when I call like this? If I remove the delete statement, it seems to
work.
TIA,
-Pete Schott
--
http://mail.python.org/mailman/listinfo/python-list
Re: About Databases...
Have to agree with others here - get a good database backend. MSSQL/Oracle - good choices commercially. Both of these offer good feature sets and have a lot of support from users. However, they are commercial apps and not necessarily free. (You can play with MS SQL Server in the Developer edition for $50 if you're interested - full feature set with minor throttling to keep from being used in production.) MySQL - fast, but not completely relational. This may or may not matter to you. For what it does, it is very fast. PostGreSQL - good feature set, open source. Not sure on speed, but it should run natively on Unix, Linux, and Win32 (not sure about BSD or others). There are quite a few others - these just come to mind off the top of my head. I'd tend to stay away from a file-based system because people tend to want some form of report off of data like this pretty quickly. Files may be fast, but they're not easy to get data from. As others have suggested, take a little time or pull in a consultant if necessary to learn SQL and build a good database to store this. You may not even need heavy normalization at first, though I'd keep in mind that repetitive data may be best stored in lookup tables. Putting this into a database will also give you access to some of the advanced set-based operations that database engines tend to be very good with. As for your questions: Numerics tend to be base types in most DB platforms. They tend to take up less space than character values and can be operated on mathematically without much trouble. I'd tend towards numeric as opposed to float if you need accuracy. With the right code in the backend modules/classes, you should be able to link anything you need to objects in the database. Best of luck, -Pete Schott [EMAIL PROTECTED] wrote: > Hello NG, > >I am still quite a newbie with Python (I intensely use wxPython, anyway). > I would like to know what are, in your opinions, the best/faster databases > that I could use in Python (and, of course, I should be able to "link" > everything > with a wxPython GUI)? Specifically, I work on Reservoir Simulation, and > usually I have to store a discrete/huge amount of data (it depends on the > oil field). As you may have understood, I know almost NOTHING about databases > ;-) > In general, my data will be numeric (floats, integers). Will a binary storage > (if it is possible) reduce the size of the DB? And what about speed in > storing/retrieving > data? > > Thank you a lot for every suggestion. > > Andrea. > -- http://mail.python.org/mailman/listinfo/python-list
Best way to parse file into db-type layout?
I've got a file that seems to come across more like a dictionary from what I can tell. Something like the following format: ###,1,val_1,2,val_2,3,val_3,5,val_5,10,val_10 ###,1,val_1,2,val_2,3,val_3,5,val_5,11,val_11,25,val_25,967,val_967 In other words, different layouts (defined mostly by what is in val_1, val_2, val_3). The ,#, fields indicate what "field" from our mainframe the corresponding value represents. Is there a good way to parse this into a DB-type format where I only pull out the values corresponding to the appropriate field numbers? Seems like converting to a dictionary of some sort would be best, but I don't quite know how I would go about that. In this case, the first field is a value unto itself - represents a "letter type" that would be associated with the rest of the record. The fields are either present or not, no placeholder if there's no value for e.g. Field #4. Thanks for any help or pointers you can give me. -Pete -- http://mail.python.org/mailman/listinfo/python-list
Re: Best way to parse file into db-type layout?
That looks promising. The field numbers are pre-defined at the mainframe level.
This may help me get to my ultimate goal which is to pump these into a DB on a
row-by-row basis ( :-P ) I'll have to do some playing around with this. I
knew that it looked like a dictionary, but wasn't sure how best to handle this.
One follow-up question: I'll end up getting multiple records for each "type".
Would I be referencing these by row[#][field#]?
Minor revision to the format is that starts like:
###,1,1,val_1,
I think right now the plan is to parse through the file and insert the pairs
directly into a DB table. Something like RowID, LetterType, date, Field#,
Value. I can get RowID and LetterType overall, date is a constant, the rest
would involve reading each pair and inserting both values into the table. Time
to hit the books a little more to get up to speed on all of this.
I really appreciate the help. I knew there had to be a better way to do this,
just wasn't sure what it was.
-Pete
John Machin <[EMAIL PROTECTED]> wrote:
> On Thu, 28 Apr 2005 23:34:31 GMT, Peter A. Schott
> <[EMAIL PROTECTED]> wrote:
>
> >I've got a file that seems to come across more like a dictionary from what I
> >can
> >tell. Something like the following format:
> >
> >###,1,val_1,2,val_2,3,val_3,5,val_5,10,val_10
> >###,1,val_1,2,val_2,3,val_3,5,val_5,11,val_11,25,val_25,967,val_967
> >
> >In other words, different layouts (defined mostly by what is in val_1, val_2,
> >val_3).
> >
> >The ,#, fields indicate what "field" from our mainframe the corresponding
> >value
> >represents.
> >
> >Is there a good way to parse this into a DB-type format where I only pull out
> >the values corresponding to the appropriate field numbers? Seems like
> >converting to a dictionary of some sort would be best, but I don't quite know
> >how I would go about that.
> >
> >In this case, the first field is a value unto itself - represents a "letter
> >type" that would be associated with the rest of the record. The fields are
> >either present or not, no placeholder if there's no value for e.g. Field #4.
>
> Here's a sketch, tested as you'll see, but totally devoid of the
> error-checking that would be necessary for any data coming from an MF.
>
> C:\junk>type schott.txt
> pers,1,xxx,2,yyy,3,zzz,100,SMITH,101,JOHN,102,ALOYSIUS,103,1969-12-31
> addr,1,qqq,2,www,3,eee,200,"""THE LODGE"", 123 MAIN ST",205,WALLA
> WALLA,206,WA
>
> C:\junk>type schott.py
> import csv
> for row in csv.reader(open('schott.txt', 'rb')):
> rectype = row[0]
> recdict = {}
> for k in range(1, len(row), 2):
> recdict[int(row[k])] = row[k+1]
> print rectype, recdict
>
> C:\junk>python schott.py
> pers {1: 'xxx', 2: 'yyy', 3: 'zzz', 100: 'SMITH', 101: 'JOHN', 102:
> 'ALOYSIUS', 103: '1969-12-31'}
> addr {1: 'qqq', 2: 'www', 3: 'eee', 200: '"THE LODGE", 123 MAIN ST',
> 205: 'WALLA WALLA', 206: 'WA'}
>
> Hint: you'll probably go nuts if you don't implement some sort of
> naming convention instead of those numbers.
>
> One way would be like this:
>
> mf_surname = 100
> mf_given_1 = 101
> ...
> mf_state = 206
>
> then you can refer to recdict[mf_state] instead of recdict[206].
>
> Going upmarket a bit:
>
> Have a mf_map = {100: 'surname', 206: 'state', } # etc etc
>
> then you do
>
> class Record(object):
> pass
>
> # for each row:
>rec = Record()
>rec.rectype = row[0]
>for k in range(1, len(row), 2):
>setattr(rec, mf_map[int(row[k])], row[k+1])
>
> Then you can refer to rec.state instead of recdict[mf_state] or
> recdict[206].
>
> Further upmarket would involve gathering basic "type" information
> about the MF fields (free text, alpha code, identifier (e.g. SSN),
> money, quantity, date, etc etc) so that you can do validations and
> format conversions as appropriate.
>
> HTH,
> John
>
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Difference between ActivePython and Python.org
Amen to this one. Found out this after struggling with no SSL module when trying to write some secure FTP and HTTPS code. Downloaded the version from Python.org and it worked like a champ. I believe you can get both and have no issues with the install, though. For my part, I decided to standardize on the main distribution from Python.org and then throw on the Win32 extensions as well. That handles the majority of my needs and gets around any licensing issues with ActivePython. -Peter Neil Hodgson <[EMAIL PROTECTED]> wrote: > S.Chang: > > > Anyone knows the difference(s) between the Python binaries from > > ActiveState and Python.org? > > As well as the differences mentioned by others, ActivePython does > not include SSL (Secure Socket Layer) and thus HTTPS support. It would > be helpful if the ActivePython "What's Included" page listed which parts > of the python.org distribution are not included. > http://aspn.activestate.com/ASPN/docs/ActivePython/2.4/about.html > > Neil -- http://mail.python.org/mailman/listinfo/python-list
Problem with FTPLib and incomplete files on some downloads
I download a lot of 4-6 KB files and regularly run into issues with files that don't get downloaded all the way or otherwise get corrupt. I do something like: RemoteList = nlstdir() for filename in RemoteList: LocalFile = open(filename, "wb") LocalFile.write( "get file code here" ) LocalFile.close() #ftplib call to delete the remote file I've tried to insert a pause into the code between the close and the remote delete, but that doesn't seem to help. For some reason it doesn't seem like the buffer is completely written or read before the remote delete is called. That leads to a file that's not 100% complete and thus can't be decrypted. Any ideas on what to look for? I don't have my exact code handy at the moment or I'd post some actual snippets. It's also not consistent or it would be a whole lot easier to troubleshoot. :-) Running Python 2.4.2 on Win32. Thanks. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Re: MSSQL LIKE and IN statements in ADO problem
Well, the raw TSQL would be: select * from tb_name where firstname like '%raj%' I think that would more translate to: name = "raj" cmd.CommandText = "SELECT * FROM tb_name WHERE firstname like '%%%s%%'" % name Perhaps issuing a print statement of the CommandText would help for future runs to determine if this is valid TSQL syntax? -Pete "gregarican" <[EMAIL PROTECTED]> wrote: > Can't you get rid of the Create Parameter part and directly pass along > the value you are looking for? Something like... > > name = 'raj' > cmd.CommandText= \ > "SELECT * FROM tb_name WHERE firstname LIKE %%%s" % name > > This way the value of the name variable gets passed along when the > CommandText method is invoked. BTW, this looks too painfully much like > Visual Basic than Python :-) Just kidding (kind of) -- http://mail.python.org/mailman/listinfo/python-list
Best way to capture output of another command-line program in Win32?
Not sure what I should do here. I know that DOS/CMD can capture the output of EXE files in Win32. I know that os.popen() has been recommended for this, but the couple of times I've tried, it didn't seem to work well. I haven't had a lot of need to do this recently, but we're trying to automate some processes right now that require us to trap the output of a command line and react accordingly. Any suggestions on the best method for trapping this output? Thanks. -Pete -- http://mail.python.org/mailman/listinfo/python-list
Re: Best way to capture output of another command-line program in Win32?
Thanks, I'll give this a try when I get a chance and will post back if we have
any issues.
-Pete
"Grig Gheorghiu" <[EMAIL PROTECTED]> wrote:
> subprocess gets my vote too.
>
> You can do something like:
>
> from subprocess import call, Popen, PIPE, STDOUT
>
> def run_cmd(cmd):
> arglist = cmd.split()
> p = Popen(arglist, stdout=PIPE, stderr=STDOUT)
> output = p.communicate()[0]
> return (p.returncode, output)
>
> rc, output = run_cmd("python setup.py test")
> if rc:
> print "Command failed"
> sys.exit(rc)
>
> Grig
--
http://mail.python.org/mailman/listinfo/python-list
Re: FTPlib
Shouldn't that be:
srcFtp.retrbinary('get ' + file[2:])
or some similar variation?
You're also missing a block telling this where to write the local file.
LocalFile = file.open(file[2:], "wb")
LocalFile.write(srcFtp.retrbinary('get ' + file[2:]) )
(or something similar)
-Pete
"Harlin Seritt" <[EMAIL PROTECTED]> wrote:
> Using ftplib from Python I am trying to get all files in a particular
> directory using ftplib and then send those same files to another ftp
> server. I have tried using commands like 'get *' and 'mget *' with no
> success.
>
> I am using the following:
>
> srcFtp = FTP(srcHost)
> srcFtp.login(srcUser, srcPass)
> srcDir = srcFtp.nlst('.')
> for file in srcDir:
> print file[2:]
> srcFtp.transfercmd('get '+file[2:])
>
> I've verified that I do have a connection with the ftp server and the
> files as 'file[2:]' are indeed the file names.
>
> Anyone know why I get the following error?
>
> podcast-1.mp3
> Traceback (most recent call last):
> File "podsync.py", line 17, in ?
> srcFtp.transfercmd('get '+file[2:])
> File "C:\Python24\lib\ftplib.py", line 345, in transfercmd
> return self.ntransfercmd(cmd, rest)[0]
> File "C:\Python24\lib\ftplib.py", line 327, in ntransfercmd
> resp = self.sendcmd(cmd)
> File "C:\Python24\lib\ftplib.py", line 241, in sendcmd
> return self.getresp()
> File "C:\Python24\lib\ftplib.py", line 216, in getresp
> raise error_perm, resp
> ftplib.error_perm: 500 Syntax error, command unrecognized.
>
> Thanks,
>
> Harlin Seritt
--
http://mail.python.org/mailman/listinfo/python-list
listdir() - any way to limit # of file entries?
I want to build a program that does some archiving. We have several programs that have been dumping files into a folder (one folder) for some time now. I want to limit the number of files returned by listdir to avoid trying to build a list with tons of entries. I then want to move those files into a zip archive for the appropriate day/month. Is there any way to build a list of the first 1000 files or so in a folder? Running Python 2.4.2 on Win32 with Win32 extensions. Thanks. -Pete Schott -- http://mail.python.org/mailman/listinfo/python-list
Re: Best way to parse file into db-type layout?
Thanks for the comments. Quick notes: 1. Yes, ultimate goal is to output various fields into Excel, but I need to enter the data into a DB as well to avoid having any strange problems. That may be handled by a completely different process. I should have been more exact in saying that my current goal is to get this data into a database. 2. I'm relatively new to Python and definitely new to trying to parse Ragged-right type CSV files that are pairs of data. I'm trying to get my syntax correct when wording my questions/phrases - please bear with me. :-) Your help (all posters) is quite appreciated. 3. regarding read/write each row - I started thinking about working my way through the dictionary. This file could have up to 1900 pairs of data per row. That's an awful lot of columns to put into a database table, especially if quite a few will be NULL. I starting thinking of stepping through each pair by each row. The rows are identified by Letter type and an account number (always the same field #). I would want to use those, and insert something like: LetterType, Account #, Field#, Field Data, (current date) I wasn't quite sure how to phrase that and apologize for being unclear. I guess if I'm doing row-by-row, I can hold that for each row without too much trouble. 4. No control over file format. I can define a lookup table that will match Field# to a real name, perhaps code that into my scripts, but I'm stuck with that. As to other posters' responses - it's ANSI-coded, CSV, No Quotes (no Commas in values). Coming from the mainframe, I don't have to worry about embedded special characters within the file as they can't be entered into these fields to start with. File always starts with letter_type, 1 - no idea why they didn't do that as 0, letter_type for consistency but it isn't my file. (Heck, I would have just specified to output all fields all the time and avoid any need to do special parsing :-) Anyway, I appreciate the help. The code looks like it will work for me and I'll be doing some basic tests to get up to speed on splitting/parsing the data and putting it into a form that will be usable by all. I may have to write certain rows out to a new file if they don't match what I'm looking for. I don't anticipate any issues there - just read the row, write it to a new file if it doesn't match my criteria. Thanks to all. Off to search for dictionaries and databases now. :-) -Pete John Machin <[EMAIL PROTECTED]> wrote: > On Fri, 29 Apr 2005 18:54:54 GMT, Peter A. Schott > <[EMAIL PROTECTED]> wrote: > > >That looks promising. > > > The field numbers are pre-defined at the mainframe level. > > Of course. Have you managed to acquire a copy of the documentation, or > do you have to reverse-engineer it? > > >This may help me get to my ultimate goal which is to pump these into a DB on > >a > >row-by-row basis ( :-P ) > > That's your *ultimate* goal? Are you running a retro-computing museum > or something? Don't you want to *USE* the data? > > > I'll have to do some playing around with this. I > >knew that it looked like a dictionary, but wasn't sure how best to handle > >this. > > > >One follow-up question: I'll end up getting multiple records for each > >"type". > > What does that mean?? If it means that more than one customer will get > the "please settle your account" letter, and more than one customer > will get the "please buy a spangled fritzolator, only $9.99" letter, > you are stating the obvious -- otherwise, please explain. > > >Would I be referencing these by row[#][field#]? > > Not too sure what you mean by that -- whether you can get away with a > (read a row, write a row) way of handling the data depends on its > structure (like what are the relationships if any between different > rows) and what you want to do with it -- both murky concepts at the > moment. > > > > >Minor revision to the format is that starts like: > >###,1,1,val_1, > > How often do these "minor revisions" happen? How flexible do you have > to be? And the extra "1" means what? Is it ever any other number? > > > > > > >I think right now the plan is to parse through the file and insert the pairs > >directly into a DB table. Something like RowID, LetterType, date, Field#, > >Value. > > Again, I'd recommend you lose the "Field#" in favour of a better > representation, ASAP. > > > I can get RowID and LetterType overall, date is a constant, the rest > >would involve reading each pair and inserting both values into the table. > >Time > >to hit the books a little more to get up to speed on all of this. > > What you need is (a) a clear appreciation of what you are trying to do > with the data at a high level (b) then develop an understanding of > what is the underlying data model (c) then and only then worry about > technical details. > > Good luck, > John > -- http://mail.python.org/mailman/listinfo/python-list
Re: Alternative Ways to install Python 2.4?
Not sure if this was ever resolved or not, but I've also had issues trying to install from a mapped drive. Once I copied the file locally and installed, it ran smoothly. -Pete Michael Goettsche <[EMAIL PROTECTED]> wrote: > On Monday 24 January 2005 00:29, "Martin v. Löwis" wrote: > > Michael Goettsche wrote: > > > I convinced my CS teacher to use Python in school. We currently have 2.2 > > > installed on a Windows 2000 Terminal server. I asked the system > > > administrator to upgrade to Python 2.4, but he didn't succeed in doing > > > it. He used the microsoft installer package, which according to him > > > crashed when starting. So my question is if there's an alternative way to > > > install it(must be easy). Would it be an option to remove 2.2 first and > > > then try to install 2.4 again? > > > > That would be an option, but I doubt it helps. Please have a look at > > > > http://www.python.org/2.4/bugs.html > > > > Most likely, you need to upgrade Visual Basic on that machine. > > > > Of course, without a precise error description, it is hard to tell. > > > > Regards, > > Martin > > Hi Martin, > > thanks for your answer, I'll let him know! > > Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: Alternative Ways to install Python 2.4?
Only thing I know to watch out for is they don't include SSL support - something the Python 2.4.1 msi file does include. I needed that for something I'm working on and had to install 2.4.1 over ActivePython to get the SSL support. -Pete David Van Mosselbeen <[EMAIL PROTECTED]> wrote: > James wrote: > > I have also some troubleshouting when i will work on mapped drives. > Let's take a example : > A Word doc on a mapped drive. Double click on it (i try to open it). > And waiting ... > waiting... > nothing do > > > If this is a bug with the standard distribution alone, maybe he can try > > ActiveState's distribution (ActivePython) instead. > > When i work on Windows XP i use Python distributed by the ActiveState's and > find that ActiveState's included really good stuff into this package. It's > a really good when you plan to use it on a Windows box. -- http://mail.python.org/mailman/listinfo/python-list
