Re: Gpg python installer

2015-04-02 Thread Ned Deily
In article 
,
 leonardo davinci  wrote:
> I am using Kleopatra(gpg for win) to verify the 3.4.3 python installer,
> Windows x86 MSI
> 
> >https://www.python.org/ftp/python/3.4.3/python-3.4.3.msi>
> www.python.org 
> /ftp/python/3.4.3/
> python-3.4.3.msi
> >. This file does
> not have a email in the digital signature and I am having trouble verifying
> the validity of the download.

Unfortunately, verifying the PGP signature of release files isn't the 
most user-friendly process, especially on Windows.  The release files 
from python.org are typically PGP-signed in armored detached signature 
files, in other words, for each release file (like python-3.4.3.msi) 
there is a separate signature file with an appended .asc extension 
(python-3.4.3.msi.asc).  If you go to the python.org downloads page 
(https://www.python.org/downloads/) and click on the release in 
question, it should take you to the page for the release 
(https://www.python.org/downloads/release/python-343/).  Near the bottom 
of the page, there is a list of downloadable files and to the right of 
each one there is a "GPG" column with a "SIG" link for each file.  
Clicking on the SIG link should download the corresponding signature 
file (python-3.4.3.msi.asc).  I'm not familiar with Kleopatra's 
interface but normally you'd want to download both the installer file 
and its asc file to the same directory/folder and then tell the GPG 
program to verify the asc file.  The PGP/GPG program will also need to 
have access to the public keys of the creators / signers of the 
downloadable files.  You will find them listed near the bottom of the 
Downloads page (https://www.python.org/downloads/).

Independently thereof, the python.org Windows installer files are also 
signed with a public-key code signing certificate that should be 
automatically verified by the Windows installer program.  (Likewise, for 
the Mac OS X installer files.)

Hope this helps!

-- 
 Ned Deily,
 n...@acm.org

-- 
https://mail.python.org/mailman/listinfo/python-list


New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Saran A
Good Morning:

I understand this error message when I run this code. However, I am curious to 
know what the most pythonic way is to convert  the list to a string? I use 
Python 2.7.

"Traceback (most recent call last):
before = dict([(f, None) for f in os.listdir(dirlist)])
TypeError: coercing to Unicode: need string or buffer, list found"


The sample code that I am trying to run is:

path = "/Users/Desktop/Projects/"
dirlist = os.listdir(path)
before = dict([(f, None) for f in os.listdir(dirlist)])

def main(dirlist):
while True:
time.sleep(10) #time between update check
after = dict([(f, None) for f in os.listdir(dirlist)])
added = [f for f in after if not f in before]
if added:
print('Successfully added new file - ready to validate')
if __name__ == "__main__": 
main() 




-- 
https://mail.python.org/mailman/listinfo/python-list


Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Chris Angelico
On Thu, Apr 2, 2015 at 11:02 PM, Saran A  wrote:
> I understand this error message when I run this code. However, I am curious 
> to know what the most pythonic way is to convert  the list to a string? I use 
> Python 2.7.
>

I don't think you actually want to convert a list into a string, here.
Tell me if I'm understanding your code's intention correctly:

> The sample code that I am trying to run is:
>
> path = "/Users/Desktop/Projects/"
> dirlist = os.listdir(path)
> before = dict([(f, None) for f in os.listdir(dirlist)])

Start up and get a full list of pre-existing files.

> def main(dirlist):
> while True:
> time.sleep(10) #time between update check

Then, every ten seconds...

> after = dict([(f, None) for f in os.listdir(dirlist)])
> added = [f for f in after if not f in before]

... get a list of files, and if there are new ones...

> if added:
> print('Successfully added new file - ready to validate')
> if __name__ == "__main__":
> main()

... print out a message.

If that's what you're trying to do, I would suggest using a directory
notification system instead. Here's one that I use on Linux:

https://github.com/Rosuav/shed/blob/master/dirwatch.py

Here's another one, this time built for Windows:

https://github.com/Rosuav/shed/blob/master/senddir.py

But even if you absolutely have to poll, like that, you'll need to
make a few code changes. The exception you're getting is symptomatic
of just one problem with the code as published. My suspicion is that
you just want to use listdir(path) rather than listdir(dirlist) - but
if you want subdirectories, then you'll need to do things a bit
differently (probably using os.walk instead).

Also: You say you're using Python 2.7. If you have no particular
reason to use 2.7, you'll do better to jump to Python 3. Your code
will probably run identically, when it's this simple.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Peter Otten
Saran A wrote:

> Good Morning:
> 
> I understand this error message when I run this code. However, I am
> curious to know what the most pythonic way is to convert  the list to a
> string? I use Python 2.7.
> 
> "Traceback (most recent call last):
> before = dict([(f, None) for f in os.listdir(dirlist)])
> TypeError: coercing to Unicode: need string or buffer, list found"
> 
> 
> The sample code that I am trying to run is:
> 
> path = "/Users/Desktop/Projects/"
> dirlist = os.listdir(path)

At this point dirlist is a list of names of the files and directories in 

"/Users/Desktop/Projects/"

Assuming that the Projects folder contains the subfolders or files
/Users/Desktop/Projects/foo, /Users/Desktop/Projects/bar and 
/Users/Desktop/Projects/baz dirlist looks like this:

["foo", "bar", "baz"]

It makes no sense to pass this list to os.listdir() as you do below:

> before = dict([(f, None) for f in os.listdir(dirlist)])

Forget about the other details in the error message; the actual problem is 
the "list found" part.

Now what would be a possible fix? Sorry, I have no idea what your intention 
is. Again, you don't need to convert your list to string, you need to decide 
what directory you want to pass to listdir(). If you have multiple such 
directories you need to invoke listdir() multiple times with a single 
directory, typically in a loop.

Bonus info:

> while True:
> time.sleep(10) #time between update check
 
This loop will never terminate.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: New to Programming - XML Processing

2015-04-02 Thread Albert-Jan Roskam

-
On Wed, Apr 1, 2015 10:00 AM CEST Mark Lawrence wrote:

>On 01/04/2015 05:27, Andrew Farrell wrote:
>> You should follow Rustom's advice before just diving into the blog post
>> I linked to. Otherwise you risk blindly following things and losing your
>> bearings when you run into bugs.
>> 
>
>Sound advice, but would you please be kind enough to intersperse your answers 
>or bottom post as top posting is heavily frowned upon here.
>
>TIA.

Would it be possible to use a script that checks every incoming mail to the 
Python mail list? Main ingredients beatfilsoup (to textify HTML mails) and 
difflib (to check whether replies contain parts of the original message, and 
that no top posting happens)

Just an idea.

Albert-Jan




>-- My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence
>
>-- https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Saran A
On Thursday, April 2, 2015 at 8:26:01 AM UTC-4, Chris Angelico wrote:
> On Thu, Apr 2, 2015 at 11:02 PM, Saran A  wrote:
> > I understand this error message when I run this code. However, I am curious 
> > to know what the most pythonic way is to convert  the list to a string? I 
> > use Python 2.7.
> >
> 
> I don't think you actually want to convert a list into a string, here.
> Tell me if I'm understanding your code's intention correctly:
> 
> > The sample code that I am trying to run is:
> >
> > path = "/Users/Desktop/Projects/"
> > dirlist = os.listdir(path)
> > before = dict([(f, None) for f in os.listdir(dirlist)])
> 
> Start up and get a full list of pre-existing files.
> 
> > def main(dirlist):
> > while True:
> > time.sleep(10) #time between update check
> 
> Then, every ten seconds...
> 
> > after = dict([(f, None) for f in os.listdir(dirlist)])
> > added = [f for f in after if not f in before]
> 
> ... get a list of files, and if there are new ones...
> 
> > if added:
> > print('Successfully added new file - ready to validate')
> > if __name__ == "__main__":
> > main()
> 
> ... print out a message.
> 
> If that's what you're trying to do, I would suggest using a directory
> notification system instead. Here's one that I use on Linux:
> 
> https://github.com/Rosuav/shed/blob/master/dirwatch.py
> 
> Here's another one, this time built for Windows:
> 
> https://github.com/Rosuav/shed/blob/master/senddir.py
> 
> But even if you absolutely have to poll, like that, you'll need to
> make a few code changes. The exception you're getting is symptomatic
> of just one problem with the code as published. My suspicion is that
> you just want to use listdir(path) rather than listdir(dirlist) - but
> if you want subdirectories, then you'll need to do things a bit
> differently (probably using os.walk instead).
> 
> Also: You say you're using Python 2.7. If you have no particular
> reason to use 2.7, you'll do better to jump to Python 3. Your code
> will probably run identically, when it's this simple.
> 
> ChrisA

@ChrisA - this is a smaller function that will take the most updated file. My 
intention is the following:

* Monitor a folder for files that are dropped throughout the day

* When a file is dropped in the folder the program should scan the file

o IF all the contents in the file have the same length (let's assume line 
length)

o THEN the file should be moved to a "success" folder and a text file written 
indicating the total number of records/lines/words processed

o IF the file is empty OR the contents are not all of the same length

o THEN the file should be moved to a "failure" folder and a text file written 
indicating the cause for failure (for example: Empty file or line 100 was not 
the same length as the rest).

Here is the code I have written:

import os
import time
import glob
import sys

def initialize_logger(output_dir):
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
 
# create console handler and set level to info
handler = logging.StreamHandler()
handler.setLevel(logging.INFO)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
 
# create error file handler and set level to error
handler = logging.FileHandler(os.path.join(output_dir, "error.log"),"w", 
encoding=None, delay="true")
handler.setLevel(logging.ERROR)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)

# create debug file handler and set level to debug
handler = logging.FileHandler(os.path.join(output_dir, "all.log"),"w")
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)

#Helper Functions for the Success and Failure Folder Outcomes, respectively

#checks the length of the file
def file_len(filename
with open(filename) as f:
for i, l in enumerate(f):
pass
return i + 1

#copies file to new destination

def copyFile(src, dest):
try:
shutil.copy(src, dest)
# eg. src and dest are the same file
except shutil.Error as e:
print('Error: %s' % e)
# eg. source or destination doesn't exist
except IOError as e:
print('Error: %s' % e.strerror)

#Failure Folder

def move_to_failure_folder_and_return_error_file():
os.mkdir('Failure')
copyFile(filename, 'Failure')
initialize_logger('rootdir/Failure')
logging.error("Either this file is empty or the lines")
 
# Success Folder Requirement
 
def move_to_success_folder_and_read(file):
os.mkdir('Success')
copyFile(filename, 'Success')
print("Success", file)
return file_len()


#This simply checks the file information by name

def fileinfo(file):
file

Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Saran A
On Thursday, April 2, 2015 at 8:26:51 AM UTC-4, Peter Otten wrote:
> Saran A wrote:
> 
> > Good Morning:
> > 
> > I understand this error message when I run this code. However, I am
> > curious to know what the most pythonic way is to convert  the list to a
> > string? I use Python 2.7.
> > 
> > "Traceback (most recent call last):
> > before = dict([(f, None) for f in os.listdir(dirlist)])
> > TypeError: coercing to Unicode: need string or buffer, list found"
> > 
> > 
> > The sample code that I am trying to run is:
> > 
> > path = "/Users/Desktop/Projects/"
> > dirlist = os.listdir(path)
> 
> At this point dirlist is a list of names of the files and directories in 
> 
> "/Users/Desktop/Projects/"
> 
> Assuming that the Projects folder contains the subfolders or files
> /Users/Desktop/Projects/foo, /Users/Desktop/Projects/bar and 
> /Users/Desktop/Projects/baz dirlist looks like this:
> 
> ["foo", "bar", "baz"]
> 
> It makes no sense to pass this list to os.listdir() as you do below:
> 
> > before = dict([(f, None) for f in os.listdir(dirlist)])
> 
> Forget about the other details in the error message; the actual problem is 
> the "list found" part.
> 
> Now what would be a possible fix? Sorry, I have no idea what your intention 
> is. Again, you don't need to convert your list to string, you need to decide 
> what directory you want to pass to listdir(). If you have multiple such 
> directories you need to invoke listdir() multiple times with a single 
> directory, typically in a loop.
> 
> Bonus info:
> 
> > while True:
> > time.sleep(10) #time between update check
>  
> This loop will never terminate.

@Peter I understand that the intention of this program is to not terminate. 
Here is what I have written so far:

I thought I would run this by you, since you offer such valuable feedback, in 
the past. Just a quick rundown on what I want my program to do:

* Monitor a folder for files that are dropped throughout the day

* When a file is dropped in the folder the program should scan the file

o IF all the contents in the file have the same length (let's assume line 
length)

o THEN the file should be moved to a "success" folder and a text file written 
indicating the total number of records/lines/words processed

o IF the file is empty OR the contents are not all of the same length

o THEN the file should be moved to a "failure" folder and a text file written 
indicating the cause for failure (for example: Empty file or line 100 was not 
the same length as the rest).

Here is the code I have written:

import os
import time
import glob
import sys

def initialize_logger(output_dir):
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
 
# create console handler and set level to info
handler = logging.StreamHandler()
handler.setLevel(logging.INFO)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
 
# create error file handler and set level to error
handler = logging.FileHandler(os.path.join(output_dir, "error.log"),"w", 
encoding=None, delay="true")
handler.setLevel(logging.ERROR)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)

# create debug file handler and set level to debug
handler = logging.FileHandler(os.path.join(output_dir, "all.log"),"w")
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)


def main(dirslist): 
while True:
for file in os.listdir(dirslist) :
return validate_files(file)
time.sleep(5)

if __name__ == "__main__": 
main() 


#Helper Functions for the Success and Failure Folder Outcomes, respectively

#checks the length of the file
def file_len(filename
with open(filename) as f:
for i, l in enumerate(f):
pass
return i + 1

#copies file to new destination

def copyFile(src, dest):
try:
shutil.copy(src, dest)
# eg. src and dest are the same file
except shutil.Error as e:
print('Error: %s' % e)
# eg. source or destination doesn't exist
except IOError as e:
print('Error: %s' % e.strerror)

#Failure Folder

def move_to_failure_folder_and_return_error_file():
os.mkdir('Failure')
copyFile(filename, 'Failure')
initialize_logger('rootdir/Failure')
logging.error("Either this file is empty or the lines")
 
# Success Folder Requirement
 
def move_to_success_folder_and_read(file):
os.mkdir('Success')
copyFile(filename, 'Success')
print("Success", file)
return file_len()

#This simply checks the file information by name

def fileinfo(file):
filename = os.path.basename(file)
rootdir = os.path.dirname(file)
lastmod = time.c

Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Chris Angelico
On Thu, Apr 2, 2015 at 11:46 PM, Saran A  wrote:
> @ChrisA - this is a smaller function that will take the most updated file. My 
> intention is the following:
>
> * Monitor a folder for files that are dropped throughout the day
>
> * When a file is dropped in the folder the program should scan the file
>
> o IF all the contents in the file have the same length (let's assume line 
> length)
>
> o THEN the file should be moved to a "success" folder and a text file written 
> indicating the total number of records/lines/words processed
>
> o IF the file is empty OR the contents are not all of the same length
>
> o THEN the file should be moved to a "failure" folder and a text file written 
> indicating the cause for failure (for example: Empty file or line 100 was not 
> the same length as the rest).
>

Sounds like a perfect job for inotify, then. Your function will be
called whenever there's a new file.

> Here is the code I have written:
>
> def initialize_logger(output_dir):
> logger = logging.getLogger()
> ...
> def file_len(filename
> with open(filename) as f:
> for i, l in enumerate(f):
> pass
> return i + 1

These functions are all getting defined inside your
initialize_logger() function. I suspect you want them to be flush left
instead.

> def copyFile(src, dest):
> try:
> shutil.copy(src, dest)
> # eg. src and dest are the same file
> except shutil.Error as e:
> print('Error: %s' % e)
> # eg. source or destination doesn't exist
> except IOError as e:
> print('Error: %s' % e.strerror)

Recommendation: Skip the try/except, and just let exceptions bubble
up. Don't just print out messages and keep going.

> def move_to_failure_folder_and_return_error_file():
> os.mkdir('Failure')
> copyFile(filename, 'Failure')
> initialize_logger('rootdir/Failure')
> logging.error("Either this file is empty or the lines")

This doesn't move the file, it copies it. Is that your intention?

Moving a file is pretty easy. Just use os.rename().

> if __name__ == '__main__':
>import sys
>validate_files(sys.argv[1:])

I've no idea what validate_files() does, as you haven't included that.

I think you could code this fairly efficiently as a simple callback
off pyinotify, or if you're not on Linux, with one of the equivalent
services. What you're doing here (watching for files, looking inside
them, and moving them when done) is pretty common around the world.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Strategy/ Advice for How to Best Attack this Problem?

2015-04-02 Thread Saran A
On Wednesday, April 1, 2015 at 7:52:27 PM UTC-4, Dave Angel wrote:
> On 04/01/2015 09:43 AM, Saran A wrote:
> > On Tuesday, March 31, 2015 at 9:19:37 AM UTC-4, Dave Angel wrote:
> >> On 03/31/2015 07:00 AM, Saran A wrote:
> >>
> >>   > @DaveA: This is a homework assignment.  Is it possible that you
> >> could provide me with some snippets or guidance on where to place your
> >> suggestions (for your TO DOs 2,3,4,5)?
> >>   >
> >>
> >>
> >>> On Monday, March 30, 2015 at 2:36:02 PM UTC-4, Dave Angel wrote:
> >>
> 
>  It's missing a number of your requirements.  But it's a start.
> 
>  If it were my file, I'd have a TODO comment at the bottom stating known
>  changes that are needed.  In it, I'd mention:
> 
>  1) your present code is assuming all filenames come directly from the
>  commandline.  No searching of a directory.
> 
>  2) your present code does not move any files to success or failure
>  directories
> 
> >>
> >> In function validate_files()
> >> Just after the line
> >>   print('success with %s on %d reco...
> >> you could move the file, using shutil.  Likewise after the failure print.
> >>
>  3) your present code doesn't calculate or write to a text file any
>  statistics.
> >>
> >> You successfully print to sys.stderr.  So you could print to some other
> >> file in the exact same way.
> >>
> 
>  4) your present code runs once through the names, and terminates.  It
>  doesn't "monitor" anything.
> >>
> >> Make a new function, perhaps called main(), with a loop that calls
> >> validate_files(), with a sleep after each pass.  Of course, unless you
> >> fix TODO#1, that'll keep looking for the same files.  No harm in that if
> >> that's the spec, since you moved the earlier versions of the files.
> >>
> >> But if you want to "monitor" the directory, let the directory name be
> >> the argument to main, and let main do a dirlist each time through the
> >> loop, and pass the corresponding list to validate_files.
> >>
> 
>  5) your present code doesn't check for zero-length files
> 
> >>
> >> In validate_and_process_data(), instead of checking filesize against
> >> ftell, check it against zero.
> >>
>  I'd also wonder why you bother checking whether the
>  os.path.getsize(file) function returns the same value as the os.SEEK_END
>  and ftell() code does.  Is it that you don't trust the library?  Or that
>  you have to run on Windows, where the line-ending logic can change the
>  apparent file size?
> 
>  I notice you're not specifying a file mode on the open.  So in Python 3,
>  your sizes are going to be specified in unicode characters after
>  decoding.  Is that what the spec says?  It's probably safer to
>  explicitly specify the mode (and the file encoding if you're in text).
> 
>  I see you call strip() before comparing the length.  Could there ever be
>  leading or trailing whitespace that's significant?  Is that the actual
>  specification of line size?
> 
>  --
>  DaveA
> >>>
> >>>
> >>
> >>> I ask this because I have been searching fruitlessly through for some 
> >>> time and there are so many permutations that I am bamboozled by which is 
> >>> considered best practice.
> >>>
> >>> Moreover, as to the other comments, those are too specific. The scope of 
> >>> the assignment is very limited, but I am learning what I need to look out 
> >>> or ask questions regarding specs - in the future.
> >>>
> >>
> >>
> >> --
> >> DaveA
> >
> > @DaveA
> >
> > My most recent commit 
> > (https://github.com/ahlusar1989/WGProjects/blob/master/P1version2.0withassumptions_mods.py)
> >  has more annotations and comments for each file.
> 
> Perhaps you don't realize how github works.  The whole point is it 
> preserves the history of your code, and you use the same filename for 
> each revision.
> 
> Or possibly it's I that doesn't understand it.  I use git, but haven't 
> actually used github for my own code.
> 
> >
> > I have attempted to address the functional requirements that you brought up:
> >
> > 1) Before, my present code was assuming all filenames come directly from 
> > the commandline.  No searching of a directory. I think that I have 
> > addressed this.
> >
> 
> Have you even tried to run the code?  It quits immediately with an 
> exception since your call to main() doesn't pass any arguments, and main 
> requires one.
> 
>  > def main(dirslist):
>  > while True:
>  > for file in dirslist:
>  >return validate_files(file)
>  >time.sleep(5)
> 
> In addition, you aren't actually doing anything to find what the files 
> in the directory are.  I tried to refer to dirlist, as a hint.  A 
> stronger hint:  look up  os.listdir()
> 
> And that list of files has to change each time through the while loop, 
> that's the whole meaning of scanning.  You don't just grab the names 
> once, you look to see what's ther

Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Saran A
On Thursday, April 2, 2015 at 9:06:49 AM UTC-4, Chris Angelico wrote:
> On Thu, Apr 2, 2015 at 11:46 PM, Saran A  wrote:
> > @ChrisA - this is a smaller function that will take the most updated file. 
> > My intention is the following:
> >
> > * Monitor a folder for files that are dropped throughout the day
> >
> > * When a file is dropped in the folder the program should scan the file
> >
> > o IF all the contents in the file have the same length (let's assume line 
> > length)
> >
> > o THEN the file should be moved to a "success" folder and a text file 
> > written indicating the total number of records/lines/words processed
> >
> > o IF the file is empty OR the contents are not all of the same length
> >
> > o THEN the file should be moved to a "failure" folder and a text file 
> > written indicating the cause for failure (for example: Empty file or line 
> > 100 was not the same length as the rest).
> >
> 
> Sounds like a perfect job for inotify, then. Your function will be
> called whenever there's a new file.
> 
> > Here is the code I have written:
> >
> > def initialize_logger(output_dir):
> > logger = logging.getLogger()
> > ...
> > def file_len(filename
> > with open(filename) as f:
> > for i, l in enumerate(f):
> > pass
> > return i + 1
> 
> These functions are all getting defined inside your
> initialize_logger() function. I suspect you want them to be flush left
> instead.
> 
> > def copyFile(src, dest):
> > try:
> > shutil.copy(src, dest)
> > # eg. src and dest are the same file
> > except shutil.Error as e:
> > print('Error: %s' % e)
> > # eg. source or destination doesn't exist
> > except IOError as e:
> > print('Error: %s' % e.strerror)
> 
> Recommendation: Skip the try/except, and just let exceptions bubble
> up. Don't just print out messages and keep going.
> 
> > def move_to_failure_folder_and_return_error_file():
> > os.mkdir('Failure')
> > copyFile(filename, 'Failure')
> > initialize_logger('rootdir/Failure')
> > logging.error("Either this file is empty or the lines")
> 
> This doesn't move the file, it copies it. Is that your intention?
> 
> Moving a file is pretty easy. Just use os.rename().
> 
> > if __name__ == '__main__':
> >import sys
> >validate_files(sys.argv[1:])
> 
> I've no idea what validate_files() does, as you haven't included that.
> 
> I think you could code this fairly efficiently as a simple callback
> off pyinotify, or if you're not on Linux, with one of the equivalent
> services. What you're doing here (watching for files, looking inside
> them, and moving them when done) is pretty common around the world.
> 
> ChrisA

@ChrisA

validate_files will:

#double check for record time and record length - logic to be written to either 
pass to Failure or Success folder respectively. I welcome your thoughts on 
this.  

def validate_files(): 
creation = time.ctime(os.path.getctime(added)) 
lastmod = time.ctime(os.path.getmtime(added)) 


Does this address the issue. I particularly like writing my own exceptions as 
they provide me with more information on what could be the root cause. I know 
that in other circumstances, try and except are not the best practice. I 
appreciate the reminder though.  

Does this modification to copyFile do the job of moving the file? I haven't 
written a test yet. 

Thanks for catching the indentation for the helper functions. 
   
def copyFile(src, dest):
> > try:
> > shutil.rename(src, dest)
> > # eg. src and dest are the same file
> > except shutil.Error as e:
> > print('Error: %s' % e)
> > # eg. source or destination doesn't exist
> > except IOError as e:
> > print('Error: %s' % e.strerror)
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Chris Angelico
On Fri, Apr 3, 2015 at 12:28 AM, Saran A  wrote:
> Does this modification to copyFile do the job of moving the file? I haven't 
> written a test yet.
>
> Thanks for catching the indentation for the helper functions.
>
> def copyFile(src, dest):
>> > try:
>> > shutil.rename(src, dest)
>> > # eg. src and dest are the same file
>> > except shutil.Error as e:
>> > print('Error: %s' % e)
>> > # eg. source or destination doesn't exist
>> > except IOError as e:
>> > print('Error: %s' % e.strerror)

You shouldn't need shutil here; just os.rename(src, dest) should do
the trick. But be careful! Now you have a function which moves a file,
and it's called "copyFile". If its purpose changes, so should its
name.

Have fun!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python to C++ in a pickle

2015-04-02 Thread richismyname
On Tuesday, July 4, 2000 at 12:00:00 AM UTC-7, Larry Whitley wrote:
> I'd like to pickle an object in python and unpickle it in C++.  The object
> in question is a python dictionary so it will have to go into a C++ map.
> Has someone done this and is willing to share?  Or, is the cPickle source
> available on the web somewhere?
> 
> Larry

Check out http://www.picklingtools.com.  It gives you C++ code for pickling
and unpickling as well as data structures for manipulating those structures.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: pysftp connection not sending files, hanging on 'put'

2015-04-02 Thread ykravetz89
On Saturday, January 24, 2015 at 11:32:43 AM UTC-5, Jules Stevenson wrote:
> Hi List,
> 
> 
> I'm trying to send some files via pysftp, authentication seems fine, but 
> actually putting the file results in it hanging, doing nothing. The files are 
> small (<200kb) so I don't believe it's an upload issue (internet access is 
> fine). Eventually the connection times out.
> 
> 
> Code:
> 
> 
> 
> if os.listdir(ftp_path):# only do if files are there
>   with pysftp.Connection(FTP_SERVER,
> 
> port=
> 
> username=FTP_USER, 
> 
> private_key=FTP_SSH_PRIVATE_KEY_PATH
> 
> ) as sftp:
> with sftp.cd(FTP_REMOTE_FOLDER):
> 
> for f in os.listdir(ftp_path):
> 
> if os.path.isfile(os.path.join(ftp_path,f)) :
> 
> # upload image to server
> 
> self.log.info("HB FTP, Start upload: "+f)
> 
> print(ftp_path+"\\"+f)
> 
> sftp.put(os.path.join(ftp_path,f))
> 
> self.log.info("HB FTP, Finished Upload: "+f)
> 
> 
> 
> Logging output here:
> 
> 
> 
> FSW_COMMS paramiko.transport 01/24/2015 04:23:58 PM: INFO: Authentication 
> (publickey) successful!
> FSW_COMMS paramiko.transport 01/24/2015 04:23:58 PM: DEBUG: [chan 1] Max 
> packet in: 32768 bytes
> FSW_COMMS paramiko.transport 01/24/2015 04:23:58 PM: DEBUG: [chan 1] Max 
> packet out: 0 bytes
> FSW_COMMS paramiko.transport 01/24/2015 04:23:58 PM: DEBUG: Secsh channel 1 
> opened.
> FSW_COMMS paramiko.transport 01/24/2015 04:23:58 PM: DEBUG: [chan 1] Sesch 
> channel 1 request ok
> FSW_COMMS paramiko.transport.sftp 01/24/2015 04:23:58 PM: INFO: [chan 1] 
> Opened sftp connection (server version 3)
> FSW_COMMS paramiko.transport.sftp 01/24/2015 04:23:58 PM: DEBUG: [chan 1] 
> normalize('.')
> FSW_COMMS paramiko.transport.sftp 01/24/2015 04:23:58 PM: DEBUG: [chan 1] 
> stat('files')
> FSW_COMMS paramiko.transport.sftp 01/24/2015 04:23:58 PM: DEBUG: [chan 1] 
> normalize('files')
> FSW_COMMS root 01/24/2015 04:23:58 PM: INFO: HB FTP, Start upload: 
> 2015-01-17-19-37-07.jpg
> D:\FSW\ftp\2015-01-17-19-37-07.jpg
> FSW_COMMS paramiko.transport.sftp 01/24/2015 04:23:58 PM: DEBUG: [chan 1] 
> open('/files/2015-01-17-19-37-07.jpg', 'wb')
> FSW_COMMS paramiko.transport.sftp 01/24/2015 04:23:58 PM: DEBUG: [chan 1] 
> open('/files/2015-01-17-19-37-07.jpg', 'wb') -> 
> 34613039393262343666383036653839
> 
> 
> Any help much appreciated,
> 
> 
> Jules

I am having the exact issue, have you found a solution?
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Strategy/ Advice for How to Best Attack this Problem?

2015-04-02 Thread Dave Angel

On 04/02/2015 09:06 AM, Saran A wrote:



Thanks for your help on this homework assignment. I started from scratch last 
night. I have added some comments that will perhaps help clarify my intentions 
and my thought process. Thanks again.

from __future__ import print_function


I'll just randomly comment on some things I see here.  You've started 
several threads, on two different forums, so it's impractical to figure 
out what's really up.



   


#Helper Functions for the Success and Failure Folder Outcomes, respectively

 def file_len(filename):


This is an indentation error, as you forgot to start at the left margin


 with open(filename) as f:
 for i, l in enumerate(f):
 pass
 return i + 1


 def copy_and_move_File(src, dest):


ditto


 try:
 shutil.rename(src, dest)


Is there a reason you don't use the move function?  rename won't work if 
the two directories aren't on the same file system.



 # eg. src and dest are the same file
 except shutil.Error as e:
 print('Error: %s' % e)
 # eg. source or destination doesn't exist
 except IOError as e:
 print('Error: %s' % e.strerror)


# Call main(), with a loop that calls # validate_files(), with a sleep after 
each pass. Before, my present #code was assuming all filenames come directly 
from the commandline.  There was no actual searching #of a directory.

# I am assuming that this is appropriate since I moved the earlier versions of 
the files.
# I let the directory name be the argument to main, and let main do a dirlist 
each time through the loop,
# and pass the corresponding list to validate_files.


path = "/some/sample/path/"
dirlist = os.listdir(path)
before = dict([(f, None) for f in dirlist)

#Syntax Error? before = dict([(f, None) for f in dirlist)
  ^
SyntaxError: invalid syntax


Look at the line in question. There's an unmatched set of brackets.  Not 
that it matters, since you don't need these 2 lines for anything.  See 
my comments on some other forum.




def main(dirlist):


bad name for a directory path variable.


 while True:
 time.sleep(10) #time between update check


Somewhere inside this loop, you want to obtain a list of files in the 
specified directory.  And you want to do something with that list.  You 
don't have to worry about what the files were last time, because 
presumably those are gone.  Unless in an unwritten part of the spec, 
you're supposed to abort if any filename is repeated over time.




 after = dict([(f, None) for f in dirlist)
 added = [f for f in after if not f in before]
 if added:
 print('Sucessfully added new file - ready to validate')
   add return statement here to pass to validate_files
if __name__ == "__main__":
 main()


You'll need an argument to call main()




#check for record time and record length - logic to be written to either pass 
to Failure or Success folder respectively

def validate_files():


Where are all the parameters to this function?


 creation = time.ctime(os.path.getctime(added))
 lastmod = time.ctime(os.path.getmtime(added))



#Potential Additions/Substitutions  - what are the implications/consequences 
for this

def move_to_failure_folder_and_return_error_file():
 os.mkdir('Failure')
 copy_and_move_File(filename, 'Failure')
 initialize_logger('rootdir/Failure')
 logging.error("Either this file is empty or there are no lines")


def move_to_success_folder_and_read(f):
 os.mkdir('Success')
 copy_and_move_File(filename, 'Success')
 print("Success", f)
 return file_len()

#This simply checks the file information by name--> is this needed anymore?

def fileinfo(file):
 filename = os.path.basename(f)
 rootdir = os.path.dirname(f)
 filesize = os.path.getsize(f)
 return filename, rootdir, filesize

if __name__ == '__main__':
import sys
validate_files(sys.argv[1:])

# -- end of file




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list


Re: A simple single line, triple-quoted comment is giving syntax error. Why?

2015-04-02 Thread Thomas 'PointedEars' Lahn
Ian Kelly wrote:

> […] Thomas 'PointedEars' Lahn […] wrote:
>> Ian Kelly wrote:
>>> Within a grammar, the question of "is an X a Y" is nonsensical in
>>> isolation. It can only be answered in relation to a parse tree.
>>> Consider the simple grammar:
>>>
>>> S -> A | B
>>> A -> x
>>> B -> x
>>>
>>> Is x an A? It depends.
>>
>> No, by the definition 2 below, that we all accepted implicitly up to this
>> point, x is *definitely* an A.
> 
> What gives you the impression that I ever accepted it?

,-
| 
| What the grammar that you quoted from shows is that STRING+ is an
| expression.

There is *no way* for you to make that statement if you did not accept 
definition (2).

>> (2) Let the statement “x is an A” be true if x can be produced in a
>> production chain starting with or including the non-terminal A
>> left-hand side –
>>
>>   x ∈ A ↔ ∃A (… ⇒ A ⇒ … ⇒ x).
> 
> Sorry, but this definition just seems entirely arbitrary to me.

It is just the formalization of the definition that we all have agreed to, 
including you.

> Mathematically, it looks nonsensical; A is a symbol, not a set.

“A” is the goal symbol of a production, so it can be interpreted as the 
superset of all set of terminals that can be produced from it, through the 
goal symbols that can be produced from it.  And all of us implicitly did 
that when we said “STRING(+) (literals) is/are (not) (an) expression(s)”.

> This question of whether "x is an A" is informal and not a topic of formal
> language theory so far as I'm aware. Can you cite some source for it?

No, because I was formalizing the ad-hoc definition by Chris Angelico in 
.

>> Now, according to these definitions, in the offered grammar x is *both*
>> an A and a B.  Because what matters is _not_ the practical result of
>> production chains (the actual parse tree), but the certainty of the
>> theoretical possibility of it.
> 
> This strikes me as being a lot like arguing, "some kites are toys, and
> some kites are birds; therefore, all kites are both toys and birds."

False analogy again.  We are discussing *in theory* a *formal* grammar.  Its 
goal symbols have *no meaning* except what can be produced from them.

> As noted above, the inaccuracy that Gregory pointed out has no bearing
> on my argument.

But it does.

> You're really going to make me spell it out, aren't you? Fine, here you
> go.
> 
> single_input -> […] -> expr -> […] -> atom -> STRING STRING
> 
> Note: the derivation contains exactly one expr node, which indirectly
> produces both STRINGs. Neither STRING in this derivation is
> individually produced from the expr.

So you have proven that which nobody ever doubted nor requested, but I 
pointed out already.  What you have still not proven is what you claimed: 
the parse tree.

I am sorry that you cannot see that your argument is strewn with gaping 
defects in logic, but I think I will stop trying to convince you of that 
now.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: A simple single line, triple-quoted comment is giving syntax error. Why?

2015-04-02 Thread sohcahtoa82
On Thursday, April 2, 2015 at 2:33:17 PM UTC-7, Thomas 'PointedEars' Lahn wrote:
> Ian Kelly wrote:
> 
> > […] Thomas 'PointedEars' Lahn […] wrote:
> >> Ian Kelly wrote:
> >>> Within a grammar, the question of "is an X a Y" is nonsensical in
> >>> isolation. It can only be answered in relation to a parse tree.
> >>> Consider the simple grammar:
> >>>
> >>> S -> A | B
> >>> A -> x
> >>> B -> x
> >>>
> >>> Is x an A? It depends.
> >>
> >> No, by the definition 2 below, that we all accepted implicitly up to this
> >> point, x is *definitely* an A.
> > 
> > What gives you the impression that I ever accepted it?
> 
> ,-
> | 
> | What the grammar that you quoted from shows is that STRING+ is an
> | expression.
> 
> There is *no way* for you to make that statement if you did not accept 
> definition (2).
> 
> >> (2) Let the statement “x is an A” be true if x can be produced in a
> >> production chain starting with or including the non-terminal A
> >> left-hand side –
> >>
> >>   x ∈ A ↔ ∃A (… ⇒ A ⇒ … ⇒ x).
> > 
> > Sorry, but this definition just seems entirely arbitrary to me.
> 
> It is just the formalization of the definition that we all have agreed to, 
> including you.
> 
> > Mathematically, it looks nonsensical; A is a symbol, not a set.
> 
> “A” is the goal symbol of a production, so it can be interpreted as the 
> superset of all set of terminals that can be produced from it, through the 
> goal symbols that can be produced from it.  And all of us implicitly did 
> that when we said “STRING(+) (literals) is/are (not) (an) expression(s)”.
> 
> > This question of whether "x is an A" is informal and not a topic of formal
> > language theory so far as I'm aware. Can you cite some source for it?
> 
> No, because I was formalizing the ad-hoc definition by Chris Angelico in 
> .
> 
> >> Now, according to these definitions, in the offered grammar x is *both*
> >> an A and a B.  Because what matters is _not_ the practical result of
> >> production chains (the actual parse tree), but the certainty of the
> >> theoretical possibility of it.
> > 
> > This strikes me as being a lot like arguing, "some kites are toys, and
> > some kites are birds; therefore, all kites are both toys and birds."
> 
> False analogy again.  We are discussing *in theory* a *formal* grammar.  Its 
> goal symbols have *no meaning* except what can be produced from them.
> 
> > As noted above, the inaccuracy that Gregory pointed out has no bearing
> > on my argument.
> 
> But it does.
> 
> > You're really going to make me spell it out, aren't you? Fine, here you
> > go.
> > 
> > single_input -> […] -> expr -> […] -> atom -> STRING STRING
> > 
> > Note: the derivation contains exactly one expr node, which indirectly
> > produces both STRINGs. Neither STRING in this derivation is
> > individually produced from the expr.
> 
> So you have proven that which nobody ever doubted nor requested, but I 
> pointed out already.  What you have still not proven is what you claimed: 
> the parse tree.
> 
> I am sorry that you cannot see that your argument is strewn with gaping 
> defects in logic, but I think I will stop trying to convince you of that 
> now.
> 
> -- 
> PointedEars
> 
> Twitter: @PointedEars2
> Please do not cc me. / Bitte keine Kopien per E-Mail.

*sigh*

https://xkcd.com/386/
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Strategy/ Advice for How to Best Attack this Problem?

2015-04-02 Thread Saran A
On Thursday, April 2, 2015 at 5:11:20 PM UTC-4, Dave Angel wrote:
> On 04/02/2015 09:06 AM, Saran A wrote:
> 
> >
> > Thanks for your help on this homework assignment. I started from scratch 
> > last night. I have added some comments that will perhaps help clarify my 
> > intentions and my thought process. Thanks again.
> >
> > from __future__ import print_function
> 
> I'll just randomly comment on some things I see here.  You've started 
> several threads, on two different forums, so it's impractical to figure 
> out what's really up.
> 
> 
> 
> >
> > #Helper Functions for the Success and Failure Folder Outcomes, respectively
> >
> >  def file_len(filename):
> 
> This is an indentation error, as you forgot to start at the left margin
> 
> >  with open(filename) as f:
> >  for i, l in enumerate(f):
> >  pass
> >  return i + 1
> >
> >
> >  def copy_and_move_File(src, dest):
> 
> ditto
> 
> >  try:
> >  shutil.rename(src, dest)
> 
> Is there a reason you don't use the move function?  rename won't work if 
> the two directories aren't on the same file system.
> 
> >  # eg. src and dest are the same file
> >  except shutil.Error as e:
> >  print('Error: %s' % e)
> >  # eg. source or destination doesn't exist
> >  except IOError as e:
> >  print('Error: %s' % e.strerror)
> >
> >
> > # Call main(), with a loop that calls # validate_files(), with a sleep 
> > after each pass. Before, my present #code was assuming all filenames come 
> > directly from the commandline.  There was no actual searching #of a 
> > directory.
> >
> > # I am assuming that this is appropriate since I moved the earlier versions 
> > of the files.
> > # I let the directory name be the argument to main, and let main do a 
> > dirlist each time through the loop,
> > # and pass the corresponding list to validate_files.
> >
> >
> > path = "/some/sample/path/"
> > dirlist = os.listdir(path)
> > before = dict([(f, None) for f in dirlist)
> >
> > #Syntax Error? before = dict([(f, None) for f in dirlist)
> >   ^
> > SyntaxError: invalid syntax
> 
> Look at the line in question. There's an unmatched set of brackets.  Not 
> that it matters, since you don't need these 2 lines for anything.  See 
> my comments on some other forum.
> 
> >
> > def main(dirlist):
> 
> bad name for a directory path variable.
> 
> >  while True:
> >  time.sleep(10) #time between update check
> 
> Somewhere inside this loop, you want to obtain a list of files in the 
> specified directory.  And you want to do something with that list.  You 
> don't have to worry about what the files were last time, because 
> presumably those are gone.  Unless in an unwritten part of the spec, 
> you're supposed to abort if any filename is repeated over time.
> 
> 
> >  after = dict([(f, None) for f in dirlist)
> >  added = [f for f in after if not f in before]
> >  if added:
> >  print('Sucessfully added new file - ready to validate')
> >add return statement here to pass to validate_files
> > if __name__ == "__main__":
> >  main()
> 
> You'll need an argument to call main()
> 
> >
> >
> > #check for record time and record length - logic to be written to either 
> > pass to Failure or Success folder respectively
> >
> > def validate_files():
> 
> Where are all the parameters to this function?
> 
> >  creation = time.ctime(os.path.getctime(added))
> >  lastmod = time.ctime(os.path.getmtime(added))
> >
> >
> >
> > #Potential Additions/Substitutions  - what are the 
> > implications/consequences for this
> >
> > def move_to_failure_folder_and_return_error_file():
> >  os.mkdir('Failure')
> >  copy_and_move_File(filename, 'Failure')
> >  initialize_logger('rootdir/Failure')
> >  logging.error("Either this file is empty or there are no lines")
> >
> >
> > def move_to_success_folder_and_read(f):
> >  os.mkdir('Success')
> >  copy_and_move_File(filename, 'Success')
> >  print("Success", f)
> >  return file_len()
> >
> > #This simply checks the file information by name--> is this needed 
> > anymore?
> >
> > def fileinfo(file):
> >  filename = os.path.basename(f)
> >  rootdir = os.path.dirname(f)
> >  filesize = os.path.getsize(f)
> >  return filename, rootdir, filesize
> >
> > if __name__ == '__main__':
> > import sys
> > validate_files(sys.argv[1:])
> >
> > # -- end of file
> >
> 
> 
> -- 
> DaveA

@DaveA

I debugged and rewrote everything. Here is the full version. Feel free to tear 
this apart. The homework assignment is not due until tomorrow, so I am 
currently also experimenting with pyinotify as well. I do have questions 
regarding how to make this function compatible with the ProcessEvent Class. I 
will create another post for this. 

What would you advise in regards to renaming the inaptl

Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Chris Angelico
On Fri, Apr 3, 2015 at 11:03 AM, Dennis Lee Bieber
 wrote:
>>o IF all the contents in the file have the same length (let's assume line 
>>length)
>>
>>o THEN the file should be moved to a "success" folder and a text file written 
>>indicating the total number of records/lines/words processed
>>
>>o IF the file is empty OR the contents are not all of the same length
>>
>>o THEN the file should be moved to a "failure" folder and a text file written 
>>indicating the cause for failure (for example: Empty file or line 100 was not 
>>the same length as the rest).
>>
> You still haven't defined how you determine the "correct length" of 
> the
> record. What if the first line is 79 characters, and all the others are 80
> characters? Do you report ALL lines EXCEPT the first as being the wrong
> length, when really it is the first line that is wrong?

Relatively immaterial here; in the first place, line length is just a
placeholder (my guess is it's more likely to be something like "CSV
files with the same number of cells on each row", or something), and
in the second place, the lines aren't the failures - if there's a
mismatch, the entire file is deemed wrong. It doesn't matter whether
it's the first line or the other lines, the file is dead.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

2015-04-02 Thread Saran A
On Thursday, April 2, 2015 at 8:03:53 PM UTC-4, Dennis Lee Bieber wrote:
> On Thu, 2 Apr 2015 05:46:57 -0700 (PDT), Saran A
>  declaimed the following:
> 
> >
> >@ChrisA - this is a smaller function that will take the most updated file. 
> >My intention is the following:
> >
> >* Monitor a folder for files that are dropped throughout the day
> >
>   I would suggest that your first prototype is to be a program that
> contains a function whose only purpose is to report on the files it finds
> -- forget about all the processing/moving of the files until you can
> successfully loop around the work of fetching the directory and handling
> the file names found (by maybe printing the names of the ones determined to
> be new since last fetch).
> 
> >* When a file is dropped in the folder the program should scan the file
> >
> >o IF all the contents in the file have the same length (let's assume line 
> >length)
> >
> >o THEN the file should be moved to a "success" folder and a text file 
> >written indicating the total number of records/lines/words processed
> >
> >o IF the file is empty OR the contents are not all of the same length
> >
> >o THEN the file should be moved to a "failure" folder and a text file 
> >written indicating the cause for failure (for example: Empty file or line 
> >100 was not the same length as the rest).
> >
>   You still haven't defined how you determine the "correct length" of the
> record. What if the first line is 79 characters, and all the others are 80
> characters? Do you report ALL lines EXCEPT the first as being the wrong
> length, when really it is the first line that is wrong?
> 
>   Also, if the files are Unicode (UTF-8, in particular) -- the byte
> length of a line could differ but the character length could be the same.
> 
> >Here is the code I have written:
> >
> >import os
> >import time
> >import glob
> >import sys
> >
> >def initialize_logger(output_dir):
> >logger = logging.getLogger()
> >logger.setLevel(logging.DEBUG)
> > 
> ># create console handler and set level to info
> >handler = logging.StreamHandler()
> >handler.setLevel(logging.INFO)
> >formatter = logging.Formatter("%(levelname)s - %(message)s")
> >handler.setFormatter(formatter)
> >logger.addHandler(handler)
> > 
> ># create error file handler and set level to error
> >handler = logging.FileHandler(os.path.join(output_dir, "error.log"),"w", 
> > encoding=None, delay="true")
> >handler.setLevel(logging.ERROR)
> >formatter = logging.Formatter("%(levelname)s - %(message)s")
> >handler.setFormatter(formatter)
> >logger.addHandler(handler)
> >
> ># create debug file handler and set level to debug
> >handler = logging.FileHandler(os.path.join(output_dir, "all.log"),"w")
> >handler.setLevel(logging.DEBUG)
> >formatter = logging.Formatter("%(levelname)s - %(message)s")
> >handler.setFormatter(formatter)
> >logger.addHandler(handler)
> >
> >#Helper Functions for the Success and Failure Folder Outcomes, respectively
> >
> >#checks the length of the file
> >def file_len(filename
> >with open(filename) as f:
> >for i, l in enumerate(f):
> >pass
> >return i + 1
> >
> >#copies file to new destination
> >
> >def copyFile(src, dest):
> >try:
> >shutil.copy(src, dest)
> ># eg. src and dest are the same file
> >except shutil.Error as e:
> >print('Error: %s' % e)
> ># eg. source or destination doesn't exist
> >except IOError as e:
> >print('Error: %s' % e.strerror)
> >
> >#Failure Folder
> >
> >def move_to_failure_folder_and_return_error_file():
> >os.mkdir('Failure')
> >copyFile(filename, 'Failure')
> >initialize_logger('rootdir/Failure')
> >logging.error("Either this file is empty or the lines")
> > 
> ># Success Folder Requirement
> > 
> >def move_to_success_folder_and_read(file):
> >os.mkdir('Success')
> >copyFile(filename, 'Success')
> >print("Success", file)
> >return file_len()
> >
> >
> >#This simply checks the file information by name
> >
> >def fileinfo(file):
> >filename = os.path.basename(file)
> >rootdir = os.path.dirname(file)
> >lastmod = time.ctime(os.path.getmtime(file))
> >creation = time.ctime(os.path.getctime(file))
> >filesize = os.path.getsize(file)
> >return filename, rootdir, lastmod, creation, filesize
> >
> >if __name__ == '__main__':
> >   import sys
> >   validate_files(sys.argv[1:])
> 
>   Yeesh... Did you even try running that?
> 
>   validate_files  is not defined
>   file_lenis at the wrong indentation
>   is syntactically garbage
>   is a big time-waste (you read 
> the file just to
> enumerate the number of lines? Why didn't you count the lines while
> checking 

Re: A simple single line, triple-quoted comment is giving syntax error. Why?

2015-04-02 Thread Ian Kelly
On Thu, Apr 2, 2015 at 3:31 PM, Thomas 'PointedEars' Lahn
 wrote:
> Ian Kelly wrote:
>
>> […] Thomas 'PointedEars' Lahn […] wrote:
>>> Ian Kelly wrote:
 Within a grammar, the question of "is an X a Y" is nonsensical in
 isolation. It can only be answered in relation to a parse tree.
 Consider the simple grammar:

 S -> A | B
 A -> x
 B -> x

 Is x an A? It depends.
>>>
>>> No, by the definition 2 below, that we all accepted implicitly up to this
>>> point, x is *definitely* an A.
>>
>> What gives you the impression that I ever accepted it?
>
> ,-
> |
> | What the grammar that you quoted from shows is that STRING+ is an
> | expression.
>
> There is *no way* for you to make that statement if you did not accept
> definition (2).

Actually, there is a very simple way: I was being sloppily imprecise
when I wrote that.  First, I was speaking only in reference to the
class of parse trees with more than one STRING, as that was the topic
under discussion. I should have been clearer about that. Second, I
never should have used the term "STRING+" there (or anywhere else in
this discussion), as that merely clouded the point I was trying to
offer. What I *meant* was that the complete sequence of produced
STRINGs -- as opposed to any individual STRING -- is an expression,
and I inappropriately used "STRING+" to denote that. (Maybe that is
the point you were trying to make when you were talking about EBNF
before.)

>> This question of whether "x is an A" is informal and not a topic of formal
>> language theory so far as I'm aware. Can you cite some source for it?
>
> No, because I was formalizing the ad-hoc definition by Chris Angelico in
> .

That URI is not useful to me as I don't use a newsreader. Is that the
message dated Sun, 22 Mar 2015 14:36:48 +1100? I don't see any
suggestion of this definition in that post.

t>>> Now, according to these definitions, in the offered grammar x is *both*
>>> an A and a B.  Because what matters is _not_ the practical result of
>>> production chains (the actual parse tree), but the certainty of the
>>> theoretical possibility of it.
>>
>> This strikes me as being a lot like arguing, "some kites are toys, and
>> some kites are birds; therefore, all kites are both toys and birds."
>
> False analogy again.  We are discussing *in theory* a *formal* grammar.  Its
> goal symbols have *no meaning* except what can be produced from them.

We're discussing a formal grammar as if it described a classification
hierarchy, which gives meaning to statements like "A STRING is an
expr". Otherwise, that statement is meaningless and can be neither
true nor false. So I think that the analogy between one such hierarchy
and another is apt.

>> You're really going to make me spell it out, aren't you? Fine, here you
>> go.
>>
>> single_input -> […] -> expr -> […] -> atom -> STRING STRING
>>
>> Note: the derivation contains exactly one expr node, which indirectly
>> produces both STRINGs. Neither STRING in this derivation is
>> individually produced from the expr.
>
> So you have proven that which nobody ever doubted nor requested, but I
> pointed out already.

As I tried to point out several posts back when I suggested that we
were in agreement, and which you flatly denied.

> What you have still not proven is what you claimed:
> the parse tree.

Because you misunderstood my claim.
-- 
https://mail.python.org/mailman/listinfo/python-list


New to Programming: Adding custom functions with ipynotify classes

2015-04-02 Thread Saran A
Hello All:

Here is the program that I am trying to write (with specs):

* Monitors a folder for files that are dropped throughout the day 

* When a file is dropped in the folder the program should scan the file 

o IF all the records in the file have the same length (line length)

o THEN the file should be moved to a "success" folder and a text file written 
indicating the total number of records processed 

o IF the file is empty OR the records are not all of the same length 

o THEN the file should be moved to a "failure" folder and a text file written 
indicating the cause for failure (for example: Empty file or line 100 was not 
the same length as the rest).  

Many on forums suggest using ipynotify. I am wondering how to combine my 
current script and add it to the ipynotify.

Below is my original script (the ipynotify script is provided after this)
 
[code]
# # # Without data to examine here, I can only guess based on this 
requirement's language that 
# # fixed records are in the input.

##I made the assumption that the directories are in the same filesystem

# # Takes the function fileinfo as a starting point and demonstrates calling a 
function from within a function.  
# I tested this little sample on a small set of files created with MD5 
checksums.  I wrote the Python in such a way as it 
# would work with Python 2.x or 3.x (note the __future__ at the top).

# # # There are so many wonderful ways of failure, so, from a development 
standpoint, I would probably spend a bit 
# # more time trying to determine which failure(s) I would want to report to 
the user, and how (perhaps creating my own Exceptions)

# # # The only other comments I would make are about safe-file handling.

# # #   #1:  Question: After a user has created a file that has failed (in
# # #processing),can the user create a file with the same name?
# # #If so, then you will probably want to look at some sort
# # #of file-naming strategy to avoid overwriting evidence of
# # #earlier failures.

# # # File naming is a tricky thing.  I referenced the tempfile module [1] and 
the Maildir naming scheme to see two different 
# # types of solutions to the problem of choosing a unique filename.

## I am assuming that all of my files are going to be specified in unicode  

## Utilized Spyder's Scientific Computing IDE to debug, check for indentation 
errors and test function suite

from __future__ import print_function

import os.path
import time
import difflib
import logging

def initialize_logger(output_dir):
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
 
# create console handler and set level to info
handler = logging.StreamHandler()
handler.setLevel(logging.INFO)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
 
# create error file handler and set level to error
handler = logging.FileHandler(os.path.join(output_dir, "error.log"),"w", 
encoding=None, delay="true")
handler.setLevel(logging.ERROR)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)

# create debug file handler and set level to debug
handler = logging.FileHandler(os.path.join(output_dir, "all.log"),"w")
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)


#This function's purpose is to obtain the filename, rootdir and filesize 

def fileinfo(f):
filename = os.path.basename(f)
rootdir = os.path.dirname(f)  
filesize = os.path.getsize(f)
return filename, rootdir, filesize

#This helper function returns the length of the file
def file_len(f):
with open(f) as f:
for i, l in enumerate(f):
pass
return i + 1

#This helper function attempts to copy file and move file to the respective 
directory
#I am assuming that the directories are in the same filesystem

# If directories ARE in different file systems, I would use the following 
helper function:

# def move(src, dest): 
# shutil.move(src, dest)

def copy_and_move_file(src, dest):
try:
os.rename(src, dest)
# eg. src and dest are the same file
except IOError as e:
print('Error: %s' % e.strerror)


path = "."
dirlist = os.listdir(path)


# Caveats of the "main" function is that it does not scale well 
#(although it is appropriate if one assumes that there will be few changes)

# It does not account for updated files existing in the directory - only new 
files "dropped" in
# (If this was included in the requirements, os.stat would be appropriate here)

 
def main(dirlist):   
before = dict([(f, 0) for f in dirlist])
while True:
time.sleep(1) #time between update check
after = dict([(f, None) for f in dirlist])
added = [f for f in after if not f in before]

Re: New to Programming - XML Processing

2015-04-02 Thread Steve Hayes
On Thu, 2 Apr 2015 05:42:18 -0700, Albert-Jan Roskam 
wrote:

>
>-
>On Wed, Apr 1, 2015 10:00 AM CEST Mark Lawrence wrote:
>
>>On 01/04/2015 05:27, Andrew Farrell wrote:
>>> You should follow Rustom's advice before just diving into the blog post
>>> I linked to. Otherwise you risk blindly following things and losing your
>>> bearings when you run into bugs.
>>> 
>>
>>Sound advice, but would you please be kind enough to intersperse your answers 
>>or bottom post as top posting is heavily frowned upon here.
>>
>>TIA.
>
>Would it be possible to use a script that checks every incoming mail to the 
>Python mail list? Main ingredients beatfilsoup (to textify 


And that badly formatted posts like this are corrected for proper line
length.

 
-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: A simple single line, triple-quoted comment is giving syntax error. Why?

2015-04-02 Thread Steven D'Aprano
On Friday 03 April 2015 08:31, Thomas 'PointedEars' Lahn wrote:

> I am sorry that you cannot see that your argument is strewn with gaping
> defects in logic, but I think I will stop trying to convince you of that
> now.

I'm sorry, I've been away for four days and have lost track of this thread. 
Have you reached an agreement about the number of angels which dance on the 
head of a pin yet, or will this argument continue for another week?

This sterile, pointless arguing about the minutia of pedantic definitions is 
not even close to useful. Honestly Thomas, Ian, nobody cares any more. If I 
were a betting man, I'd bet that neither of you can describe, in one 
sentence, what is the actual disagreement between the two of you, let alone 
why it matters for Python programming.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: A simple single line, triple-quoted comment is giving syntax error. Why?

2015-04-02 Thread Chris Angelico
On Fri, Apr 3, 2015 at 4:40 PM, Steven D'Aprano
 wrote:
> This sterile, pointless arguing about the minutia of pedantic definitions is
> not even close to useful. Honestly Thomas, Ian, nobody cares any more. If I
> were a betting man, I'd bet that neither of you can describe, in one
> sentence, what is the actual disagreement between the two of you, let alone
> why it matters for Python programming.

I know that it started in response to my statement that string literal
concatenation wasn't an expression as such, but I have no idea what
either side of the current debate is, nor how it affects my
statement's validity. I _had_ hoped it would reach some sort of
conclusion - I may be right, I may be wrong, but it'd be nice to know
which. So far... no idea.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: generator/coroutine terminology

2015-04-02 Thread Steven D'Aprano
On Wednesday 01 April 2015 00:18, Albert van der Horst wrote:

> In article <55062bda$0$12998$c3e8da3$54964...@news.astraweb.com>,
> Steven D'Aprano   wrote:

>>The biggest difference is syntactic. Here's an iterator which returns a
>>never-ending sequence of squared numbers 1, 4, 9, 16, ...
>>
>>class Squares:
>>def __init__(self):
>>self.i = 0
>>def __next__(self):
>>self.i += 1
>>return self.i**2
>>def __iter__(self):
>>return self
> 
> You should give an example of usage. As a newby I'm not up to
> figuring out the specification from source for
> something built of the mysterious __ internal
> thingies.
> (I did experiment with Squares interactively. But I didn't get
> further than creating a Squares object.)


Ah, sorry about that!

Usage is:

it = Squares()  # create an iterator
print(next(it))  # print the first value
x = next(it)  # extract the second
while x < 100:
print(x)
x = next(it)


Beware of doing this:

for x in Squares():
print(x)

since Squares is an *infinite* generator, it will continue for ever if you 
let it. Fortunately you can hit Ctrl-C to interrupt the for loop at any 
point.

In Python 2, you will need to rename __next__ to just next without the 
double-leading-and-trailing underscores.


>>Here's the same thing written as a generator:
>>
>>def squares():
>>i = 1
>>while True:
>>yield i**2
>>i += 1

And for this one:

it = squares()  # create the iterator
print(next(it))  # print the first value
x = next(it)  # extract the second
while x < 100:
print(x)
x = next(it)


Usage is pretty much exactly the same.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: A simple single line, triple-quoted comment is giving syntax error. Why?

2015-04-02 Thread Ian Kelly
On Thu, Apr 2, 2015 at 11:40 PM, Steven D'Aprano
 wrote:
> This sterile, pointless arguing about the minutia of pedantic definitions is
> not even close to useful. Honestly Thomas, Ian, nobody cares any more. If I
> were a betting man, I'd bet that neither of you can describe, in one
> sentence, what is the actual disagreement between the two of you, let alone
> why it matters for Python programming.

Oh, either of us *could* describe it in one sentence; it's just that
our descriptions of how we disagree would not agree. ;-)
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: generator/coroutine terminology

2015-04-02 Thread Paul Rubin
alb...@spenarnc.xs4all.nl (Albert van der Horst) writes:
> You should give an example of usage. As a newby I'm not up to
> figuring out the specification from source for
> something built of the mysterious __ internal
> thingies.

In reality because of generator expressions, the yield statement, and
some useful built-in generators in the itertools module, you rarely
have to use those special methods.  

I'd advise reading through the itertools module documentation from
beginning to end, trying to understand what everything does.  Even if
some are not that useful, it will help convey the mode of thinking that
went into these features.

At a somewhat deeper level you might like the SICP book: 

  http://mitpress.mit.edu/sicp/

It's somewhat old now and it's about Scheme rather than Python, but it
builds up the relevant concepts quite nicely.

Regarding the squares example, consider this even simpler generator:

  def count(n):
   while True:
 yield n
 n += 1

so count(1) yields 1, 2, 3, 4 ...

This is a very useful generator but you don't need to write it since
it's included in the itertools module as itertools.count.  Its initial
value defaults to 0.  So the  squares generator can be written:

   def squares():
  return (i*i for i in itertools.count(0))

Now if you want all the squares less than 100 (i.e. 0, 1, 4, 9, ..., 81):

  wanted = itertools.takewhile(lambda x: x<100, squares())

You can print that out as a list:

   print(list(wanted))

The built-in sum function consumes an iterator, so you can add up the
squares less than 100:

  print(sum(itertools.takewhile(lambda x: x<100, squares(

this prints 205 which is 1+4+9+16+25+36+49+64+81.

These features fit together quite elegantly and code like this flows off
the fingertips naturally once you've used to it.
-- 
https://mail.python.org/mailman/listinfo/python-list