Re: Problem with pickle and restarting a program
peace writes: > On Thursday, March 20, 2014 1:20:03 AM UTC-7, dieter wrote: > ... >> You may want to use debugging to determine what goes on in detail. > ... > I tried doing that. I still could not figure out what was wrong. Thank you. Debugging is often not easy. An essential strategy is "divide and conquer": i.e. you split the complete scenario into segments and analyse each segment to find out where the bad thing happens. Related to pickle, there is one point to lock at closely: where you "dump" the data. Verify, that you are dumping the correct (expected) data. If the data is not as you expect at that point, you must analyse the first part (from where the data was produced up to the dumping point); on the other hand, if the dumped data is correct, you would verify that "load" restores this same data (this will be very likely the case) and if it does why it became wrong afterward. -- https://mail.python.org/mailman/listinfo/python-list
which async framework? - a summary
On 14/03/2014 00:36, Tim Chase wrote: On 2014-03-14 00:25, Chris Withers wrote: I've been pleasantly surprised by the succinct, well reasoned and respectful replies from each of the communities! As one who doesn't lurk on the other lists, is there a nice executive summary of their responses? Well, of course, each person recommended their own framework. The short version was: - tornado is a web server, if you're not using it, the async stuff is better developed elsewhere - asyncio is very new, not a lot of protocols out in the wild and not much battle testing - twisted is been around the block a lot, huge amounts of support and an active, knowledgeable community. Can feel crufty and old in parts, mind The community bit swung it for me, so I've opted for Twisted, if I ever get a chance to work on the project in question ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk -- https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
Le samedi 22 mars 2014 05:59:34 UTC+1, Mark H. Harris a écrit :
> On 3/21/14 11:46 PM, Chris Angelico wrote:
>
> > (Side point: You have your 0d and your 0a backwards; the Unix line
>
> > ending is U+000A, and the Windows default is U+000D U+000A.)
>
>
>
> Yeah, I know... smart apple.
>
>
>
> > How are you going to make people change? What are you going to make
>
> > them change to? Who controls this standard, and how do you convince
>
> > all OSes to comply with it?
>
>
>
> Well, we're already doing this to some extent; baby steps. Well, we
>
> have open document standards (evolving) and we have a really good sense
>
> for unicode (and python is being a genuine leader there) and the
>
> flat-file is just another open document (very simple no doubt), not
>
> different from a standards viewpoint than rft, odt, {whatever}; txt?
>
>
>
> My idea is that as we are morphing open document standards we need
>
> to keep the "flat-file" in mind too. The ASCII ship has sailed too.
>
> Unicode is in, ASCII is out (for all intents and purposes) except at
>
> Microsoft---and its time to rethink what a "flat" unicode text file
>
> really is. That's all.
>
>
No offense. A good start would be to understand "unicode"
instead of bashing MS.
jmf
--
https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On Fri, Mar 21, 2014 at 8:06 PM, Rustom Mody wrote: > Two: A comprehension variable is not bound but reassigned across the > comprehension. This problem remains in python3 and causes weird behavior when > lambdas are put in a comprehension Because Python as a language only has the concept of assignment, not binding. I think it would be weird and confusing if variables worked this way in comprehensions and nowhere else. > >>> fl = [lambda y : x+y for x in [1,2,3]] > >>> [fl[i](2) for i in [0,1,2]] > [5, 5, 5] You can get the desired effect by adding a layer of indirection: >>> fl = [(lambda x: lambda y: x+y)(x) for x in [1,2,3]] >>> [f(2) for f in fl] [3, 4, 5] If that's too ugly then give the wrapper a proper name: >>> def make_function(x): ... return lambda y: x+y ... >>> fl = [make_function(x) for x in [1,2,3]] >>> [f(2) for f in fl] [3, 4, 5] There is also the default argument trick: >>> fl = [lambda y, *, x=x: x+y for x in [1,2,3]] >>> [f(2) for f in fl] [3, 4, 5] -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On Fri, 21 Mar 2014 23:51:38 -0500, Mark H Harris wrote:
> Lambda is a problem, if only because it causes confusion. What's the
> problem? Glad you asked. The constructs DO NOT work the way most people
> would expect them to, having limited knowledge of python!
Why is that a problem? Would you consider it a problem that people who
don't understand BASIC can't understand BASIC? ("I don't get the
difference between GOTO and GOSUB. Obviously GOSUB is a problem and
should be throw out.") Do you think that the fact that people who have
never used a TI-89 calculator before may have trouble used a TI-89
calculator means that TI-89 calculators are "a problem"? (I've taught
school children who didn't know how to use their calculator.)
The problem is ignorance, not the calculator, and not lambda.
You've told us that "most people" (which people? dentists? lawyers?
illiterate tribesmen from the Amazon? professional programmers?) are
surprised by lamba's behaviour, but you haven't actually told us what
behaviour is surprising, or what "most people" expect lambda to do.
Perhaps you ought to?
> I ran into
> this thing about seven years ago (when I was studying Haskell, and
> Scheme) and I wanted to see how "pure functional" python was (well, not
> at all really).
Python is not a pure functional language, but you can write functional
code in it. If you want a pure functional language, you know where to
find one.
(I wonder whether, say, Haskell has the people coming along and demanding
that it should become more like Python?)
> I can see uses for python's lambda. But, honestly, I think python could
> deprecate its use and in five years just remove it from the language;
> along with filter, map, and reduce !
Python could deprecate many things. It would make Python a worse language.
By the way, are you aware that lambda is *identical* to def except for
three things?
- lambda is an expression, not a statement like def;
- the body of a function created with lambda is limited to a
single expression, not a block;
- the function object created with lambda has a generic name.
So unless the surprising behaviour about lambda that you're about to tell
us about relates to one of those three things, I *guarantee* that
functions created with def have the same surprising behaviour.
--
Steven D'Aprano
http://import-that.dreamwidth.org/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On Sat, 22 Mar 2014 01:24:33 -0400, Terry Reedy wrote: > On 3/22/2014 12:30 AM, Mark H Harris wrote: >> On 3/21/14 11:15 PM, Chris Angelico wrote: >>> It compounds. One reply makes for double spacing... two makes >>> quadruple, three means we have seven wasted lines between every pair >>> of real lines. That gets pretty annoying. And considering that most >>> people who reply without cleaning up the lines also keep the entire >>> quoted text (and usually top-post as well), this gets big fast. > > Before Mark started asking people adjust to the foibles of gg, we used > to get such posts. I refused to read them. I have not seen one lately, Luck you. I see them quite frequently. > say maybe his nudging has had some positive effect. > >> Yes, I can see that readily/ I get it, it just seems that fixing >> it >> at the source (gg) is the answer; > > I completely agree. However, Google seems immune to suggestions, > including requests that it try to stop being a major source of spam > posts. Remember, we are not Google's customers. We are Google's product. The customers are the advertisers. [...] > If I were in charge of the software used for this list, I would replace > Mark with a custom addition to return mis-formated posts (more blank > lines than not) with instructions on how to fix them. But I am not. Wouldn't it be less obnoxious and more useful to pass the posts through a filter that deletes the annoying blank lines? -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On 3/22/2014 5:50 AM, Steven D'Aprano wrote: On Sat, 22 Mar 2014 01:24:33 -0400, Terry Reedy wrote: If I were in charge of the software used for this list, I would replace Mark with a custom addition to return mis-formated posts (more blank lines than not) with instructions on how to fix them. But I am not. Wouldn't it be less obnoxious and more useful to pass the posts through a filter that deletes the annoying blank lines? I have thought of that too, and may have suggested it. It would be slightly harder as a decision would be required as to which to delete. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Github down?
Dan Sommers wrote: On Fri, 21 Mar 2014 14:51:54 +0100, Chris “Kwpolska” Warrick wrote: (though GitHub could qualify as social media for some…) +1 QOTW https://xkcd.com/624/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
Steven D'Aprano : > This makes perfect sense: by the time you call the functions, the name x > has been rebound to the value 3. > [...] > Now I'm not sure precisely how Haskell implements this trick, but it > suggests to me that it creates a different closure each time around > the loop of the comprehension. That could end up being very expensive. Haskell does not rebind variables. I guess Haskell simply physically substitutes object references for each syntactic occurrence of a variable. If Python worked analogously, the following loop: for i, x in enumerate([1, 2, 3]): f[i] = lambda y: x + y would unroll as follows: f[0] = lambda y: 1 + y f[1] = lambda y: 2 + y f[2] = lambda y: 3 + y That is actually how classic lambda calculus does it. Variables are substituted, not "bound" or "assigned". They are syntactic slots. There is no heap, there is no stack, there is no memory apart from the expression itself. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
Ian Kelly : > You can get the desired effect by adding a layer of indirection: > fl = [(lambda x: lambda y: x+y)(x) for x in [1,2,3]] A trick to remember! Variable lifetime reduction by function invocation. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On 22/03/2014 02:06, Rustom Mody wrote: The same in haskell: Prelude> let fl = [\ y -> x + y | x <- [1,2,3]] Prelude> [(fl!!i) 0 | i<- [0,1,2]] [1,2,3] My really big complaint about Python is that it's nothing like CORAL 66. I think I'll raise this on python ideas in an attempt to get this glaring deficiency corrected. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On 22/03/2014 03:58, Mark H Harris wrote: On 3/21/14 5:44 PM, Mark Lawrence wrote: I'm pleased to see that you have answers. In return would you either use the mailing list https://mail.python.org/mailman/listinfo/python-list or read and action this https://wiki.python.org/moin/GoogleGroupsPython to prevent us seeing double line spacing and single line paragraphs, thanks. I perceive that this is your singular pet peeve, or, you were elected by the python community some time ago to police the line-end problem ? It's a pet peeve as:- a) trying to read something that's the fourth level of reply or higher to the original gets to be almost impossible as it's all white space and no substance. b) better tools exist c) the work around is shown on the Python wiki, not on the crappy, bug ridden gg site itself. Wow, it's like a sauna in here :) I doubt that the Python community would elect me to do anything. Anyhow I start my new job in the diplomatic corp next week. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On 22/03/2014 08:54, wxjmfa...@gmail.com wrote:
Le samedi 22 mars 2014 05:59:34 UTC+1, Mark H. Harris a écrit :
On 3/21/14 11:46 PM, Chris Angelico wrote:
(Side point: You have your 0d and your 0a backwards; the Unix line
ending is U+000A, and the Windows default is U+000D U+000A.)
Yeah, I know... smart apple.
How are you going to make people change? What are you going to make
them change to? Who controls this standard, and how do you convince
all OSes to comply with it?
Well, we're already doing this to some extent; baby steps. Well, we
have open document standards (evolving) and we have a really good sense
for unicode (and python is being a genuine leader there) and the
flat-file is just another open document (very simple no doubt), not
different from a standards viewpoint than rft, odt, {whatever}; txt?
My idea is that as we are morphing open document standards we need
to keep the "flat-file" in mind too. The ASCII ship has sailed too.
Unicode is in, ASCII is out (for all intents and purposes) except at
Microsoft---and its time to rethink what a "flat" unicode text file
really is. That's all.
No offense. A good start would be to understand "unicode"
instead of bashing MS.
jmf
How apt given how this thread has moved :)
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
---
This email is free from viruses and malware because avast! Antivirus protection
is active.
http://www.avast.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On Saturday, March 22, 2014 3:39:21 AM UTC+2, Terry Reedy wrote: > Does your .b2 install work? Can you delete it thru the programs list? I uninstalled it before this entire adventure. -- https://mail.python.org/mailman/listinfo/python-list
help with for loop----python 2.7.2
I am trying to get all the element data from the rss below.
The only thing I am pulling is the first element.
I don't understand why the for loop does not go through the entire rss.
Here is my code
try:
from urllib2 import urlopen
except ImportError:
from urllib.request import urlopen
from bs4 import BeautifulSoup
soup = BeautifulSoup(urlopen('http://bl.ocks.org/mbostock.rss'))
#print soup.find_all('item')
#print (soup)
for item in soup.find_all('item'):
#for item in soup:
title = soup.find('title').text
link = soup.find('link').text
item = soup.find('item').text
print item
print title
print link
--
https://mail.python.org/mailman/listinfo/python-list
Re: Installing ssdeep on Portable Python /advice
http://ssdeep.sourceforge.net/usage.html the installation described in aboved document is for Linux only. Well, I need experiment and see errors. Regards, > - Original Message - > From: Mark H Harris > Sent: 03/22/14 05:32 AM > To: python-list@python.org > Subject: Re: Installing ssdeep on Portable Python /advice > > On 3/21/14 9:51 PM, Mark H Harris wrote: > > On 3/20/14 7:16 PM, laguna...@mail.com wrote: > > >> $ tar -zxvf ssdeep-2.10.tar.gz > >> $ cd ssdeep-2.10&& ./configure&& make&& sudo make install > > >> I need install it on PortablePython for Windows, so it's not > >> clear how to make this: where should be placed ssdeep Windows > >> binary files, that Python application can found it? > > > It is strongly recommended that ssdeep be run on windows from > > precompiled binaries. Building from sources is not recommended because > > of dependencies and environment... > > > Google is our frined; > > > http://ssdeep.sourceforge.net/usage.html > > > Actually, I restated that wrong. The doc says that building from sources > on windows is NOT supported. > > marcus > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: help with for loop----python 2.7.2
On Sat, Mar 22, 2014 at 5:21 AM, wrote:
> I am trying to get all the element data from the rss below.
> The only thing I am pulling is the first element.
> I don't understand why the for loop does not go through the entire rss.
> Here is my code
[SNIP]
> for item in soup.find_all('item'):
> #for item in soup:
> title = soup.find('title').text
> link = soup.find('link').text
> item = soup.find('item').text
The three find method calls in the for loop are searching from the
document root (the "soup" variable), not from the item you're
currently iterating at. Try changing these to calls of item.find. And
note that calling one of the results "item" will replace the loop
variable. That won't affect the iteration, but it's bad practice to
refer to two different things by the same local name.
--
https://mail.python.org/mailman/listinfo/python-list
terminate a program gracefully from a thread
Hi,
I have a script (see below) that I want to terminate after X seconds.
The main loop of the program is waiting for user input.
The program enters the main loop and I try to shut down the program
after X seconds from a thread but I can't figure out how to do it. The
program should also do some cleanup before termination, so the shut
down must be graceful.
The code below is a simplified version. The whole idea is the
following: I have a script that has grown quite big over time. It
needs to read several data files, so when I start it for the first
time, it takes about 3-4 seconds to launch. The next start is much
faster since, I guess, the OS has done some caching. I use this script
a lot and the first slow launch bothers me. My idea: after booting, I
want to start the script in the background in suicide mode. OS does
the caching, so when I need it, it starts quickly.
See the code below with comments.
Thanks,
Laszlo
===
import atexit
import sys
import time
from threading import Thread
import os
def suicide(wait):
time.sleep(wait)
print("should exit now...")
sys.exit() # exits this thread but not the main thread
#os._exit(0)# no cleanup with this :(
def cleanup():
# I want it to run before termination.
print("cleaning up...")
def main():
Thread(target=suicide, kwargs={'wait': 3}).start()
#
while True:
try:
inp = raw_input("in> ")
print(inp)
except (KeyboardInterrupt, EOFError):
print()
sys.exit()
#
if __name__ == "__main__":
atexit.register(cleanup)
main()
--
https://mail.python.org/mailman/listinfo/python-list
Re:terminate a program gracefully from a thread
Jabba Laci Wrote in message: > Hi, > > I have a script (see below) that I want to terminate after X seconds. > The main loop of the program is waiting for user input. > The program enters the main loop and I try to shut down the program > after X seconds from a thread but I can't figure out how to do it. The > program should also do some cleanup before termination, so the shut > down must be graceful. > > The code below is a simplified version. The whole idea is the > following: I have a script that has grown quite big over time. It > needs to read several data files, so when I start it for the first > time, it takes about 3-4 seconds to launch. The next start is much > faster since, I guess, the OS has done some caching. I use this script > a lot and the first slow launch bothers me. My idea: after booting, I > want to start the script in the background in suicide mode. OS does > the caching, so when I need it, it starts quickly. > You need a flag to indicate that a particular invocation is the dummy one (background). So use that same flag either to suppress starting the thread, or to avoid the unwanted raw_input. If you had no blocking I/o, you could use a simple global to notify all the threads. Or use a signal to abort the main thread if it's stuck in raw_input. Alternatively, rethink the need to preload at boot time. Any caching the OS does is likely to only last a few minutes, depending on load. So maybe you can make the real load seem to be quicker by displaying the gui right away, but doing the time-consuming part in a thread. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re: terminate a program gracefully from a thread
> You need a flag to indicate that a particular invocation is the > dummy one (background). So use that same flag either to suppress > starting the thread, or to avoid the unwanted raw_input. > > Alternatively, rethink the need to preload at boot time. Any > caching the OS does is likely to only last a few minutes, > depending on load. So maybe you can make the real load seem to be > quicker by displaying the gui right away, but doing the > time-consuming part in a thread. Hi, Thanks for the idea. Right, if it's started in suicide mode, then there is no need to enter the raw_input. Thanks, Laszlo -- https://mail.python.org/mailman/listinfo/python-list
Re: terminate a program gracefully from a thread
On Sun, Mar 23, 2014 at 12:18 AM, Dave Angel wrote:
> Alternatively, rethink the need to preload at boot time. Any
> caching the OS does is likely to only last a few minutes,
> depending on load. So maybe you can make the real load seem to be
> quicker by displaying the gui right away, but doing the
> time-consuming part in a thread.
I second this motion. Don't preload stuff on boot like that; if you
want the data preloaded, actually load the program and keep it
running. Depending on the OS/FS cache like that is a bit better than
what seems to proliferate on Windows ("MS Office fast-load" being one
of the worst offenders), where there's this constant fight between
boot time and program load time, with RAM usage going nuts in between.
By (ab)using the disk cache like this, you make your cache work
beautifully until some app actually needs all that RAM, and then your
cache will be dropped; which is a good thing for the system, but it
means you'll have completely wasted the loading effort. Much better to
be more explicit about it, and probably just accept that first startup
is slower.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: newbie - Does IDLE care about sitecustomize.py?
Could you figure this out? On Wednesday, November 16, 2005 10:37:09 PM UTC+8, bobu...@yahoo.com wrote: > I have the following test script in the file customize.py > > # C:\Python24\Lib\site-packages\sitecustomize.py > print "test text from sitecustomize" > > If start Python from command prompt I get > > C:\Python24>python > test in sitecustomize > Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] > on win32 > Type "help", "copyright", "credits" or "license" for more information. > > which shows that sitecustomize.py works > > > Now if I start IDLE I only get > > IDLE 1.1.1 > >>> > > which shows that that IDLE doesn't care about sitecustomize.py > > Am I missing something or how do you customize if you are using IDLE? -- https://mail.python.org/mailman/listinfo/python-list
Re: newbie - Does IDLE care about sitecustomize.py?
On Sun, Mar 23, 2014 at 12:41 AM, wrote: > Could you figure this out? > > On Wednesday, November 16, 2005 10:37:09 PM UTC+8, bobu...@yahoo.com wrote: >> [ chomp ] You're responding to a decade-old post, you're posting from Google Groups, and you haven't added any information to the thread at all. The original post was talking about Python 2.4, which is well and truly out of support (unless you're running RHEL5), so the very first thing you should do is see if there's still a corresponding issue with a current version of Python. The next thing to do is to make sure you don't offend with form (always offend people with substance, it's more satisfying), so either don't use Google Groups, or clean up what you post so it comes out looking correct. And following the standard interleaved style, rather than top-posting, will also help. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
User prompt as file to read
I'm trying to create a program that will prompt the user for a list of text
files to read from, then read those text files and build a dictionary of all
the unique words found. Then finally put those unique words into another file
and make it alphabetical order.
What I've got:
import string
s = input("Enter a file name: ") + ".txt"
filepath = "I:\\" + s
# remove all punctuation marks and make lower case
s_nopunct = "".join(c for c in s if c not in string.punctuation).lower()
# convert to a sorted list of unique words via set comprehension
list_unique = sorted(list({word for word in s_nopunct.split()}))
print("\nSorted list of unique words in sentence:")
print(list_unique)
with open("C:\\Users\\Desktop\\words.dat", "w") as f:
for x in list_unique:
f.write(x + "\n")
I need help making it so that the user is prompted to enter at least 3 files.
And also, I tried making those unique words to write to another file (I got it
that far), but how do I make it more of an arbitrary path (rather than the
C:\Users etc) since I need it so that anyone can run that program and
write/find to that file.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python - Caeser Cipher Not Giving Right Output
On Mar 20, 2014 9:59 PM, "Dave Angel" wrote: > > dtran...@gmail.com Wrote in message: > > And I was wondering how I would add the partenthesis because I tried: > > > > return numtochar(c1 + c2 (%26)) and it gave me an error. > > Please help us to help you by actually showing the traceback. > Doesn't matter in this case, but... > > What order do you want the add and the modulo to happen? Use the > parentheses to say so. Ah, but he did. x = y + z (mod m) is a perfectly well-formed math equation. Not the OP's fault if it doesn't work that way in Python. -- https://mail.python.org/mailman/listinfo/python-list
Re:User prompt as file to read
kjaku...@gmail.com Wrote in message:
> I'm trying to create a program that will prompt the user for a list of text
> files to read from, then read those text files and build a dictionary of all
> the unique words found. Then finally put those unique words into another file
> and make it alphabetical order.
Specify python version and os. I assume python 3 and Windows.
>
> What I've got:
>
> import string
>
> s = input("Enter a file name: ") + ".txt"
> filepath = "I:\\" + s
So you've got a filename. You're not using it for anything.
Where's your open? Where's your read or readline?
>
> # remove all punctuation marks and make lower case
> s_nopunct = "".join(c for c in s if c not in string.punctuation).lower()
Are you sure you want a single string nearly the total size of
your 3 files? Could be huge. Might be better to do it
incrementally.
It's a lot safer to include the characters you want, instead of
excluding some of the ones you don't. And many valid words
contain punctuation such as apostrophe.
>
> # convert to a sorted list of unique words via set comprehension
> list_unique = sorted(list({word for word in s_nopunct.split()}))
>
> print("\nSorted list of unique words in sentence:")
> print(list_unique)
>
> with open("C:\\Users\\Desktop\\words.dat", "w") as f:
> for x in list_unique:
> f.write(x + "\n")
>
> I need help making it so that the user is prompted to enter at least 3 files.
Need a while loop for that.
> And also, I tried making those unique words to write to another file (I got
> it that far), but how do I make it more of an arbitrary path (rather than the
> C:\Users etc) since I need it so that anyone can run that program and
> write/find to that file.
>
That could be another input, or it could be a command line
parameter. Your choice.
--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On Fri, 21 Mar 2014 22:58:37 -0500, Mark H Harris wrote:
> I notice (since moving my stuff to Thunderbird two weeks back) the
> double spacing you keep squawking about, but I don't find it the big
> nuisance you're talking about; ok, so we have to scroll a bit further.
It's not the scrolling that interferes
with readability, it's the interruption
to the flow of text by having excess
blank lines within a paragraph of text.
> I am honestly convinced that this might even be a python problem. More
> likely than not, gg is written in python, and this is the goofy line-end
> character problem we have to deal with when we read lines in python.
Well, that's certainly a novel idea. Why you think that Google Groups is
written in Python? Does every post from GG end with "Powered By Python"?
> Why do we suck in the new-line character as though it were part of the
> line?
Because it is the only sensible way to handle it. If you don't, then
there is no way to distinguish between a file that ends with a newline
and one which doesn't.
> This is asinine behavior. The new-line is a "file" delimiter
> character and NOT intended to be part of the line.
Line endings are terminators: they end the line. Whether you consider the
terminator part of the line or not is a matter of opinion (is the cover
of a book part of the book?) but consider this:
If you say that the end of lines are *not* part of the line, then
that implies that some parts of the file are not inside any line
at all. And that would be just weird.
> Thinking this through a bit
Yes, that helps :-)
> I've noticed that a blank line comes back
> with a '\n' which differentiates it from file end which comes back
> "without" the new-line. So, it appears that python is using the
> new-line character (or lack there-of) to have meaning which the new=line
> in a unix file was never suppose to carry.
I don't understand what meaning you are referring to here. Blank lines
comes back as a \n because a blank line *is* a \n with nothing before it.
Python isn't doing anything funny here, at least not on Linux. If you
open a text editor, and type:
spam ENTER ENTER ENTER ENTER eggs ENTER
(where ENTER means to hit the Enter key, not the letters E N T E R) and
then save, your editor will show the words "spam" and "eggs" separated by
three blank lines. If you open that file in a hex editor, you will see
something like:
73 70 61 6d 0a 0a 0a 0a 65 67 67 73 0a
Notice the four 0a bytes in a row? That gives you three blank lines.
Python is no adding any extra newlines or putting them where they aren't,
so I don't really understand what point you're trying to make here.
> If python would just return EOF like every other language at file end,
> and a test line without '\n' tacked to the end of it, this little snag
> with gg would probably go away. What say you?
There is no evidence that Google Group's difficulty is because of Python.
More likely it is related to the translation between "rich text"
formatted using HTML, and plain text.
By the way, Python *does* return EOF at the end of the file. It is just
that EOF in Python is spelled "the empty string" instead of some other
special value. Both Ruby and Lua behave like Python, returning the empty
string on end of file:
steve@orac:~$ irb
irb(main):001:0> f = File.open('/tmp/io.txt')
=> #
irb(main):002:0> f.read()
=> "hello"
irb(main):003:0> f.read()
=> ""
[steve@ando ~]$ lua
Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
> fp = io.open('/tmp/io.txt', 'r')
> print(fp:read("*all"))
hello
> print(fp:read("*all"))
>
Similarly, Rust returns None:
http://static.rust-lang.org/doc/0.9/std/io/trait.Reader.html#tymethod.read
And Java's java.io.BufferedReader is also similar, returning null.
--
Steven D'Aprano
http://import-that.dreamwidth.org/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano wrote: > Line endings are terminators: they end the line. Whether you consider the > terminator part of the line or not is a matter of opinion (is the cover > of a book part of the book?) but consider this: > > If you say that the end of lines are *not* part of the line, then > that implies that some parts of the file are not inside any line > at all. And that would be just weird. Not so weird IMO. A file is not a concatenation of lines; it is a stream of bytes. Now, if you ask Python to read you 512 bytes from a binary file, and then ask for another 512 bytes, and so on until you reach the end, then it would indeed be VERY weird if there were parts of the file that weren't in the returned (byte) strings. But if you ask for a line, and then another line, and another line, then it's quite reasonable to interpret U+000A as "line separation" rather than "line termination", and not return it. (Both interpretations make sense. I just wish the most obvious form of iteration gave the cleaner/tidier version, or at very least that there be some really obvious way to ask for lines-without-endings.) Imagine the output of GNU find as a series of records. You can ask for those to be separated by newlines (the default, or -print), or by NULs (with the -print0 command). In either case, the records do not *contain* that value, they're separated by it; the records consist of file names. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On Saturday, March 22, 2014 2:39:56 PM UTC+5:30, Ian wrote: > On Fri, Mar 21, 2014 at 8:06 PM, Rustom Mody wrote: > > Two: A comprehension variable is not bound but reassigned across the > > comprehension. This problem remains in python3 and causes weird behavior > > when > > lambdas are put in a comprehension > Because Python as a language only has the concept of assignment, not > binding. I think it would be weird and confusing if variables worked > this way in comprehensions and nowhere else. Bizarre viewpoint! When you do this: > There is also the default argument trick: > >>> fl = [lambda y, *, x=x: x+y for x in [1,2,3]] > >>> [f(2) for f in fl] > [3, 4, 5] how is that not-a-binding solution? More generally, insofar as variable-scopes can be made and exited, there is binding. Its just that imperative languages have - assignment wherein the shape of the environment is preserved but its content is changed - there are binding-constructs -- functions, methods, classes etc etc -- which leave extant bindings intact but create/remove new ones. Ok, functional languages have only the latter. But only the former?? Beyond assembly language I dont know what that would/could be... -- https://mail.python.org/mailman/listinfo/python-list
Re: Python - Caeser Cipher Not Giving Right Output
On Saturday, March 22, 2014 7:52:28 PM UTC+5:30, Ian wrote: > On Mar 20, 2014 9:59 PM, "Dave Angel" wrote: > > dtra...@gmail.com Wrote in message: > > > And I was wondering how I would add the partenthesis because I tried: > > > return numtochar(c1 + c2 (%26)) and it gave me an error. > > Please help us to help you by actually showing the traceback. > > Doesn't matter in this case, but... > > What order do you want the add and the modulo to happen? Use the > > parentheses to say so. > Ah, but he did. x = y + z (mod m) is a perfectly well-formed math equation. > Not the OP's fault if it doesn't work that way in Python. ! !I forgot that parse! -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about Source Control
In article ,
Tim Chase wrote:
>On 2014-03-18 21:38, Terry Reedy wrote:
>> At least with hg, one should best test the code in the working
>> directory *before* committing to the local repository.
>
>I don't know if this is a hg-vs-git way of thinking, but I tend to
>frequently commit things on a private development branch regardless
>of brokenness, but once I get it working, I flatten & clean up those
>changes ("rebase" in git terms, which I believe has been adopted as a
>standardly-available-but-not-enabled-by-default module in hg) into
>logical units of change-sets that I then test individually before
>applying them to my more public-facing branch. This produces clean
>history that makes it easy for others to see what's going on.
I see it this way that all code is broken to at least a small
extent, so it is stupid to not to save into source control because
code is not yet flawless.
I use RCS (!) for my eulerproject.net programs, and
I save every small milestone. There is just one important rule,
if you save code that has (severe) restrictions, keep
track of it in the submission message, otherwise you loose your
bearings.
Basically the first ten of so versions (before the tag WINNER)
just don't solve the problem.
(Of course euler problems are hard, three rewrites are not
uncommon.)
I compare the in between versions with the nails they put in
the mountainside in climbing. It is a point below which you'll
never need to slide back.
>
>-tkc
Groetjes Albert
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
--
https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On 22/03/2014 09:09, Ian Kelly wrote: On Fri, Mar 21, 2014 at 8:06 PM, Rustom Mody wrote: Two: A comprehension variable is not bound but reassigned across the comprehension. This problem remains in python3 and causes weird behavior when lambdas are put in a comprehension Because Python as a language only has the concept of assignment, not binding. I think it would be weird and confusing if variables worked this way in comprehensions and nowhere else. My understanding has always been that an expression of the rhs is bound to a name of the lhs. So is my understanding wrong, or is the above wrong, or are we talking at cross purposes, or what? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about Source Control
In article , Gregory Ewing wrote: >Chris Angelico wrote: >> You can then offer a non-source-control means of downloading that >> specific revision. > >Just keep in mind the downside that you can't then >push or pull your changes directly back into the main >repository. You can generate a patch file for the >project maintainer to apply, however. Hg makes it >very easy to produce a patch file between any two >revisions. > >Also, unless the project is truly ancient, the >whole history might not be as big as you expect. >The code presumably grew to its present size >incrementally, in an approximately monotonic >manner, so the sum of all the diffs is probably >about the same order of magnitude as the current >code size. > >As an experiment, I just cloned a copy of the >CPython repository, and it's about 300MB. A >tarball of Python 3.2 that I downloaded and >compiled earlier is about 75MB. That's a ratio >of about 4, and CPython is a pretty ancient >project! This post made me worry for the first time about one project of mine (ciforth). It started in 2000 with an msdos assembler file, and after several hundreds version it has accumulated doc's and test's and is now usable on linux, windows whatnot. Since 2000 the cvs style archive has grown to 2 megabyte, for a current version of 400 kbyte. I kept the smallest of changes, and at times was very happy I did. Bottom line, the grow of a source archive cannot keep up with LAN and Internet speeds and hard disk sizes. > >-- >Greg Groetjes Albert -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst -- https://mail.python.org/mailman/listinfo/python-list
RE: Controlling buffer alignment in file.read()
Haralanov, Mitko wrote: >> For control at that level you'd be better off using >> direct system calls, i.e. os.open() and os.read(), >> then you can read exacty the number of bytes you want. >> > > The problem is not controlling the number of bytes read. That part seems > to be working. The issue is that the buffer into which the data is placed > needs to be of certain alignment (8byte-aligned). Python does not seem to > have a way that allows me to control that. > > Thanks, > - Mitko Did you try to set buffering parameter to 0 (unbuffered) ? Eventually going to os.open() which map tp low level (eventually followed by an os.fdopen()) http://docs.python.org/2/library/os.html#os.open A+ Laurent. -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
Mark Lawrence : > On 22/03/2014 09:09, Ian Kelly wrote: >> Because Python as a language only has the concept of assignment, not >> binding. I think it would be weird and confusing if variables worked >> this way in comprehensions and nowhere else. > > My understanding has always been that an expression of the rhs is > bound to a name of the lhs. So is my understanding wrong, or is the > above wrong, or are we talking at cross purposes, or what? Hard to say without knowing more of your understanding. Even Scheme doesn't have purely classical variable binding because variables can be assigned to. You will notice that right away when you try to implement a lisp dialect; a purely binding (substituting) implementation fails with set!/setq because there is no variable to assign a value to. Python variables are named memory slots you can peek and poke into. Not so in a purely functional language where variables are physically removed from the code before the code is executed. Example: def redouble(x): return x + x redouble(17 + 4) If Python were simply binding variables in the purely functional sense, the interpreter would first evaluate 17 + 4 and then make a copy of the "redouble" function substituting the result (21) for each syntactic occurrence of x: def __redouble__234001942(): return 21 + 21 The interpreter would then proceed to evaluate the variable-less function instance. If you leave out assignment, there is really no difference between the two models. Only if you don't have assignment, you don't need to complicate your computational model with memory. Instead, you deal with things like continuations. In fact, even "def" violates the classic functional paradigm. For example, to calculate 10's factorial without assignments, you can do this (even in Python): (lambda n: (lambda fac: fac(fac, n)) \ (lambda fac, n: 1 if n < 2 else n * fac(fac, n - 1)))(10) Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
The foll is fairly standard fare in denotational semantics -- please excuse the length! In order to understand (formally) the concept of 'variable' we need to have at the least a concept of name(or identifier) -> value mapping. This mapping is called an 'environment' If we stop at that we get the 'simplest' (at least from a mathematical pov!!) language -- λ calculus. However programming languages also need to be implemented -- typically on von Neumann machines. So we make 'Variable' a composition of two functions: Env : Identifier -> Location Store : Location -> Value This seems fine and dandy until we realize that if the compositon is of non one-one onto functions all kinds of troubles result; eg -- Two locations going to one value -- aliasing -- Store partial at a location -- uninitialized variable etc etc the most innocuous looking… -- assignment becomes a higher order function because it converts a starting id -> -> value mapping to a new mapping Seeing all these difficulties, some religious zealots (aka functional programmers) decide that this composite mapping is the root of the problem -- throw it out -- no aliasing, no assignment, no time. [Minor problem -- How to implement -- remains!] The non-religious bigots (also called C programmers) see that managing the Env at one time and the Store at a later time (the two times are usually called compile-time and run-time but like bell-bottoms these words are currently unfashionable!) can produce very effective engineering expedience. So strictly speaking whenever we have variables we have binding [Probably mathematica is an exception... dunno for sure] More loosely and conventionally if the mapping is a single direct one: Env: Variable -> Value as in λ calculus, functional languages etc, they are called binding-languages To distinguish, the 'other' languages which need a compose of Environment and Store are called variously: - imperative language - reference semantics - conventional (imperative) variable (vs mathematical variable) etc IOW in most (normal) languages we have constructs to change the shape of the environment and we have constructs to change the content of the environment. The single pre-eminent way for the latter is assignment, function is the typical way for the former. [Unfortunately this is not universally true: In C we have initialized variables that look like assignment but is not. In python the exception is what Ian calls the default variable trick: x=x would be a rather asinine assignment. But its not an assignment at all, it just looks like one] So no... > My understanding has always been that an expression of the rhs is bound > to a name of the lhs. So is my understanding wrong, or is the above > wrong, or are we talking at cross purposes, or what? assignment changes the content of the env, functions change the shape -- which latter you may call binding. -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about Source Control
On 2014-03-22 17:32, Albert van der Horst wrote:
> >I don't know if this is a hg-vs-git way of thinking, but I tend to
> >frequently commit things on a private development branch regardless
> >of brokenness, but once I get it working, I flatten & clean up
> >those changes ("rebase" in git terms, which I believe has been
> >adopted as a standardly-available-but-not-enabled-by-default
> >module in hg) into logical units of change-sets that I then test
> >individually before applying them to my more public-facing
> >branch. This produces clean history that makes it easy for others
> >to see what's going on.
>
> I see it this way that all code is broken to at least a small
> extent, so it is stupid to not to save into source control because
> code is not yet flawless.
I agree that skipping the commits just because it might be broken is
a foolish idea. However, with most VCS tools, your commits can look
something like
Baseline -> A -> B -> C -> D -> E -> F -> G {head/tip}
but A, C, & E all comprise one conceptual change, while B & G are
another, and D & F cancel each other out. You can either cherry-pick
those changes or rebase (with git or the hg plugin) so that the
history looks like
Baseline -> (ACE) -> (BG) {head/tip}
With git, the history isn't actually discarded immediately, but
rather just gets a parallel version (which may or may not have a
reference to it; if it doesn't it will get cleaned up about a month
later according to your garbage-collection settings) so your repo
ends up looking something like
Baseline -> A -> B -> C -> D -> E -> F -> G {orphaned or named}
\---> (ACE) -> (BG) {head/tip}
You can then publish that conceptually clean (and hopefully
tested&working) branch while simultaneously having the full history
in the event you need it. That said, I almost never want the
intermediate work product once I have a final clean version, so I
just let git GC that for me.
-tkc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Question about Source Control
Hi,
I can recommend the book "Pragmatic Guide to Git". Very practical and to the
point:
http://www.amazon.com/Pragmatic-Guide-Git-Programmers/dp/1934356727/ref=sr_1_1/184-0142481-0484062?ie=UTF8&qid=1395518159&sr=8-1&keywords=pragmatic+guide+to+git
I addition, I read a big fat super-exhaustive book, I believe it' s this one
(there are two Git books with a bat!):
http://www.amazon.com/Version-Control-Git-collaborative-development/dp/1449316387/ref=sr_1_2/184-0142481-0484062?ie=UTF8&qid=1395518159&sr=8-2&keywords=pragmatic+guide+to+git
The former is for common tasks that are not common enough to remember right
away. The latter is for reference.
I only have experience with git and subversion. I like git much better. But any
SCM is better than none at all.
Regards,
Albert-Jan
~~
All right, but apart from the sanitation, the medicine, education, wine, public
order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~
On Saturday, March 22, 2014 7:49 PM, Tim Chase
wrote:
On 2014-03-22 17:32, Albert van der Horst wrote:
>> >I don't know if this is a hg-vs-git way of thinking, but I tend to
>> >frequently commit things on a private development branch regardless
>> >of brokenness, but once I get it working, I flatten & clean up
>> >those changes ("rebase" in git terms, which I believe has been
>> >adopted as a standardly-available-but-not-enabled-by-default
>> >module in hg) into logical units of change-sets that I then test
>> >individually before applying them to my more public-facing
>> >branch. This produces clean history that makes it easy for others
>> >to see what's going on.
>>
>> I see it this way that all code is broken to at least a small
>> extent, so it is stupid to not to save into source control because
>> code is not yet flawless.
>
>I agree that skipping the commits just because it might be broken is
>a foolish idea. However, with most VCS tools, your commits can look
>something like
>
> Baseline -> A -> B -> C -> D -> E -> F -> G {head/tip}
>
>but A, C, & E all comprise one conceptual change, while B & G are
>another, and D & F cancel each other out. You can either cherry-pick
>those changes or rebase (with git or the hg plugin) so that the
>history looks like
>
> Baseline -> (ACE) -> (BG) {head/tip}
>
>With git, the history isn't actually discarded immediately, but
>rather just gets a parallel version (which may or may not have a
>reference to it; if it doesn't it will get cleaned up about a month
>later according to your garbage-collection settings) so your repo
>ends up looking something like
>
> Baseline -> A -> B -> C -> D -> E -> F -> G {orphaned or named}
> \---> (ACE) -> (BG) {head/tip}
>
>You can then publish that conceptually clean (and hopefully
>tested&working) branch while simultaneously having the full history
>in the event you need it. That said, I almost never want the
>intermediate work product once I have a final clean version, so I
>just let git GC that for me.
>
>
>-tkc
>
>
>
>
>
>
>
>
>
>
>
>
>
>--
>https://mail.python.org/mailman/listinfo/python-list
>
>
>--
https://mail.python.org/mailman/listinfo/python-list
Re: newbie - Does IDLE care about sitecustomize.py?
On 3/22/2014 9:41 AM, vikram.deni...@gmail.com wrote: Could you figure this out? On Wednesday, November 16, 2005 10:37:09 PM UTC+8, bobu...@yahoo.com wrote: I have the following test script in the file customize.py # C:\Python24\Lib\site-packages\sitecustomize.py print "test text from sitecustomize" If start Python from command prompt I get C:\Python24>python test in sitecustomize Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Note: running on Windows Type "help", "copyright", "credits" or "license" for more information. which shows that sitecustomize.py works Now if I start IDLE I only get IDLE 1.1.1 On Windows, the default mode of running Idle is without a console window, with pythonw.exe. Site.py and sitecustomize.py are run before anything else and any output is discarded. I propose to note this where sitecustomize.py is explained. http://bugs.python.org/issue21026 -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: terminate a program gracefully from a thread
On 3/22/2014 8:40 AM, Jabba Laci wrote: I have a script (see below) that I want to terminate after X seconds. The main loop of the program is waiting for user input. The program enters the main loop and I try to shut down the program after X seconds from a thread but I can't figure out how to do it. The program should also do some cleanup before termination, so the shut down must be graceful. Although you have gotten comments specific to your problem, this type of thing should be easy (in just one thread) with asyncio (new in 3.4, backport on PyPI). -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Help needed to create a Python extension library for an existing shared memory hash table library
Hi Python fans, I just released my first open source project ever called SharedHashFile [1]. It's a shared memory hash table written in C. Some guy on Quora asked [2] whether there's an extension library for Python coming out. I would like to do one but I know little about Python. I was wondering if anybody in this group has experience writing extension libraries for Python? Could you help? Or could we collaborate? Thanks and all the best, Simon [1] https://github.com/simonhf/sharedhashfile [2] http://www.quora.com/Inter-Process-Communication-1/Whats-a-good-library-to-share-a-key-value-store-across-multiple-processes-through-shared-memory -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
Thanks to all those who answered. - Vasudev -- https://mail.python.org/mailman/listinfo/python-list
RICHARD LEAKEY RECENTLY ARRESTED -- THE THRINAXODON TIMES REPORTS YOU CRAP, YOU CALL OUT BULLSHIT!
>LOOK, MOM! A BIRD, A PLANE, A THRINAXODON! > RICHARD LEAKEY WAS RECENTLY ARRESTED IN CONNECTION TO EVOLUTIONARY SCANDALS. > THRINAXODON CAUGHT THE WHOLE SCENE! > ONLY ONE GUESS TO WHAT RICHARD LEAKEY WAS UP TO WILL SHUT YOUR MOUTHS! > RICHARD LEAKEY WAS BRAINWASHING CHILDREN INTO THE SCAM OF HUMAN EVOLUTION JUST TO MAKE BIG BUCKS... > == EVIDENCE THAT HUMANS LIVED IN THE DEVONIAN: https://groups.google.com/group/sci.bio.paleontology/browse_thread/thread/6f501c469c7af24f# https://groups.google.com/group/sci.bio.paleontology/browse_thread/thread/3aad75c16afb0b82# -- ---Thrinaxodon -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On Sat, 22 Mar 2014 05:26:26 -, Rustom Mody wrote: Well almost... Except that the 'loop' I am talking of is one of def loop(): return [yield (lambda: x) for x in [1,2,3]] or return (yield (lambda: x) for x in [1,2,3]) or just plain ol (lambda x: for x in [1,2,3]) IOW loop is an imperative construct, comprehensions are declarative I'm sorry, you've made a logical leap too far here. I understand loops being imperative, but how are comprehensions declarative? What do they declare that the loop equivalent doesn't. You've made a great deal of the "for" in a comprehension not having the same meaning as the "for" in a loop. That may well be true in the equivalent Haskell constructs (I don't speak or write Haskell), but I think you are wrong in Python. If so, please stop trying to write Haskell in Python; you'll be as happy as the friend of mine I've finally persuaded to stop writing Fortran in Python, I promise! -- Rhodri James *-* Wildebeest Herder to the Masses -- https://mail.python.org/mailman/listinfo/python-list
Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary
On Sun, 23 Mar 2014 02:09:20 +1100, Chris Angelico wrote:
> On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano
> wrote:
>> Line endings are terminators: they end the line. Whether you consider
>> the terminator part of the line or not is a matter of opinion (is the
>> cover of a book part of the book?) but consider this:
>>
>> If you say that the end of lines are *not* part of the line, then
>> that implies that some parts of the file are not inside any line at
>> all. And that would be just weird.
>
> Not so weird IMO. A file is not a concatenation of lines; it is a stream
> of bytes.
But a *text file* is a concatenation of lines. The "text file" model is
important enough that nearly all programming languages offer a line-based
interface to files, and some (Python at least, possibly others) make it
the default interface so that iterating over the file gives you lines
rather than bytes -- even in "binary" mode.
> Now, if you ask Python to read you 512 bytes from a binary
> file, and then ask for another 512 bytes, and so on until you reach the
> end, then it would indeed be VERY weird if there were parts of the file
> that weren't in the returned (byte) strings. But if you ask for a line,
> and then another line, and another line, then it's quite reasonable to
> interpret U+000A as "line separation" rather than "line termination",
> and not return it. (Both interpretations make sense. I just wish the
> most obvious form of iteration gave the cleaner/tidier version, or at
> very least that there be some really obvious way to ask for
> lines-without-endings.)
There is: call strip('\n') on the line after reading it. Perl and Ruby
spell it chomp(). Other languages may spell it differently. I don't know
of any language that automatically strips newlines, probably because you
can easily strip the newline from the line, but if the language did it
for you, you cannot reliably reverse it.
> Imagine the output of GNU find as a series of
> records. You can ask for those to be separated by newlines (the default,
> or -print), or by NULs (with the -print0 command). In either case, the
> records do not *contain* that value, they're separated by it; the
> records consist of file names.
I have no problem with that: when interpreting text as a record with
delimiters, e.g. from a CSV file, you normally exclude the delimiter.
Sometimes the line terminator does double-duty as a record delimiter as
well.
Reading from a file is considered a low-level operation. Reading
individual bytes in binary mode is the lowest level; reading lines in
text mode is the next level, built on top of the lower binary mode. You
build higher protocols on top of one or the other of that mode, e.g.
"read a zip file" would be built on top of binary mode, "read a csv file"
would be built on top of text mode.
As a low-level protocol, you ought to be able to copy a file without
changing it by reading it in then writing it out:
for blob in infile:
outfile.write(blob)
ought to work whether you are in text mode or binary mode, so long as the
infile and outfile are opened in the same mode. If Python were to strip
newlines, that would no longer be the case.
(Even high-level protocols should avoid unnecessary modifications to
files. One of the more annoying, if not crippling, limitations to the
configparser module is that reading an INI file in, then writing it out
again destroys the high-level structure of the file: comments and blank
lines are stripped, and records may be re-ordered.)
--
Steven D'Aprano
http://import-that.dreamwidth.org/
--
https://mail.python.org/mailman/listinfo/python-list
Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)
On Sun, Mar 23, 2014 at 12:07 PM, Steven D'Aprano
wrote:
> On Sun, 23 Mar 2014 02:09:20 +1100, Chris Angelico wrote:
>
>> On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano
>> wrote:
>>> Line endings are terminators: they end the line. Whether you consider
>>> the terminator part of the line or not is a matter of opinion (is the
>>> cover of a book part of the book?) but consider this:
>>>
>>> If you say that the end of lines are *not* part of the line, then
>>> that implies that some parts of the file are not inside any line at
>>> all. And that would be just weird.
>>
>> Not so weird IMO. A file is not a concatenation of lines; it is a stream
>> of bytes.
>
> But a *text file* is a concatenation of lines. The "text file" model is
> important enough that nearly all programming languages offer a line-based
> interface to files, and some (Python at least, possibly others) make it
> the default interface so that iterating over the file gives you lines
> rather than bytes -- even in "binary" mode.
And lines are delimited entities. A text file is a sequence of lines,
separated by certain characters.
>> (Both interpretations make sense. I just wish the
>> most obvious form of iteration gave the cleaner/tidier version, or at
>> very least that there be some really obvious way to ask for
>> lines-without-endings.)
>
> There is: call strip('\n') on the line after reading it. Perl and Ruby
> spell it chomp(). Other languages may spell it differently. I don't know
> of any language that automatically strips newlines, probably because you
> can easily strip the newline from the line, but if the language did it
> for you, you cannot reliably reverse it.
That's not a tidy way to iterate, that's a way to iterate and then do
stuff. Compare:
for line in f:
# process line with newline
for line in f:
line = line.strip("\n")
# process line without newline, as long as it doesn't have \r\n or something
for line in f:
line = line.split("$")
# process line as a series of dollar-delimited fields
The second one is more like the third than the first. Python does not
offer a tidy way to do the common thing, which is reading the content
of the line without its terminator.
>> Imagine the output of GNU find as a series of
>> records. You can ask for those to be separated by newlines (the default,
>> or -print), or by NULs (with the -print0 command). In either case, the
>> records do not *contain* that value, they're separated by it; the
>> records consist of file names.
>
> I have no problem with that: when interpreting text as a record with
> delimiters, e.g. from a CSV file, you normally exclude the delimiter.
> Sometimes the line terminator does double-duty as a record delimiter as
> well.
So why is the delimiter excluded when you treat the file as CSV, but
included when you treat the file as lines of text?
> Reading from a file is considered a low-level operation. Reading
> individual bytes in binary mode is the lowest level; reading lines in
> text mode is the next level, built on top of the lower binary mode. You
> build higher protocols on top of one or the other of that mode, e.g.
> "read a zip file" would be built on top of binary mode, "read a csv file"
> would be built on top of text mode.
I agree that reading a binary file is the lowest level. Reading a text
file is higher level, but to me "reading a text file" means "reading a
binary file and decoding it into Unicode text", and not "... and
dividing it into lines". Bear in mind that reading a CSV file can be
built on top of a Unicode decode, but not on a line-based iteration
(in case there are newlines inside quotes).
> As a low-level protocol, you ought to be able to copy a file without
> changing it by reading it in then writing it out:
>
> for blob in infile:
> outfile.write(blob)
>
>
> ought to work whether you are in text mode or binary mode, so long as the
> infile and outfile are opened in the same mode. If Python were to strip
> newlines, that would no longer be the case.
All you need is a "writeln" method that re-adds the newline, and then
it's correctly round-tripping, based on what you've already stated
about the file: that it's a series of lines of text. It might not be a
byte-equivalent round-trip if you're changing newline style, any more
than it already won't be for other reasons (file encoding, for
instance). By reading the file as a series of Unicode lines, you're
declaring that it contains lines of Unicode text, not arbitrary bytes,
and so a valid representation of those lines of Unicode text is a
faithful reproduction of the file. If you want a byte-for-byte
identical file, open it in binary mode to do the copy; that's what we
learn from FTPing files between Linux and Windows.
> (Even high-level protocols should avoid unnecessary modifications to
> files. One of the more annoying, if not crippling, limitations to the
> configparser module is that reading an INI file in, then writing it out
> again destroy
Re: Question about Source Control
Albert-Jan Roskam Wrote in message: > In addition to posting in html format, you have also set the font size too small for me to easily read. Reason number 12 for posting in text mode in a text newsgroup. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re: Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)
On 23Mar2014 12:37, Chris Angelico wrote: > On Sun, Mar 23, 2014 at 12:07 PM, Steven D'Aprano > wrote: > > On Sun, 23 Mar 2014 02:09:20 +1100, Chris Angelico wrote: > >> On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano > >> wrote: > >>> Line endings are terminators: they end the line. Whether you consider > >>> the terminator part of the line or not is a matter of opinion (is the > >>> cover of a book part of the book?) but consider this: > >>> > >>> If you say that the end of lines are *not* part of the line, then > >>> that implies that some parts of the file are not inside any line at > >>> all. And that would be just weird. > >> > >> Not so weird IMO. A file is not a concatenation of lines; it is a stream > >> of bytes. > > > > But a *text file* is a concatenation of lines. The "text file" model is > > important enough that nearly all programming languages offer a line-based > > interface to files, and some (Python at least, possibly others) make it > > the default interface so that iterating over the file gives you lines > > rather than bytes -- even in "binary" mode. > > And lines are delimited entities. A text file is a sequence of lines, > separated by certain characters. [...snip...] As far as I'm concerned, a text file is a sequence lines, each of which is _terminated_ by a newline (or the OS end-of-line flavour). So I say "terminated by", not "separated by". Plenty of people use editors that consider end-of-line to be a separator and not a terminator, leading to supposed text files lacking trailing newlines (or end-of-line of OS). I consider this sloppy and error prone. I like to be able to read a file and if it lacks a final newline then I have a good clue that the file was incompletely written. Editors (and other tools) that won't enforce a trailing newline as omitting an easy way to give a fairly robust indication of completion at no benefit to the user. (Not to mention the visual annoyance of "cat file" when there's no trailing newline.) So I'm happy to write code that errors if a line lacks a trailing newline, and thus I consider the newline to be an intergral part of the line. Having passed that sanity check, for most machine readable text formats I'm usually happy to use: line = line.rstrip() to get the salient part of the line. (Of course, lines extended with slosh-extension or the like need pickier handling.) Cheers, -- Cameron Simpson If at first you don't succeed, your sky-diving days are over. - Paul Blumstein, pa...@harley.tti.com, DoD #36 -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On Sat, Mar 22, 2014 at 6:32 PM, Rhodri James wrote: > On Sat, 22 Mar 2014 05:26:26 -, Rustom Mody > wrote: > >> Well almost... >> Except that the 'loop' I am talking of is one of >> def loop(): >> return [yield (lambda: x) for x in [1,2,3]] >> or >> return (yield (lambda: x) for x in [1,2,3]) >> or just plain ol >> (lambda x: for x in [1,2,3]) >> IOW loop is an imperative construct, comprehensions are declarative > > > I'm sorry, you've made a logical leap too far here. I understand loops > being imperative, but how are comprehensions declarative? What do they > declare that the loop equivalent doesn't. I'm with Rustom on this point. A list comprehension is a syntax for building a list by declaring a transformation from some other iterable object. Forget comprehensions for a moment and think of literals. Would you not consider this to be declarative? x = [1, 2, 3] A comprehension is syntactically similar to a literal, with just a different type of construction in mind. Where I disagree is on the question of whether Python should therefore break its established closure rules for lambdas that are nested inside comprehensions versus functions that are not. It breaks the equivalence between comprehensions and loops, and to my mind it introduces significant complexity for relatively little gain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On Sunday, March 23, 2014 8:16:28 AM UTC+5:30, Ian wrote: > On Sat, Mar 22, 2014 at 6:32 PM, Rhodri James wrote: > > wrote: > >> Well almost... > >> Except that the 'loop' I am talking of is one of > >> def loop(): > >> return [yield (lambda: x) for x in [1,2,3]] > >> or > >> return (yield (lambda: x) for x in [1,2,3]) > >> or just plain ol > >> (lambda x: for x in [1,2,3]) > >> IOW loop is an imperative construct, comprehensions are declarative > > I'm sorry, you've made a logical leap too far here. I understand loops > > being imperative, but how are comprehensions declarative? What do they > > declare that the loop equivalent doesn't. > I'm with Rustom on this point. A list comprehension is a syntax for > building a list by declaring a transformation from some other iterable > object. Forget comprehensions for a moment and think of literals. > Would you not consider this to be declarative? > x = [1, 2, 3] > A comprehension is syntactically similar to a literal, with just a > different type of construction in mind. Aha! Very elegantly put! > Where I disagree is on the question of whether Python should therefore > break its established closure rules for lambdas that are nested inside > comprehensions versus functions that are not. No... see below > It breaks the > equivalence between comprehensions and loops, and to my mind it > introduces significant complexity for relatively little gain. [I am not completely sure whether the following can be proved/is true] 1. One can change lambda's closure rules which would amount to "significant complexity for relatively little gain" 2. One can change comprehension rules to not reassign to the running comprehension running varible but to rebind, using a recursive function as the simulation of the comprehension rather than a for loop 3. 2 is semantically equivalent to 1 - trivially for normal (ie non-lambda containing) expressions - and also for lambda containing expressions if your default-argument trick is implemented by the python compiler [This is the claim I am not completely sure of and would love to hear of/if counter examples] Assuming its true: One *semantically specifies* a comprehension with a recursive function, not a for loop One *implements* a comprehension with the standard use of append method inside a for as the expansion of a comprehension with the extra caveat that interior lambdas are automatically wrapped inside a default-argument binding for all outer comprehension variables. A vanilla python programmer need not know anything about this any more than a vanilla C programmer knows about - strength reduction - code hoisting - loop unrolling etc that goes on inside an optimizing C compiler -- https://mail.python.org/mailman/listinfo/python-list
Re: Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)
On Sat, Mar 22, 2014 at 9:16 PM, Rustom Mody wrote: > [I am not completely sure whether the following can be proved/is true] > > 1. One can change lambda's closure rules which would amount to > "significant complexity for relatively little gain" > > 2. One can change comprehension rules to not reassign to the > running comprehension running varible but to rebind, using a recursive > function as the simulation of the comprehension rather than a for loop > > 3. 2 is semantically equivalent to 1 Well, if you accept that 1) amounts to significant complexity for relatively little gain, and if 2) is semantically equivalent to 1), then it follows that 2) also amounts to significant complexity for relatively little gain. My statement was in regard to the complexity of the language, not the implementation of the language. >- trivially for normal (ie non-lambda containing) expressions >- and also for lambda containing expressions if your > default-argument trick is implemented by the python compiler > [This is the claim I am not completely sure of and would love to hear > of/if counter examples] The disadvantage of the default argument trick is that it does modify the function's signature, by adding an extra argument with a default, so they're not entirely equivalent. > Assuming its true: > > One *semantically specifies* a comprehension with a recursive function, > not a for loop The problem with this is a cognitive one. The comprehension *looks like* a for loop, not a recursive function. It is natural to reason about it as if it were a for loop, not a recursive function. This is that added complexity I was talking about. > A vanilla python programmer need not know anything about this any > more than a vanilla C programmer knows about > - strength reduction > - code hoisting > - loop unrolling > etc > > that goes on inside an optimizing C compiler Until they go to unroll their comprehension into a for loop because they need to add some imperative construct to it, and their code breaks as a result. From there you'll get the programmers who will add functions with side effects to their comprehensions because they want the comprehension binding behavior while still performing imperative tasks within the loop. -- https://mail.python.org/mailman/listinfo/python-list
