Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Gregory Ewing : > Marko Rauhamaa wrote: >> How about "return"? > > How about "goto"? :-) > > That's not entirely an unserious suggestion -- if you > really believe that a "goto with arguments" is a good > feature for a language to have, you shouldn't be > afraid to spell it as such. > > def quicksort(array, start, end): > midp = partition(array, start, end) > if midp <= (start+end)//2: > quicksort(array, start, midp) > goto quicksort(array, midp+1, end) > else: > quicksort(array, midp+1, end) > goto quicksort(array, start, midp) This works already now: def quicksort(array, start, end): midp = partition(array, start, end) if midp <= (start+end)//2: quicksort(array, start, midp) return quicksort(array, midp+1, end) else: quicksort(array, midp+1, end) return quicksort(array, start, midp) It might even be tail-call optimized by Python. Only you can't count on it because the language spec doesn't guarantee it. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 07/14/2015 05:20 AM, Steven D'Aprano wrote: > On Tue, 14 Jul 2015 12:05 am, Chris Angelico wrote: > >> If you want to make an assertion that iterative >> code requires equivalent warping to tail-recursive code, I want to see >> an example of it. Is that difficult? > Of course it is. If it wasn't difficult, people would post examples instead > of getting all defensive about being asked for examples. Shrug! I have seen this game played before. It has been played since before python had an if-expression. It always goes as follows. Some one mentions a feature python doesn't has and that could be useful. The python status-quo people will argue that it is unnecessary. Any example that will be given to illustrate the advantage will be nit picked appart for any possible flaw, while mostly ignoring the flaws in the status quo. Before python had an if-expression, examples were asked too and they convinced noone. It wasn't until a core developer was bitten by an elusive bug, caused by using the then advised "and or" combination, that python got an if-expression and that an if-expression was considered useful by the python community in general. I expect any example given, to be used as a contest by those reading, for finding the least warped alternative and then using that to dismiss the example. So I really don't feel compelled to search through my code in the hope of finding an example that would persuade someone. -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
Can't install/uninstall Python
Hello. I have a problem installing/uninstalling python. I’m trying to reinstall Python 2.7.9/2.7.10, but during the installation, it didn’t let me, which gave me this: "There is a problem with this Windows Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor." (The message wanted me to contact you for this.) Is there any way to fix this problem?-- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 14/07/2015 07:29, Gregory Ewing wrote: Marko Rauhamaa wrote: Ian Kelly : Another point in favor of an explicit tail-call keyword. Then one couldn't make that mistake. How about "return"? How about "goto"? :-) Why not "goto"? It's use is scattered throughout the cpython code, and to my knowledge there's no paper that says *NEVER* use it. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On 14/07/2015 03:16, Michael Torrie wrote: On 07/13/2015 08:42 AM, Grant Edwards wrote: If it didn't have to run on Windows, I'd pick pygtk over wx. I've never tried qt. PyQt is very nice to work with. In some respects it's not as Pythonic as PyGTK. It feels a lot like transliterated C++ code, which it is. But it's a powerful toolkit and looks great on all supported platforms. If the licensing terms of PyQt are not to your liking, PySide is fairly close to PyQt (a few quirks that can be worked around), though I'm not sure how much love it's receiving lately. Like wx, or Gtk, you would have to ship some extra dlls with your project for Windows and OS X. Looks good for PySide http://lists.qt-project.org/pipermail/pyside/2015-July/002313.html :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa wrote: > Ian Kelly : > >> On Mon, Jul 13, 2015 at 11:25 PM, Chris Angelico wrote: >>> (Also, side point: Python can't actually optimize the above function, >>> because it actually means "call quicksort, then discard its return >>> value and return None". A true tail call has to return the result of >>> the recursive call, and Python lacks a way to say "this function will >>> always return None". But that's trivial.) >> >> Another point in favor of an explicit tail-call keyword. Then one >> couldn't make that mistake. > > How about "return"? I think you miss my point entirely. "return" doesn't mean tail-call optimize; it just means to return the result. This is what led to the confusion responsible for the bug that Chris pointed out in the first place. With a keyword that explicitly means "perform tail-call optimization *and* return", the association of the keyword with the optimization is much clearer, and the programmer is much less likely to mistakenly omit it. -- https://mail.python.org/mailman/listinfo/python-list
Re: str.index() and str.find() versus only list.index()
On Tue, Jul 14, 2015 at 7:13 AM, Chris Angelico wrote: > On Tue, Jul 14, 2015 at 2:58 PM, Steven D'Aprano > wrote: > > On Tuesday 14 July 2015 14:07, Ian Kelly wrote: > > > >> On Mon, Jul 13, 2015 at 9:23 PM, Steven D'Aprano > >> wrote: > >>> Correct. But rather than removing it, it would be better to take a leaf > >>> out of re.match's book and return None as the sentinel. That would > >>> eliminate the "using -1 as a valid index" bug. > >> > >> Better IMO to just have the one non-redundant method that raises an > >> exception rather than returning anything that could possibly be > >> interpreted as a string index. > > > > > > Well, maybe, but if you got rid of str.find, the first thing people > would do > > is recreate it: > > > > def find(*args): > > try: > > return str.index(*args) > > except ValueError: > > return -1 > > > > > > Having a version of str.index that returns a sentinel is just too damn > > handy. > > Same as dictionaries have [] and .get(), although find doesn't allow > you to change the sentinel. > > Maybe that is the solution? Add a keyword-only argument to find to change the sentinel? -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Tue, Jul 14, 2015 at 1:26 AM, Antoon Pardon wrote: > On 07/14/2015 05:20 AM, Steven D'Aprano wrote: >> On Tue, 14 Jul 2015 12:05 am, Chris Angelico wrote: >> >>> If you want to make an assertion that iterative >>> code requires equivalent warping to tail-recursive code, I want to see >>> an example of it. Is that difficult? >> Of course it is. If it wasn't difficult, people would post examples instead >> of getting all defensive about being asked for examples. > > Shrug! I have seen this game played before. It has been played since before > python had an if-expression. It always goes as follows. Some one mentions > a feature python doesn't has and that could be useful. The python status-quo > people will argue that it is unnecessary. Any example that will be given > to illustrate the advantage will be nit picked appart for any possible flaw, > while mostly ignoring the flaws in the status quo. > > Before python had an if-expression, examples were asked too and they > convinced > noone. It wasn't until a core developer was bitten by an elusive bug, caused > by using the then advised "and or" combination, that python got an > if-expression > and that an if-expression was considered useful by the python community > in general. > > I expect any example given, to be used as a contest by those reading, > for finding > the least warped alternative and then using that to dismiss the example. > > So I really don't feel compelled to search through my code in the hope > of finding > an example that would persuade someone. And yet, Python somehow manages to gain new features with each release. The reason why most proposals get rejected is because most proposals are bad. If every idea that came along just got accepted, we'd have a disastrous hodge-podge of a language. In the case of TCO, Guido rejected the feature six years ago and hasn't shown any sign of changing his mind. He considered the arguments for it, and he decided that for Python, the benefits don't outweigh the drawbacks. Maybe his mind could still be changed, but it's going to take somebody to make a very convincing case for the feature, and if you're not even willing to put forth some examples then you don't have a case at all. -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Ian Kelly : > On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa wrote: >> How about "return"? > > I think you miss my point entirely. "return" doesn't mean tail-call > optimize; Why not? > it just means to return the result. Same difference. > This is what led to the confusion responsible for the bug that Chris > pointed out in the first place. With a keyword that explicitly means > "perform tail-call optimization *and* return", That could well be the explicit definition of the "return" statement in Python without changing the behavior of any working Python program today. > the association of the keyword with the optimization is much clearer, > and the programmer is much less likely to mistakenly omit it. The programmer shouldn't be controlling tail call optimizations. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 07/14/2015 10:34 AM, Ian Kelly wrote: > On Tue, Jul 14, 2015 at 1:26 AM, Antoon Pardon > wrote: >> I expect any example given, to be used as a contest by those reading, >> for finding >> the least warped alternative and then using that to dismiss the example. >> >> So I really don't feel compelled to search through my code in the hope >> of finding >> an example that would persuade someone. > And yet, Python somehow manages to gain new features with each release. Sure, but what happens in this list, has an insignificant role in driving that process. > The reason why most proposals get rejected is because most proposals > are bad. If every idea that came along just got accepted, we'd have a > disastrous hodge-podge of a language. But I'm not proposing TCO to be accepted. I was just pointing out a reason for preferring it in particular circumstances and only because some people here suggested there could be no such reasons. > In the case of TCO, Guido rejected the feature six years ago and > hasn't shown any sign of changing his mind. He considered the > arguments for it, and he decided that for Python, the benefits don't > outweigh the drawbacks. Maybe his mind could still be changed, but > it's going to take somebody to make a very convincing case for the > feature, and if you're not even willing to put forth some examples > then you don't have a case at all. Indeed I am not trying to make a case. I was just protesting the sugestion that there are no circumstances in which having TCO could be an advantage. That me suggesting such circumstances do exist gets turned into me trying to make a case for including TCO into the language is one important reason why I am not inclined to search for examples. Because all I would try to do is illustrate how TCO would be an advantage in that case en how it would probably be received is as if I would try and present that example as a compelling case for introducing TCO into python. For what it is worth, in general I find both the warping necessary for turning iterative code into tail-recursive code and vice versa not that big a deal. I just find that the solutions for my problems are sometimes more easily expressed one way and sometimes the other way and I would prefer I didn't need to warp. -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Tue, Jul 14, 2015 at 3:41 PM, Marko Rauhamaa wrote: > Tail-call optimization has nothing to do with converting algorithms into > iterations. It's a prosaic trick of dropping an unneeded stack frame > before making a function call. > >> The claim that TCO means you don't need stack space for all those >> levels of recursion doesn't work if you still need stack space for all >> those levels of recursion *before* you get to the tail call. > > Nobody is making that claim. Actually, that claim was made - that Python's stack would overflow if you didn't optimize tail calls away. I don't feel like digging up through the history to find out who first made the claim, but it was made in this thread. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Tue, Jul 14, 2015 at 3:41 PM, Paul Rubin wrote:
> Chris Angelico writes:
>> That's a prime example of recursion... but not of recursion that can
>> be tail-call optimized into iteration. It's an example of forking
>> recursion, where one call can result in multiple calls (same with tree
>> traversal); it calls itself to sort the first part and the last part,
>> and while you might be able to turn the second call into a goto, you
>> still need stack space to maintain the first call. The claim that TCO
>> means you don't need stack space for all those levels of recursion
>> doesn't work if you still need stack space for all those levels of
>> recursion *before* you get to the tail call.
>
> You partition the array into two parts, one of which has at most N/2
> elements. You push that smaller one onto the stack and recurse, so at
> the next recursive level you push at most an N/4 part, then N/8 and so
> on. In that way the total recursion depth is O(log N). If N=1e6 you
> enter 20 levels of recursion which is no big deal.
>
> If you also push the larger part on the stack, it can have size as high
> as N-1, so you can end up pushing N-1, N-2, N-3, etc. If N=1e6 you
> overflow the stack when you've barely gotten started.
Ah, good point. So this is a way of forcing the recursion to be capped
at log N, preventing the worst-case time from also carrying worst-case
stack usage. Fair enough. Okay! Great! We have one example where TCO
can be helpful, because it means your two calls look identical. Well,
nearly identical... since one of them has to be "return
quicksort(...)" but the other has to be just "quicksort(...)". So in
order to make sure it's a true optimizable tail call, they end up
having to look different. Plus there's the whole traceback problem.
I'm still interested in the explicit "replace current stack frame with
this call" operation. Calling it "goto" seems wrong, as most languages
with goto restrict it to _within_ a function, whereas this would be
_across_ functions:
int fail_and_bail_demo(int arg, char *whatever)
{
int ret = 1; /* failure */
if (arg < 0) goto failure;
int stuff_done = do_stuff(whatever);
if (stuff_done < arg) goto failure;
if (do_more_stuff(arg, whatever)) goto failure;
ret = 0; /* If we get here, success! */
failure:
clean_up();
return ret;
}
Contrast this proposal:
def func():
pass
def setup_and_call():
setup()
transfer func, (), {}
The transfer of control has to be to the beginning of some other
function. So I'd rather it _not_ be called goto, but rather something
more evoking the notion of "hand control over to that function over
there, pretending that I've already returned". It's very much like the
Unix exec* family of functions, but Python already uses the name
'exec' in a very different way.
Done explicitly like this, it avoids the problem of bad tracebacks,
because by definition you would use it only when you want the current
stack frame to be dropped. I wonder how hard it would be to hack this
into CPython... Anyone feel like tackling it?
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Can't install/uninstall Python
On Tue, Jul 14, 2015 at 7:20 AM, Thomas Via wrote: > I’m trying to reinstall Python 2.7.9/2.7.10, but during the installation, it > didn’t let me, which gave me this: > > "There is a problem with this Windows Installer package. A program required > for this install to complete could not be run. Contact your support > personnel or package vendor." Hi! What version of Windows are you running? And what architecture (32-bit or 64-bit)? Exactly what file did you download, and from where? The more information you can provide, the easier it will be to figure out what's going on. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: python 3.5.0b3 (32-bit) exe setup
On Tue, Jul 14, 2015 at 8:15 AM, Colin Dick wrote: > when I run the downloaded version of this pgm "python-3.5.0b3.exe" > > there are no visible selection box's except "cancel" on the > first screen shown > > see the attached screen dump > > its running on a win xp sp3 as a vmware vm > As of Python 3.5, Windows XP is no longer a supported platform. Since there has also been a change of compiler for 3.5, it wouldn't surprise me if there are odd problems where something just doesn't work. Can you reproduce the problem on Win 7 or newer? If it's just an XP problem, I suspect the devs won't be very much interested in fixing it (although if _you_ fix it, they may be willing to incorporate the change, assuming it doesn't break anything else). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Chris Angelico : > On Tue, Jul 14, 2015 at 3:41 PM, Marko Rauhamaa wrote: >> Tail-call optimization has nothing to do with converting algorithms into >> iterations. It's a prosaic trick of dropping an unneeded stack frame >> before making a function call. >> >>> The claim that TCO means you don't need stack space for all those >>> levels of recursion doesn't work if you still need stack space for all >>> those levels of recursion *before* you get to the tail call. >> >> Nobody is making that claim. > > Actually, that claim was made - that Python's stack would overflow if > you didn't optimize tail calls away. I don't feel like digging up > through the history to find out who first made the claim, but it was > made in this thread. Nobody is making the claim that optimizing tail calls *always* saves you from stack overflows. Optimizing tail calls *sometimes* saves you from stack overflows. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Chris Angelico : > Done explicitly like this, it avoids the problem of bad tracebacks, > because by definition you would use it only when you want the current > stack frame to be dropped. I wonder how hard it would be to hack this > into CPython... Anyone feel like tackling it? I would rather optimize by default and disable optimizations with --debug or equivalent. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> Done explicitly like this, it avoids the problem of bad tracebacks, >> because by definition you would use it only when you want the current >> stack frame to be dropped. I wonder how hard it would be to hack this >> into CPython... Anyone feel like tackling it? > > I would rather optimize by default and disable optimizations with > --debug or equivalent. That assumes that it's simply an optimization. This is a distinct semantic change. Would you, for instance, want all of your arithmetic to be rounded to integer because integers are faster - and then disable this "optimization" with a flag that permits floating-point arithmetic? Certainly not. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Tue, Jul 14, 2015 at 2:33 AM, Marko Rauhamaa wrote: > Ian Kelly : > >> On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa wrote: >>> How about "return"? >> >> I think you miss my point entirely. "return" doesn't mean tail-call >> optimize; > > Why not? > >> it just means to return the result. > > Same difference. > >> This is what led to the confusion responsible for the bug that Chris >> pointed out in the first place. With a keyword that explicitly means >> "perform tail-call optimization *and* return", > > That could well be the explicit definition of the "return" statement in > Python without changing the behavior of any working Python program > today. To the compiler. It won't be viewed that way be the programmer, however. So they'll forget that the "return" is necessary to the optimization and just write quicksort() instead of return quicksort(). >> the association of the keyword with the optimization is much clearer, >> and the programmer is much less likely to mistakenly omit it. > > The programmer shouldn't be controlling tail call optimizations. The programmer controls it regardless. return foo() # TCO result = foo() # No TCO return result # No TCO -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Chris Angelico : > On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: >> I would rather optimize by default and disable optimizations with >> --debug or equivalent. > > That assumes that it's simply an optimization. This is a distinct > semantic change. No, tail call optimization doesn't change the behavior of the program, for the worse anyway. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: >>> I would rather optimize by default and disable optimizations with >>> --debug or equivalent. >> >> That assumes that it's simply an optimization. This is a distinct >> semantic change. > > No, tail call optimization doesn't change the behavior of the program, > for the worse anyway. It does, because you lose traceback records. That's pretty significant when you come to try to debug something. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 07/14/2015 03:43 PM, Chris Angelico wrote: > On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: I would rather optimize by default and disable optimizations with --debug or equivalent. >>> That assumes that it's simply an optimization. This is a distinct >>> semantic change. >> No, tail call optimization doesn't change the behavior of the program, >> for the worse anyway. > It does, because you lose traceback records. That's pretty significant > when you come to try to debug something. I doubt it would be really significant. Not compared to writing it iteratively. When you write it iteratively, you don't get to keep a traceback record per time you go throught the loop. So the traceback records you loose in tale call elimination would be the traceback records you wouldn't have anyway in the iterative version. So how would this be significant? -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 2015-07-14, Ian Kelly wrote: > And yet, Python somehow manages to gain new features with each release. > > The reason why most proposals get rejected is because most proposals > are bad. If every idea that came along just got accepted, we'd have a > disastrous hodge-podge of a language. And PHP already has that niche nailed down. -- Grant Edwards grant.b.edwardsYow! World War III? at No thanks! gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Wed, Jul 15, 2015 at 12:02 AM, Antoon Pardon wrote: > On 07/14/2015 03:43 PM, Chris Angelico wrote: >> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: >>> Chris Angelico : >>> On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: > I would rather optimize by default and disable optimizations with > --debug or equivalent. That assumes that it's simply an optimization. This is a distinct semantic change. >>> No, tail call optimization doesn't change the behavior of the program, >>> for the worse anyway. >> It does, because you lose traceback records. That's pretty significant >> when you come to try to debug something. > > I doubt it would be really significant. Not compared to writing it > iteratively. > When you write it iteratively, you don't get to keep a traceback record per > time > you go throught the loop. So the traceback records you loose in tale call > elimination > would be the traceback records you wouldn't have anyway in the iterative > version. > > So how would this be significant? You're proposing making _every_ instance of "return func(...)" into this kind of thing. That's not always recursion, and it's certainly not always clear what called what to get you there. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On 2015-07-14, Michael Torrie wrote: > On 07/13/2015 08:42 AM, Grant Edwards wrote: >> If it didn't have to run on Windows, I'd pick pygtk over wx. I've >> never tried qt. > > PyQt is very nice to work with. In some respects it's not as Pythonic > as PyGTK. It feels a lot like transliterated C++ code, which it is. > But it's a powerful toolkit and looks great on all supported platforms. > If the licensing terms of PyQt are not to your liking, PySide is fairly > close to PyQt (a few quirks that can be worked around), though I'm not > sure how much love it's receiving lately. Like wx, or Gtk, you would > have to ship some extra dlls with your project for Windows and OS X. Why would you have to ship "extra" libraries for Windows? Extra compared to what? When I compared bundled apps for Windows using wx and Tk, you had to ship more libraries using Tk than you did with wx. Maybe that's changed... -- Grant Edwards grant.b.edwardsYow! On the road, ZIPPY at is a pinhead without a gmail.compurpose, but never without a POINT. -- https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On 07/14/2015 08:06 AM, Grant Edwards wrote: > On 2015-07-14, Michael Torrie wrote: >> On 07/13/2015 08:42 AM, Grant Edwards wrote: >>> If it didn't have to run on Windows, I'd pick pygtk over wx. I've >>> never tried qt. >> >> PyQt is very nice to work with. In some respects it's not as Pythonic >> as PyGTK. It feels a lot like transliterated C++ code, which it is. >> But it's a powerful toolkit and looks great on all supported platforms. >> If the licensing terms of PyQt are not to your liking, PySide is fairly >> close to PyQt (a few quirks that can be worked around), though I'm not >> sure how much love it's receiving lately. Like wx, or Gtk, you would >> have to ship some extra dlls with your project for Windows and OS X. > > Why would you have to ship "extra" libraries for Windows? Extra > compared to what? When I compared bundled apps for Windows using wx > and Tk, you had to ship more libraries using Tk than you did with wx. > Maybe that's changed... You make a good point. Although Tk is considered part of the standard Python library (though optional), Tk not only requires some dlls, it also embeds the tcl language interpreter as well. So by some measures, Tk in Python is pretty heavy, though I have no idea what the size it would add to an application bundle actually is. I've always thought it was a bit crazy how when you use Tk, you're actually using the Tcl language as well, even though you're driving it all from Python. I believe there were attempts to separate Tk from Tcl, and allow Perl/Tk to replace all the Tcl code with Perl code. Qt weighs in at between 5 and 15 MB of dlls, depending on how much of it you are using, and 32 or 64-bit. Gtk weighs in at about 10-15 MB also, and that's over a lot more little files and directories for things like image loaders. -- https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On 14/07/2015 16:21, Michael Torrie wrote: On 07/14/2015 08:06 AM, Grant Edwards wrote: On 2015-07-14, Michael Torrie wrote: On 07/13/2015 08:42 AM, Grant Edwards wrote: If it didn't have to run on Windows, I'd pick pygtk over wx. I've never tried qt. PyQt is very nice to work with. In some respects it's not as Pythonic as PyGTK. It feels a lot like transliterated C++ code, which it is. But it's a powerful toolkit and looks great on all supported platforms. If the licensing terms of PyQt are not to your liking, PySide is fairly close to PyQt (a few quirks that can be worked around), though I'm not sure how much love it's receiving lately. Like wx, or Gtk, you would have to ship some extra dlls with your project for Windows and OS X. Why would you have to ship "extra" libraries for Windows? Extra compared to what? When I compared bundled apps for Windows using wx and Tk, you had to ship more libraries using Tk than you did with wx. Maybe that's changed... You make a good point. Although Tk is considered part of the standard Python library (though optional), Tk not only requires some dlls, it also embeds the tcl language interpreter as well. So by some measures, Tk in Python is pretty heavy, though I have no idea what the size it would add to an application bundle actually is. I've always thought it was a bit crazy how when you use Tk, you're actually using the Tcl language as well, even though you're driving it all from Python. I believe there were attempts to separate Tk from Tcl, and allow Perl/Tk to replace all the Tcl code with Perl code. Surely if Tk is optional then IDLE is also optional, as IDLE depends on Tk? But I thought that IDLE was always supplied with Python, so am I missing something, or am I simply plain wrong, or what? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On Wed, 15 Jul 2015 01:43 am, Mark Lawrence wrote: > Surely if Tk is optional then IDLE is also optional, as IDLE depends on > Tk? But I thought that IDLE was always supplied with Python, so am I > missing something, or am I simply plain wrong, or what? If you try to run IDLE on a system that doesn't have Tk/Tcl installed, you get an exception and IDLE won't run. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Ian Kelly :
> On Tue, Jul 14, 2015 at 2:33 AM, Marko Rauhamaa wrote:
>> The programmer shouldn't be controlling tail call optimizations.
>
> The programmer controls it regardless.
>
> return foo() # TCO
>
> result = foo() # No TCO
> return result # No TCO
Not necessarily. Compile this function with gcc ("gcc -S -O2 atoi.c"):
int atoi(const char *str, int n)
{
if (str && *str)
n = atoi(str + 1, n * 10 + *str - '0');
return n;
}
(Example modified from http://stackoverflow.com/questions/34125/wh
ich-if-any-c-compilers-do-tail-recursion-optimization#220660>.)
You'll see that the generated code is tail-call-optimized.
Marko
--
https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On 2015-07-14, Mark Lawrence wrote: > On 14/07/2015 16:21, Michael Torrie wrote: >>> Why would you have to ship "extra" libraries for Windows? Extra >>> compared to what? When I compared bundled apps for Windows using wx >>> and Tk, you had to ship more libraries using Tk than you did with wx. >>> Maybe that's changed... >> >> You make a good point. Although Tk is considered part of the standard >> Python library (though optional), Tk not only requires some dlls, it >> also embeds the tcl language interpreter as well. So by some measures, >> Tk in Python is pretty heavy, though I have no idea what the size it >> would add to an application bundle actually is. I've always thought it >> was a bit crazy how when you use Tk, you're actually using the Tcl >> language as well, even though you're driving it all from Python. I >> believe there were attempts to separate Tk from Tcl, and allow Perl/Tk >> to replace all the Tcl code with Perl code. > > Surely if Tk is optional It is. > then IDLE is also optional, It is. > as IDLE depends on Tk? But I thought that IDLE was always supplied > with Python, That depends on who supplied the Python. Most/all of the ready-made Windows builds include Tk, but there's no reason you can't build it without Tk not install IDLE. But, you can't assume that all Windows machines have _any_ Python build installed. > so am I missing something, or am I simply plain wrong, or what? My point is that on Windows, _Python_ is optional. If you want to ship a stand-alone .exe file, then you've got to ship both Python andT Tcl/Tk or ship both Python and wx. In my experience (which was a couple years ago), Windows wx libs were smaller than Windows Tcl+Tk libs. wx on Windows uses native widgets. Tcl/Tk doesn't: it includes not only the Tcl language implementation, it also includes low-level pixel-twiddling implementations of all the widgets. Comparing the size of Tcl+Tk and wx/Gtk doesn't really make sense either since we're talking about MS Windows targets. Gtk isn't involved. wxWindows on MS Windows runs on top of native widgets, not on top of Gtk. -- Grant Edwards grant.b.edwardsYow! VICARIOUSLY experience at some reason to LIVE!! gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Chris Angelico : > On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: >> No, tail call optimization doesn't change the behavior of the >> program, for the worse anyway. > > It does, because you lose traceback records. That's pretty significant > when you come to try to debug something. Doesn't count. Optimization can do weird things to code and make debugging challenging. That's why you usually tell the compiler not to optimize the debug builds. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Chris Angelico : > You're proposing making _every_ instance of "return func(...)" into > this kind of thing. Yes. > That's not always recursion, Correct. > and it's certainly not always clear what called what to get you there. Correct. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Tue, 14 Jul 2015 06:33 pm, Marko Rauhamaa wrote:
> Ian Kelly :
>
>> On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa
>> wrote:
>>> How about "return"?
>>
>> I think you miss my point entirely. "return" doesn't mean tail-call
>> optimize;
>
> Why not?
Because then you lose most of your debugging information when an exception
occurs and a stack trace is printed.
>> it just means to return the result.
>
> Same difference.
If "return" was the same as "return and TCO", we wouldn't be having this
discussion, because Python would have had TCO for over 20 years. So the two
must clearly be different.
And I think that's the solution. If we had two key words, let's say "return"
and "treturn" ("tail-return"), where treturn performed TCO when possible,
then the programmer could TCO some functions and not others, according to
whether they feared bugs in the function more or less than hitting the
recursion limit.
That's better than a global setting or switch, and better than a TCO
decorator, since it would even allow you to TCO some paths through a
function but not others.
I'm undecided about what should happen if you use treturn in a non-tail call
situation. Should it simply fall back to regular return? Or should it raise
an exception? (Preferably at compile time, if not possible, at runtime.)
The downside is that this would put responsibility on the programmer to
decide whether to TCO or not. Well, potential downside. Using a TCO
decorator or switch to enable it also puts responsibility on the
programmer, but that's usually considered a good thing.
>> This is what led to the confusion responsible for the bug that Chris
>> pointed out in the first place. With a keyword that explicitly means
>> "perform tail-call optimization *and* return",
>
> That could well be the explicit definition of the "return" statement in
> Python without changing the behavior of any working Python program
> today.
Really? If it doesn't change the behaviour, why bother making the change?
It would change the behaviour of code. Code which currently breaks for
sufficiently large data may no longer break. That's a plus. Code which
currently generates tracebacks -- which may actually be *intended*
behaviour and not a bug, e.g. in test suites -- will change their behaviour
and be significantly less useful. That's a minus.
>> the association of the keyword with the optimization is much clearer,
>> and the programmer is much less likely to mistakenly omit it.
>
> The programmer shouldn't be controlling tail call optimizations.
Why not? The programmer should control ALL optimizations that have a
semantic effect. I think constant folding is okay[1], everything else is
dubious and should be under the control of the programmer, not the compiler
writer.
I find it rather unnerving to think that I've written code to do something
in a certain way, and somebody else (the compiler writer) decides to
silently change that to something which may or may not have the same
effect. The *intent* is that it should have the same effect, but in
practice that's often not the case. Compiler optimizations often break
floating point algorithms (e.g. inappropriately deciding that x + y - x
must equal y) or introduce security bugs.
Here's an example where an overzealous but standards-conforming optimization
introduced a security bug:
http://code.google.com/p/nativeclient/issues/detail?id=245
According to the compiler writer, the code before and after the optimization
had precisely the same meaning. According to the developers. who should
know because they wrote it, the optimized code was missing a rather large
function: sanitising untrusted data.
C, in my opinion, is the poster child for how a programming language should
NOT be designed, and I don't want Python to go down the same path. John
Regehr writes:
"... there are compilers (like GCC) where integer overflow behaved a certain
way for many years and then at some point the optimizer got just a little
bit smarter and integer overflows suddenly and silently stopped working as
expected. This is perfectly OK as far as the standard goes. While it may be
unfriendly to developers, it would be considered a win by the compiler team
because it will increase benchmark scores."
http://blog.regehr.org/archives/213
So, no, I don't want the compiler making optimizations I can't control.
[1] Only because Python gives us no standard way to control the floating
point rounding mode.
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On Wed, Jul 15, 2015 at 3:28 AM, Grant Edwards wrote: > Comparing the size of Tcl+Tk and wx/Gtk doesn't really make sense > either since we're talking about MS Windows targets. Gtk isn't > involved. wxWindows on MS Windows runs on top of native widgets, not > on top of Gtk. Well, let's look at three straight-forward options: 1) Python program uses tkinter 2) Python program uses wxPython 3) Python program uses PyGTK 4) (optionally) Python program uses PyGObject Then you package your script up with all of its dependencies - Python interpreter, GUI toolkit, and everything that they depend on - and see how big it is. That's the comparison that matters; everything else is implementation detail. I had been under the impression that tkinter + Tcl + Tk + etc etc etc was smaller than wxPython + wxWidgets + etc etc etc, but it's entirely possible I'm flat wrong on that. It doesn't matter how Python is normally shipped, it matters how big it's going to be when you make that single-executable package. But I still think it's better to *not* do that, because you bind your deps to your code release cycle. If you make this kind of package, most people won't know how to unwrap it and get the Python scripts out of it, so the only thing they can do is upgrade to the latest release that you've made. So when Python 2.7.11 comes out, maybe with some crucial security patch that an end user needs, s/he has to wait for you to make a new package that incorporates the latest Python. If you ship just the .py files, you're responsible for your own code and nothing else. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Steven D'Aprano : > C, in my opinion, is the poster child for how a programming language > should NOT be designed, and I don't want Python to go down the same > path. John Regehr writes: > > "... there are compilers (like GCC) where integer overflow behaved a > certain way for many years and then at some point the optimizer got > just a little bit smarter and integer overflows suddenly and silently > stopped working as expected. This is perfectly OK as far as the > standard goes. While it may be unfriendly to developers, it would be > considered a win by the compiler team because it will increase > benchmark scores." The problem there is in the C standard, not the compiler that complies with the standard. I don't like the way integer overflows are explicitly undefined in modern C. Similarly, I don't like the way tail call behavior is undefined in Python. Neither blemish gives me much trouble in practice. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Wed, 15 Jul 2015 03:29 am, Marko Rauhamaa wrote: > Chris Angelico : > >> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa >> wrote: >>> No, tail call optimization doesn't change the behavior of the >>> program, for the worse anyway. >> >> It does, because you lose traceback records. That's pretty significant >> when you come to try to debug something. > > Doesn't count. Says you. > Optimization can do weird things to code and make > debugging challenging. That's why you usually tell the compiler not to > optimize the debug builds. Here's a real scenario for you to chew on: testing. Test suites should be considered an application where the developer is the end user, and the point of the application is to verify that the code passes tests, and if they don't, present the user (i.e. the developer) with sufficient information to debug the problem (i.e. full stack traces). So how do you test your optimized code? If you turn optimizations on globally, you're testing something which may or may not have the same behaviour as the code you wrote. Compiler bugs exist, so it is more important than ever to ensure that the post-optimization code passes the test suite. But if it fails, you no longer get the stack traces you need to debug the problem. But if you turn optimizations off globally, you're not testing the optimized code any longer. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: > I don't like the way integer overflows are explicitly undefined in > modern C. > > Similarly, I don't like the way tail call behavior is undefined in > Python. Where in the Python spec is it stated that tail call behaviour is undefined? The behaviour of the 'return' statement is well defined: it evaluates its expression (or None), *then* removes the top of the call stack and passes control back to the caller: https://docs.python.org/3/reference/simple_stmts.html#the-return-statement This implies that during the evaluation of its expression, the current function's call stack entry is still present. Tail call behaviour is therefore well defined: it is identical to any other expression evaluation, and then the final result is passed back to the caller. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Steven D'Aprano writes: > Here's an example where an overzealous but standards-conforming optimization > introduced a security bug: > http://code.google.com/p/nativeclient/issues/detail?id=245 In that particular example, the refactored code simply looks wrong. -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Wed, 15 Jul 2015 03:43 am, Marko Rauhamaa wrote: > The problem there is in the C standard, not the compiler that complies > with the standard. > > I don't like the way integer overflows are explicitly undefined in > modern C. > > Similarly, I don't like the way tail call behavior is undefined in > Python. Tail call behaviour is not undefined in Python. There is a strong convention (followed by all implementations I know of) to follow the reference implementation's (CPython) behaviour, which is not to optimize tail calls. But even if an implementation chooses to not follow that, it's not *undefined behaviour*. It's just implementation-dependent behaviour. No Python compiler is permitted to format your hard drive because you perform a tail call. > Neither blemish gives me much trouble in practice. Given how even the best, cleverest, and most security conscious C programmers get repeatedly bitten by C undefined behaviour, I'm confident that if you've written any non-trivial C code, it almost certainly has bugs because of undefined behaviour, you've just never noticed them yet. Perhaps the specific version of the compiler you use doesn't optimize that case, or you've never tried it with data that exposes the introduced bugs. "Making the landmine a much much worse place to be is the fact that there is no good way to determine whether a large scale application is free of undefined behavior, and thus not susceptible to breaking in the future. There are many useful tools that can help find some of the bugs, but nothing that gives full confidence that your code won't break in the future." http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_14.html So, if you make it a habit to religiously compile your C applications with all warnings turned on, and you actually read *and fix* them all, AND you scan the applications with Valgrind, the Clang static analyser, and whatever other tools you can find, then you *might* have good reason to be reasonably confident that your code is *mostly* free of C undefined behaviour bugs. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On 2015-07-14, Chris Angelico wrote: > On Wed, Jul 15, 2015 at 3:28 AM, Grant Edwards > wrote: >> Comparing the size of Tcl+Tk and wx/Gtk doesn't really make sense >> either since we're talking about MS Windows targets. Gtk isn't >> involved. wxWindows on MS Windows runs on top of native widgets, not >> on top of Gtk. > > Well, let's look at three straight-forward options: > > 1) Python program uses tkinter > 2) Python program uses wxPython > 3) Python program uses PyGTK > 4) (optionally) Python program uses PyGObject > > Then you package your script up with all of its dependencies - Python > interpreter, GUI toolkit, and everything that they depend on - and see > how big it is. That's the comparison that matters; everything else is > implementation detail. > > I had been under the impression that tkinter + Tcl + Tk + etc etc etc > was smaller than wxPython + wxWidgets + etc etc etc, but it's entirely > possible I'm flat wrong on that. The last time I did exactly that comparison, the tk option was larger. That was 2-3 years ago, and it's quite possible the py2exe configuration I was using for the tk option could have been optimized more to get rid of libraries that py2exe thought were needed but were in fact not needed. Once the download gets below about 25MB, nobody seems to care about the size. > It doesn't matter how Python is normally shipped, it matters how big > it's going to be when you make that single-executable package. Right. And in practice, I find that doesn't actually matter either. Python+ is small compared to a lot of the things people are used to downloading. > But I still think it's better to *not* do that, because you bind your > deps to your code release cycle. If you make this kind of package, > most people won't know how to unwrap it and get the Python scripts out > of it, so the only thing they can do is upgrade to the latest release > that you've made. So when Python 2.7.11 comes out, maybe with some > crucial security patch that an end user needs, s/he has to wait for > you to make a new package that incorporates the latest Python. If you > ship just the .py files, you're responsible for your own code and > nothing else. Maybe your Windows audience is different than mine. Mine absolutely will not tolerate anything other than a single myapp-setup.exe or myapp.msi file. They will not can not install Python, update Python, or update the Python scripts independently of the exe/msi single-blob. If you depend on an external Python installation or make a bundled one visible to the user, you're just digging yourself a hole. They'll screw up the external Python installation somehow or they'll uninstall a bundled but exposed Python installation and then call in complaining the app doesn't work. -- Grant Edwards grant.b.edwardsYow! Jesuit priests are at DATING CAREER DIPLOMATS!! gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Steven D'Aprano : > Tail call behaviour is not undefined in Python. There is a strong > convention (followed by all implementations I know of) to follow the > reference implementation's (CPython) behaviour, which is not to > optimize tail calls. But even if an implementation chooses to not > follow that, it's not *undefined behaviour*. It's just > implementation-dependent behaviour. But in Scheme, tail call elimination is mandatory and thus can be relied on. Unless that promise is made, you can't write code that depends on it. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Wed, Jul 15, 2015 at 4:37 AM, Marko Rauhamaa wrote: > Steven D'Aprano : > >> Tail call behaviour is not undefined in Python. There is a strong >> convention (followed by all implementations I know of) to follow the >> reference implementation's (CPython) behaviour, which is not to >> optimize tail calls. But even if an implementation chooses to not >> follow that, it's not *undefined behaviour*. It's just >> implementation-dependent behaviour. > > But in Scheme, tail call elimination is mandatory and thus can be relied > on. Unless that promise is made, you can't write code that depends on I think I'm beginning to understand. Tail call elimination is necessary to cope with a system of programming that has deliberately excluded certain other options (eg mutables, in pure functional languages), but is not as important in a more fully-featured language. Constant folding would be the same way; it's not absolutely critical, and if Python doesn't support it for some particular expression then it's not going to kill you, but in SPL [1] it would be a vital efficiency improvement. Scheme needs TCE because of the way you have to write code for Scheme to be even capable of executing it; Python doesn't, because you can write your code to not need TCE. [1] https://en.wikipedia.org/wiki/Shakespeare_(programming_language)#Constants_and_Assignment_of_Values ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: python 3.5.0b3 (32-bit) exe setup
On 7/14/2015 7:56 AM, Chris Angelico wrote: On Tue, Jul 14, 2015 at 8:15 AM, Colin Dick wrote: when I run the downloaded version of this pgm "python-3.5.0b3.exe" there are no visible selection box's except "cancel" on the first screen shown see the attached screen dump its running on a win xp sp3 as a vmware vm As of Python 3.5, Windows XP is no longer a supported platform. It is unsupported by Microsoft, and hence by CPython. Since there has also been a change of compiler for 3.5, to the new 2015 Microsoft compiler, which I am very sure does not support XP, as per above. People running XP should install the latest 3.4. it wouldn't surprise me if there are odd problems where something just doesn't work. Can you reproduce the problem on Win 7 or newer? If it's just an XP problem, I suspect the devs won't be very much interested in fixing it (although if _you_ fix it, they may be willing to incorporate the change, assuming it doesn't break anything else). I think not. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Chris Angelico : > Tail call elimination is necessary to cope with a system of > programming that has deliberately excluded certain other options (eg > mutables, in pure functional languages), Tail call elimination is necessary for the functional programming style. While Scheme strongly promotes it, functional programming style crops up in virtually all programming languages nowadays. > but is not as important in a more fully-featured language. It is not all that important in Python for idiomatic reasons, not for some "full-featuredness". > Scheme needs TCE because of the way you have to write code for Scheme > to be even capable of executing it; Python doesn't, because you can > write your code to not need TCE. You can write Sheme perfectly procedurally, but why would you do that when you have Python to satisfy your procedural urges? Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: python 3.5.0b3 (32-bit) exe setup
On Wed, Jul 15, 2015 at 5:09 AM, Terry Reedy wrote: >> it wouldn't surprise >> me if there are odd problems where something just doesn't work. Can >> you reproduce the problem on Win 7 or newer? If it's just an XP >> problem, I suspect the devs won't be very much interested in fixing it >> (although if _you_ fix it, they may be willing to incorporate the >> change, assuming it doesn't break anything else). > > > I think not. Yeah, I wasn't particularly sanguine [1] on that last point. ChrisA [1] Sanguine. Hopeful. Point of interest, it also means bloody, which is a word a lot of British people use in describing Win XP. -- https://mail.python.org/mailman/listinfo/python-list
Re: beginners choice: wx or tk?
On 7/14/2015 11:43 AM, Mark Lawrence wrote: On 14/07/2015 16:21, Michael Torrie wrote: You make a good point. Although Tk is considered part of the standard Python library (though optional), Python-coded tkinter depends on C-coded _tkinter and both are in the stdlib, which means the code is in the CPython hg repository. tcl/tk is an external dependency of _tkinter and is not is the stdlib. There are a few other stdlib modules with such external dependencies. The Windows installers include the external dependencies. AFAIK, Linux distributions do not, but expect the dependencies to be already present or separately downloaded. I believe the situation is mixed on Mac. Ned Deily is working on include the latest tcl/tk with the Mac installer, but this is apparently not trivial. Surely if Tk is optional then IDLE is also optional, as IDLE depends on Tk? Idle is in the stdlib and depends on tkinter (and maybe _tkinter), so it indirectly depends on tk. Turtle and turtledemo and some tools/scripts also depend on tkinter. > But I thought that IDLE was always supplied with Python, It is unless a distributor removes it, perhaps to bundle with tcl/tk, _tkinter, tkinter, turtle, and turtledemo. Importing tkinter imports _tkinter, which fails if it cannot find tcl/tk. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 13/07/2015 23:46, Terry Reedy wrote: Optimizing specific tail calls is tricker. For one thing, calls to a recursion-replacement function, such as recur, add a stack frame that must also be popped. A CPython bytecode manimpuation solution was given years ago as a recipe on the ActiveState Python Cookbook site. MacroPy at pypi.python.org "provides a mechanism for user-defined functions (macros) to perform transformations on the abstract syntax tree(AST) of Python code at module import time". Among many included examples, it has a tco decorator. A direct link https://github.com/lihaoyi/macropy#tail-call-optimization for anybody who is interested. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 15/07/2015 00:40, Mark Lawrence wrote: On 13/07/2015 23:46, Terry Reedy wrote: Optimizing specific tail calls is tricker. For one thing, calls to a recursion-replacement function, such as recur, add a stack frame that must also be popped. A CPython bytecode manimpuation solution was given years ago as a recipe on the ActiveState Python Cookbook site. MacroPy at pypi.python.org "provides a mechanism for user-defined functions (macros) to perform transformations on the abstract syntax tree(AST) of Python code at module import time". Among many included examples, it has a tco decorator. A direct link https://github.com/lihaoyi/macropy#tail-call-optimization for anybody who is interested. And I've just stumbled across this https://github.com/baruchel/tco which was only put up two days ago. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 7/14/2015 1:15 AM, Paul Rubin wrote: def quicksort(array, start, end): midp = partition(array, start, end) if midp <= (start+end)//2: quicksort(array, start, midp) quicksort(array, midp+1, end) else: quicksort(array, midp+1, end) quicksort(array, start, midp) I assume you know how quicksort and its partition step work. The idea is you partition the array around the pivot element (midp is the index of that element), then recursively sort the two partitions, doing the smaller partition as a recursive call and the larger one as a tail call, so that you use O(log n) stack depth instead of potentially O(n). Thank you for the example. It I have ever seen how to minimize the max stack needed for first calls, I have forgotten. First some comment: 1. There is no terminal condition, so the above will loop forever. The body should start with 'if not terminal(start, end):' where terminal is the actual expression. I did not write it because it depends on whether 'end' is the highest index or one past it. 2. There are no tail calls (call followed by return), so a tail-call optimizer will not optimize this. A recur() function might be able to. 3. Mutation is anathema to functional programming, so a functional programmer would never write and version of this, at least not without holding his nose. The tail-like calls in each branch can be avoided with assignment and gobacks. In Python, we go-back with while loops. (IE, 'while' = 'label' + 'if' + 'goto'.) With minimal change to the above, we get (untested) def quicksort(array, start, end): while not terminal(start, end): midp = partition(array, start, end) if midp <= (start+end) // 2: quicksort(array, start, midp) start = midp+1 else: quicksort(array, midp+1, end) end = midp I can understand that someone might prefer to break the symmetry of the paired calls by wrapping the second with recur() (assuming that such a function is sensibly feasible on *all* implementations). In the other hand, I prefer the reduced-noise explicit assignment that highlights the one parameter that gets rebound. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Why pay DICE When TheGongzuo.com is !! FREE !!
Greetings! You been Invited as a Beta User for TheGongzuo.com ( Absolutely Extended Trial). We bring to you TheGongzuo.com, Top notch highly talented IT (Information Technology / Software Industry) skilled Professional Candidates, where you can find more than a resume. For a NO COST TRIAL all you have to do is: * Register & Post only 1 Active Job. * Start interacting with more 100,000 IT professionals and Innovators/Ideators. Before you get started , there are few things you should know, 1. We are getting better :- We will be updating our site with new material daily basis and we welcome your feedback on newly added material. Please send us your feedback by using Feedback option on the site or send us an email @ supp...@thegongzuo.com. 2. We like Criticism :- As a fresh site, We are sure you can find lots to Criticize - remember , though we would prefer constructive criticism, so we can make site better. Please send us your valuable criticism by email @ supp...@thegongzuo.com. Please make us a part of your distribution list, We will start sending resumes to you. If you do not wish to be a part of our mailing list just reply to us with unsubscribe from this list or else it will consider it as your consent to be part of our distribution list. Thanks for trying TheGongzuo.com and please let us know , What you think ! Thanks, TheGongzuo.com Team ! Disclaimer: This message is sent in compliance with the new email bill section 301. Under Bill S.1618 TITLE III passed by the 105th US Congress, this message cannot be considered SPAM as long as we include the way to be removed, Paragraph (a)(c) of S.1618, further transmissions to you by the sender of this email will be stopped if you by send a response of "REMOVE" in the subject line of the email, we will remove you immediately from subscription. -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 7/14/2015 10:02 AM, Antoon Pardon wrote: On 07/14/2015 03:43 PM, Chris Angelico wrote: On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: No, tail call optimization doesn't change the behavior of the program, for the worse anyway. It does, because you lose traceback records. That's pretty significant when you come to try to debug something. I doubt it would be really significant. Not compared to writing it iteratively. When you write it iteratively, you don't get to keep a traceback record per time you go throught the loop. So the traceback records you loose in tale call elimination would be the traceback records you wouldn't have anyway in the iterative version. To repeat: loosing tracebacks is probably fine when the function has a single *recursive* tail call. This are precise the cases when it is simple and perhaps preferable to use a loop. But *recursive* tail calls are only a small fraction of all tail calls. So most of the time, the loss *would* be significant. Consider In moda: def f(a): return modb.g(a-3) In modb: def g(b): return modc.h(b*(b-1)) In modc: def h(c): return 1.0/c from moda import f ... (500 lines later) f(3) and your traceback has been reduced to In modc: line 1 return 1.0/c ZeroDivisionError: ... ??? -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On 7/14/2015 1:52 PM, Chris Angelico wrote: On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: I don't like the way integer overflows are explicitly undefined in modern C. Similarly, I don't like the way tail call behavior is undefined in Python. Where in the Python spec is it stated that tail call behaviour is undefined? The behaviour of the 'return' statement is well defined: it evaluates its expression (or None), *then* removes the top of the call stack and passes control back to the caller: https://docs.python.org/3/reference/simple_stmts.html#the-return-statement I do not see anything there explicitly about call stacks. This implies that during the evaluation of its expression, the current function's call stack entry is still present. Tail call behaviour is therefore well defined: it is identical to any other expression evaluation, and then the final result is passed back to the caller. In the blog post http://neopythonic.blogspot.co.uk/2009/04/tail-recursion-elimination.html that everyone discussing this issue should read, Guido said that if a tail call is not within a try statement, tco would be fine except for the implementation issue of helpful tracebacks, which he cares greatly about. However, as both he and the linked doc say, a tail call within a try: statement is not a tail call, in that more python code within the function may yet be executed. It is not hard to construct an example in which tco would mess up a traceback message that is displayed. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
I think I'm beginning to understand. Tail call elimination is necessary to cope with a system of programming that has deliberately excluded certain other options (eg mutables, in pure functional languages), Tail recursion is the functional way to write a while loop. To put is another way, a while loop is within stackframe recursion. Ditto for for loops, which are specialized while loops. Recursion, first (or car), rest (or cdr), and linked lists (non-mutational stacks with the top at the front) work together as a tightly coordinated system. (rest ll) is equivalent to ll[1:] except that rest does not mutate anything. When Python's ll.pop(0) (or ll.pop() removes and returns (first ll), it does mutate ll. Since ll becomes what was previously (rest ll), there is no need for a separate list.rest method. Python's next(it) is also does the job of both first + rest. As with pop(0), it removes the first item of it, mutates it so it represents the rest of the items in the collection it represents, and then returns the initial item. Again, there is no need for a separate rest(it) funciton. The fundamental computational idea in both systems, beneath the differing syntax and religious wars, is that we can process a collection by processing one item at a time. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Why pay DICE When TheGongzuo.com is !! FREE !!
On Tue, 14 Jul 2015 17:31:31 -0700 (PDT), trentonwesle...@gmail.com wrote: >Greetings! >You been Invited as a Beta User for TheGongzuo.com ( Absolutely Extended >Trial). >We bring to you TheGongzuo.com, Top notch highly talented IT (Information >Technology / Software Industry) skilled Professional Candidates, So what does it actually DO? I'm assuming that it's some kind of enhancement for Python, but why would anyone actually use it? -- Steve Hayes from Tshwane, South Africa Web: http://www.khanya.org.za/stevesig.htm Blog: http://khanya.wordpress.com E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
On Wed, Jul 15, 2015 at 11:01 AM, Terry Reedy wrote: > On 7/14/2015 1:52 PM, Chris Angelico wrote: >> >> On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: >>> >>> I don't like the way integer overflows are explicitly undefined in >>> modern C. >>> >>> Similarly, I don't like the way tail call behavior is undefined in >>> Python. >> >> >> Where in the Python spec is it stated that tail call behaviour is >> undefined? The behaviour of the 'return' statement is well defined: it >> evaluates its expression (or None), *then* removes the top of the call >> stack and passes control back to the caller: >> >> https://docs.python.org/3/reference/simple_stmts.html#the-return-statement > > > I do not see anything there explicitly about call stacks. There isn't anything explicitly about call stacks, but it clearly states that the expression is evaluated, and *then* processing of the return statement occurs. Unlike C, Python doesn't have "undefined behaviour"; there are some things which are "implementation-defined", but that's a different concept. But the spec is reasonably clear by implication, in my opinion; if tail calls are allowed to remove stack entries, that definition will need to be edited. (That's not to say it can't be, of course. But it would be a change.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Marko Rauhamaa wrote: It might even be tail-call optimized by Python. Only you can't count on it because the language spec doesn't guarantee it. The language spec might permit it, but the BDFL has explicitly expressed a dislike for the idea of implicit tail call removal, so it's unlikely to ever happen in CPython. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
xlrd 0.9.4 released!
Hi All, I'm pleased to announce the release of xlrd 0.9.4: http://pypi.python.org/pypi/xlrd/0.9.4 This release includes the following changes: - Automated tests are now run on Python 3.4 - Use ElementTree.iter() if available, not deprecated getiterator() when parsing xlsx files. - Fix #106 : Exception Value: unorderable types: Name() < Name() - Create row generator expression with Sheet.get_rows() - Fix for forward slash file separator and lowercase names within xlsx internals. Thanks to the following for their contributions to this release: - Corey Farwell - Jonathan Kamens - Deepak N - Brandon R. Stoner - John McNamara If you find any problems, please ask about them on the python-ex...@googlegroups.com list, or submit an issue on GitHub: https://github.com/python-excel/xlrd/issues Full details of all things Python and Excel related can be found here: http://www.python-excel.org/ cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk -- https://mail.python.org/mailman/listinfo/python-list
Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
Gregory Ewing : > Marko Rauhamaa wrote: >> It might even be tail-call optimized by Python. Only you can't count >> on it because the language spec doesn't guarantee it. > > The language spec might permit it, but the BDFL has explicitly > expressed a dislike for the idea of implicit tail call removal, so > it's unlikely to ever happen in CPython. Permitting wouldn't be enough. The other problem for tail call elimination is the requirement that functions return None by default. Smooth tail call elimination would require that Python leave the default return value unspecified. Marko -- https://mail.python.org/mailman/listinfo/python-list
