Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-18 Thread Gregory Ewing
Antoon Pardon wrote: It seems a bit strange that with the immense value that is given to stack frames, that these wouldn't be available somehow. There was discussion recently about the idea of providing access to the frame of a suspended generator, so that debuggers can inspect the stack of an

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Terry Reedy
On 7/17/2015 7:43 AM, Antoon Pardon wrote: On 07/17/2015 01:05 PM, Chris Angelico wrote: def gen(): yield stuff yield more stuff for stuff in gen(): bomb with exception The error didn't happen in the generator, so I wouldn't expect to see it in the traceback. Yes something l

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Chris Angelico
On Fri, Jul 17, 2015 at 10:54 PM, Antoon Pardon wrote: > On 07/17/2015 01:49 PM, Chris Angelico wrote: >> On Fri, Jul 17, 2015 at 9:43 PM, Antoon Pardon >> wrote: >> >> >>> Sure, but in this case, the generator is still active. The Runtime >>> would be able to jump to and somehow activates it's s

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Antoon Pardon
On 07/17/2015 01:49 PM, Chris Angelico wrote: > On Fri, Jul 17, 2015 at 9:43 PM, Antoon Pardon > wrote: > > >> Sure, but in this case, the generator is still active. The Runtime >> would be able to jump to and somehow activates it's stack record >> for the next value. So why would we expect it to

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Chris Angelico
On Fri, Jul 17, 2015 at 9:43 PM, Antoon Pardon wrote: > On 07/17/2015 01:05 PM, Chris Angelico wrote: >> On Fri, Jul 17, 2015 at 8:48 PM, Antoon Pardon >> wrote: >>> Just wondering, are traceback records of generators available? They are >>> if an exception is raised in the generator itself, but

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Antoon Pardon
On 07/17/2015 01:05 PM, Chris Angelico wrote: > On Fri, Jul 17, 2015 at 8:48 PM, Antoon Pardon > wrote: >> Just wondering, are traceback records of generators available? They are >> if an exception is raised in the generator itself, but what if an exception >> is raised in the loop that is driven

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Chris Angelico
On Fri, Jul 17, 2015 at 8:48 PM, Antoon Pardon wrote: > Just wondering, are traceback records of generators available? They are > if an exception is raised in the generator itself, but what if an exception > is raised in the loop that is driven by a generator. They don't appear in > the standard s

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-17 Thread Antoon Pardon
On 07/16/2015 06:43 PM, Chris Angelico wrote: > On Fri, Jul 17, 2015 at 12:32 AM, Antoon Pardon > wrote: > >> What is unclear about "as it is generally produced on stderr"? That you >> can do a whole lot of stuff, doesn't mean that this whole lot of stuff is >> part of what generally happens. When

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Rustom Mody
On Wednesday, July 15, 2015 at 4:54:51 PM UTC+5:30, Marko Rauhamaa wrote: > Ned Batchelder: > > > On Wednesday, July 15, 2015 at 6:56:10 AM UTC-4, Marko Rauhamaa wrote: > >> Ned Batchelder : > >> > I don't understand this, can you explain more? Are you saying that the > >> > Python specification s

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Gregory Ewing
Joonas Liik wrote: https://docs.python.org/2/library/sys.html#sys.setrecursionlimit so as per the docs the programmer has no real control over how much stack his program can have. all you can say is "let me ignore the safeguard a little longer, i hope i wont crash the program" that is not the sam

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Gregory Ewing
Antoon Pardon wrote: On 07/16/2015 12:43 AM, Gregory Ewing wrote: Whatever value you choose for N, keeping only the first/last N traceback frames will lead to someone tearing their hair out. I would say, that someone should get over himself. Traceback are not the only or even the most useful

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Terry Reedy
On 7/16/2015 7:45 AM, Chris Angelico wrote: On Thu, Jul 16, 2015 at 5:31 PM, Antoon Pardon wrote: Traceback are not the only or even the most useful tool for debugging code. The current stack trace doesn't even contain the value's of the variables on the stack. So in case of Terry Reedy's ex

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Ethan Furman
On 07/16/2015 12:14 PM, Joonas Liik wrote: You are giving the programmer a choice between "run out of stack and crash" and "mutilate interpreter internals and crash or zero out the hard drive" this is not a real choice.. +1 -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
On 16 July 2015 at 21:58, Steven D'Aprano wrote: > On Fri, 17 Jul 2015 03:34 am, Joonas Liik wrote: > >> Now i admit that it is possible to have infinite recursion but it is >> also possiblew to have infinite loops. and we don't kill your code >> after 1000 iterations of a while loop so why should

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Steven D'Aprano
On Fri, 17 Jul 2015 03:34 am, Joonas Liik wrote: > Now i admit that it is possible to have infinite recursion but it is > also possiblew to have infinite loops. and we don't kill your code > after 1000 iterations of a while loop so why should we treat recursion > any differently? Because a while

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 4:23 AM, Joonas Liik wrote: > On 16 July 2015 at 20:49, Chris Angelico wrote: >> >> This sounds like a denial-of-service attack. If you can state that no >> reasonable document will ever have more than 100 levels of nesting, >> then you can equally state that cutting the p

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
On 16 July 2015 at 20:49, Chris Angelico wrote: > > This sounds like a denial-of-service attack. If you can state that no > reasonable document will ever have more than 100 levels of nesting, > then you can equally state that cutting the parser off with a tidy > exception if it exceeds 100 levels

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 3:34 AM, Joonas Liik wrote: > That all sounds reasonable. However that can be looked another way. > Soppose you have some code that traverses some tree, a strange > imbalanced tree (say from some xml) > > It is, semantically at least, a reasonable aproach to process such a

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
On 16 July 2015 at 20:03, Chris Angelico wrote: > > The trouble with that is that it can quickly run you out memory when > you accidentally trigger infinite recursion. A classic example is a > simple wrapper function... > > def print(msg): > print(ctime()+" "+msg) > > With the recursion limit

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Ethan Furman
On 07/16/2015 09:43 AM, Chris Angelico wrote: True. That said, though, it's not a justification for dropping stack frames; even in the form that's printed to stderr, there is immense value in them. It may be possible to explicitly drop frames that a programmer believes won't be useful, but a gen

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 2:50 AM, Joonas Liik wrote: > Wouldn't it be possible to have like a dynamically > sized stack so that you can grow it endlessly > with some acceptable overhead.. > > That would pretty much take care of the stack-overflow > argument without many painful side effects on > th

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Joonas Liik
Wouldn't it be possible to have like a dynamically sized stack so that you can grow it endlessly with some acceptable overhead.. That would pretty much take care of the stack-overflow argument without many painful side effects on the semantics at least.. -- https://mail.python.org/mailman/listinf

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Fri, Jul 17, 2015 at 12:32 AM, Antoon Pardon wrote: > On 07/16/2015 04:00 PM, Chris Angelico wrote: >> On Thu, Jul 16, 2015 at 11:56 PM, Antoon Pardon >> wrote: >>> Fine, I should have been more clear. >>> >>> The stack trace as it is generally produced on stderr after an uncought >>> exceptio

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Antoon Pardon
On 07/16/2015 04:00 PM, Chris Angelico wrote: > On Thu, Jul 16, 2015 at 11:56 PM, Antoon Pardon > wrote: >> Fine, I should have been more clear. >> >> The stack trace as it is generally produced on stderr after an uncought >> exception, doesn't contain the values of the variables on the stack. > S

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Thu, Jul 16, 2015 at 11:56 PM, Antoon Pardon wrote: > On 07/16/2015 01:45 PM, Chris Angelico wrote: >> On Thu, Jul 16, 2015 at 5:31 PM, Antoon Pardon >> wrote: >>> >>> I would say, that someone should get over himself. >>> Traceback are not the only or even the most useful >>> tool for debuggi

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Antoon Pardon
On 07/16/2015 01:45 PM, Chris Angelico wrote: > On Thu, Jul 16, 2015 at 5:31 PM, Antoon Pardon > wrote: >> >> I would say, that someone should get over himself. >> Traceback are not the only or even the most useful >> tool for debugging code. The current stack trace >> doesn't even contain the val

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Chris Angelico
On Thu, Jul 16, 2015 at 5:31 PM, Antoon Pardon wrote: > On 07/16/2015 12:43 AM, Gregory Ewing wrote: > >> Antoon Pardon wrote: >>> But it doesn't need to be all or nothing. How about the following >>> possibility. >>> When the runtime detects a serie of tail calls, it will keep the bottom >>> th

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-16 Thread Antoon Pardon
On 07/16/2015 12:43 AM, Gregory Ewing wrote: > Antoon Pardon wrote: >> But it doesn't need to be all or nothing. How about the following >> possibility. >> When the runtime detects a serie of tail calls, it will keep the bottom three >> and the top three backtrace records of the serie. > Whatever

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Mark Lawrence
On 15/07/2015 23:34, Gregory Ewing wrote: Mark Lawrence wrote: IIRC the realms of the C setjmp and longjmp. Not really the same thing. A longjmp chops the stack back, whereas a tail call avoids putting something on the stack to begin with. Thanks for that :) -- My fellow Pythonistas, ask n

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Antoon Pardon wrote: But it doesn't need to be all or nothing. How about the following possibility. When the runtime detects a serie of tail calls, it will keep the bottom three and the top three backtrace records of the serie. Whatever value you choose for N, keeping only the first/last N trac

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Mark Lawrence wrote: IIRC the realms of the C setjmp and longjmp. Not really the same thing. A longjmp chops the stack back, whereas a tail call avoids putting something on the stack to begin with. -- Greg -- https://mail.python.org/mailman/listinfo/python-list

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Chris Angelico wrote: Which really says that TCO is impossible if you have any sort of clean-up or deallocation to be done after the call begins. The only way to truly turn your tail call into a GOTO is to do all your cleanup first. Indeed. In compilers that implement TCO, there's quite a lot m

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Ian Kelly : > On Tue, Jul 14, 2015 at 11:27 AM, Marko Rauhamaa wrote: >> You'll see that the generated code is tail-call-optimized. > > This can't be done generally, though. What if, instead of a local > variable, the assignment were to a dict item? Even if the dict itself > is a local variable,

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Ian Kelly
On Tue, Jul 14, 2015 at 11:27 AM, Marko Rauhamaa wrote: > Ian Kelly : > >> On Tue, Jul 14, 2015 at 2:33 AM, Marko Rauhamaa wrote: >>> The programmer shouldn't be controlling tail call optimizations. >> >> The programmer controls it regardless. >> >> return foo() # TCO >> >> result = foo(

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Chris Angelico
On Wed, Jul 15, 2015 at 7:13 PM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> I'm still interested in the explicit "replace current stack frame with >> this call" operation. Calling it "goto" seems wrong, as most languages >> with goto restrict it to _within_ a function, > > > This just sugg

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Chris Angelico
On Wed, Jul 15, 2015 at 10:00 PM, MRAB wrote: > On 2015-07-15 12:22, Mark Lawrence wrote: >> >> On 15/07/2015 10:13, Gregory Ewing wrote: >>> >>> Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seem

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread MRAB
On 2015-07-15 12:22, Mark Lawrence wrote: On 15/07/2015 10:13, Gregory Ewing wrote: Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seems wrong, as most languages with goto restrict it to _within_ a function,

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Ned Batchelder : > On Wednesday, July 15, 2015 at 6:56:10 AM UTC-4, Marko Rauhamaa wrote: >> Ned Batchelder : >> > I don't understand this, can you explain more? Are you saying that the >> > Python specification shouldn't specify what x becomes?: >> > >> > def who_knows(): >> > pass >>

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Mark Lawrence
On 15/07/2015 10:13, Gregory Ewing wrote: Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seems wrong, as most languages with goto restrict it to _within_ a function, This just suggests to me is that most lang

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Ned Batchelder
On Wednesday, July 15, 2015 at 6:56:10 AM UTC-4, Marko Rauhamaa wrote: > Ned Batchelder : > > > On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: > >> The other problem for tail call elimination is the requirement that > >> functions return None by default. Smooth tail call el

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Antoon Pardon
On 07/15/2015 12:55 PM, Marko Rauhamaa wrote: > Ned Batchelder : > >> On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: >>> The other problem for tail call elimination is the requirement that >>> functions return None by default. Smooth tail call elimination would >>> require t

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Ned Batchelder : > On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: >> The other problem for tail call elimination is the requirement that >> functions return None by default. Smooth tail call elimination would >> require that Python leave the default return value unspecified

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Ned Batchelder
On Wednesday, July 15, 2015 at 2:44:55 AM UTC-4, Marko Rauhamaa wrote: > Gregory Ewing : > > > Marko Rauhamaa wrote: > > >> It might even be tail-call optimized by Python. Only you can't count > >> on it because the language spec doesn't guarantee it. > > > > The language spec might permit it, bu

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Marko Rauhamaa
Gregory Ewing : > A tail call *is* a goto. That's how you implement one in assembly > language -- you write a jump instruction instead of a call > instruction. The jump doesn't have to be to the same function. In functional programming languages you might not even have a call stack. Instead, you

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Gregory Ewing
Chris Angelico wrote: I'm still interested in the explicit "replace current stack frame with this call" operation. Calling it "goto" seems wrong, as most languages with goto restrict it to _within_ a function, This just suggests to me is that most language designers are not very imaginative. :-

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Antoon Pardon
On 07/15/2015 02:41 AM, Terry Reedy wrote: > On 7/14/2015 10:02 AM, Antoon Pardon wrote: >> On 07/14/2015 03:43 PM, Chris Angelico wrote: >>> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa >>> wrote: > No, tail call optimization doesn't change the behavior of the program, for the worse

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-15 Thread Antoon Pardon
On 07/14/2015 07:48 PM, Steven D'Aprano wrote: > On Wed, 15 Jul 2015 03:29 am, Marko Rauhamaa wrote: > >> Chris Angelico : >> >>> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa >>> wrote: No, tail call optimization doesn't change the behavior of the program, for the worse anyway. >>> I

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Gregory Ewing : > Marko Rauhamaa wrote: >> It might even be tail-call optimized by Python. Only you can't count >> on it because the language spec doesn't guarantee it. > > The language spec might permit it, but the BDFL has explicitly > expressed a dislike for the idea of implicit tail call remo

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Gregory Ewing
Marko Rauhamaa wrote: It might even be tail-call optimized by Python. Only you can't count on it because the language spec doesn't guarantee it. The language spec might permit it, but the BDFL has explicitly expressed a dislike for the idea of implicit tail call removal, so it's unlikely to ev

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Wed, Jul 15, 2015 at 11:01 AM, Terry Reedy wrote: > On 7/14/2015 1:52 PM, Chris Angelico wrote: >> >> On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: >>> >>> I don't like the way integer overflows are explicitly undefined in >>> modern C. >>> >>> Similarly, I don't like the way tail cal

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
I think I'm beginning to understand. Tail call elimination is necessary to cope with a system of programming that has deliberately excluded certain other options (eg mutables, in pure functional languages), Tail recursion is the functional way to write a while loop. To put is another way, a

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
On 7/14/2015 1:52 PM, Chris Angelico wrote: On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: I don't like the way integer overflows are explicitly undefined in modern C. Similarly, I don't like the way tail call behavior is undefined in Python. Where in the Python spec is it stated tha

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
On 7/14/2015 10:02 AM, Antoon Pardon wrote: On 07/14/2015 03:43 PM, Chris Angelico wrote: On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: No, tail call optimization doesn't change the behavior of the program, for the worse anyway. It does, because you lose traceback records. That's

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Terry Reedy
On 7/14/2015 1:15 AM, Paul Rubin wrote: def quicksort(array, start, end): midp = partition(array, start, end) if midp <= (start+end)//2: quicksort(array, start, midp) quicksort(array, midp+1, end) else: quicksort(array, midp+1, end) quicks

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Mark Lawrence
On 15/07/2015 00:40, Mark Lawrence wrote: On 13/07/2015 23:46, Terry Reedy wrote: Optimizing specific tail calls is tricker. For one thing, calls to a recursion-replacement function, such as recur, add a stack frame that must also be popped. A CPython bytecode manimpuation solution was given y

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Mark Lawrence
On 13/07/2015 23:46, Terry Reedy wrote: Optimizing specific tail calls is tricker. For one thing, calls to a recursion-replacement function, such as recur, add a stack frame that must also be popped. A CPython bytecode manimpuation solution was given years ago as a recipe on the ActiveState Pyt

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > Tail call elimination is necessary to cope with a system of > programming that has deliberately excluded certain other options (eg > mutables, in pure functional languages), Tail call elimination is necessary for the functional programming style. While Scheme strongly promotes

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Wed, Jul 15, 2015 at 4:37 AM, Marko Rauhamaa wrote: > Steven D'Aprano : > >> Tail call behaviour is not undefined in Python. There is a strong >> convention (followed by all implementations I know of) to follow the >> reference implementation's (CPython) behaviour, which is not to >> optimize t

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Steven D'Aprano : > Tail call behaviour is not undefined in Python. There is a strong > convention (followed by all implementations I know of) to follow the > reference implementation's (CPython) behaviour, which is not to > optimize tail calls. But even if an implementation chooses to not > follo

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Steven D'Aprano
On Wed, 15 Jul 2015 03:43 am, Marko Rauhamaa wrote: > The problem there is in the C standard, not the compiler that complies > with the standard. > > I don't like the way integer overflows are explicitly undefined in > modern C. > > Similarly, I don't like the way tail call behavior is undefined

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Paul Rubin
Steven D'Aprano writes: > Here's an example where an overzealous but standards-conforming optimization > introduced a security bug: > http://code.google.com/p/nativeclient/issues/detail?id=245 In that particular example, the refactored code simply looks wrong. -- https://mail.python.org/mailman/

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Wed, Jul 15, 2015 at 3:43 AM, Marko Rauhamaa wrote: > I don't like the way integer overflows are explicitly undefined in > modern C. > > Similarly, I don't like the way tail call behavior is undefined in > Python. Where in the Python spec is it stated that tail call behaviour is undefined? The

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Steven D'Aprano
On Wed, 15 Jul 2015 03:29 am, Marko Rauhamaa wrote: > Chris Angelico : > >> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa >> wrote: >>> No, tail call optimization doesn't change the behavior of the >>> program, for the worse anyway. >> >> It does, because you lose traceback records. That's pr

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Steven D'Aprano : > C, in my opinion, is the poster child for how a programming language > should NOT be designed, and I don't want Python to go down the same > path. John Regehr writes: > > "... there are compilers (like GCC) where integer overflow behaved a > certain way for many years and then

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Steven D'Aprano
On Tue, 14 Jul 2015 06:33 pm, Marko Rauhamaa wrote: > Ian Kelly : > >> On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa >> wrote: >>> How about "return"? >> >> I think you miss my point entirely. "return" doesn't mean tail-call >> optimize; > > Why not? Because then you lose most of your debug

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > You're proposing making _every_ instance of "return func(...)" into > this kind of thing. Yes. > That's not always recursion, Correct. > and it's certainly not always clear what called what to get you there. Correct. Marko -- https://mail.python.org/mailman/listinfo/pyth

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: >> No, tail call optimization doesn't change the behavior of the >> program, for the worse anyway. > > It does, because you lose traceback records. That's pretty significant > when you come to try to debug something. Does

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Ian Kelly : > On Tue, Jul 14, 2015 at 2:33 AM, Marko Rauhamaa wrote: >> The programmer shouldn't be controlling tail call optimizations. > > The programmer controls it regardless. > > return foo() # TCO > > result = foo() # No TCO > return result # No TCO Not necessarily. Compile

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Wed, Jul 15, 2015 at 12:02 AM, Antoon Pardon wrote: > On 07/14/2015 03:43 PM, Chris Angelico wrote: >> On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: >>> Chris Angelico : >>> On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: > I would rather optimize by default and dis

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Grant Edwards
On 2015-07-14, Ian Kelly wrote: > And yet, Python somehow manages to gain new features with each release. > > The reason why most proposals get rejected is because most proposals > are bad. If every idea that came along just got accepted, we'd have a > disastrous hodge-podge of a language. And P

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Antoon Pardon
On 07/14/2015 03:43 PM, Chris Angelico wrote: > On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: I would rather optimize by default and disable optimizations with --debug or equivalent. >>> That as

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Tue, Jul 14, 2015 at 11:38 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: >>> I would rather optimize by default and disable optimizations with >>> --debug or equivalent. >> >> That assumes that it's simply an optimization. This is a d

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: >> I would rather optimize by default and disable optimizations with >> --debug or equivalent. > > That assumes that it's simply an optimization. This is a distinct > semantic change. No, tail call optimization doesn't ch

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Ian Kelly
On Tue, Jul 14, 2015 at 2:33 AM, Marko Rauhamaa wrote: > Ian Kelly : > >> On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa wrote: >>> How about "return"? >> >> I think you miss my point entirely. "return" doesn't mean tail-call >> optimize; > > Why not? > >> it just means to return the result. >

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Tue, Jul 14, 2015 at 10:28 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> Done explicitly like this, it avoids the problem of bad tracebacks, >> because by definition you would use it only when you want the current >> stack frame to be dropped. I wonder how hard it would be to hack this >> i

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > Done explicitly like this, it avoids the problem of bad tracebacks, > because by definition you would use it only when you want the current > stack frame to be dropped. I wonder how hard it would be to hack this > into CPython... Anyone feel like tackling it? I would rather opt

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Chris Angelico : > On Tue, Jul 14, 2015 at 3:41 PM, Marko Rauhamaa wrote: >> Tail-call optimization has nothing to do with converting algorithms into >> iterations. It's a prosaic trick of dropping an unneeded stack frame >> before making a function call. >> >>> The claim that TCO means you don't

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Tue, Jul 14, 2015 at 3:41 PM, Paul Rubin wrote: > Chris Angelico writes: >> That's a prime example of recursion... but not of recursion that can >> be tail-call optimized into iteration. It's an example of forking >> recursion, where one call can result in multiple calls (same with tree >> tra

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Chris Angelico
On Tue, Jul 14, 2015 at 3:41 PM, Marko Rauhamaa wrote: > Tail-call optimization has nothing to do with converting algorithms into > iterations. It's a prosaic trick of dropping an unneeded stack frame > before making a function call. > >> The claim that TCO means you don't need stack space for all

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Antoon Pardon
On 07/14/2015 10:34 AM, Ian Kelly wrote: > On Tue, Jul 14, 2015 at 1:26 AM, Antoon Pardon > wrote: >> I expect any example given, to be used as a contest by those reading, >> for finding >> the least warped alternative and then using that to dismiss the example. >> >> So I really don't feel compel

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Ian Kelly : > On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa wrote: >> How about "return"? > > I think you miss my point entirely. "return" doesn't mean tail-call > optimize; Why not? > it just means to return the result. Same difference. > This is what led to the confusion responsible for

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Ian Kelly
On Tue, Jul 14, 2015 at 1:26 AM, Antoon Pardon wrote: > On 07/14/2015 05:20 AM, Steven D'Aprano wrote: >> On Tue, 14 Jul 2015 12:05 am, Chris Angelico wrote: >> >>> If you want to make an assertion that iterative >>> code requires equivalent warping to tail-recursive code, I want to see >>> an exa

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Ian Kelly
On Mon, Jul 13, 2015 at 11:57 PM, Marko Rauhamaa wrote: > Ian Kelly : > >> On Mon, Jul 13, 2015 at 11:25 PM, Chris Angelico wrote: >>> (Also, side point: Python can't actually optimize the above function, >>> because it actually means "call quicksort, then discard its return >>> value and return

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Mark Lawrence
On 14/07/2015 07:29, Gregory Ewing wrote: Marko Rauhamaa wrote: Ian Kelly : Another point in favor of an explicit tail-call keyword. Then one couldn't make that mistake. How about "return"? How about "goto"? :-) Why not "goto"? It's use is scattered throughout the cpython code, and to

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Antoon Pardon
On 07/14/2015 05:20 AM, Steven D'Aprano wrote: > On Tue, 14 Jul 2015 12:05 am, Chris Angelico wrote: > >> If you want to make an assertion that iterative >> code requires equivalent warping to tail-recursive code, I want to see >> an example of it. Is that difficult? > Of course it is. If it wasn't

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-14 Thread Marko Rauhamaa
Gregory Ewing : > Marko Rauhamaa wrote: >> How about "return"? > > How about "goto"? :-) > > That's not entirely an unserious suggestion -- if you > really believe that a "goto with arguments" is a good > feature for a language to have, you shouldn't be > afraid to spell it as such. > > def quic

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Gregory Ewing
Marko Rauhamaa wrote: Ian Kelly : Another point in favor of an explicit tail-call keyword. Then one couldn't make that mistake. How about "return"? How about "goto"? :-) That's not entirely an unserious suggestion -- if you really believe that a "goto with arguments" is a good feature for

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Marko Rauhamaa
Ian Kelly : > On Mon, Jul 13, 2015 at 11:25 PM, Chris Angelico wrote: >> (Also, side point: Python can't actually optimize the above function, >> because it actually means "call quicksort, then discard its return >> value and return None". A true tail call has to return the result of >> the recur

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Ian Kelly
On Mon, Jul 13, 2015 at 11:25 PM, Chris Angelico wrote: > (Also, side point: Python can't actually optimize the above function, > because it actually means "call quicksort, then discard its return > value and return None". A true tail call has to return the result of > the recursive call, and Pyth

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Paul Rubin
Chris Angelico writes: > That's a prime example of recursion... but not of recursion that can > be tail-call optimized into iteration. It's an example of forking > recursion, where one call can result in multiple calls (same with tree > traversal); it calls itself to sort the first part and the la

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Marko Rauhamaa
Chris Angelico : >> def quicksort(array, start, end): >> midp = partition(array, start, end) >> if midp <= (start+end)//2: >> quicksort(array, start, midp) >> quicksort(array, midp+1, end) >> else: >> quicksort(array, midp+1, end) >> quicksort(array

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Chris Angelico
On Tue, Jul 14, 2015 at 3:15 PM, Paul Rubin wrote: > Chris Angelico writes: >> I'm not sure why the transition to another state has to be recursive. > > It's not recursive: it's more like a goto with arguments, and a tail > call expresses it nicely. Hmm, maybe, but I'm not sure that the transiti

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Chris Angelico
On Tue, Jul 14, 2015 at 3:15 PM, Paul Rubin wrote: > It's difficult given how subjective the concept of warping is. What's > straightforward to someone else sounds likely to look warped to you and > vice versa. But how does this look: > > def quicksort(array, start, end): > midp = partiti

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Paul Rubin
Chris Angelico writes: > I'm not sure why the transition to another state has to be recursive. It's not recursive: it's more like a goto with arguments, and a tail call expresses it nicely. > Maybe this is something where previous experience makes you more > comfortable with a particular style,

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Terry Reedy
On 7/13/2015 3:07 PM, Marko Rauhamaa wrote: Or, translated into (non-idiomatic) Python code: def common_prefix_length(bytes_a, bytes_b): def loop(list_a, list_b, common_length): if not list_a: re

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Steven D'Aprano
On Tue, 14 Jul 2015 12:05 am, Chris Angelico wrote: > If you want to make an assertion that iterative > code requires equivalent warping to tail-recursive code, I want to see > an example of it. Is that difficult? Of course it is. If it wasn't difficult, people would post examples instead of gett

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Terry Reedy
On 7/13/2015 7:22 AM, Jussi Piitulainen wrote: Ian Burnette writes: A post I did not receive, but want to comment on. I've recently been playing around with Clojure, and I really like the way they've overcome the JVM not supporting TRE. The way Clojure solves this problem is to have a built i

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Marko Rauhamaa
Ethan Furman : > I would love to see good functional examples as it is definitely a > weak spot for me. Oh, if you want to go functional, you should look at idiomatic Scheme code: (define (common-prefix-length bytes-a bytes

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Ethan Furman
Antoon, I think Chris is arguing in good faith; certainly asking for examples should be a reasonable request. Even if he is not, we would have better discussions and other participants would learn more if you act like he is. I would love to see good functional examples as it is definitely a

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

2015-07-13 Thread Antoon Pardon
On 07/13/2015 04:05 PM, Chris Angelico wrote: > On Mon, Jul 13, 2015 at 11:55 PM, Antoon Pardon > wrote: >> On 07/13/2015 02:34 PM, Chris Angelico wrote: > Warping your code around a recursive solution > to make it into a perfect tail call usually means making it look a lot > less impr

  1   2   >