Re: An idiom for code generation with exec

2008-06-25 Thread eliben
On Jun 23, 6:44 am, eliben <[EMAIL PROTECTED]> wrote: > Thanks for all the replies in this post. Just to conclude, I want to > post a piece of code I wrote to encapsulate function creation in this > way: > > def create_function(code): > """ Create and return the function defined in code. >

Re: An idiom for code generation with exec

2008-06-25 Thread jhermann
Since nobody mentioned textwrap.dedent yet as an alternative to the old "if 1:" trick, I thought I should do so. :) -- http://mail.python.org/mailman/listinfo/python-list

Re: An idiom for code generation with exec

2008-06-23 Thread Maric Michaud
Le Tuesday 24 June 2008 07:18:47 eliben, vous avez écrit : > > If code generation is not the best, and I fail to see any performance > > issue that could explain such a choice, except a misunderstanding of > > what "compilation" means in python, just don't use it, use closures or > > callable insta

Re: An idiom for code generation with exec

2008-06-23 Thread Terry Reedy
eliben wrote: And while we're on the topic of what compilation means in Python, It depends on the implementation. I'm not sure I fully understand the difference between compiled (.pyc) code and exec-ed code. Is the exec-ed code turned to bytecode too, i.e. it will be as efficient as comp

Re: An idiom for code generation with exec

2008-06-23 Thread eliben
> If code generation is not the best, and I fail to see any performance issue > that could explain such a choice, except a misunderstanding of > what "compilation" means in python, just don't use it, use closures or > callable instances, there are many way to achieve this. And while we're on the t

Re: An idiom for code generation with exec

2008-06-23 Thread Fuzzyman
On Jun 21, 7:52 am, Peter Otten <[EMAIL PROTECTED]> wrote: > eliben wrote: > > On Jun 20, 2:44 pm, Peter Otten <[EMAIL PROTECTED]> wrote: > >> eliben wrote: > >> > Additionally, I've found indentation to be a problem in such > >> > constructs. Is there a workable way to indent the code at the level

Re: An idiom for code generation with exec

2008-06-23 Thread Bruno Desthuilliers
Maric Michaud a écrit : Le Monday 23 June 2008 09:22:29 Bruno Desthuilliers, vous avez écrit : With some help from the guys at IRC I came to realize your way doesn't do the same. It creates a function that, when called, creates 'foo' on globals(). This is not exactly what I need. I possibly mes

Re: An idiom for code generation with exec

2008-06-23 Thread Maric Michaud
Le Monday 23 June 2008 09:22:29 Bruno Desthuilliers, vous avez écrit : > > With some help from the guys at IRC I came to realize your way doesn't > > do the same. It creates a function that, when called, creates 'foo' on > > globals(). This is not exactly what I need. > > I possibly messed up a cou

Re: An idiom for code generation with exec

2008-06-23 Thread Bruno Desthuilliers
eliben a écrit : d = {} execcode in globals(), d return d['foo'] My way: return function(compile(code, '', 'exec'), globals()) With some help from the guys at IRC I came to realize your way doesn't do the same. It creates a function that, when called, creates 'foo' on globals(). This

Re: An idiom for code generation with exec

2008-06-22 Thread eliben
Thanks for all the replies in this post. Just to conclude, I want to post a piece of code I wrote to encapsulate function creation in this way: def create_function(code): """ Create and return the function defined in code. """ m = re.match('\s*def\s+([a-zA-Z_]\w*)\s*\(', code) if m

Re: An idiom for code generation with exec

2008-06-22 Thread Scott David Daniels
[EMAIL PROTECTED] wrote: > On 20 juin, 21:44, eliben <[EMAIL PROTECTED]> wrote: ... >> The generic version has to make a lot of decisions at runtime, based >> on the format specification. >> Extract the offset from the spec, extract the length. ... ... > Just my 2 cents. Truth is that as long as i

Re: An idiom for code generation with exec

2008-06-21 Thread George Sakkis
On Jun 21, 9:40 am, eliben <[EMAIL PROTECTED]> wrote: > > > I see. In my case I only evaluate function definitions with 'exec', so > > > I only need to de-indent the first line, and the others can be > > > indented because they're in a new scope anyway. What you suggest works > > > for arbitrary c

Re: An idiom for code generation with exec

2008-06-21 Thread eliben
> > I see. In my case I only evaluate function definitions with 'exec', so > > I only need to de-indent the first line, and the others can be > > indented because they're in a new scope anyway. What you suggest works > > for arbitrary code and not only function definitions. It's a nice > > trick wi

Re: An idiom for code generation with exec

2008-06-21 Thread Lie
On Jun 21, 2:02 pm, eliben <[EMAIL PROTECTED]> wrote: > On Jun 21, 8:52 am, Peter Otten <[EMAIL PROTECTED]> wrote: > > > > > eliben wrote: > > > On Jun 20, 2:44 pm, Peter Otten <[EMAIL PROTECTED]> wrote: > > >> eliben wrote: > > >> > Additionally, I've found indentation to be a problem in such > >

Re: An idiom for code generation with exec

2008-06-21 Thread eliben
On Jun 21, 8:52 am, Peter Otten <[EMAIL PROTECTED]> wrote: > eliben wrote: > > On Jun 20, 2:44 pm, Peter Otten <[EMAIL PROTECTED]> wrote: > >> eliben wrote: > >> > Additionally, I've found indentation to be a problem in such > >> > constructs. Is there a workable way to indent the code at the level

Re: An idiom for code generation with exec

2008-06-21 Thread Peter Otten
eliben wrote: > On Jun 20, 2:44 pm, Peter Otten <[EMAIL PROTECTED]> wrote: >> eliben wrote: >> > Additionally, I've found indentation to be a problem in such >> > constructs. Is there a workable way to indent the code at the level of >> > build_func, and not on column 0 ? >> >> exec"if 1:" + code.

Re: An idiom for code generation with exec

2008-06-20 Thread eliben
On Jun 20, 2:44 pm, Peter Otten <[EMAIL PROTECTED]> wrote: > eliben wrote: > > Additionally, I've found indentation to be a problem in such > > constructs. Is there a workable way to indent the code at the level of > > build_func, and not on column 0 ? > > exec"if 1:" + code.rstrip() > > Peter Why

Re: An idiom for code generation with exec

2008-06-20 Thread eliben
>d = {} > execcode in globals(), d > return d['foo'] > > My way: > > return function(compile(code, '', 'exec'), globals()) > With some help from the guys at IRC I came to realize your way doesn't do the same. It creates a function that, when called, creates 'foo' on globals(). This is not

Re: An idiom for code generation with exec

2008-06-20 Thread eliben
> So you are saying that for example "if do_reverse: data.reverse()" is > *much* slower than "data.reverse()" ? I would expect that checking the > truthness of a boolean would be negligible compared to the reverse > itself. Did you try converting all checks to identity comparisons with > None ? I m

Re: An idiom for code generation with exec

2008-06-20 Thread [EMAIL PROTECTED]
On 20 juin, 21:44, eliben <[EMAIL PROTECTED]> wrote: > On Jun 20, 3:19 pm, George Sakkis <[EMAIL PROTECTED]> wrote: > (snip) > > It's still not clear why the generic version is so slower, unless you > > extract only a few selected fields, not all of them. Can you post a > > sample of how you used

Re: An idiom for code generation with exec

2008-06-20 Thread [EMAIL PROTECTED]
On 20 juin, 21:41, eliben <[EMAIL PROTECTED]> wrote: > > [1] except using compile to build a code object with the function's > > body, then instanciate a function object using this code, but I'm not > > sure whether it will buy you much more performance-wise. I'd personnaly > > prefer this because

Re: An idiom for code generation with exec

2008-06-20 Thread George Sakkis
On Jun 20, 3:44 pm, eliben <[EMAIL PROTECTED]> wrote: > On Jun 20, 3:19 pm, George Sakkis <[EMAIL PROTECTED]> wrote: > > > > > On Jun 20, 8:03 am, eliben <[EMAIL PROTECTED]> wrote: > > > > On Jun 20, 9:17 am, Bruno Desthuilliers > > > [EMAIL PROTECTED]> wrote: > > > > eliben a écrit :> Hello, > >

Re: An idiom for code generation with exec

2008-06-20 Thread eliben
On Jun 20, 3:19 pm, George Sakkis <[EMAIL PROTECTED]> wrote: > On Jun 20, 8:03 am, eliben <[EMAIL PROTECTED]> wrote: > > > > > On Jun 20, 9:17 am, Bruno Desthuilliers > > [EMAIL PROTECTED]> wrote: > > > eliben a écrit :> Hello, > > > > > In a Python program I'm writing I need to dynamically genera

Re: An idiom for code generation with exec

2008-06-20 Thread eliben
> [1] except using compile to build a code object with the function's > body, then instanciate a function object using this code, but I'm not > sure whether it will buy you much more performance-wise. I'd personnaly > prefer this because I find it more explicit and readable, but YMMV. > How is com

Re: An idiom for code generation with exec

2008-06-20 Thread eliben
> FWIW, when I had a similar challenge for dynamic coding, I just > generated a py file and then imported it. This technique was nice > because can also work with Pyrex or Psyco. > I guess this is not much different than using exec, at the conceptual level. exec is perhaps more suitable when you

Re: An idiom for code generation with exec

2008-06-20 Thread Dan Yamins
On Fri, Jun 20, 2008 at 3:17 AM, Bruno Desthuilliers <[EMAIL PROTECTED]> wrote: Just to make things clear: you do know that you can dynamically build > functions without exec, do you ? Actually, I don't know how to do this, but would like to. Can you point me to a place where I can read more a

Re: An idiom for code generation with exec

2008-06-20 Thread Bruno Desthuilliers
eliben a écrit : On Jun 20, 9:17 am, Bruno Desthuilliers wrote: eliben a écrit :> Hello, In a Python program I'm writing I need to dynamically generate functions[*] (snip) [*] I know that each time a code generation question comes up people suggest that there's a better way to achieve this

Re: An idiom for code generation with exec

2008-06-20 Thread Raymond Hettinger
On Jun 20, 5:03 am, eliben <[EMAIL PROTECTED]> wrote: > I've rewritten it using a dynamically generated procedure > for each field, that does hard coded access to its data. For example: > > def get_counter(packet): >   data = packet[2:6] >   data.reverse() >   return data > > This gave me a huge sp

Re: An idiom for code generation with exec

2008-06-20 Thread George Sakkis
On Jun 20, 8:03 am, eliben <[EMAIL PROTECTED]> wrote: > On Jun 20, 9:17 am, Bruno Desthuilliers > [EMAIL PROTECTED]> wrote: > > eliben a écrit :> Hello, > > > > In a Python program I'm writing I need to dynamically generate > > > functions[*] > > > (snip) > > > > [*] I know that each time a code g

Re: An idiom for code generation with exec

2008-06-20 Thread Peter Otten
eliben wrote: > Additionally, I've found indentation to be a problem in such > constructs. Is there a workable way to indent the code at the level of > build_func, and not on column 0 ? exec "if 1:" + code.rstrip() Peter -- http://mail.python.org/mailman/listinfo/python-list

Re: An idiom for code generation with exec

2008-06-20 Thread eliben
On Jun 20, 9:17 am, Bruno Desthuilliers wrote: > eliben a écrit :> Hello, > > > In a Python program I'm writing I need to dynamically generate > > functions[*] > > (snip) > > > [*] I know that each time a code generation question comes up people > > suggest that there's a better way to achieve thi

Re: An idiom for code generation with exec

2008-06-20 Thread Bruno Desthuilliers
eliben a écrit : Hello, In a Python program I'm writing I need to dynamically generate functions[*] (snip) [*] I know that each time a code generation question comes up people suggest that there's a better way to achieve this, without using exec, eval, etc. Just to make things clear: you