Re: [Python-Dev] PEP 391 ready for review

2009-11-26 Thread Vinay Sajip 
Terry Reedy  udel.edu> writes:

> 
[snip]

Terry,

Thanks for your helpful comments. I have checked in some changes to the PEP
(r76533) which take into account the comments you made.

Regards,

Vinay Sajip

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 391 ready for review

2009-11-26 Thread Vinay Sajip 
Dj Gilcrease  gmail.com> writes:

> 
> I would recommend removing the class keyword and replacing it with the
> () as you have in the custom examples or replacing () with class so it
> is uniform across all config options
[snip]
> 
> This just strikes me as odd to have to remember for built in handlers
> I need to use the class keyword and for my custom handlers I need to
> use (). My preference would be the class keyword vs the odd (), as
> that is what I am defining, the class to be used for this handler.

If you are using the built in handlers, you will be using a class - hence the
key 'class' is used. If you are using a custom factory, then the system does not
force you to use a class - you can use any callable (e.g. a function which
constructs an instance and sets some attributes of it before returning it), and
the use of '()' indicates that you're not being forced to use a class. Of course
you can, since a class is a callable, but you're not restricted to classes here.

> The ext:// I think should be py:// when defining objects to be access
> via pythons normal import mechanisms and ext:// for resolving external
> processes or system calls
> 

The mechanism I proposed can be extended or changed as you suggest according to
the norms of a specific developer environment (e.g. organization policies), but
the basic system as I see it doesn't (and shouldn't) care about whether a
specific value resolves to an internal (e.g. provided by stdlib) value or an
external (e.g. provided by 3rd-party lib) value. The resolution process would be
exactly the same in either case. Or perhaps I misunderstood what you meant?


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Building a Windows MSI for Python /trunk

2009-11-26 Thread Martin v. Löwis 
> First, the script that finds & builds the external dependencies has two
> minor problems.
> 
>  * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
>tcl8.* and tk8.* to grab the license text.  I changed the glob strings
>appropriately and that seemed to work.

Not sure what "it" is that put it there; perhaps you are referring to
the Tools/buildbot scripts?

Unfortunately, the naming convention that these scripts establish
doesn't quite work, as Tix would fail to build under these conventions.
So in my own checkout, I manually renamed the trees, and that's what
msi.py refers to.

>  * Tix isn't downloaded/installed/built automatically like everything else,
>and msi.py looks for its license file, too.  I just removed the
>Tix reference.  I can't figure out how to build Tix appropriately; any
>tips?

See above. I keep forgetting how to build Tix; one set of command lines
is in PCbuild/build_tkinter.py.

> Second, the buildmsi.bat file refers to python26a3.hhp instead of
> python27a0.hhp.

Yes, that script hasn't been updated for a long time, ever since we
stopped having automated builds.

Ideally, the script would find the Python version on its own.

> Third, I could not get _tkinter to build properly, although it wasn't fatal
> to the endeavor.  It couldn't find ..\..\tcltk\lib\tcl85.lib, although
> tcl85g.lib existed.

You'll need to build release versions of Tcl/Tk/Tix.

> Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
> had to figure out with google -- things like nasm/nasmw, for example.  Are
> these documented somewhere, or would it be helpful to document them?

See PCbuild/readme.txt.

> Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;)

Thanks, fixed.

> I'd love to get this build process working completely automatically and
> 100% correctly, too.
> 
> Hat tip to Trent Nelson, who helped me figure out where the scripts are
> and what other things I needed...

Feel free to submit patches. There may be a misunderstanding, though, in
that buildmsi.bat isn't actually used for anything, and isn't "meant" to
be run by users, but instead by build slaves.

One consequence is that these scripts build in debug mode (perhaps
except for buildmsi), whereas end users would typically want to build in
release mode.

Another consequence is that different committers actually use different
procedures. Trent created much of the Tools/buildbot setup, so he is
obviously in favor of that strategy. Christian Heimes created
PCbuild/build_tkinter, so he would probably prefer to use that instead.
When I took over Windows builds from Tim Peters, PCbuild/readme.txt
was the official reference, and I try to stick to that.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 391 ready for review

2009-11-26 Thread Nick Coghlan 
Terry Reedy wrote:
> if not hasattr(factory, '__call__'): factory = __import__(factory)

That won't quite work since the string generally isn't referring to a
module, and due to the quirk of __import__ returning the top level
module since that is what the interpreter needs to bind to a name as
part of a normal import statement.

This would need to look more like:

if not hasattr(factory, '__call__'):
  mod_name, dot, attr = factory.rpartition('.')
  if not dot:
raise ValueError("")
  module = imputil.import_module(mod_name)
  factory = getattr(module, attr)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 391 ready for review

2009-11-26 Thread Nick Coghlan 
Nick Coghlan wrote:
> Terry Reedy wrote:
>> if not hasattr(factory, '__call__'): factory = __import__(factory)
> 
> That won't quite work since the string generally isn't referring to a
> module, and due to the quirk of __import__ returning the top level
> module since that is what the interpreter needs to bind to a name as
> part of a normal import statement.
> 
> This would need to look more like:
> 
> if not hasattr(factory, '__call__'):
>   mod_name, dot, attr = factory.rpartition('.')
>   if not dot:
> raise ValueError("")
>   module = imputil.import_module(mod_name)
>   factory = getattr(module, attr)

s/imputil/importlib/ (Oops...)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Building a Windows MSI for Python /trunk

2009-11-26 Thread C. Titus Brown 
On Thu, Nov 26, 2009 at 09:54:35AM +0100, "Martin v. L?wis" wrote:
> > First, the script that finds & builds the external dependencies has two
> > minor problems.
> > 
> >  * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
> >tcl8.* and tk8.* to grab the license text.  I changed the glob strings
> >appropriately and that seemed to work.
> 
> Not sure what "it" is that put it there; perhaps you are referring to
> the Tools/buildbot scripts?

Yes, sorry, Tools/buildbot/external*.bat, as in the diff.

> Unfortunately, the naming convention that these scripts establish
> doesn't quite work, as Tix would fail to build under these conventions.
> So in my own checkout, I manually renamed the trees, and that's what
> msi.py refers to.

OK, thanks!

> >  * Tix isn't downloaded/installed/built automatically like everything else,
> >and msi.py looks for its license file, too.  I just removed the
> >Tix reference.  I can't figure out how to build Tix appropriately; any
> >tips?
> 
> See above. I keep forgetting how to build Tix; one set of command lines
> is in PCbuild/build_tkinter.py.

Ok, thanks!

> > Second, the buildmsi.bat file refers to python26a3.hhp instead of
> > python27a0.hhp.
> 
> Yes, that script hasn't been updated for a long time, ever since we
> stopped having automated builds.
> 
> Ideally, the script would find the Python version on its own.

Right...

> > Third, I could not get _tkinter to build properly, although it wasn't fatal
> > to the endeavor.  It couldn't find ..\..\tcltk\lib\tcl85.lib, although
> > tcl85g.lib existed.
> 
> You'll need to build release versions of Tcl/Tk/Tix.

Yes, I saw a reference to that online, but I wasn't sure if that was the
problem -- is the naming convention really that 'tcl85g.lib' is the debug
lib!?

> > Oh, and there were a bunch of missing commands that (as a non-Windows 
> > xpert) I
> > had to figure out with google -- things like nasm/nasmw, for example.  Are
> > these documented somewhere, or would it be helpful to document them?
> 
> See PCbuild/readme.txt.

OK, thanks.

> > I'd love to get this build process working completely automatically and
> > 100% correctly, too.
> > 
> > Hat tip to Trent Nelson, who helped me figure out where the scripts are
> > and what other things I needed...
> 
> Feel free to submit patches. There may be a misunderstanding, though, in
> that buildmsi.bat isn't actually used for anything, and isn't "meant" to
> be run by users, but instead by build slaves.

Well, yes, although wouldn't we want the scripts to run without any user
editing, for the build slaves?

> One consequence is that these scripts build in debug mode (perhaps
> except for buildmsi), whereas end users would typically want to build in
> release mode.

Right.

Is there a reason the buildbot builds were stopped? I've added binary
file uploads to pony-build, so that platform-specific files can be
uploaded to build results -- see, e.g.

http://lyorn.idyll.org/ctb/pb-dev/p/python/10768/files/

or

http://lyorn.idyll.org/ctb/pb-dev/p/pygr/10805/files/

and I thought daily -latest builds for Python latest on Windows and Mac would
be a good demo...

Incidentally, the pony-build results & file upload scripts can be used from
within buildbot, if people are interested.

> Another consequence is that different committers actually use different
> procedures. Trent created much of the Tools/buildbot setup, so he is
> obviously in favor of that strategy. Christian Heimes created
> PCbuild/build_tkinter, so he would probably prefer to use that instead.
> When I took over Windows builds from Tim Peters, PCbuild/readme.txt
> was the official reference, and I try to stick to that.

OK, I see.  Thanks!  So if I can bring the scripts into concordance with
that it might be valuable?

--titus
-- 
C. Titus Brown, [email protected]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Building a Windows MSI for Python /trunk

2009-11-26 Thread Martin v. Löwis 
>> You'll need to build release versions of Tcl/Tk/Tix.
> 
> Yes, I saw a reference to that online, but I wasn't sure if that was the
> problem -- is the naming convention really that 'tcl85g.lib' is the debug
> lib!?

Yes; Tcl apparently follows Unix conventions here rather than Windows
conventions (where the debug build would end with "d").

>> Feel free to submit patches. There may be a misunderstanding, though, in
>> that buildmsi.bat isn't actually used for anything, and isn't "meant" to
>> be run by users, but instead by build slaves.
> 
> Well, yes, although wouldn't we want the scripts to run without any user
> editing, for the build slaves?

Sure. For the scripts that are in actual use, that's also the case. I
stopped working on buildmsi because it was just too tedious to get
right.

> Is there a reason the buildbot builds were stopped?

It was too difficult to get right, plus nobody was interested in
using the daily MSI files.

At some point, it was broken for several months (IIRC), with nobody
reporting that breakage. So when we noticed, we just turned the service
off.

> and I thought daily -latest builds for Python latest on Windows and Mac would
> be a good demo...

I'll guess that you will find the same: nobody is really interested
in *using* daily builds of Python.

> OK, I see.  Thanks!  So if I can bring the scripts into concordance with
> that it might be valuable?

Most certainly, yes.

Regards,
Martin

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] wpython is back

2009-11-26 Thread Cesare Di Mauro 
Hi Mart

I'm back with some news about wpython. I completed all the work that I was
committed to do till the end of the year. I made a lot of changes to the
code, that I'll report here.

First, I added several conditional compilation sections that enable or
disable almost every optimization I introduced into the project. Everything
is controlled by a new include file, wpython.h, which holds a lot of
#DEFINEs for each one of them.
Every #DEFINE has a brief explanation, and some report an example with
Python code disassembled, showing what happens.
It can be useful both to document the code (also to access to the interested
parts), and to let people test the effect of all optimizations. There are
also a couple of #DEFINEs which are useful to enable or disable all
superinstructions, or to make wpython work like CPython (with all new
optimizations and superinstructions disabled).

Full tracing support required a big effort, due to the missing
SETUP_LOOP/POP_BLOCK instructions used in FOR_ITER blocks. It was a pain in
the neck to let them work, but I think I have found a good solution fot it.
If I remember correctly, Collin asked in the past about performance with
testing enabled. I believe that speed is comparable to CPython, since I can
trace FOR_ITER blocks enter/exit with very little time spent intercepting
them; stack unrolling (for forward jumps cases) is fast too.

Restoring Python object model required much of the work. I reverted all the
changes that I made to many PyObjects, and just added some accessory code
only to a few of them. There are no more hacks, and code is quite polite;
only CodeObject required one line of code change in the hash function, to
let it calculate hash correctly for the constants tuple (because it can hold
lists and dictionaries now, which usally aren't hashable).
Every file in Include/ and Objects/ that I modified has only 1 diff (except
frameobject.c, for tracing code), so it's easy so see what is changed and
the extra helper functions that I added to introduce lists and dictionaries
in the consts tuple.

In the meanwhile I've added a little optimization for lists and dictionaries
used in for loops. Writing this:

def f():
for x in ['a', 'b', 'c']: print x

generates the following (word)code with the previous wpython:

LOAD_CONST (['a', 'b', 'c'])
DEEP_LIST_COPY
GET_ITER
FOR_ITER

because ['a', 'b', 'c'] is a mutable object, and a copy must be made before
using it.

Now it'll be:

LOAD_CONST (['a', 'b', 'c'])
GET_ITER
FOR_ITER

So code is reduced and memory consumption too, because there's no need clone
the list. The trick works only for lists and dictionaries that holds
non-mutable objects, but I found it's a common pattern in Python code.

I've also updated the source to the latest Python 2.x version, 2.6.4.

All tests pass, both with Debug and Release code, on Visual Studio Express
with 32 bit code (I can't compile 64 bits versions with it).

There are only a few open issues.

test_syntax.py required some changes in the doctest (adding full filesystem
path) to let them pass correctly. It's really strange, but... works now!

test_compile.py has 2 tests disabled in test_compile_ast:

#['', """for n in [1, 2, 3]:\n print n\n"""],
#[fname, fcontents],

that's because there's no support for constants (except Num_kind and
Str_kind) in the current ASTs code. However code compiles well, except that
it cannot make use of the new constant folding code.

I haven't updated Doc/library/dis.rst, which is exactly the same of CPython.
I'll do it when I stop introducing or changing opcodes.

Right now wpython requires manual patching of Include/Python-ast.h, with the
following lines:

enum _expr_kind {BoolOp_kind=1, BinOp_kind=2, UnaryOp_kind=3, Lambda_kind=4,
[...]
List_kind=18, Tuple_kind=19, Const_kind=20};

enum _expr_const {no_const=0, mutable_const=1, content_const=3,
pure_const=7};

struct _expr {

enum _expr_kind kind;
union {
[...]

struct {
object c;
enum _expr_const constant;
} Const;

} v;
int lineno;
int col_offset;
};

They are need to let ast.c handle constants for the new constant folding
code.
I greatly appreciate any help to let it be generated automatically with ASDL
grammar.


That's all about the new code. Now the weird and stupid part. A few days I
got a new gmail account, but accidentally I removed the google account that
I've used to create the wpython at Google Code. I definitely lost project
ownership, so I can't tag the old code and put the new one in trunk.
I'll thank very much if someone that works or has contacts with Google can
ask for moving ownership from my old account (cesare at pronto do it) to my
new (the one which I've using now to write this mail), so I'll commit ASAP.
Alternatively, I need to create a new project at Google Code.

I hope that the community will appreciate the work (when I'll upload it :-).
I know that it's a young project, but I think it's mature enough to take a
look at

Re: [Python-Dev] Building a Windows MSI for Python /trunk

2009-11-26 Thread David Bolen 
"Martin v. Löwis"  writes:

> It was too difficult to get right, plus nobody was interested in
> using the daily MSI files.
>
> At some point, it was broken for several months (IIRC), with nobody
> reporting that breakage. So when we noticed, we just turned the service
> off.

Sounds right - the daily MSIs were built on my slave from Sep 2007
into 2008 somewhere, but it had been failing for 6 months or more when
the task was turned off last December.  I'm not sure precisely when
the failures started (the buildbot status page was also down for a
while starting in March), but it had been failing for a long time.

I believe the failures at the time were tcl related - maybe some of
the MSI buildbot-related scripts getting out of date with ongoing
development or something.  Sounds like it might be the same stuff
Titus is running into.  It didn't seem worth the effort to track down
at the time, given that nobody had seemed to notice the lack of
builds.

-- David

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] wpython is back

2009-11-26 Thread Georg Brandl 
Cesare Di Mauro schrieb:
> Hi Mart
> 
> I'm back with some news about wpython. I completed all the work that I
> was committed to do till the end of the year. I made a lot of changes to
> the code, that I'll report here..

For those of us without an elephant's memory, could you please also recap
what exactly wpython is, and maybe what the goals and intentions are?

cheers,
Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] wpython is back

2009-11-26 Thread Guido van Rossum 
On Thu, Nov 26, 2009 at 3:41 PM, Georg Brandl  wrote:
> Cesare Di Mauro schrieb:
>> Hi Mart
>>
>> I'm back with some news about wpython. I completed all the work that I
>> was committed to do till the end of the year. I made a lot of changes to
>> the code, that I'll report here..
>
> For those of us without an elephant's memory, could you please also recap
> what exactly wpython is, and maybe what the goals and intentions are?

It's a Python implementation that uses wordcode instead of bytecode.

http://code.google.com/p/wpython/

I don't see any benchmarks though.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] wpython is back

2009-11-26 Thread Cesare Di Mauro 
2009/11/27 Guido van Rossum 

> It's a Python implementation that uses wordcode instead of bytecode.
>
> http://code.google.com/p/wpython/
>
> I don't see any benchmarks though.
>

You'll find some at page 28
here
.

Mart made more interesting
oneswith
Unladen benchmarks.

Cesare
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com