Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Glenn Linderman]

On 3/19/2011 7:38 PM, Mark Hammond wrote:

Thanks for the feedback!


And thanks for more complete explanations.

Sadly I was offline when writing my first response, and couldn't view 
the man page for execve you referred to.  Having just read it, I think 
it would be total gibberish to a Windows user who has only learned 
Python 2.x programming, and is now faced with migrating to 3.x, and 
wants to use this launcher to help make that a gradual process, instead 
of the full, instant migration required by the installation of Python 
3.x wiping out the associations from Python 2.x leaving his system with 
2.x unusable (from a naïve point of view).  I have not found any support 
either from the installer or the documentation (but maybe I missed some 
things) that tell how to make Python 2.x and 3.x coexist happily on 
Windows, short of simply reinstalling the one you want to use.


A Windows user who has only learned Python 2.x programming would not 
necessarily have ever heard of execve, would not realize execve(2) means 
it is from the 2nd chapter of the Unix man pages, meaning an API call, 
would not know C programming, would not know what #include  
means, would not understand syntax like "const" or "[]".  The 
description is a bit friendlier, if they get that far, but they may 
still be quite confused by wondering how to create and provide argv and 
envp in the right forms, wondering what SIGCHLD, PID, set-user-ID bit, 
SIG_IGN, SIG_DFL, SIGTRAP, set-group-ID, effective id, a.out, binary 
executable, shared library stubs, ld.so(8), ELF executable, PT_INTERP, 
and /lib/ld-unix.so.1 (and .2) are.


In short, anyone that is a Unix C programmer can easily understand this 
stuff, but anyone that is a Windows Python-only programmer will feel 
like understanding the launcher by reading that man page is similar in 
effort to getting their masters degree, and that if it is that hard to 
slowly migrate their 300 Python scripts to use Python 3.x while still 
using Python 2.x for those that are not converted, that they might as 
well just stick with Python 2.x.


Yes, the argument could be made that parts of Python also require some 
Unix knowledge and basic C knowledge to use effectively... but there are 
many Python programs that can be written without using the parts that 
require that knowledge.


That said, I don't think it would be an onerous task to extract a 
reasonable description of #! usage from the execve man page, and 
eliminate the Unix masters degree requirement.  In fact, only the 
following two paragraph fragments seem to be relevant to #! lines:


... a script starting with a line of the form "*#! /interpreter/* 
[arg]".  ... the interpreter must be a valid pathname for an 
executable which is not itself a script, which will be invoked as 
*interpreter* [arg] /filename/.


A maximum line length of 127 characters is allowed for the first line 
in a #! executable shell script.




More below in context...


On 19/03/2011 7:44 PM, Glenn Linderman wrote:

Not all of the ideas below are complementary to each other, some are
either or, to allow different thoughts to be inspired or different
directions to be taken.

Thanks for starting a PEP.

On 3/18/2011 11:02 PM, Mark Hammond wrote:


 The launcher should be as simple as possible (but no simpler.)


The launcher could be simpler if it isn't used for launching interactive
interpreters as well as script references via Windows associations (more
about that after the next quote).

The launcher could be simpler if it just read through the file of its
first parameter until it finds a line starting with #@ (process as a
Windows version of Unix #!) or starting without a # (error case). This
avoids the need parse such lines.


I don't agree with that at all.  The rules for the shebang parsing are 
very simple - see the man-page I reference in the pep - the file must 
start with the characters '#!' and the entire command must be < 127 
chars.  The only real "parsing" needed is to check if the specified 
command starts with one of 2 fixed strings.  I believe this is simpler 
than parsing multiple lines of the file.


Ah, but Mark!  You are confusing (and maybe I do in some of my comments 
too) the complexity of the launcher versus the complexity of describing 
the launcher.  It is really the complexity of describing the launcher 
(and remember, Unix doesn't need a launcher, it has #!, so you need to 
describe it to Windows users for Windows users) that matters most.  I 
find I often rewrite portions of my programs when I write the 
documentation to make them easier to describe.  And some say the docs 
should be written first, but then I find that I have to rewrite the docs 
to make the code possible :)  Whatever the order, the goal should be to 
make the program as useful as possible to a variety of users, with as 
little complexity as possible in the documentation.


So I still claim that it is easier to tell a Windows user that the 
launcher looks at comme

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 20/03/2011 6:22 PM, Glenn Linderman wrote:

On 3/19/2011 7:38 PM, Mark Hammond wrote:

...

A Windows user who has only learned Python 2.x programming would not
necessarily have ever heard of execve, would not realize execve(2) means
it is from the 2nd chapter of the Unix man pages, meaning an API call,
would not know C programming, would not know what #include 
means, would not understand syntax like "const" or "[]". The description
is a bit friendlier, if they get that far, but they may still be quite
confused by wondering how to create and provide argv and envp in the
right forms, wondering what SIGCHLD, PID, set-user-ID bit, SIG_IGN,
SIG_DFL, SIGTRAP, set-group-ID, effective id, a.out, binary executable,
shared library stubs, ld.so(8), ELF executable, PT_INTERP, and
/lib/ld-unix.so.1 (and .2) are.


Sorry, I didn't mean to imply we would refer people to the execve man 
page, but simply that the man page defines the behaviour from the POV of 
the PEP.



Ah, but Mark! You are confusing (and maybe I do in some of my comments
too) the complexity of the launcher versus the complexity of describing
the launcher.


I believe it will be simple to describe it -as you mention, the execve 
man page needs only a couple of sentences - I believe with just a few 
more it can be documented with the same clarity and quality as the rest 
of the Python docs for Windows users.



The PEP could therefore be clearer by discussing the CPython
implementation strategy, even if it points out that other
implementations may need a different strategy.


I believe it states fairly clearly that the reference implementation 
defines much of this, and the reference implementation I referred to 
does indeed implement (and therefore define) it.  The PEP does make 
mention of other implementations.



So I think we are agreeing here, mostly. And you mentioned having the
CPython implementation PythonX.Y get on the PATH somehow. So just keep
the python launcher off the PATH, remove its then unnecessary option
syntax, and we agree totally :)


Not quite - my point is that *if* a CPython implementation is on the 
path (which many users arrange), then there is no advantage to using the 
launcher for that specific Python.  I personally don't have any CPython 
implementation on my path (actually, that isn't totally true - cygwin's 
Python is for my command-prompts), and therefore would find the 
behaviour described in the launcher incredibly useful.  This scratches 
my itch and unless the option does harm for people who choose not to use 
it, I see no problem with supporting it.



Does anyone see any benefit to using a launcher, if PythonX.Y is on the
PATH? I don't.


Fair enough.  But users who do not have a PythonX.Y on the PATH (even if 
they have another X.Y on it) will.



Per the beginning of this email, I think you need to stay far away from
referencing the execve man page in a description of a Windows launcher,
except perhaps as a footnote.


The PEP is not intended to be user documentation, but if it is to be 
used in that way I'd be happy to document it directly in the PEP.



Not using the launcher from the command line, and it not having any
available options, would sidestep this issue nicely.


I don't see what issue needs to be side-stepped.  IMO the feature can be 
clearly described (even if my initial PEP didn't do so) and easily 
avoided if not desired.



No, the workaround of using fully-qualified executable names in the
shebang line makes the script now dysfunctional on Unix.


No - as the PEP says, if it can be expected to work on Unix, it should 
be expected to work on Windows.  Otherwise, the script is inherently 
dysfunctional on Unix, so the fully-qualified path is perfectly reasonable.



I don't think you'll get "always", because of environments that use
different implementations on different platforms. To get always, you
need the flexibility to specify implementation and version for each
platform separately... two lines.


Sorry, but I just don't go for this at all.  Please don't continue to 
advocate for that in a PEP I am sponsoring - just propose your own.



Let's enumerate... and maybe if we keep talking we'll increase the
agreement list and decrease the disagreement list.

New ideas in this message:
1) If launcher doesn't find a #!/#@ line, it should run python2 to
support scripts without such lines.


Sorry if the PEP didn't make it clear, but that is how the reference 
implementation works.  You might like to try it.



Agreements:
1) Python needs PythonX.Y to be on the PATH


Nope.


2) Need a launcher to handle Windows associations that otherwise only
allow one version for one extension
3) Need a separate launcher for .py and .pyw


Yep.


4) Support for as many environments as possible, and as many
installations as possible for each environment.


Nope - "supports standard Python installations on all Windows 
installations".  Note the "all".



What else? Can I borrow and tweak your reference impleme

[Python-Dev] PyLongObject "safe" changes?

[Cesare Di Mauro]

Hi

I have little knowledge of some Python (3.2) internals on objects' internal
structure handling.

Suppose that I have any PyLongObject object (even internal / shared ones)
and that
- I need to change some or all of its internal values (size, sign, digits)
in a "critical section";
- the critical section is C code only;
- no CPython APIs will be called (all work is on local vars and object's
internal values);
- the object internal structure is completely restored before exiting from
the critical section.

Is it thread / interpreter safe or something dirty can happen?

Cesare
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Terry Reedy]

On 3/20/2011 3:22 AM, Glenn Linderman wrote:
On 3/19/2011 7:38 PM, Mark Hammond wrote:
[snip]

As both a writer and reader, I would like to just add, for instance,

#! python3

(or 3.3 or whatever) and have the launcher do the 'right thing'.
It seems to me that that really should be enough info for *nix,
but given that it is not, a program to add the required extra noise  for 
a particular distribution to the first line of every module in a package 
should not be too difficult.


The above would be easy for anyone to write and understand.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

> Out of curiosity, what is your objection to having the child process?  I
> guess it must be the system resources needed for the parent process
> while it waits for the child to terminate so the exit code can be
> reflected correctly "up stream"?

I see three problems with creating child processes:
- WaitForSingleObject and GetExitProcessCode must work correctly. I
  think this is possible to achieve
- applications using the debug API, PSAPI, etc. will be confused if
  the action all happens in a child process. I can accept that they
  have to adjust, though.
- TerminateProcess must work correctly, and I see no way to achieve
  that (i.e. terminating the parent process won't terminate the child)

> I would have 3 main problems with dynamically loading Python into the
> launcher process:
> 
> * It may be difficult or impossible to do for implementations other than
> CPython.

That wouldn't bother me.

> * There are subtle differences when Python is loaded by an executable in
> the "install" directory versus by an executable that is not.
> Specifically, I'm thinking about the differences in how the default
> sys.path is populated.

This would need to be fixed. I.e. the launcher would have to arrange it
so it gives the illusion of having been started from the executable.

> * The registry doesn't currently tell us exactly where pythonxx.dll can
> be located, but does tell is where python.exe is.  Thus things would get
> a little more complex as we go snooping for the location of the DLL.

The launcher can know how all released Python versions behaved, so it
can adjust.

> Actually, it does say '...the "console" version of the launcher should
> be associated with .py files and the "windows" version associated with
> .pyw files.'

Ah, ok, I missed that.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

>>> Does it also support loading a different interpreter version than the
>>> one it is running?
>>
>> Ouch, no. I guess you're right, any Python based solution will need to
>> create a child process.
> 
> Why would that be true? Shouldn't this launcher just be a basic wrapper
> that cobbles together the arguments for an eventual os.exec*() call?

Windows doesn't support exec*(3).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyLongObject "safe" changes?

[Martin v. Löwis]

> Is it thread / interpreter safe or something dirty can happen?

It is safe, thanks to the global interpreter lock (a.k.a. GIL).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 20/03/2011 8:36 PM, Terry Reedy wrote:

On 3/20/2011 3:22 AM, Glenn Linderman wrote:
On 3/19/2011 7:38 PM, Mark Hammond wrote:
[snip]

As both a writer and reader, I would like to just add, for instance,

#! python3

(or 3.3 or whatever) and have the launcher do the 'right thing'.


If there was general agreement, I'd be happy for the PEP to support that 
variation.


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 20/03/2011 8:37 PM, "Martin v. Löwis" wrote:
...

... problems with creating child processes:
- applications using the debug API, PSAPI, etc. will be confused if
   the action all happens in a child process. I can accept that they
   have to adjust, though.


Some of these uses probably shouldn't use the launcher directly - eg, 
ISAPI apps and COM objects which have a separate registration step could 
register a specific python.exe - the installation script itself could by 
launched by the launcher, but that script could continue to use 
sys.executable to record the specific python.exe.


A side effect of this though is that if a newer version of python is 
installed such that "py.exe register.py" now used the later version, the 
existing registration would still use the older version until 
re-registered.  I think that is reasonable though.



- TerminateProcess must work correctly, and I see no way to achieve
   that (i.e. terminating the parent process won't terminate the child)


I understand this concern (and meant to say this in reply to Eugene's 
mail), but wonder if it isn't similar to the above - that if some 
program wants to launch python and note the id of the child process so 
it can terminate the child later, then python.exe could continue to be 
used.  OTOH, if you are terminating the process by using Process Manager 
or something else to find a 'python.exe', that will continue to work and 
the parent py.exe will automatically silently terminate.



That wouldn't bother me.


* There are subtle differences when Python is loaded by an executable in
the "install" directory versus by an executable that is not.
Specifically, I'm thinking about the differences in how the default
sys.path is populated.


This would need to be fixed. I.e. the launcher would have to arrange it
so it gives the illusion of having been started from the executable.


It isn't immediately obvious how that could be done though - although I 
admit I haven't looked into it.


But in general, I'd be happy for py.exe to avoid CreateProcess for 
CPython if that could be reasonably done and was the consensus...


Thanks,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Antoine Pitrou]

Le dimanche 20 mars 2011 à 00:06 -0400, R. David Murray a écrit :
> On Thu, 17 Mar 2011 14:33:00 +0100,  wrote:
> > On Thu, 17 Mar 2011 09:24:26 -0400
> > "R. David Murray"  wrote:
> > > 
> > > It would be great if rebase did work with share, that would make a
> > > push race basically a non-issue for me.
> > 
> > rebase as well as strip destroy some history, meaning some of your
> > shared clones may end up having their working copy based on a
> > non-existent changeset. I'm not sure why rebase would be worse that
> > strip in that regard, though.
> 
> Well, it turns out that this completely doesn't work, though at first
> it appeared to (and so I pushed).
> 
> I had a push race, so I did hg pull; hg rebase.

Have you tried hg pull --rebase instead?
If you "hg rebase" by hand, I think you have to tell it the right set of
revisions to rebase.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Ben Finney]

Terry Reedy  writes:

> As both a writer and reader, I would like to just add, for instance,
>
> #! python3
>
> (or 3.3 or whatever) and have the launcher do the 'right thing'. It
> seems to me that that really should be enough info for *nix,

For clarity: the reason it's not enough information is because the
kernel doesn't go hunting for the interpreter. The shebang line needs to
specify the full filesystem path of the interpreter program.

-- 
 \ “The Vatican is not a state.… a state must have territory. This |
  `\ is a palace with gardens, about as big as an average golf |
_o__) course.” —Geoffrey Robertson, 2010-09-18 |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Stefan Behnel]

Nick Coghlan, 12.03.2011 12:43:

I posted my rough notes and additional write-ups for Wednesday's VM
summit and Thursday's language summit:

http://www.boredomandlaziness.org/2011/03/python-vm-summit-rough-notes.html
http://www.boredomandlaziness.org/2011/03/python-vm-summit-somewhat-coherent.html
http://www.boredomandlaziness.org/2011/03/python-language-summit-rough-notes.html
http://www.boredomandlaziness.org/2011/03/python-language-summit-highlights.html


It appears that there has been little mention of Cython at the summit, 
despite of the speed of CPython being a major topic, according to the 
notes. I can see several areas where Cython could help in speeding up core 
CPython, one of the most obvious being compilation of standard library 
modules, and JIT-like compilation of Python modules at import time.


Nick mentioned that there was a discussion of making C-only modules 
available as alternative Python implementations. I'd like to suggest the 
opposite as well: make pure Python stdlib modules optionally compilable to 
fast C code using Cython. The obvious advantages are a) speed and b) a 
single code base for both Python and C modules.


Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Jesse Noller]

On Sun, Mar 20, 2011 at 7:19 AM, Stefan Behnel  wrote:
> Nick Coghlan, 12.03.2011 12:43:
>>
>> I posted my rough notes and additional write-ups for Wednesday's VM
>> summit and Thursday's language summit:
>>
>>
>> http://www.boredomandlaziness.org/2011/03/python-vm-summit-rough-notes.html
>>
>> http://www.boredomandlaziness.org/2011/03/python-vm-summit-somewhat-coherent.html
>>
>> http://www.boredomandlaziness.org/2011/03/python-language-summit-rough-notes.html
>>
>> http://www.boredomandlaziness.org/2011/03/python-language-summit-highlights.html
>
> It appears that there has been little mention of Cython at the summit,
> despite of the speed of CPython being a major topic, according to the notes.
> I can see several areas where Cython could help in speeding up core CPython,
> one of the most obvious being compilation of standard library modules, and
> JIT-like compilation of Python modules at import time.
>
> Nick mentioned that there was a discussion of making C-only modules
> available as alternative Python implementations. I'd like to suggest the
> opposite as well: make pure Python stdlib modules optionally compilable to
> fast C code using Cython. The obvious advantages are a) speed and b) a
> single code base for both Python and C modules.
>
> Stefan

The reason why there was no mention is probably because no one
intimately familiar with Cython was there, and if they were - it was
not brought up. If Cython supports PyPy - and Jython, and IronPython,
your proposal makes sense. The reason for "pure" python implementation
is so that other implementations can share the exact same standard
library we have today. I don't see adding a cython version as helping
that.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Antoine Pitrou]

On Sun, 20 Mar 2011 07:32:34 -0400
Jesse Noller  wrote:
> 
> The reason why there was no mention is probably because no one
> intimately familiar with Cython was there, and if they were - it was
> not brought up. If Cython supports PyPy - and Jython, and IronPython,
> your proposal makes sense. The reason for "pure" python implementation
> is so that other implementations can share the exact same standard
> library we have today.

Well, realistically, they don't. Some functionality just isn't
satisfied with a slow Python implementation. The io module is a primary
example of that.
(but I don't think a Cython version of io would be fast enough, either)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Stefan Behnel]

Jesse Noller, 20.03.2011 12:32:

On Sun, Mar 20, 2011 at 7:19 AM, Stefan Behnel wrote:

It appears that there has been little mention of Cython at the summit,
despite of the speed of CPython being a major topic, according to the notes.
I can see several areas where Cython could help in speeding up core CPython,
one of the most obvious being compilation of standard library modules, and
JIT-like compilation of Python modules at import time.

Nick mentioned that there was a discussion of making C-only modules
available as alternative Python implementations. I'd like to suggest the
opposite as well: make pure Python stdlib modules optionally compilable to
fast C code using Cython. The obvious advantages are a) speed and b) a
single code base for both Python and C modules.


The reason why there was no mention is probably because no one
intimately familiar with Cython was there, and if they were - it was
not brought up. If Cython supports PyPy - and Jython, and IronPython,
your proposal makes sense. The reason for "pure" python implementation
is so that other implementations can share the exact same standard
library we have today. I don't see adding a cython version as helping
that.


Cython is meant to compile Python code. A "cython version" would just be a 
pure Python module, usable with all other implementations, but with type 
annotations that make it compile to more optimal C code. Type annotations 
can be provided in an external file (.pxd), using decorators and/or using 
Python 3 annotation syntax. There's also a port of Cython to IronPython 
being written. Additionally, other implementations (such as PyPy) could 
start using the available type declarations in order to improve their 
optimisation capabilities as well.


CPython, specifically, can also continue to use a multi-step implementation 
as it's currently being done, i.e. split the implementation into a pure 
Python module with optional compilation, a Cython-specific partial 
reimplementation in optimised only-mostly-Python code, and potentially a C 
module for the external code that must really be implemented in pure C for 
whatever reason. That would already lower the entry level for non-C-affine 
contributors by moving more code into plain Python and (mostly Python) 
Cython code, instead of requiring proficiency in both C and CPython's C-API 
in order to work on the code at all.


Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Stefan Behnel]

Antoine Pitrou, 20.03.2011 12:40:

On Sun, 20 Mar 2011 07:32:34 -0400 Jesse Noller wrote:


The reason why there was no mention is probably because no one
intimately familiar with Cython was there, and if they were - it was
not brought up. If Cython supports PyPy - and Jython, and IronPython,
your proposal makes sense. The reason for "pure" python implementation
is so that other implementations can share the exact same standard
library we have today.


Well, realistically, they don't. Some functionality just isn't
satisfied with a slow Python implementation. The io module is a primary
example of that.


Agreed.



(but I don't think a Cython version of io would be fast enough, either)


I assume you mean a compiled version of the pure Python implementation? No, 
that likely won't be faster and would certainly be much slower than the 
current C implementation. There's not much Cython can optimise in Python 
code that mostly does Python-level I/O operations.


However, I'd argue that if the C implementation of the "io" module had been 
written in Cython code instead of C, it would have ended up being about as 
fast (or maybe slightly faster due to the reduced calling overhead in 
Cython), but substantially shorter, more readable and accessible for Python 
users and consequently easier to maintain. Likely also easier to write in 
the first place, as most of the code had already been written for the pure 
Python version, which makes it a "copy&optimise" rather than a "read&rewrite".


Maybe that would make a good topic for the GSoC: rewrite some of the C 
modules in CPython in Cython, either in pure Python with Cython type 
annotations, or in Cython code, depending on the type of module and its 
speed requirements. Rewriting C code in Cython usually goes quite quickly, 
so the outcome would likely be several modules for one GSoC participant.


IMHO, taking modules that currently only have a C implementation due to 
performance constraints and rewriting them in Cython is a much more 
worthwhile thing to do than adding an alternative pure Python 
implementation that other Python runtimes wouldn't use anyway. And at least 
IronPython could soon benefit directly from a Cython implementation as well.


Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Jesse Noller]

On Sun, Mar 20, 2011 at 7:40 AM, Antoine Pitrou  wrote:
> On Sun, 20 Mar 2011 07:32:34 -0400
> Jesse Noller  wrote:
>>
>> The reason why there was no mention is probably because no one
>> intimately familiar with Cython was there, and if they were - it was
>> not brought up. If Cython supports PyPy - and Jython, and IronPython,
>> your proposal makes sense. The reason for "pure" python implementation
>> is so that other implementations can share the exact same standard
>> library we have today.
>
> Well, realistically, they don't. Some functionality just isn't
> satisfied with a slow Python implementation. The io module is a primary
> example of that.
> (but I don't think a Cython version of io would be fast enough, either)
>
> Regards
>
> Antoine.

The intention (and pep that is in progress) is to break out the
standard library so that it *is* shared, and used by all of the
implementations. We should not go out of our way to make this
difficult, and provide pure python implementation whenever we can.
This was universally agreed to at the summit at least, and I believe
Brett Cannon is going to work on a pep that solidifies it.

The other implementations (PyPy, in particular) have run into serious
issues with the optimized C versions of modules being placed into the
core, and the Python fallbacks being torn out.

jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Jesse Noller]

...snip

> IMHO, taking modules that currently only have a C implementation due to
> performance constraints and rewriting them in Cython is a much more
> worthwhile thing to do than adding an alternative pure Python implementation
> that other Python runtimes wouldn't use anyway. And at least IronPython
> could soon benefit directly from a Cython implementation as well.
>
> Stefan

The other python implementation expressed serious interest, and are
willing to help support a shared standard library, and shared modules.
So saying they "won't use them anyway" may apply to things such as the
io module, but is far from the truth for the entire standard library.

You're also asking that we depend on Cython within core, which while
it has it's benefits, I think warrants a PEP and a working
implementation to show the potential speedups you're talking about
before we can agree to include it/depend on it.

jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Dj Gilcrease]

On Sun, Mar 20, 2011 at 5:37 AM, "Martin v. Löwis"  wrote:
> I see three problems with creating child processes:
> - WaitForSingleObject and GetExitProcessCode must work correctly. I
>  think this is possible to achieve
> - applications using the debug API, PSAPI, etc. will be confused if
>  the action all happens in a child process. I can accept that they
>  have to adjust, though.
> - TerminateProcess must work correctly, and I see no way to achieve
>  that (i.e. terminating the parent process won't terminate the child)

Before Mark wrote up this pep I had started experimenting with how to
make the launcher and I was able to get it to launch python while
exiting py.exe and as far as I could tell it behaved just as if I had
launched the app directly by double clicking it.

http://pastebin.com/vxcrC8Tn

I tested it with long running scripts and a full application (openrpg)
that has many external dependencies and in both cases py.exe exited
and the script/app ran as expected leaving the python console open
until the script/app exited on it's own. I did not test adding
debugger breakpoint in the code though to see how it behaved though.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 19/03/11 19:59, Raymond Hettinger wrote:
> I think if we're going to require a complex workflow, then we're
> going to have to expect a lot of questions.  And those questions
> shouldn't be brushed-off with "go read the tutorial, we have no time
> for you" or words to that effect.

I think we are doing some antipatterns with our current approach,
battling the tools instead of "joining them".

Would be nice to document a complete case workflow, from patch thinking
to the final push, dealing with all the little details, collapsing
outgoing commits, "+1 head", merging real conflicts, linealizing
history, moving the patch thru branches, etc.

But the fact is that some of us are still experimenting.

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBTYX64Jlgi5GaxT1NAQLaXwQAhtZyuIZ1wYXx7yS41+bzIpZHEMbj3uCN
iA/OzE1cAY55/opNHqIu1OAONKD/Y4kAm6cU/Qs2prRL2OsMwml1zPsG2Aono2iN
SIqAN1YS7fagOEcDeKmPo6LbHTBpq9wuv53LBiW7xM1/ut2e5ou9DMfpf5bK6xP6
bbR4ESTGzyU=
=NNNV
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Stefan Behnel]

Jesse Noller, 20.03.2011 13:51:

...snip


IMHO, taking modules that currently only have a C implementation due to
performance constraints and rewriting them in Cython is a much more
worthwhile thing to do than adding an alternative pure Python implementation
that other Python runtimes wouldn't use anyway. And at least IronPython
could soon benefit directly from a Cython implementation as well.

Stefan


The other python implementation expressed serious interest, and are
willing to help support a shared standard library, and shared modules.
So saying they "won't use them anyway" may apply to things such as the
io module, but is far from the truth for the entire standard library.


I guess someone would have to look through the stdlib and make a list of 
suitable candidates for Cython compilation and/or Python/Cython/C 
reimplementations at this point.




You're also asking that we depend on Cython within core


It's a substantial dependency, sure. The question is: what's more work, 
having to install a specific version of Cython in order to work on CPython, 
or having to get fluent in C + C-API first?


I like the way the Jython people put it, slightly adapted into "We write C 
so you don't have to".




which while
it has it's benefits, I think warrants a PEP and a working
implementation to show the potential speedups you're talking about
before we can agree to include it/depend on it.


We will have a Cython core developers workshop next weekend. Maybe we can 
find a bit of time there to run the then-merged-together-bleeding-edge 
Cython over the standard library and see what that gives.


It's not easy to find good benchmarks, though. Most of what the 
PyPy/Unladen developers settled on so far isn't very interesting for the 
way Cython works. It's usually a bit of work to make the code substantially 
faster by providing enough type annotations to let the compiler drop 
critical Python code into C. We did that for a couple of benchmarks, but 
lost interest as there wasn't much to win - all we could show was that, by 
changing the code enough to make it violate the usual constraints of a fair 
benchmark comparison, you could make it run as fast as C code without 
writing C code. Not much news to us and nothing that would integrate in any 
acceptable way into speed.pypy.org.


If anyone knows about a good benchmark for a currently pure Python standard 
library module, preferably a smaller, self-contained one that's somewhat 
computationally intensive, I'd be happy to hear about it.


Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Jesse Noller]

On Sun, Mar 20, 2011 at 9:39 AM, Stefan Behnel  wrote:
> Jesse Noller, 20.03.2011 13:51:
>>
>> ...snip
>>
>>> IMHO, taking modules that currently only have a C implementation due to
>>> performance constraints and rewriting them in Cython is a much more
>>> worthwhile thing to do than adding an alternative pure Python
>>> implementation
>>> that other Python runtimes wouldn't use anyway. And at least IronPython
>>> could soon benefit directly from a Cython implementation as well.
>>>
>>> Stefan
>>
>> The other python implementation expressed serious interest, and are
>> willing to help support a shared standard library, and shared modules.
>> So saying they "won't use them anyway" may apply to things such as the
>> io module, but is far from the truth for the entire standard library.
>
> I guess someone would have to look through the stdlib and make a list of
> suitable candidates for Cython compilation and/or Python/Cython/C
> reimplementations at this point.
>
>
>> You're also asking that we depend on Cython within core
>
> It's a substantial dependency, sure. The question is: what's more work,
> having to install a specific version of Cython in order to work on CPython,
> or having to get fluent in C + C-API first?
>
> I like the way the Jython people put it, slightly adapted into "We write C
> so you don't have to".
>
>
>> which while
>> it has it's benefits, I think warrants a PEP and a working
>> implementation to show the potential speedups you're talking about
>> before we can agree to include it/depend on it.
>
> We will have a Cython core developers workshop next weekend. Maybe we can
> find a bit of time there to run the then-merged-together-bleeding-edge
> Cython over the standard library and see what that gives.
>
> It's not easy to find good benchmarks, though. Most of what the PyPy/Unladen
> developers settled on so far isn't very interesting for the way Cython
> works. It's usually a bit of work to make the code substantially faster by
> providing enough type annotations to let the compiler drop critical Python
> code into C. We did that for a couple of benchmarks, but lost interest as
> there wasn't much to win - all we could show was that, by changing the code
> enough to make it violate the usual constraints of a fair benchmark
> comparison, you could make it run as fast as C code without writing C code.
> Not much news to us and nothing that would integrate in any acceptable way
> into speed.pypy.org.
>
> If anyone knows about a good benchmark for a currently pure Python standard
> library module, preferably a smaller, self-contained one that's somewhat
> computationally intensive, I'd be happy to hear about it.
>
> Stefan

The nice thing is about those benchmarks - from PyPy and Unladen
Swallow's work - is that they do reflect real world applications, and
tend away from pure numerical micro benchmarks (how fast you can do
fibonacci is almost worthless). So if we want to add more to that
suite of benchmarks, great - but those will be some of the benchmarks
against which speed improvements would be measured against.

There's also a plan (coming out of the vm summit/language summit) to
turn speed.pypy.org into a shared "performance.python.org" resource,
and showcase the speed of CPython, Jython, IronPython and PyPy side by
side, using common benchmarks. I am currently working on acquiring the
needed resources from the PSF side to get us to the point where the
real work can begin.

If you can show that Cython lowers the barrier to writing fast(er)
Python extensions, without dropping into pure C, and we don't cripple
the ability for the other implementation to share the code, then by
all means, I personally recommend a PEP.

jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Antoine Pitrou]

On Sun, 20 Mar 2011 14:39:20 +0100
Stefan Behnel  wrote:
> 
> If anyone knows about a good benchmark for a currently pure Python standard 
> library module, preferably a smaller, self-contained one that's somewhat 
> computationally intensive, I'd be happy to hear about it.

You can take a look at difflib, sre_compile, weak containers
(WeakrefSet, WeakValueDict, WeakKeyDict).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Paul Moore]

On 20 March 2011 09:58, Mark Hammond  wrote:
> On 20/03/2011 8:36 PM, Terry Reedy wrote:
>>
>> On 3/20/2011 3:22 AM, Glenn Linderman wrote:
>> On 3/19/2011 7:38 PM, Mark Hammond wrote:
>> [snip]
>>
>> As both a writer and reader, I would like to just add, for instance,
>>
>> #! python3
>>
>> (or 3.3 or whatever) and have the launcher do the 'right thing'.
>
> If there was general agreement, I'd be happy for the PEP to support that
> variation.

I would probably use this. I'd also be happy with Unix-style
#!/usr/bin/python3. I would not use a Windows-only line at all.

Anyone advocating including full paths in a Windows-specific line
(can't remember who it was, sorry - Glen?) should remember that unlike
Unix, there is NO standard location for the Python interpreter on
Windows. The installer's default is only that - a default. (And TBH,
in my view, not that good a default, merely the best of a series of
bad options...) My Python installs are usually in \Apps\Pythonxy on
some drive - which one depends on how my drives are laid out and how
much free space I have. So an absolute path wouldn't even be portable
across all of my own PCs! Personally, I'd rather not have such a line
at all (and find another approach) than use an absolute path...

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[R. David Murray]

On Sun, 20 Mar 2011 12:17:25 +0100, Antoine Pitrou  wrote:
> Le dimanche 20 mars 2011 à 00:06 -0400, R. David Murray a écrit :
> > On Thu, 17 Mar 2011 14:33:00 +0100,  wrote:
> > > On Thu, 17 Mar 2011 09:24:26 -0400
> > > "R. David Murray"  wrote:
> > > >
> > > > It would be great if rebase did work with share, that would make a
> > > > push race basically a non-issue for me.
> > >
> > > rebase as well as strip destroy some history, meaning some of your
> > > shared clones may end up having their working copy based on a
> > > non-existent changeset. I'm not sure why rebase would be worse that
> > > strip in that regard, though.
> >
> > Well, it turns out that this completely doesn't work, though at first
> > it appeared to (and so I pushed).
> >
> > I had a push race, so I did hg pull; hg rebase.
> 
> Have you tried hg pull --rebase instead?
> If you "hg rebase" by hand, I think you have to tell it the right set of
> revisions to rebase.

That wouldn't help me, since my goal is to rebase merges between
branches.  I think Mercurial just doesn't support that.

I think I'm going to have to give in and do "merge heads" when
someone else has pushed since my last pull.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[R. David Murray]

On Sun, 20 Mar 2011 00:06:01 -0400, "R. David Murray"  
wrote:
> It would have been nice if rebase had refused to run given that
> there were merges, since it clearly doesn't work in that case.

To clarify, that should have read "given that there were merges
*between branches*".  Cleaning merges on the same branch is
the goal of rebase, after all :)

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Antoine Pitrou]

Le dimanche 20 mars 2011 à 11:52 -0400, R. David Murray a écrit :
> On Sun, 20 Mar 2011 00:06:01 -0400, "R. David Murray"  
> wrote:
> > It would have been nice if rebase had refused to run given that
> > there were merges, since it clearly doesn't work in that case.
> 
> To clarify, that should have read "given that there were merges
> *between branches*".  Cleaning merges on the same branch is
> the goal of rebase, after all :)

I think that would be the kind of issue working bringing on the
Mercurial mailing-list.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Guido van Rossum]

Hi Stefan,

I'm glad to see Cython picking up steam and trying to compete with
CPython, PyPy, and possibly others. It's true that few in the core
development group know much about Cython -- essentially my own
understanding is still that it's like Pyrex, which was a
mostly-Python-compatible syntax for writing C extensions, but
certainly not able to compile all existing Python code successfully,
due to small syntactic incompatibilities (e.g. new reserved words). I
also thought that Cython was mostly popular is scientific computing
circles. But I may well be wrong on all counts. Please enlighten me.

I'd like to hear more about Cython's compatibility -- e.g. does it
compile Django? If so, is it any faster? What versions of Python does
it currently support? How does it do on the benchmark suite used by
PyPy (originated with Unladen Swallow)? IMO it's up to Cython to prove
its worth.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Guido van Rossum]

What is "rebase"? Why does everyone want it and hate it at the same time?


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Would someone please help me merge these changes to other branches?

[R. David Murray]

On Sun, 20 Mar 2011 10:10:02 -0500, s...@pobox.com wrote:
> > I pushed two changes to cpython yesterday and today.  I don't know how to
> > merge them to the 2.7 and 3.2 branches, and don't want to even try for
> > fear of screwing something up.  If what I read in python-dev yesterday is
> > correct, I did things incorrectly by starting with cpython instead of 2.7
> > or 3.2.
> 
> Actually, the newline= change only needs to go to 3.2.  The RFC 4180 change
> can also go to 2.7.

I'd already started on this fix, so I incorporated your changes (except
for the RFC 4180 one) and made them on 3.1, 3.2, and harmonized default.
So this should all be good now, except for the RFC change.  I guess I'll
see about doing that one later.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Georg Brandl]

On 20.03.2011 16:21, Guido van Rossum wrote:
> What is "rebase"? Why does everyone want it and hate it at the same time?

Basically, rebase is a way to avoid having pointless merge commits on the
same branch.

Let's say you have the following history in your local repository:

... --- X --- A --- B

where X is the last commit that is in the remote repository, and
A and B are commits you made locally.  Now you pull from the remote,
and see that others have pushed more history in the meantime.  The
graph now looks this:

  A --- B
 /
... --- X --- C --- D --- E

To get your history pushed, you need to make a merge commit:

  A --- B .
 / \
... --- X --- C --- D --- E --- M

Then you can push A, B and M to the remote.

Now, "hg pull --rebase" prevents that by "re-basing" the A-B history
line onto the latest remote head.  After rebasing, the history looks
like this:

... --- X --- C --- D --- E --- A' --- B'

without the need to merge before the push.

Obviously, if the merge above gave conflicts, the rebasing process
will also give conflicts.  In both cases you need to solve them
before continuing.

The reason why rebasing is not universally applied is that the
rebased changesets are different from the original ones (therefore
I wrote A' and B') -- even if the diff is the same, the parents
are not, and therefore the changeset id (hash) changes.  This is
called "changing history", and frowned upon by purists.  In reality
it works fine if you know the limits: rebasing really only should be
applied if the changesets are not already known somewhere else,
only in the local repo you're working with.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Georg Brandl]

On 20.03.2011 16:39, Georg Brandl wrote:

> The reason why rebasing is not universally applied is that the
> rebased changesets are different from the original ones (therefore
> I wrote A' and B') -- even if the diff is the same, the parents
> are not, and therefore the changeset id (hash) changes.  This is
> called "changing history", and frowned upon by purists.  In reality
> it works fine if you know the limits: rebasing really only should be
> applied if the changesets are not already known somewhere else,
> only in the local repo you're working with.

Ah, and it apparently also doesn't work in more complex situations
where you already merged your changes to another branch.  This is where
David got stuck, I guess.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[R. David Murray]

On Sun, 20 Mar 2011 16:39:50 -, Georg Brandl  wrote:
> The reason why rebasing is not universally applied is that the
> rebased changesets are different from the original ones (therefore
> I wrote A' and B') -- even if the diff is the same, the parents
> are not, and therefore the changeset id (hash) changes.  This is
> called "changing history", and frowned upon by purists.  In reality
> it works fine if you know the limits: rebasing really only should be
> applied if the changesets are not already known somewhere else,
> only in the local repo you're working with.

And, as I discovered, only if they are on a single branch.  Which is
something *none* of the documentation I've read has mentioned.  Perhaps
that's because named branches are relatively new?

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Thomas Wouters]

On Sun, Mar 20, 2011 at 17:39, Georg Brandl  wrote:

> The reason why rebasing is not universally applied is that the
> rebased changesets are different from the original ones (therefore
> I wrote A' and B') -- even if the diff is the same, the parents
> are not, and therefore the changeset id (hash) changes.  This is
> called "changing history", and frowned upon by purists.  In reality
> it works fine if you know the limits:


It's frowned upon by more than just purists, and it works "in reality" as
fine as handing out your creditcard and personal information over the
internet; you can't always tell the result is bad, and it can be very
painful finding out.

Jaywalking-and-unprotected-sex-metaphors-available-on-demand'ly y'rs,
-- 
Thomas Wouters 

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Georg Brandl]

On 20.03.2011 16:50, Thomas Wouters wrote:
> 
> 
> On Sun, Mar 20, 2011 at 17:39, Georg Brandl  > wrote:
> 
> The reason why rebasing is not universally applied is that the
> rebased changesets are different from the original ones (therefore
> I wrote A' and B') -- even if the diff is the same, the parents
> are not, and therefore the changeset id (hash) changes.  This is
> called "changing history", and frowned upon by purists.  In reality
> it works fine if you know the limits:
> 
> 
> It's frowned upon by more than just purists, and it works "in reality" as fine
> as handing out your creditcard and personal information over the internet; you
> can't always tell the result is bad, and it can be very painful finding out.

Well, YMMV.  But instead of spreading FUD you might want to state *why*.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Terry Reedy]

On 3/20/2011 10:51 AM, Antoine Pitrou wrote:

On Sun, 20 Mar 2011 14:39:20 +0100
Stefan Behnel  wrote:


If anyone knows about a good benchmark for a currently pure Python standard
library module, preferably a smaller, self-contained one that's somewhat
computationally intensive, I'd be happy to hear about it.


You can take a look at difflib,


I think this is a good example. difflib.SequenceMatcher is linear to 
quadratic (says Tim) in the number of items and can take seconds or 
longer in real-life-sized use cases. It is all indexing, comparison, 
looping, and logic, so I think it would be an interesting test case. 
There is no C version now, so a sped-up version could be immediately 
useful to some people.



sre_compile, weak containers
(WeakrefSet, WeakValueDict, WeakKeyDict).


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Thomas Wouters]

On Sun, Mar 20, 2011 at 17:59, Georg Brandl  wrote:

> On 20.03.2011 16:50, Thomas Wouters wrote:
> >
> >
> > On Sun, Mar 20, 2011 at 17:39, Georg Brandl  > > wrote:
> >
> > The reason why rebasing is not universally applied is that the
> > rebased changesets are different from the original ones (therefore
> > I wrote A' and B') -- even if the diff is the same, the parents
> > are not, and therefore the changeset id (hash) changes.  This is
> > called "changing history", and frowned upon by purists.  In reality
> > it works fine if you know the limits:
> >
> >
> > It's frowned upon by more than just purists, and it works "in reality" as
> fine
> > as handing out your creditcard and personal information over the
> internet; you
> > can't always tell the result is bad, and it can be very painful finding
> out.
>
> Well, YMMV.  But instead of spreading FUD you might want to state *why*.


I could also ask you what makes you think changing history is frowned upon
by *just purists*, or why on earth anyone would think it's a good idea
regardless of how practical or theoretical they are.

As for my dislike, your reason (changing history) and Davids apparent
problem seem obvious enough. Aren't they? If you don't rebase correctly,
stuff goes wrong. Doing it correctly is harder than people seem to think.
You can see what's going on with 'hg outgoing -p', but I, as an observer,
have no way of knowing whether you used that and were aware of what exactly
you were pushing.

Merges can also contain useful information about how *future* merges of the
same changes should be handled. This doesn't matter much to changes with
short turnaround (or changes that are continuously reverted and re-applied)
but can make things a lot simpler if you have lots of diverging work in
multiple branches -- and all the getting used to Mercurial aside, I'm sure
that is what we're actually doing it for.

Even when done correctly, in situations where merges are automatic and do
not involve the changed code in any way, I don't like rebasing a single
changeset because it makes it less apparent when the actual work was
started. It's not any worse than diffing and re-applying a changeset, but
for example if your change modified a particular call pattern appearing in
multiples places, and another place started using the same pattern somewhere
in your merges that you made go away, did you intentionally forget to fix
the pattern or was it an accident? This isn't a problem that is *caused* by
rebasing, but it certainly isn't helped by it. Rebasing multiple changesets
is worse, because the intermediate changesets are a real lie rather than
some fudging of history; your repository was never in that state, that
changeset never looked that way and it was never built or tested that way.
No matter that it *usually* doesn't matter, or even *almost never* matters,
it *can* matter.

Merging and merge changesets are a fact of DVCSes, and while I (as a grumpy
luddite misanthrope) greatly prefer the automatic (and mostly silent) merge
as BitKeeper does it, in the long run the actual merging and the merge
changesets are unavoidable and something to get used to, not dodged around
(at least not at this cost.)
-- 
Thomas Wouters 

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Stefan Behnel]

Antoine Pitrou, 20.03.2011 15:51:

On Sun, 20 Mar 2011 14:39:20 +0100, Stefan Behnel wrote:


If anyone knows about a good benchmark for a currently pure Python standard
library module, preferably a smaller, self-contained one that's somewhat
computationally intensive, I'd be happy to hear about it.


You can take a look at difflib, sre_compile, weak containers
(WeakrefSet, WeakValueDict, WeakKeyDict).


Ok, I skipped through the difflib module and it looked like a reasonable 
candidate. So I gave its Py3.2 version a run and except for triggering one 
class scoping bug in Cython, it compiled nicely with our latest generators 
branch. Working around that bug was easy (renaming the affected variables) 
and allowed me to run a diff on a short text file (68 lines, Cython's own 
README.txt). For simplicity, and to actually get a diff, I compared the 
duplicated original lines to their sorted list. Here are the timings:


Python 3.2 (locally compiled with suitable CFLAGS, Linux 64bit):

$ PYTHONPATH=py python3 -m timeit \
 -s 'from difflib import ndiff; \
 t1= 2 * open("../README.txt").readlines(); t2=sorted(t1);' \
 'list(ndiff(t1, t2))'
100 loops, best of 3: 3.36 msec per loop

Cython:

$ PYTHONPATH=so python3 -m timeit \
-s 'from difflib import ndiff; \
t1= 2 * open("../README.txt").readlines(); t2=sorted(t1);' \
'list(ndiff(t1, t2))'
100 loops, best of 3: 2.08 msec per loop

That's more than a third faster, without any additional code optimisations 
or typing. It's not the first time I see Cython compiled plain Python code 
become about 30% faster, so I guess that's about the factor that 
computational code can expect before starting to add static types.


I did a bit of profiling and added a couple of type declarations to the 
top-2 methods, which grew the speed-up to about 50%, but there's quite some 
space left for improvements here, especially by overriding class 
definitions into extension types (which would speed up attribute access and 
method calls in Cython). Also, a couple of optimisationisms in the code 
prevent Cython from doing its own optimisions, e.g. there's critical code 
that takes off bound methods for "some_list.__contains__" or 
"some_dict.get". I didn't change those. They are faster in CPython, but 
Cython could otherwise translate them into the corresponding C-API calls 
directly.


I currently don't have psyco available to compare, but I think that would 
be interesting as well.


Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

> Before Mark wrote up this pep I had started experimenting with how to
> make the launcher and I was able to get it to launch python while
> exiting py.exe and as far as I could tell it behaved just as if I had
> launched the app directly by double clicking it.

In this code, py.exe terminates way before the actual Python script
terminates, right?

So if somebody would launch a python script with py.exe, they would
think it was completed even though it would still be running.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Avoiding whitespace conflicts (was: I am now lost - committed, pulled, merged, what is "collapse"?)

[skip]

Ben> Ned Deily  writes:
>> As a side note, if you are prone to accidentally adding extra
>> whitespace (like I am), you can add the whitespace check hook into
>> your local copy of Mercurial so that you will be warned about
>> whitespace problems immediately when you commit things to your local
>> repos rather than later when you try to push them up to the
>> python.org repo.

Ben> Even earlier that the attempt to commit, one can configure the text
Ben> editor to show whitespace in order to be aware of it at the point
Ben> where it's easiest to change:

...

I wasn't aware that TABs were verboten in .rst files.  If svn was configured
to reject TABs I simply never stumbled on it with my edits.  I did some
searching but found nothing in the dev guide to suggest there were any
restrictions on the whitespace in .rst files.  Now that I know, it's not
difficult to avoid.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Avoiding whitespace conflicts

[Georg Brandl]

On 20.03.2011 18:05, s...@pobox.com wrote:
> 
> Ben> Ned Deily  writes:
> >> As a side note, if you are prone to accidentally adding extra
> >> whitespace (like I am), you can add the whitespace check hook into
> >> your local copy of Mercurial so that you will be warned about
> >> whitespace problems immediately when you commit things to your local
> >> repos rather than later when you try to push them up to the
> >> python.org repo.
> 
> Ben> Even earlier that the attempt to commit, one can configure the text
> Ben> editor to show whitespace in order to be aware of it at the point
> Ben> where it's easiest to change:
> 
> ...
> 
> I wasn't aware that TABs were verboten in .rst files.  If svn was configured
> to reject TABs I simply never stumbled on it with my edits.  I did some
> searching but found nothing in the dev guide to suggest there were any
> restrictions on the whitespace in .rst files.  Now that I know, it's not
> difficult to avoid.

I probably thought it would be obvious, since reST relies on structural
indentation just like Python.  I've now added this in "Documenting Python".

cheers,
Georg


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Michael Foord]

On 20/03/2011 09:58, Mark Hammond wrote:

On 20/03/2011 8:36 PM, Terry Reedy wrote:

On 3/20/2011 3:22 AM, Glenn Linderman wrote:
On 3/19/2011 7:38 PM, Mark Hammond wrote:
[snip]

As both a writer and reader, I would like to just add, for instance,

#! python3

(or 3.3 or whatever) and have the launcher do the 'right thing'.


If there was general agreement, I'd be happy for the PEP to support 
that variation.




Yes please. :-)

Michael


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[John Arbash Meinel]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 3/20/2011 5:06 AM, R. David Murray wrote:
> On Thu, 17 Mar 2011 14:33:00 +0100,  wrote:
>> On Thu, 17 Mar 2011 09:24:26 -0400
>> "R. David Murray"  wrote:
>>>
>>> It would be great if rebase did work with share, that would make a
>>> push race basically a non-issue for me.
>>
>> rebase as well as strip destroy some history, meaning some of your
>> shared clones may end up having their working copy based on a
>> non-existent changeset. I'm not sure why rebase would be worse that
>> strip in that regard, though.
> 
> Well, it turns out that this completely doesn't work, though at first
> it appeared to (and so I pushed).
> 
> I had a push race, so I did hg pull; hg rebase.  Then I looked at the
> log, and I could (apparently) see my change sets on the top of the
> stack.  So I pushed.  Victor then asked why one of my commits deleted
> Tools/demo/README, and then the next commit restored it.
> 
> What I was attempting to push was a doc change in 3.1 that I had then
> merged to 3.2 and default.  What I saw when looking closer at the log
> (after Victor pointed it out) was that my merge commits had lost their
> parents.
> 
> I thought that at worst a rebase would screw up my local history, but
> apparently I managed to push some sort of damaged history.  The doc
> change only got applied to default, since that's the branch I
> happened to be in when I did the rebase.
> 
> Needless to say, I'm avoiding rebase henceforth.

AIUI, rebase defaults to always omitting merge changesets. Under the
assumption that the branch you would merge is the one you are targeting
to rebase upon. So those merges are 'not interesting' once you are
rebased. Obviously this has failure cases (when the branch being merged
is not the one you are targeting.)

John
=:->

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2GSdMACgkQJdeBCYSNAAMf5wCgox24+LoJRKtzJHmCFTWcZnjI
MwIAniISqH9xDR/9g5EiXEsg5Wk66jeN
=39Oi
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[Terry Reedy]

What you felt like doing after doing the rest;-?

I believe your question and its answers have helped me understand hg 
better for when I dive in. Thanks.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[Stephen J. Turnbull]

Jesus Cea writes:

 > I think we are doing some antipatterns with our current approach,
 > battling the tools instead of "joining them".

Yes.  That is deliberate; see PEP 0374.  I admit I personally didn't
foresee the issues Nick describes with the flow of patches from one
branch to another.  Also, the "collapse" stage was completely
unexpected to me (and wasn't stated as a requirement for PEP 0374),
although in hindsight it seems natural for Python.

The problem with "joining" the tools is that is a euphemism for "RTFM,
OK?"  People who are already working with hg and similar tools in
their daily lives can probably make that transition in Python's
workflow pretty easily, but they are not going to be very good at
providing scripts for the DVCS newbies because they don't have them;
they Just Do It.  The DVCS newbies, OTOH, were told that the workflow
would change as little as possible, and we (the PEP 0374 authors) did
our best to make that possible.  Unfortunately, it hasn't worked as
well as hoped.  I, for one, am sorry about that.

I'm coming to the conclusion that those who say that Mercurial
documentation should be found at the Mercurial project are wrong.  I
think there's a reasonably strong case (based on the explicit promise
of PEP 0374 that workflows would change as little as possible) for a
follow-on informational PEP providing verbal, and maybe automated,
scripts for the various operations needed in the Python workflow.

OTOH, people who are having problems with the workflow imposed by
Mercurial need to recognize that Subversion basically ripped a big
hole in the QA aspect of Python's workflow.  As Nick points out,
Subversion merges create new versions in the repository that *never
existed in any developer's workspace* and therefore was never tested
before committing.  This is somewhat mitigated by buildbot testing,
but that is mostly unit testing and inherently is not very good at
catching problems due to interactions across modules.  That is, it's
not that Subversion provided a simpler way of doing the work.  Rather,
it hid the fact that certain work was not being done at all.  hg
exposes this fact.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[R. David Murray]

On Mon, 21 Mar 2011 03:44:34 +0900, "Stephen J. Turnbull"  
wrote:
> I'm coming to the conclusion that those who say that Mercurial
> documentation should be found at the Mercurial project are wrong.  I

+1.  I think the dev docs should explain anything needed to do the
basic Python workflow, even if that feels like repeating parts of the
Mercurial docs.  (If it is a large chunk, then a pointer to the exactly
relevant section of the Mercurial docs would be appropriate.)  On the
other hand, I'm not offering to do the work, so I'll happily accept
what has been produced so far (thanks, Antoine!).

> think there's a reasonably strong case (based on the explicit promise
> of PEP 0374 that workflows would change as little as possible) for a
> follow-on informational PEP providing verbal, and maybe automated,
> scripts for the various operations needed in the Python workflow.

Sounds good, but someone needs to write them.  You'll note that I've been
asking for this for a while, but as it turns out it wasn't really possible
to write them "correctly" until we started actually using the system.
I'm still not comfortable enough to write them myself, but after I've
used my chosen workflow for a while I'll do my best to write it up in
more detail than my earlier post as a blog post or something.

> OTOH, people who are having problems with the workflow imposed by
> Mercurial need to recognize that Subversion basically ripped a big
> hole in the QA aspect of Python's workflow.  As Nick points out,
> Subversion merges create new versions in the repository that *never
> existed in any developer's workspace* and therefore was never tested
> before committing.  This is somewhat mitigated by buildbot testing,
> but that is mostly unit testing and inherently is not very good at
> catching problems due to interactions across modules.  That is, it's
> not that Subversion provided a simpler way of doing the work.  Rather,
> it hid the fact that certain work was not being done at all.  hg
> exposes this fact.

Given that I generally did an svn up and a final regrtest run before
committing in SVN, the hole here for me was small (whatever changesets
came in after my up and before my commit).  Given that I do an hg pull/up
and a final regrtest run before starting my merge dance and am unlikely
to *re*run the regrtest after having to do an hg pull/merge heads,
exactly the same size hole is going to exist in my hg workflow.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[skip]

Stephen> Subversion merges create new versions in the repository that
Stephen> *never existed in any developer's workspace* and therefore was
Stephen> never tested before committing.  This is somewhat mitigated by
Stephen> buildbot testing, but that is mostly unit testing and
Stephen> inherently is not very good at catching problems due to
Stephen> interactions across modules.  That is, it's not that Subversion
Stephen> provided a simpler way of doing the work.  Rather, it hid the
Stephen> fact that certain work was not being done at all.  hg exposes
Stephen> this fact.

Can you provide an example of this (real or hypothetical)?  It seems to me
that if all you have is unit tests it matters little, if at all, whether you
catch test failures before a checkin (in your local hg repo) or after (using
svn + a buildbot run).  The workflow of the version control system isn't
going to magically expose semantic interactions across modules.  It seems to
me that at best the new workflow means there is a better chance of catching
textual conflicts before checkin than before.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[skip]

Terry> What you felt like doing after doing the rest;-?

I still f*cked everything up.  R. David Murray came to my rescue.  Even he
muffed it and had to backtrack I think, at least based on one of the
comments I saw on one of the involved tracker issues.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Stefan Behnel]

[warning, long post ahead]

Guido van Rossum, 20.03.2011 17:17:

Hi Stefan,


Hi!


I'm glad to see Cython picking up steam and trying to compete with
CPython, PyPy, and possibly others.


We do, although our main focus is much more on targeted manual optimisation 
rather than whole applications. For example, we currently only boot-strap 
seven out of some 60 modules of Cython itself into C code, as those are the 
most critical parts (mostly related to parsing and syntax tree traversal) 
and the rest simply takes too long to run through the C compiler for too 
little overall benefit. Compiling only the top-4 pure Python modules of the 
Cython compiler, optimised with externally provided type annotations, lets 
the overall compiler run about twice as fast.




It's true that few in the core
development group know much about Cython -- essentially my own
understanding is still that it's like Pyrex, which was a
mostly-Python-compatible syntax for writing C extensions, but
certainly not able to compile all existing Python code successfully,
due to small syntactic incompatibilities (e.g. new reserved words).


We decide the available syntax/keywords based on the file extension (.py vs 
.pyx), and you can compile (most) Python 3 code by putting a comment at the 
top of your source file that specifies the language level (or by passing 
"-3" on the command line).


A nice feature is that we support CPython 2.3 through 3.2 with the same 
generated C code file, so you can ship the quality assured generated C code 
and then put your trust into a C compiler to do the right thing for a given 
runtime on a given machine, without requiring normal users to have (the 
right version of) Cython installed on their side.


Cython always works at a per-module level, does a bit of function/closure 
local static type inference and optimistic optimisations and has a few 
intentional semantic differences compared to Python, such as mostly fixed 
builtins (unless re-assigned directly inside of a module, or deemed 
uninteresting, such as "open"). This allows the compiler to apply a huge 
set of optimisations to the syntax tree and the generated C code that 
substantially speed up the usage of builtin types as well as common idioms 
for looping and iteration.


We still can't compile "all existing Python code", though, as there are 
several known bugs regarding Python semantics. However, we recently managed 
to get pretty close to Python language feature completeness and hope to 
finish that up soon.


In short: if it works, it works, and it usually only gets better. ;-)



I also thought that Cython was mostly popular is scientific computing
circles. But I may well be wrong on all counts. Please enlighten me.


It does have a large and quickly growing user base in scientific computing. 
I would even say that Cython is growing into one of the main features that 
Python has in that field, right next to NumPy/SciPy, which are also 
starting to migrate parts of their code to Cython to improve its 
portability and maintainability. Cython is a great way to get numeric code 
up to speed without descending all the way down into a low-level language.


Cython is IMHO also the most advanced FFI that CPython has, much faster 
than any other wrapper tool and also more natural to use than ctypes. We 
natively support interaction with C and C++ code, and there's external 
support (fwrap) for Fortran as well, as that's extremely important in the 
high-performance computing area.


Finally, Cython is a programming language in its own right, that aims to 
close the not-so-small gap between Python and C/C++, both in terms of 
performance and usability. For example, we support many Python programming 
idioms on C data types, such as for-loop iteration over sliced pointers, 
and tightly adapt the code generated for a given language construct to the 
data types it operates on. While code can often be rewritten in a more 
C-ish way if performance truly demands it, we rather try to avoid that need 
by constantly improving and tailoring the generated code instead.


'nough sold? :-)

There are also Cython language semantics that we are still working on, such 
as support for some tricky C++ features. We generally try to follow Python 
semantics whenever possible, but need to accept in some cases that C/C++ 
have language semantics that we cannot hide from the user or that are 
actually helpful for users (yes, that really happens :). Advancing the 
language in these fields is pretty interesting business.




I'd like to hear more about Cython's compatibility -- e.g. does it
compile Django?


Never been that ambitious. As I said, Cython works at the module level, and 
you'd likely only compile some performance critical modules of an 
application anyway, in order to keep the overall code development more 
flexible.


A common approach is to profile an application, take the top-k modules and 
try to compile them in Cython. If they fail to compile, adapt the code

Re: [Python-Dev] Hg: inter-branch workflow

[Tim Delaney]

On 21 March 2011 04:32, Thomas Wouters  wrote:

>
> Merging and merge changesets are a fact of DVCSes, and while I (as a grumpy
> luddite misanthrope) greatly prefer the automatic (and mostly silent) merge
> as BitKeeper does it, in the long run the actual merging and the merge
> changesets are unavoidable and something to get used to, not dodged around
> (at least not at this cost.)
>

Agreed 100%. I really don't understand the antipathy to branching and
merging. I understand the desire to distribute a collapsed changeset - it's
changing history, but as long as you're absolutely sure that it's never been
shared to another repository, *even another local one* then you're OK.

As far as I can see, people here are fighting against the tools, and as a
result making life much much harder for themselves. Patch, Export/Import,
Mercurial Queues - they're all recipes for ending up with an inconsistent
history across repos in the hands of inexperienced users (and as much as I
consider myself a decent user of Mercurial, I'm very much a neophyte when it
comes to MQ).

The way I set up Mercurial for my team was to have a single master
repository, which was then cloned as many times as desired by developers.
Clones were also made to do automated builds/tests in. Nothing was *ever*
done in the master's workspace (it was always empty).

All work was done in a named branch per task named for the task number. You
could collapse changesets so long as they had not been shared (though we
preferred to keep the entire history - made code reviews and pairing much
easier).

The advantages I saw of this:

1. It's a much simpler workflow. You pull and push between repositories,
update to the named branch you're working on, and commit. You only need to
merge *from* your task branch in most cases, except when merging a change
between feature branches (e.g. named branch -> 3.1 -> 3.2 -> 3.3).

The only complication is if you start your work in e.g. 3.2 but it needs to
be backported to 3.1. In that case, you can either do an export/import/dummy
merge forwards to get the change onto 3.1, or if the change has never been
shared you could *transplant* the named branch onto the 3.1 feature branch
(but that is a history change, using the transplant extension, so should
only be done if the named branch has never been shared outside that repo).
For this reason, you should always branch from the earliest possible
location - you can always merge forwards, but backwards is hard because of
the extra associated history). If the task is for a bugfix, always try to
branch from the point where the bug originated.

2. Work sits in named branches, and can be reviewed there. All you have to
say in the issue tracker is "branch X in repo URL". Work doesn't need to be
merged to the feature branches until it's been reviewed.

This does mean that a branch can be based on an outdated version of the
feature branch. A merge from the feature branch will fix that, but does make
reviewing harder. This is the main reason I've seen to want to rebase - but
in general I still think it's more trouble than it's worth. Live with the
merges - if you really want to, you can always create *another* named branch
to do the merge in e.g. for the first merge from feature branch to task 1234

hg update 1234
hg branch 1234_merged_with_3.2
hg merge 3.2
hg commit -m "Initial merge 1234_merged_with_3.2"

For the second and later merges:

hg update 1234_merged_with_3.2
hg merge 3.2
hg commit -m "Merged 3.2 to 1234_merged_with_3.2"
hg merge 1234
hg commit -m "Merged 1234 to 1234_merged_with_3.2"

Then you have a "clean" branch to review, and a branch that will merge
cleanly with the feature branch. This combined branch is the one that you
will then merge to the feature branch.

3. When a branch is finished, it can just be closed, then merged to the
feature branch(es). Note that you do want to merge the close changeset as
well - this prevents leaving extra heads around.

4. When a branch is abandoned, it can just be closed. It stays in the
history, so if someone wants to take it up again they've got the previous
work sitting there and can either resume, or use it to work out what went
wrong. Note that closing a branch does nothing except hide it from hg heads
and hg branches.

5. It's very easy to just pull or push a single branch (although I'm
personally in favour of all branches eventually ending up in the master
repository). Pulling a single branch from another developer's repo is a good
way to work on a branch together.

6. Working on someone else's code is much much easier. Say I have an
environment set up, with various ignored configuration files, etc. A good
example would be an Eclipse workspace. You really don't want to have to set
up a separate workspace for each task or feature you're working on (which
you would need to do if you're using separate clones for each task). If you
have each task on a named branch, you can just hg update 1234 and your
existing workspace is now read

Re: [Python-Dev] Hg: inter-branch workflow

[Stephen J. Turnbull]

Thomas Wouters writes:
 > On Sun, Mar 20, 2011 at 17:59, Georg Brandl  wrote:
 > > On 20.03.2011 16:50, Thomas Wouters wrote:
 > > > On Sun, Mar 20, 2011 at 17:39, Georg Brandl  > > > wrote:
 > > >
 > > > The reason why rebasing is not universally applied is that the
 > > > rebased changesets are different from the original ones (therefore
 > > > I wrote A' and B') -- even if the diff is the same, the parents
 > > > are not, and therefore the changeset id (hash) changes.  This is
 > > > called "changing history", and frowned upon by purists.  In reality
 > > > it works fine if you know the limits: rebasing really only should be
 > > > applied if the changesets are not already known somewhere else,
 > > > only in the local repo you're working with.

This is not complete list.  The other important limit (pointed out by
Nick and Thomas) is that it creates changesets that have not yet been
tested, and therefore need to be tested or collapsed (if you want
bisect searches to work properly).

 > > > It's frowned upon by more than just purists, and it works "in
 > > > reality" as fine as handing out your creditcard and personal
 > > > information over the internet; you can't always tell the result
 > > > is bad, and it can be very painful finding out.

That's a problem with any merging, not just with rebasing, though.

 > I could also ask you what makes you think changing history is frowned upon
 > by *just purists*, or why on earth anyone would think it's a good idea
 > regardless of how practical or theoretical they are.

In practice, people who rebase often rebase the same branch to
mainline frequently.  If so, they detect most conflicts with mainline
as they occur, and this is often more useful context than knowing only
that "this branch was developed in a different context from the
current mainline tip", which is the "purist" reason most frequently
given for disliking rebasing.

That is, in practice people often resolve merge conflicts statically,
just by looking at the diff3 output.  If a developer is rebasing
frequently, on the other hand, they are probably also more aware of
the current context of the mainline, and why the conflicting changes
were made.  This dynamic context may improve the conflict resolutions.

 > As for my dislike, your reason (changing history) and Davids apparent
 > problem seem obvious enough. Aren't they? If you don't rebase correctly,
 > stuff goes wrong. Doing it correctly is harder than people seem to
 > think.

I don't think it is, at least not in theory: just run the test suite
on all rebased changesets.  As argued above, the case for frequent
rebasing is as least as strong as the case for merging a branch
developed entirely in its original context, and only dealing with
conflicts at the very end, quite out of context.

 > Merges can also contain useful information about how *future* merges of the
 > same changes should be handled.

That hg rebasing (and git rebasing, for that matter) doesn't preserve
this information is a design bug (ask any Darcs user :-).  It is
important in practice, of course.

 > Rebasing multiple changesets is worse, because the intermediate
 > changesets are a real lie rather than some fudging of history; your
 > repository was never in that state, that changeset never looked
 > that way and it was never built or tested that way.

Ah, but merging and rebasing are done locally.  The repository *is* in
that state now, and building and testing the relevant changesets can
be done automatically.  (This may be problematic in practice due to
the push race, of course.)  So it's just a matter of project policy to
require those tests before pushing.  People who are bugged by the push
race can do an ordinary merge, and provide less testing thereby.

 > Merging and merge changesets are a fact of DVCSes,

Actually, of any decentralized development.  DVCSes simply provide
tools to make you aware of merges, rather than hiding problems behind
"coalesced" patches that don't even know their parent version.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Nick Coghlan]

On Mon, Mar 21, 2011 at 2:59 AM, Georg Brandl  wrote:
> Well, YMMV.  But instead of spreading FUD you might want to state *why*.

David isn't the only one this kind of thing blew up on during the
sprints (although in my case, it may have been "hg rollback" rather
than rebasing).

Essentially, if using the cascaded local repositories workflow (i.e.
local 3.1 -> local 3.2 -> local default -> hg.python.org/cpython), by
the time you go to do the push to remote, your local history already
includes a change on 3.1, a 3.1->3.2 merge and a 3.2->default merge.
The history changing commands appear to have trouble coping with that
correctly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

Following up myself here...
On 20/03/2011 9:25 PM, Mark Hammond wrote:

On 20/03/2011 8:37 PM, "Martin v. Löwis" wrote:
...
Some of these uses probably shouldn't use the launcher directly - eg,
ISAPI apps and COM objects which have a separate registration step could
register a specific python.exe - the installation script itself could by
launched by the launcher, but that script could continue to use
sys.executable to record the specific python.exe.


The above raises an interesting question - if the launcher executed 
Python in-process, what would sys.executable be?  I can imagine there 
are few scenarios where it would be desirable to have it refer to the 
launcher and a number of scenarios where it would be undesirable and 
possibly break existing scripts.


Consider when a script with a "../python3" shebang line is started via 
the launcher, but that script itself uses sys.executable to launch its 
own child.  If sys.executable was the launcher, that child would likely 
be started using a python 2.x interpreter...


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 20/03/2011 11:56 PM, Dj Gilcrease wrote:
...

Before Mark wrote up this pep I had started experimenting with how to
make the launcher and I was able to get it to launch python while
exiting py.exe and as far as I could tell it behaved just as if I had
launched the app directly by double clicking it.


In addition to Martin's point, this approach would mean the exit code of 
the child process probably isn't available to whoever started the 
launcher.  For example, a batch file that tried to use something like 
"if errorlevel ..." to check the result of the Python process would 
probably not work as expected.


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Georg Brandl]

On 20.03.2011 18:32, Thomas Wouters wrote:
> 
> 
> On Sun, Mar 20, 2011 at 17:59, Georg Brandl  > wrote:
> 
> On 20.03.2011 16:50, Thomas Wouters wrote:
> >
> >
> > On Sun, Mar 20, 2011 at 17:39, Georg Brandl  
> > >> wrote:
> >
> > The reason why rebasing is not universally applied is that the
> > rebased changesets are different from the original ones (therefore
> > I wrote A' and B') -- even if the diff is the same, the parents
> > are not, and therefore the changeset id (hash) changes.  This is
> > called "changing history", and frowned upon by purists.  In reality
> > it works fine if you know the limits:
> >
> >
> > It's frowned upon by more than just purists, and it works "in reality" 
> as fine
> > as handing out your creditcard and personal information over the 
> internet; you
> > can't always tell the result is bad, and it can be very painful finding 
> out.
> 
> Well, YMMV.  But instead of spreading FUD you might want to state *why*.
> 
> 
> I could also ask you what makes you think changing history is frowned upon by
> *just purists*, or why on earth anyone would think it's a good idea regardless
> of how practical or theoretical they are.

The "purists" (which may have been a slight hyperbole) came from the fact that
usually the only argument you're presented with is "but, but, you're changing
history!" -- and I usually reply "it's my private history, and it's none of your
concern if I change it".  I will still have to take the blame if my changesets
break the build, so or so.

> As for my dislike, your reason (changing history) and Davids apparent problem
> seem obvious enough. Aren't they?

Yes, it shows that you should know about the limits of rebase.  In particular,
it is not suited to the forward-merging branch workflow when these merges
already occurred.

> If you don't rebase correctly, stuff goes
> wrong. Doing it correctly is harder than people seem to think. You can see
> what's going on with 'hg outgoing -p', but I, as an observer, have no way of
> knowing whether you used that and were aware of what exactly you were pushing.

That isn't specific to rebased changesets though, but to all changesets.  (See
for example the recently committed changeset including merge conflict markers.)
And by pushing linear history, I'm even making it easier for others to review,
since merge commits are a real pain to review.

> Merges can also contain useful information about how *future* merges of the 
> same
> changes should be handled. This doesn't matter much to changes with short
> turnaround (or changes that are continuously reverted and re-applied) but can
> make things a lot simpler if you have lots of diverging work in multiple
> branches -- and all the getting used to Mercurial aside, I'm sure that is what
> we're actually doing it for.

Yes, and for large parallel work it totally makes sense to use all the history
tracking that hg gives us.

> Even when done correctly, in situations where merges are automatic and do not
> involve the changed code in any way, I don't like rebasing a single changeset
> because it makes it less apparent when the actual work was started.

You need to remember that not all changesets are a) large or b) involve a lot of
code in different places.  Typical bugfix commits to Python touch one module,
its docs, and the NEWS file.  Others are even smaller.  It is utterly irrelevant
if I fixed a typo in the urllib docs when os.lseek was still leaking references,
or when it was already fixed.

> It's not any
> worse than diffing and re-applying a changeset, but for example if your change
> modified a particular call pattern appearing in multiples places, and another
> place started using the same pattern somewhere in your merges that you made go
> away, did you intentionally forget to fix the pattern or was it an accident?
> This isn't a problem that is *caused* by rebasing, but it certainly isn't 
> helped
> by it.

Sure, you will be given a chance to fix all new call sites in the merge commit.
For this kind of change it makes much sense to merge.

> Rebasing multiple changesets is worse, because the intermediate
> changesets are a real lie rather than some fudging of history; your repository
> was never in that state, that changeset never looked that way and it was never
> built or tested that way. No matter that it *usually* doesn't matter, or even
> *almost never* matters, it *can* matter.

There we have the "purist" point again.

> Merging and merge changesets are a fact of DVCSes, and while I (as a grumpy
> luddite misanthrope) greatly prefer the automatic (and mostly silent) merge as
> BitKeeper does it, in the long run the actual merging and the merge changesets
> are unavoidable and something to get used to, not dodged around (at least not 
> at
> this cost.)

I th

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Dj Gilcrease]

On Sun, Mar 20, 2011 at 1:56 PM, "Martin v. Löwis"  wrote:
> So if somebody would launch a python script with py.exe, they would
> think it was completed even though it would still be running.

Yes py.exe exits way before the python script, but the console stays
open, though now that I test it by running it via the command line and
not just launch it it directly I see what you mean the console returns
to usable then output starts being sent to it.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Westley Martínez]

On Sun, 2011-03-20 at 05:36 -0400, Terry Reedy wrote:
> On 3/20/2011 3:22 AM, Glenn Linderman wrote:
> On 3/19/2011 7:38 PM, Mark Hammond wrote:
> [snip]
> 
> As both a writer and reader, I would like to just add, for instance,
> 
> #! python3
> 
> (or 3.3 or whatever) and have the launcher do the 'right thing'.
> It seems to me that that really should be enough info for *nix,
> but given that it is not, a program to add the required extra noise  for 
> a particular distribution to the first line of every module in a package 
> should not be too difficult.
> 
> The above would be easy for anyone to write and understand.
> 
That would break the whole point of platform-independence.  UNIX won't
understand that, and I don't think we're gonna be able to convince all
the kernel developers to implement this simplified shebang.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

> The above raises an interesting question - if the launcher executed
> Python in-process, what would sys.executable be?  I can imagine there
> are few scenarios where it would be desirable to have it refer to the
> launcher and a number of scenarios where it would be undesirable and
> possibly break existing scripts.

Interesting question. What is it in COM and ISAPI applications?

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[R. David Murray]

On Sun, 20 Mar 2011 15:35:02 -0700, Westley =?ISO-8859-1?Q?Mart=EDnez?= 
 wrote:
> On Sun, 2011-03-20 at 05:36 -0400, Terry Reedy wrote:
> > On 3/20/2011 3:22 AM, Glenn Linderman wrote:
> > On 3/19/2011 7:38 PM, Mark Hammond wrote:
> > [snip]
> > 
> > As both a writer and reader, I would like to just add, for instance,
> > 
> > #! python3
> > 
> > (or 3.3 or whatever) and have the launcher do the 'right thing'.
> > It seems to me that that really should be enough info for *nix,
> > but given that it is not, a program to add the required extra noise  for 
> > a particular distribution to the first line of every module in a package 
> > should not be too difficult.
> > 
> > The above would be easy for anyone to write and understand.
> > 
> That would break the whole point of platform-independence.  UNIX won't
> understand that, and I don't think we're gonna be able to convince all
> the kernel developers to implement this simplified shebang.

+1.  If one of the points is to be compatible with PEP 394, then
the above seems like a bad idea.

However, I don't use windows, so I don't know if my vote should
count ;)

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Greg Ewing]

Mark Hammond wrote:

The above raises an interesting question - if the launcher executed 
Python in-process, what would sys.executable be?


I think it should be the actual Python executing at that
moment, not the launcher. This is the least change from
current behaviour and therefore least likely to break
existing code.

If some new code that's aware of the launcher wants to
use it, a new attribute such as sys.launcher could be
provided to make it easy to find. Or maybe a launch()
function that runs a specified script using launcher
semantics.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 21/03/2011 10:32 AM, "Martin v. Löwis" wrote:

The above raises an interesting question - if the launcher executed
Python in-process, what would sys.executable be?  I can imagine there
are few scenarios where it would be desirable to have it refer to the
launcher and a number of scenarios where it would be undesirable and
possibly break existing scripts.


Interesting question. What is it in COM and ISAPI applications?


ISAPI isn't a good example - that uses DLLs.  For COM objects hosted in 
a .exe it will be pythonw.exe.  For Windows services it will be 
pythonservice.exe (which is part of pywin32 and is located simply by 
assuming it is next to win32service.pyd)


Regardless, I don't think they are very likely to break - my concern is 
more for "normal" scripts which construct a child process cmdline/argv 
using sys.executable...


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

>> #! python3
[...]
> That would break the whole point of platform-independence.

a) the PEP makes no point about platform-independence.
   it is a pure Windows thing, and has no stated objective
   to being platform-independent. So loss of platform-independence
   may not be deliberate, but is certainly acceptable.
b) platform-independence is not lost as the launcher would continue
   to support /usr/bin-based paths. So platform-independence would
   be an opt-in feature, at the choice of script authors.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

Am 21.03.2011 00:52, schrieb Mark Hammond:
> On 21/03/2011 10:32 AM, "Martin v. Löwis" wrote:
>>> The above raises an interesting question - if the launcher executed
>>> Python in-process, what would sys.executable be?  I can imagine there
>>> are few scenarios where it would be desirable to have it refer to the
>>> launcher and a number of scenarios where it would be undesirable and
>>> possibly break existing scripts.
>>
>> Interesting question. What is it in COM and ISAPI applications?
> 
> ISAPI isn't a good example - that uses DLLs. 

Actually, this is precisely the example i was looking at.

> For COM objects hosted in
> a .exe it will be pythonw.exe.  For Windows services it will be
> pythonservice.exe (which is part of pywin32 and is located simply by
> assuming it is next to win32service.pyd)
> 
> Regardless, I don't think they are very likely to break - my concern is
> more for "normal" scripts which construct a child process cmdline/argv
> using sys.executable...

If we find out how other cases of just loading the DLL deal with
sys.executable, it may help us in guiding how it should be set in the
launcher.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Tim Delaney]

On 21 March 2011 08:16, Tim Delaney  wrote:

>
> For the second and later merges:
>
> hg update 1234_merged_with_3.2
> hg merge 3.2
> hg commit -m "Merged 3.2 to 1234_merged_with_3.2"
> hg merge 1234
> hg commit -m "Merged 1234 to 1234_merged_with_3.2"
>

Of course, you should probably do the "hg merge 1234" before "hg merge 3.2"
to avert the case that you actually "hg update 1234" here ...

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

Am 21.03.2011 00:43, schrieb Greg Ewing:
> Mark Hammond wrote:
> 
>> The above raises an interesting question - if the launcher executed
>> Python in-process, what would sys.executable be?
> 
> I think it should be the actual Python executing at that
> moment, not the launcher. This is the least change from
> current behaviour and therefore least likely to break
> existing code.

I remain -1 on any proposal that uses subprocesses. It absolutely must
be the launcher that is running Python. In fact, I'd call it
"python.exe".

Because of this issue, it may turn out infeasible to actually implement
the PEP. If Mark takes a stance in this issue, I'd request that the
opposite position is appropriately reflected (and rebutted) in the PEP.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Greg Ewing]

Mark Hammond wrote:

In addition to Martin's point, this approach would mean the exit code of 
the child process probably isn't available to whoever started the 
launcher.


Maybe I've missed something in this discussion, but is there
any reason the launcher can't just exec the relevant python?
Then the pid, exit status, etc. will all be as you would
expect if you'd invoked it directly.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 21/03/2011 11:10 AM, "Martin v. Löwis" wrote:

Am 21.03.2011 00:52, schrieb Mark Hammond:

On 21/03/2011 10:32 AM, "Martin v. Löwis" wrote:

The above raises an interesting question - if the launcher executed
Python in-process, what would sys.executable be?  I can imagine there
are few scenarios where it would be desirable to have it refer to the
launcher and a number of scenarios where it would be undesirable and
possibly break existing scripts.


Interesting question. What is it in COM and ISAPI applications?


ISAPI isn't a good example - that uses DLLs.


Actually, this is precisely the example i was looking at.


For COM objects hosted in
a .exe it will be pythonw.exe.  For Windows services it will be
pythonservice.exe (which is part of pywin32 and is located simply by
assuming it is next to win32service.pyd)

Regardless, I don't think they are very likely to break - my concern is
more for "normal" scripts which construct a child process cmdline/argv
using sys.executable...


If we find out how other cases of just loading the DLL deal with
sys.executable, it may help us in guiding how it should be set in the
launcher.


I guess it is all about context; in an ISAPI application or in a COM 
object implemented in a DLL, sys.executable will be the name of the host 
process (which is determined in PC/getpathp.c by calling 
GetModuleFileName with a NULL handle).  People writing ISAPI apps or COM 
objects are likely to understand the implications of them existing in 
some process which isn't python[w].exe so will ignore sys.executable and 
use some other technique to locate a python[w].exe if they need it.


"Normal" scripts are less likely to do so - a code search at google for 
sys.executable shows it used in ways which would possibly break if 
sys.executable referred to a launcher and the launcher didn't use the 
"current" version of Python as the default.  Often these are in tests or 
setup.py, but we would need to avoid any such breakage.


[Which isn't to suggest we can't avoid such breakage with an "in-process 
launcher" - it just isn't immediately clear how we would do so]


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

>> In addition to Martin's point, this approach would mean the exit code
>> of the child process probably isn't available to whoever started the
>> launcher.
> 
> Maybe I've missed something in this discussion, but is there
> any reason the launcher can't just exec the relevant python?

Windows doesn't support exec.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Terry Reedy]

On 3/20/2011 6:35 PM, Westley Martínez wrote:

On Sun, 2011-03-20 at 05:36 -0400, Terry Reedy wrote:



As both a writer and reader, I would like to just add, for instance,

#! python3

(or 3.3 or whatever) and have the launcher do the 'right thing'.
It seems to me that that really should be enough info for *nix,
but given that it is not, a program to add the required extra noise  for
a particular distribution to the first line of every module in a package
should not be too difficult.

The above would be easy for anyone to write and understand.


That would break the whole point of platform-independence.  UNIX won't
understand that, and I don't think we're gonna be able to convince all
the kernel developers to implement this simplified shebang.


Given that the default for code developed on windows is NO #! line (and 
that may also be true for new programmers in *nix too, for all I know), 
I should think that the above would be an improvement. It does not 
require new Python programmers on Windows to know anything about *nix 
(pseudo?)paths, nor about the '/' versus '\' fuss. One should only need 
to know all that if one wishes to distribute to *nix *and* to have 
script files directly executable. (I presume 'python x.py' does not 
require a #! line on *nix either.)


For people who do want to do that, I already suggested that there be a 
program that scans through a directory tree, looking for files that 
start with '#! " and replacing that when found with '#!/usr/bin/env ' or 
whatever. That could be done once, before distribution. And it would not 
even require understanding of the substitution.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 21/03/2011 11:15 AM, "Martin v. Löwis" wrote:

Am 21.03.2011 00:43, schrieb Greg Ewing:

Mark Hammond wrote:


The above raises an interesting question - if the launcher executed
Python in-process, what would sys.executable be?


I think it should be the actual Python executing at that
moment, not the launcher. This is the least change from
current behaviour and therefore least likely to break
existing code.


I remain -1 on any proposal that uses subprocesses. It absolutely must
be the launcher that is running Python. In fact, I'd call it
"python.exe".


For clarity, could you please tell us which scenarios you have in mind 
that cause you to take that position, and why those scenarios couldn't 
ignore the existence of the launcher and stick with doing exactly what 
they do now (which presumably is to use python.exe directly)?


Thanks,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

>> I remain -1 on any proposal that uses subprocesses. It absolutely must
>> be the launcher that is running Python. In fact, I'd call it
>> "python.exe".
> 
> For clarity, could you please tell us which scenarios you have in mind
> that cause you to take that position, and why those scenarios couldn't
> ignore the existence of the launcher and stick with doing exactly what
> they do now (which presumably is to use python.exe directly)?

I'm primarily bothered about the failure to implement TerminateProcess
correctly. I don't actually know what use cases would be affected, other
then saying that anything launching py.exe could be affect, in
particular applications using ShellExecuteEx. I don't think it is
feasible to change them all to launch something different instead; some
may be out of our control. Potential candidates would be web servers.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 21/03/2011 12:27 PM, "Martin v. Löwis" wrote:

I remain -1 on any proposal that uses subprocesses. It absolutely must
be the launcher that is running Python. In fact, I'd call it
"python.exe".


For clarity, could you please tell us which scenarios you have in mind
that cause you to take that position, and why those scenarios couldn't
ignore the existence of the launcher and stick with doing exactly what
they do now (which presumably is to use python.exe directly)?


I'm primarily bothered about the failure to implement TerminateProcess
correctly. I don't actually know what use cases would be affected, other
then saying that anything launching py.exe could be affect, in
particular applications using ShellExecuteEx. I don't think it is
feasible to change them all to launch something different instead; some
may be out of our control. Potential candidates would be web servers.


This fear seems overblown to me as I doubt any such programs exist.  A 
program could not use ShellExecuteEx on an arbitrary file extension and 
hope to obtain a handle it can use to kill the process - the docs make 
it clear no handle is returned in a number of common cases and implies 
that even if you get a handle back you should not kill it (as it 
explicitly states that a single process may be the target for any number 
of such calls.)  For the same reason, you can't even wait on it unless 
you know the implementation of the associated program.


So this scenario would need to be a program which used ShellExecuteEx 
only to launch Python programs and therefore felt it could rely on the 
ability to (a) always get a handle back and (b) use TerminateProcess to 
kill it.  ISTM the likelihood of such a program existing is so low that 
it should not kill the chances of this PEP being accepted if the 
sub-process model is the only reasonable way to implement the launcher.


ie, let's say we are forced to choose between the following 3 options:

* No launcher at all (the status-quo), causing demonstrable breakage in 
Windows file associations whenever Python 2.x and Python 3.x scripts 
exist on the same box.


* An in-process launcher which caused breakage in a number of reasonably 
common scenarios for Python programmers, and such breakage could easily 
be demonstrated.


* An out-of-process launcher which caused breakage for the hypothetical 
program mentioned above, of which no instance can be found and no 
breakage actually demonstrated.


I personally would conclude that the last option is the least worst 
scenario by a wide margin.


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Martin v. Löwis]

> I personally would conclude that the last option is the least worst
> scenario by a wide margin.

Ok, I let this rest. Can you please add a summary of this discussion to
the PEP? (also, can you please check in the PEP, and give it a number?)

Thanks,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Curt Hagenlocher]

On Sun, Mar 20, 2011 at 6:27 PM, "Martin v. Löwis"  wrote:
>
> I'm primarily bothered about the failure to implement TerminateProcess
> correctly. I don't actually know what use cases would be affected, other
> then saying that anything launching py.exe could be affect, in
> particular applications using ShellExecuteEx. I don't think it is
> feasible to change them all to launch something different instead; some
> may be out of our control. Potential candidates would be web servers.

This can be worked around, depending on the level of inelegance you're
willing to accept. A third watchdog process could wait on the launcher
being terminated, and use that to terminate the child Python.exe
process. Alternatively, the launcher could inject a sentinel thread
into the Python.exe process using CreateRemoteThread. Both approaches
would work for any target implementation or bitness of Python.

-Curt
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[Guido van Rossum]

On Sun, Mar 20, 2011 at 12:40 PM, R. David Murray  wrote:
> On Mon, 21 Mar 2011 03:44:34 +0900, "Stephen J. Turnbull" 
>  wrote:
>> I'm coming to the conclusion that those who say that Mercurial
>> documentation should be found at the Mercurial project are wrong.  I
>
> +1.  I think the dev docs should explain anything needed to do the
> basic Python workflow, even if that feels like repeating parts of the
> Mercurial docs.  (If it is a large chunk, then a pointer to the exactly
> relevant section of the Mercurial docs would be appropriate.)  On the
> other hand, I'm not offering to do the work, so I'll happily accept
> what has been produced so far (thanks, Antoine!).
>
>> think there's a reasonably strong case (based on the explicit promise
>> of PEP 0374 that workflows would change as little as possible) for a
>> follow-on informational PEP providing verbal, and maybe automated,
>> scripts for the various operations needed in the Python workflow.
>
> Sounds good, but someone needs to write them.  You'll note that I've been
> asking for this for a while, but as it turns out it wasn't really possible
> to write them "correctly" until we started actually using the system.
> I'm still not comfortable enough to write them myself, but after I've
> used my chosen workflow for a while I'll do my best to write it up in
> more detail than my earlier post as a blog post or something.

I think it ought to be added to the dev guide instead of a PEP.
Otherwise I agree -- the workflows are somewhat unique to Python. I
personally sure haven't grokked them yet, not having done anything
that required merging yet. (TBH I never got the hang of our SVN merge
flow either.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Guido van Rossum]

On Sun, Mar 20, 2011 at 1:59 PM, Stefan Behnel  wrote:
> [warning, long post ahead]
[all snipped]

Thanks for the clarifications. I now have a much better understanding
of what Cython is. But I'm not sold. For one, your attitude about
strict language compatibility worries me when it comes to the stdlib.
Also, I don't know how big it is, but it seems putting the cart before
the horse to use it to optimize the stdlib. Cython feels much less
mature than CPython; but the latter should only have dependencies that
themselves change even slower than CPython. I also am unclear on how
exactly you're supporting the different semantics in Python 2 vs. 3
without recompiling.

OTOH I think you've got the perfect audience in the scientific Python
world. Have you tried replacing selected stdlib modules with their
Cython-optimized equivalents in some of the NumPy/SciPy distros? (E.g.
what about Enthought's Python distros?) Depending on how well that
goes I might warm up to Cython more!

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg: inter-branch workflow

[Éric Araujo]

> I have been avoiding hg import because my understanding is that it
> defaults to commit, and I don't see that it has any advantage over patch
> itself.
“hg import” understands the extended diff format, which patch does not.
 (That format has been described a number of times already, see
http://mercurial.selenic.com/wiki/GitExtendedDiffFormat.)

“hg import --no-commit” is basically a patch command that understands
the extended format.  (Pro tip: it can be abbreviated to “hg import
--no-c”, as Mercurial accepts unambiguous abbreviations of commands and
options.)

“hg import” can also use metadata in changeset files, that is, diffs
with a special header containing the parent, date, author and commit
message, such as produced by “hg export”.  (Having symmetric names
without compatibility would be nothing short of criminal, but I’ve found
Mercurial satisfyingly consistent in those matters.)

> I'll have to check that out.
Remember that “hg help command” is always here, and that the Mercurial
wiki and search engines have a lot of complementary information.

Regards
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[Éric Araujo]

Le 20/03/2011 02:59, Ned Deily a écrit :
> On a Unix-y system, here is one way to do it (no warranty on the 
> installation instructions!):

With all due respect, the instructions are overly complicated, and may
also run afoul of the system conventions (sudo will put files in
directories that should not be touched for example).

Here’s my take:

Get the code
  $ cd where/ever
  $ hg clone http://hg.python.org/hooks

Edit your user ~/.hgrc of the cpython clones .hg/hgrc to enable the hook
  [hooks]
  pretxncommit.whitespace =
python:/home/you/where/ever/hooks/checkwhitespace.py:check_whitespace_single

Regards
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Deprecating non-Py_ssize_t use of PyArg_ParseTuple

[Martin v. Löwis]

Since Python 2.5, we maintain two versions of PyArg_ParseTuple:
one outputting int; the other one outputting Py_ssize_t.

The former should have been removed in 3.0, but this was forgotten.

Still, I would like people to move over to the new version, so
that extension modules will typically support 64-bit collections
well. Therefore, I'd like to propose that the int version is deprecated
in 3.3.

Given the recent discussion about backwards compatibility: what's
the best approach? What warning should be emitted, if any?
(the warning would only be generated if an s# or similar format
 was actually encountered - not just if merely PyArg_ParseTuple is
 called).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[Ned Deily]

In article <4d86be29.8070...@netwok.org>,
 Éric Araujo  wrote:
> Le 20/03/2011 02:59, Ned Deily a écrit :
> > On a Unix-y system, here is one way to do it (no warranty on the 
> > installation instructions!): 
> With all due respect, the instructions are overly complicated, and may
> also run afoul of the system conventions (sudo will put files in
> directories that should not be touched for example).

That depends on your configuration but ...

> Here’s my take:
> 
> Get the code
>   $ cd where/ever
>   $ hg clone http://hg.python.org/hooks
> 
> Edit your user ~/.hgrc of the cpython clones .hg/hgrc to enable the hook
>   [hooks]
>   pretxncommit.whitespace =
> python:/home/you/where/ever/hooks/checkwhitespace.py:check_whitespace_single

that's *much* better, thanks!  The hg documentation does not make it 
clear that, in the absolute path case, PYTHONPATH does not need to be 
manipulated, which is what I was trying to avoid.

Perhaps you could add those instructions to the devguide?

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Greg Ewing]

Martin v. Löwis wrote:


Windows doesn't support exec.


Hmmm. In that case, if the launcher works by loading a pythonXY.dll,
I'd say that sys.executable should point to whatever version of
python.exe corresponds to that dll.

Generally, things should be made to look as much as possible as
if that python.exe had been executed in the first place.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[Stephen J. Turnbull]

s...@pobox.com writes:

 > Stephen> interactions across modules.  That is, it's not that Subversion
 > Stephen> provided a simpler way of doing the work.  Rather, it hid the
 > Stephen> fact that certain work was not being done at all.  hg exposes
 > Stephen> this fact.

 > Can you provide an example of this (real or hypothetical)?

What happened to you is a real example.  With svn it would go
through, and any problems would be caught later.  With hg, it doesn't.

This is basically the statistical idea of false positives vs. false
negatives.  svn tends to generate false negatives (tests don't fail
until after the merge), while hg tends to generate false positives
(merges fail because they aren't "fast-forwards").

 > It seems to me that if all you have is unit tests it matters
 > little, if at all, whether you catch test failures before a checkin
 > (in your local hg repo) or after (using svn + a buildbot run).

In the long run, that's right.  In the short run the choice is between
imposing a substantially higher burden of merging and testing on
individual developers in order to get a higher quality shared branches
versus imposing a detectably higher burden on the project as whole in
terms of red buildbots.  It clearly matters to the individual
developers, and core developers are likely to have different interests
here from the occasional committers.

The burden on the project could get *very* high; that's why the Linux
kernel went to Bitkeeper, then git, rather than Subversion.  It was
necessary to impose as much burden on the branches as possible before
merging to trunk.  But then, Python doesn't have anywhere near the
change rate that the kernel does (Lennart R's woes notwithstanding).

I'm not going to take a position on what's best for the Python
project; I've drunk the DVCS Kool-Aid and am very much in Jesus's
"let's work with the tools, not against them" camp.  But there's an
uncertain amoung of bias in that opinion, and I'm not a committer so
can't speak from experience with the Python workflow.

 > The workflow of the version control system isn't going to magically
 > expose semantic interactions across modules.  It seems to me that
 > at best the new workflow means there is a better chance of catching
 > textual conflicts before checkin than before.

No, at best the DVCS workflow forces the developer on a branch to
merge and test the revisions that will actually be added to the
repository, and perhaps notice system-level anomolies before pushing.
Whether that is significant in practice is as yet a matter of faith
for me.  But again that's a personal opinion, not yet a recommendation
for the project.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

On 21/03/2011 3:50 PM, Greg Ewing wrote:

Martin v. Löwis wrote:


Windows doesn't support exec.


Hmmm. In that case, if the launcher works by loading a pythonXY.dll,
I'd say that sys.executable should point to whatever version of
python.exe corresponds to that dll.

Generally, things should be made to look as much as possible as
if that python.exe had been executed in the first place.


Yeah, that would have to be the plan - if there are ways it *doesn't* 
look like python.exe had been executed it is likely to cause problems.


Initially I thought we could just adjust sys.executable after it is set 
to the "inappropriate" default value, but that would be tricky - this is 
called by Python during Py_Main() - which also executes the specified 
script.  Thus, the launcher would either:


* Call Py_Initialize(), adjust sys.executable, then call Py_Main().  I'm 
not sure what side-effects this would have though - Py_Main does a 
number of things before calling Py_Initialize() and having it called 
before those things are done might have other side-effects.


* Re-implement Py_Main and adjust sys.executable immediately after it is 
set.  This would be a huge burden given the size of that routine and the 
number of different versions we want to support.


The first option is the only one I'd really consider reasonable and I'll 
probably experiment with it once we have enough agreement that the C 
implementation can proceed - but I'm not confident enough we can make it 
work correctly to base the entire success or failure of the PEP on that 
detail.


Any other ideas/thoughts about this?

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Greg Ewing]

Guido van Rossum wrote:

Cython feels much less
mature than CPython; but the latter should only have dependencies that
themselves change even slower than CPython.


You might be slightly more amenable to Pyrex, then, which
changes at a much more conservative pace!

They appear superficially similar, but there is a difference
in underlying philosophy. Pyrex makes no claim to be a
Python compiler; rather, it's a tool for bridging between
the Python and C worlds. As such, strict adherence to
Python syntax and semantics, or supporting all Python
features, or performing heroic optimisations on pure
Python code, are not design goals.

So it wouldn't make sense to try to compile any of the
pure Python parts of the stdlib with Pyrex. However, it
could make sense to use it for some of the modules
currently written in C, or for future modules that
would otherwise be written in C, to make them easier to
write, read and maintain.

Think of it as a tool for writing modules in C without
having to worry about reference counts or exception
handling.

If you think you might seriously consider using Pyrex,
I could translate a module or two from the stdlib so
you can see how the code might look.


I also am unclear on how
exactly you're supporting the different semantics in Python 2 vs. 3
without recompiling.


I gather that Cython generates code with a lot of #ifs
for doing things differently depending on the Python
version.

I haven't decided exactly how to handle Python 3 in
Pyrex yet, but I have some ideas. I'm not sure whether
it will generate dual-version C files or not. It
probably wouldn't matter much for stdlib usage, though,
because it only needs to generate C code good enough
for a particular release of Python.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [GSoC] Porting on RPM3

[Prashant Kumar]

Hello,
My name is  Prashant Kumar and I've worked on porting few python
libraries(distutils2, configobj) and I've been looking at the ideas
list for GSoC for a project related to porting.

I came across [1]  and found it interesting. It mentions that some
of the work has already been done; I would like to look at the code
repository for the same, could someone provide me the link for the
same?


Regards,
pkumar

[1] http://wiki.python.org/moin/RPMOnPython3
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com