Re: [Python-Dev] xmlrpclib and communication verbosity

2011-02-03 Thread Erez Sh
Oleg wrote:


>   Why not logging?
>

It seems to me that most of python's library modules don't use the logging
module, and I didn't want to make style judgments.
I actually prefer logging.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] News of the faulthandler project

2011-02-03 Thread Victor Stinner
Hi,

Since the end of last december, I'm still working on my fault handler
project:
https://github.com/haypo/faulthandler

You can use it to get more information after a crash or if you program
hangs somewhere. It helps if you don't have access to other debugging
tool (eg. install gdb7+python-gdb.py on Windows is not trivial today) or
if you cannot interact with your program (eg. on a buildbot).

The last version works on Python 2.5, 2.6, 2.7, 3.1 and 3.2, on Windows,
Linux and FreeBSD. It can display the Python backtrace on a fatal fault
(SIGSEGV, SIGFPE, SIGILL, SIGBUS), after a delay in seconds, when an
user signal is received (eg. SIGUSR1) or explicitly (call directly the
dumpbacktrace() function).

By default, it is disabled: you have to call faulthandler.enable() to
install the signal handlers. You can choose in which file the backtrace
is written (sys.stderr by default) and if it displays the backtrace of
the current thread or of all threads. If you use the delay: you can
choose to repeat the operation (dump the backtrace each delay seconds).

The project is now a module, so it is no more enabled by default. It is
more configurable, and has more features. It has a better API (so it was
a good idea to not include it in Python 3.2).

I plan to integrate this project into Python 3.3. I hope that it can
help to debug some buildbots issues, but also any segfault in your
programs.

Note: faulthandler.register() (dump the backtrace when an user signal is
raised) is only available in the development version.

--

The project is not perfect yet:

 - I have to write something to be able to enable the faulthandler
before starting your program (write a program for that?)
 - faulthandler.dumpbacktrace_later() uses alarm() which interrupts the
current system call when SIGALARM is raised: it may be a problem (it's
maybe not a problem if you try to debug a program hang)
 - I don't know if something should be done on a fork()
 - SIGABRT is not handled
 - The module is unloaded using Py_AtExit(): it cannot release
references because the unload function is called too late

--

There are similar projects, tipper and crier, using a signal handler
implemented in Python or a signal handler implemented in Python. These
projects give more information (eg. local variables) and more control on
how the informations are written, but I think that there are less
reliable: it doesn't work if Python hangs (eg. deadlock) and signal
handlers implemented in Python are asynchronous. And there are unable to
catch fatal faults (eg. SIGSEGV).

http://pypi.python.org/pypi/tipper/
https://gist.github.com/737056

Victor

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] News of the faulthandler project

2011-02-03 Thread Nick Coghlan
On Thu, Feb 3, 2011 at 11:05 PM, Victor Stinner
 wrote:
>  - I have to write something to be able to enable the faulthandler
> before starting your program (write a program for that?)

I don't know enough about signal handling to help with your other
remaining concerns, but an appropriate "-X" command line option seem
like a reasonable way to activate it before main starts running (-X is
currently documented as reserved for use by other implementations, but
it's really more a "implementation dependent options" marker)

(+1 on the general idea, though)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Michael Urman
On Thu, Feb 3, 2011 at 00:30, "Martin v. Löwis"  wrote:
> Another challenge with shared location merge modules is upgrades:
> the Python installer currently doesn't use stable component IDs;
> I think this would cause problems for users of the merge module.
> Providing stable component IDs is a challenge since it's difficult
> to version the files on disk.
>
> Not sure why Michael thinks that a private location merge module
> would provide no benefits to the user of the merge module.

I hadn't thought it through fully, but the preceding paragraph really
gets to the core of the problem. The maintenance nightmare is security
updates for private location installations by third parties. The only
MSI-friendly way to update that code is through releasing an updated
merge module and having the consuming application also release an
update that uses it. Since Python's components use fresh GUIDs each
time, this would require a "major" upgrade; "minor" upgrades would
cause Windows Installer to throw fits.

Technically this is a problem with the component generation in Python,
and for that in particular, a move to WiX could be very helpful. They
have stable component code generation which keys off of location,
name, platform, etc., but only works for single-file components.

For contrast, I don't see a shared-location merge module as offering
benefits beyond a silently redistributable msi package. The shared
location is better about following component code rules (re-use in
private areas is an allowed gray area), and there are people out there
who consider the reference counting through merge modules to be
superior. I find the resulting complexity in the consuming package's
installation to be more of a down-side.

>> I work on open source projects myself, and we always provide
>> both a merge module and a normal msi installer. It's very little
>> extra work (in WiX at least) to create both.
>
> But what's the quality of these? Ideally, I'd like to create a single
> merge module which, at the option of the user of the merge module,
> produces either a shared or a private installation. Is that still
> only little extra work in Wix?

I've never tried to make a configurable merge module in WiX, but I
think that's the only option if you want a single merge module to
allow both. It should be a one-time authoring overhead with [1]. Using
them is pretty straightforward within the Merge elements [2].

[1] http://wix.sourceforge.net/manual-wix3/wix_xsd_configuration.htm
[2] http://wix.sourceforge.net/manual-wix2/wix_xsd_configurationdata.htm

Michael
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Py_tp_getset in ABI?

2011-02-03 Thread Egon Smiwa

Hi all,
I'm trying to convert my embedding code to your new ABI,
but I cannot find the ABI slot for tp_getset in typeslots.h
(while tp_methods are supported). Is the support of tp_getset
not yet determined? (Because I cannot find this in the PEP 384)

Thank you!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] News of the faulthandler project

2011-02-03 Thread Reid Kleckner
On Thu, Feb 3, 2011 at 8:05 AM, Victor Stinner
 wrote:
>  - SIGABRT is not handled

Why not?  That seems useful for debugging assertion failures, although
most C code in Python raises exceptions rather than asserting.

I'm guessing it's because it aborts the process after printing the
backtrace.  You could just clear the signal handler before aborting.

Reid
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Floris Bruynooghe
On 3 February 2011 15:38, Michael Urman  wrote:
> Technically this is a problem with the component generation in Python,
> and for that in particular, a move to WiX could be very helpful. They
> have stable component code generation which keys off of location,
> name, platform, etc., but only works for single-file components.

At work we keep the required stable UUIDs in an ConfigParser-format
file checked in to the VCS for that purpose.

FWIW our build system uses WiX (v2) currently and if I where to redo
it, I'd change to msilib and not WiX v3.  But never change working
systems.

Python.org supplied merge modules would be nice though, if the
upgrade/security issue can be resolved.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] News of the faulthandler project

2011-02-03 Thread Victor Stinner
Le jeudi 03 février 2011 à 12:22 -0500, Reid Kleckner a écrit :
> On Thu, Feb 3, 2011 at 8:05 AM, Victor Stinner
>  wrote:
> >  - SIGABRT is not handled
> 
> Why not?

Just because I forgot to handle it. But I don't know if it is a good
thing to display the Python backtrace on abort() or not. Python uses
abort() on Py_FatalError().

Victor

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Hoyt, David
> I hadn't thought it through fully, but the preceding paragraph really
> gets to the core of the problem. The maintenance nightmare is security
> updates for private location installations by third parties. The only
> MSI-friendly way to update that code is through releasing an updated
> merge module and having the consuming application also release an
> update that uses it. Since Python's components use fresh GUIDs each
> time, this would require a "major" upgrade; "minor" upgrades would
> cause Windows Installer to throw fits.

That's exactly right and why I said earlier that patches w/ merge modules are a 
pain/nightmare. But I also said that the security patching w/ a merge module 
would then depend on the party who has integrated the merge module into their 
product -- not on you guys. I don't think most (or any) parties are trying to 
do incremental minor updates w/ python right now anyway. With us, we just want 
a single, well-tested and known python implementation installed w/ our product 
-- we basically don't trust the user to get the right version and install it 
correctly. You obviously can't hold their hand every step of the way, at some 
point you have to let go -- but this is one thing that we could possibly 
control.

> Technically this is a problem with the component generation in Python,
> and for that in particular, a move to WiX could be very helpful. They
> have stable component code generation which keys off of location,
> name, platform, etc., but only works for single-file components.

The general recommendation regarding msi packages is that you always, always do 
single-file components (one of the major reasons being for patching purposes). 
If you use WiX' heat application to autogenerate WiX files from directories, 
you'll see that it always produces single-file components.

> For contrast, I don't see a shared-location merge module as offering
> benefits beyond a silently redistributable msi package. The shared
> location is better about following component code rules (re-use in
> private areas is an allowed gray area), and there are people out there
> who consider the reference counting through merge modules to be
> superior. I find the resulting complexity in the consuming package's
> installation to be more of a down-side.

I think a merge module (shared or private) generally reduces complexity unless 
you already have a bootstrapper for other packages. Including one in WiX is 
very simple (if you're already familiar w/ msi's):









If you use something like the VS installer projects, you just have to use the 
GUI to add a reference to it and you're done.

A shared merge module might pose a problem if you want to have multiple python 
versions installed. At the least, you'd need to change the component guid w/ 
each major release (e.g. 2.x -> 3.x, but not 2.7 -> 2.8 -> 2.9, etc.). I'd 
recommend w/ sticking to a private one that doesn't modify the PATH (should you 
choose to create one) and then you're free to keep or change the component 
guids.

Can python operate inside a sandboxed C:\Program Files\\ 
directory?

> I've never tried to make a configurable merge module in WiX, but I
> think that's the only option if you want a single merge module to
> allow both. It should be a one-time authoring overhead with [1]. Using
> them is pretty straightforward within the Merge elements [2].
>
> [1] http://wix.sourceforge.net/manual-wix3/wix_xsd_configuration.htm
> [2] http://wix.sourceforge.net/manual-wix2/wix_xsd_configurationdata.htm

I wouldn't try to make it configurable (nor does it really need to be) beyond 
what it already should do -- that is, just be able to set the base target 
directory, which is already done for you. And as I just suggested, I wouldn't 
go for both -- declare the merge module to be private and that's that.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Hoyt, David
> At work we keep the required stable UUIDs in an ConfigParser-format
> file checked in to the VCS for that purpose.
>
> FWIW our build system uses WiX (v2) currently and if I where to redo
> it, I'd change to msilib and not WiX v3.  But never change working
> systems.

No need to do that if you're using heat -- I haven't used WiX v2 so I can't 
speak to its relative merits, but v3.5 (which was just officially released) is 
pretty good and much more feature complete (according to rob mensching's blog). 
I'd recommend re-evaluating it. I'm not a Microsoft fanboy, btw. But I am 
getting my group (or trying to, at least) to migrate away from older, stale 
installer projects (e.g. Visual Studio will be dropping support for any further 
installer project improvements in the future) and into WiX because that's where 
the momentum is and it also keeps up-to-date w/ the latest msi changes. That 
and I was tired of the install looking like an intern did it (no offense to 
interns -- I was one once. (c: ). A good, polished product should (IMO) also 
have a good, polished installer. Especially since that's one of the customer's 
first views/impressions of your product.

> Python.org supplied merge modules would be nice though, if the
> upgrade/security issue can be resolved.

Good to know I'm not entirely alone. (c:
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Martin v. Löwis
> Technically this is a problem with the component generation in Python,
> and for that in particular, a move to WiX could be very helpful. They
> have stable component code generation which keys off of location,
> name, platform, etc., but only works for single-file components.

That would be no reason to move to Wix. Integrating the same algorithm
in msilib is trivial.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Martin v. Löwis
Am 03.02.2011 18:58, schrieb Floris Bruynooghe:
> On 3 February 2011 15:38, Michael Urman  wrote:
>> Technically this is a problem with the component generation in Python,
>> and for that in particular, a move to WiX could be very helpful. They
>> have stable component code generation which keys off of location,
>> name, platform, etc., but only works for single-file components.
> 
> At work we keep the required stable UUIDs in an ConfigParser-format
> file checked in to the VCS for that purpose.

That's also the path I'd take. But I'm unsure how versioning would work,
in particular if I have per-directory components, and files get added
(which typically shouldn't, but might happen in bugfix releases).

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] News of the faulthandler project

2011-02-03 Thread Antoine Pitrou
On Thu, 03 Feb 2011 21:52:40 +0100
Victor Stinner  wrote:
> Le jeudi 03 février 2011 à 12:22 -0500, Reid Kleckner a écrit :
> > On Thu, Feb 3, 2011 at 8:05 AM, Victor Stinner
> >  wrote:
> > >  - SIGABRT is not handled
> > 
> > Why not?
> 
> Just because I forgot to handle it. But I don't know if it is a good
> thing to display the Python backtrace on abort() or not. Python uses
> abort() on Py_FatalError().

I think that precisely makes it a good idea.  It's much better to know
where a fatal error was triggered from if you want to have a chance of
at least working around it.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Martin v. Löwis
> The general recommendation regarding msi packages is that you always,
> always do single-file components (one of the major reasons being for
> patching purposes).

Can you please cite a source for that recommendation? Preferably
some MSDN documentation.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Py_tp_getset in ABI?

2011-02-03 Thread Martin v. Löwis
Am 03.02.2011 16:43, schrieb Egon Smiwa:
> Hi all,
> I'm trying to convert my embedding code to your new ABI,
> but I cannot find the ABI slot for tp_getset in typeslots.h
> (while tp_methods are supported). Is the support of tp_getset
> not yet determined? 

Not sure what I thought - it seems that tp_getset and tp_members
is plain missing. This is puzzling because I clearly meant to include
PyMemberDef and PyGetSetDef (and they are included).

Unless somebody reminds me why they would have to be excluded, I
would like to add them still.

Adding them after 3.2 would be forward-compatible (?), i.e.
all modules build for 3.2 would continue to work in 3.2.1.
However, modules using them would work in 3.2.1, but fail in
3.2.0 (with a RuntimeError exception)

So I think I would preferably add these before 3.2 is released.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python merge module

2011-02-03 Thread Hoyt, David
> Can you please cite a source for that recommendation? Preferably
> some MSDN documentation.

http://msdn.microsoft.com/en-us/library/aa368269(v=vs.85).aspx 
http://wix.sourceforge.net/manual-wix3/add_a_file.htm

Specifically, starting in bold, where it says "In general, you should restrict 
yourself to a single file per component. The Windows Installer is designed to 
support thousands of components in a single installer, so unless you have a 
very good reason, keep to one file per component. Every component must have its 
own unique GUID. Failure to follow these two basic rules can lead to many 
problems down the road when it comes to servicing."

A little before that, it states: "The component element describes a set of 
resources (usually files, registry entries, and shortcuts) that need to be 
installed as a single unit. This is separate from whether the set of items 
consist of a logical feature the user can select to install ... While it may 
not seem like a big deal when you are first authoring your installer, 
components play a critical role when you decide to build patches at a later 
date."

I didn't say it's a must, but experience lends you to following the 
recommendation.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Py_tp_getset in ABI?

2011-02-03 Thread Georg Brandl
Am 03.02.2011 22:46, schrieb "Martin v. Löwis":
> Am 03.02.2011 16:43, schrieb Egon Smiwa:
>> Hi all,
>> I'm trying to convert my embedding code to your new ABI,
>> but I cannot find the ABI slot for tp_getset in typeslots.h
>> (while tp_methods are supported). Is the support of tp_getset
>> not yet determined? 
> 
> Not sure what I thought - it seems that tp_getset and tp_members
> is plain missing. This is puzzling because I clearly meant to include
> PyMemberDef and PyGetSetDef (and they are included).
> 
> Unless somebody reminds me why they would have to be excluded, I
> would like to add them still.
> 
> Adding them after 3.2 would be forward-compatible (?), i.e.
> all modules build for 3.2 would continue to work in 3.2.1.
> However, modules using them would work in 3.2.1, but fail in
> 3.2.0 (with a RuntimeError exception)
> 
> So I think I would preferably add these before 3.2 is released.

I'm okay with you making this fix (and the one for #11067) before
3.2.0 is released.

Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] r88331 - in python/branches/py3k/Doc/howto: index.rst pyporting.rst

2011-02-03 Thread Nick Coghlan
On Fri, Feb 4, 2011 at 8:01 AM, brett.cannon  wrote:
> +Capturing the Currently Raised Exception
> +
> +One change between Python 2 and 3 that will require changing how you code is
> +accessing the currently raised exception.  In Python 2 the syntax to access 
> the
> +current exception is::
> +
> +    try:
> +        raise Exception()
> +    except Exception, exc:
> +        # Current exception is 'exc'
> +        pass
> +
> +This syntax changed in Python 3 to::
> +
> +    try:
> +        raise Exception()
> +    except Exception as exc:
> +        # Current exception is 'exc'
> +        pass

Note that you can write it the Python 3 way in 2.6+ as well (this was
new syntax, so there weren't any backwards compatibility issues with
adding it). You only need to do the sys.exc_info dance if you need to
support 2.5 or earlier.

Other notes:
- explicit relative imports work in 2.6+ without needing a future import
- absolute imports are the default in 2.7

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] r88331 - in python/branches/py3k/Doc/howto: index.rst pyporting.rst

2011-02-03 Thread Nick Coghlan
On Fri, Feb 4, 2011 at 8:01 AM, brett.cannon  wrote:
> +Stop Using :mod:`doctest`
> +'
> +While 2to3 tries to port doctests properly, it's a rather tough thing to do. 
> It
> +is probably best to simply convert your critical doctests to :mod:`unittest`.

This advice strikes me as being *way* too strong. Perhaps something like:

Consider limiting use of :mod:`doctest`
===

While 2to3 tries to port doctests properly, it's a rather tough thing
to do. If your test suite is heavily doctest dependent, then you may
end up spending a lot of time manually fixing doctests. The two major
avenues for dealing with this are to either port doctest based tests
over to the unittest module (making them significantly easier for 2to3
to handle) or else to follow the guidelines below for writing 2/3
compatible source code in all doctests (making it so they should run
unmodified on both Python versions).


Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] r88331 - in python/branches/py3k/Doc/howto: index.rst pyporting.rst

2011-02-03 Thread Steven D'Aprano

Nick Coghlan wrote:

On Fri, Feb 4, 2011 at 8:01 AM, brett.cannon  wrote:

+Capturing the Currently Raised Exception
+
+One change between Python 2 and 3 that will require changing how you code is
+accessing the currently raised exception.  In Python 2 the syntax to access the
+current exception is::


I think that the semantic change is *much* more important that the 
syntax change.


In 2.6:

>>> try:
... len(None)
... except TypeError as e:
... pass
...
>>> e
TypeError("object of type 'NoneType' has no len()",)


In 3.1:

>>> try:
... len(None)
... except TypeError as e:
... pass
...
>>> e
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'e' is not defined



--
Steven

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] r88331 - in python/branches/py3k/Doc/howto: index.rst pyporting.rst

2011-02-03 Thread Brett Cannon
On Thu, Feb 3, 2011 at 15:10, Nick Coghlan  wrote:
> On Fri, Feb 4, 2011 at 8:01 AM, brett.cannon  
> wrote:
>> +Stop Using :mod:`doctest`
>> +'
>> +While 2to3 tries to port doctests properly, it's a rather tough thing to 
>> do. It
>> +is probably best to simply convert your critical doctests to 
>> :mod:`unittest`.
>
> This advice strikes me as being *way* too strong. Perhaps something like:

I will change it to make sure that it states that you may want to port
your doctests if all you have is one massive set, but I do not think
it is "*way* too strong". Massive doctest inputs are bad enough as it
is to edit when you don't have a shift in syntax (e.g., I have a patch
waiting for 3.3 which causes entire test suites to skip because they
are a massive doctest and it is not reasonable nor easy to make
something conditional based on whether a trace function is set).
Trying to port them to new syntax is just that much harder (and a
complaint I came across online while researching the HOWTO).

-Brett

>
> Consider limiting use of :mod:`doctest`
> ===
>
> While 2to3 tries to port doctests properly, it's a rather tough thing
> to do. If your test suite is heavily doctest dependent, then you may
> end up spending a lot of time manually fixing doctests. The two major
> avenues for dealing with this are to either port doctest based tests
> over to the unittest module (making them significantly easier for 2to3
> to handle) or else to follow the guidelines below for writing 2/3
> compatible source code in all doctests (making it so they should run
> unmodified on both Python versions).
>
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [email protected]   |   Brisbane, Australia
> ___
> Python-checkins mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-checkins
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Issue #11051: system calls per import

2011-02-03 Thread Rian Hunter
Hello

Speaking from experience from my observations on millions of machines the 
stat() call is *very slow* when compared to readdir(), FindNextFile(), 
getdirentriesattr(), etc. When we switched from a file system indexer that 
stat()ed every file to one that read directories we noticed an average speedup 
of about 10x.

You can probably attribute this to the fact that in file system indexing the 
raw system call volume is much lower (not having to stat() each file, just read 
the directories) but also due to the fact that there is much less HD seeking 
(stat() has to jump around the HD, usually all directory entries fit in one 
block). If you only need to test for the existence of multiple files and don't 
need the extra information that stat() gives you, it might make sense to avoid 
the context switch/IO overhead.  

Rian

On Jan 31, 2011, at 4:43 AM, Antoine Pitrou wrote:

> On Mon, 31 Jan 2011 00:08:25 -0800
> Guido van Rossum  wrote:
>> 
>> (Basically I am biased to believe that stat() is a pretty slow system
>> call -- this may just be old NFS lore though.)
> 
> I don't know about NFS, but starting a Python interpreter located on a
> Samba share from a Windows VM is quite slow too.
> I think Martin is right for the common case: on a local filesystem on a
> modern Unix, stat() is certainly very fast. Remote or
> distributed filesystems seem to be more of a problem.
> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/rian%40dropbox.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com