execnet-1.1: cross-interpreter distributed execution library
execnet-1.1 is a backward compatible beta release of the popular (>53000 pypi downloads of 1.0.9) cross-interpreter execution library. If you are in need of connecting Python2 and Python3 and/or want to throw PyPy in your deployment mix, then you might want to join Quora and many others and try out execnet. execnet provides a share-nothing model with channel-send/receive communication and distributed execution across many Python interpreters across version, platform and network barriers. See below for changes and see here for extensive documentation and tested examples: http://codespeak.net/execnet Particular thanks to Ronny Pfannschmidt for a lot of internal cleanups and to Alex Gaynor for providing feature patches. Have fun, holger 1.1 (compared to 1.0.9) - introduce execnet.dumps/loads providing serialization between python interpreters, see http://codespeak.net/execnet/basics.html#dumps-loads - group.remote_exec now supports kwargs as well - support per channel string coercion configuration, helping with dealing with mixed Python2/Python3 environments. - Popen2IO.read now reads correct amounts of bytes from nonblocking fd's - added a ``dont_write_bytecode`` option to Popen gateways, this sets the ``sys.dont_write_bytecode`` flag on the spawned process, this only works on CPython 2.6 and higher. Thanks to Alex Gaynor. - added a pytest --broken-isp option to skip tests that assume DNS queries for unknown hosts actually are resolved as such (Thanks Alex Gaynor) - fix issue 1 - decouple string coercion of channels and gateway - fix issue #2 - properly reconfigure the channels string coercion for rsync, so it can send from python2 to python3 - refactor socketserver, so it can be directly remote_exec'd for starting a socket gateway on a remote -- http://mail.python.org/mailman/listinfo/python-list
Re: Dr. Dobb's Python-URL! - weekly Python news and links (Dec 30)
On Fri, Dec 31, 2004 at 19:18 +0100, Alex Martelli wrote: > Cameron Laird <[EMAIL PROTECTED]> wrote: >... > > Yippee! The martellibot promises to explain Unicode for Pythoneers. > > http://groups-beta.google.com/group/comp.lang.python/msg/6015a5a05c206712 > > Uh -- _did_ I? Eeep... I guess I did... mostly, I was pointing to > Holger Krekel's very nice recipe (not sure he posted it to the site as > well as submitting it for the printed edition, but, lobby _HIM_ about > that;-). FWIW, i added the recipe back to the online cookbook. It's not perfectly formatted but still useful, i hope. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/361742 cheers, holger P.S: happy new year. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python evolution: Unease
Hi Roman, On Wed, Jan 05, 2005 at 00:44 +0300, Roman Suzi wrote: > Python could have honest support of concepts. Everything else will be > available with them. > > That is the whole point that Python supports GP. It is only one step > to do concepts right (and GvR it seems want type-checking into Python 3.0 > anyway), so support for concepts/concept checking is very logical, > isn't it? As an innocent by-dropper into this thread: a) do you know Armin Rigo's "semantic model" page dealing with concepts? http://arigo.tunes.org/semantic_models.html b) do you have some concise definition of what you mean by "concept"? Myself, i appreciate the combination of testing and python's flexibility so much that i don't long for type declarations, at least not to the degree that would warrant syntax additions. Now for interfaces, for me this references more a documentation issue than a syntax one: I'd like to have a full-featured runtime browser that allows to quickly navigate cross-referenced life python objects. Back and forth in execution time, preferably. This probably requires the interpreter to help, track and record more information (which is one of the possibilities with PyPy). It doesn't neccessarily require any new syntax though. And I don't really see the point of adding interface hierarchies to already large frameworks. To me this adds to - what i call - "naming complexity", i.e. the number of names a programmer needs to remember to deal with a library or framework. For me, it's not really the syntax that is the problem with interfaces in Zope3 or twisted, but the sheer amount of names (each indicating certain concepts and behaviours) i am confronted with. Not sure why i am saying all this here but have fun, holger -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError of a module instance
On Mon, Dec 26, 2005 at 17:41 +, Paolino wrote: > I'd like to catch AttributeError on the module level,so that I can > declare default bindings for useds defore definition.How is this to be > done?Thanks for help. It cannot be done directly but with a small hack. This is the idea: import sys class A: def __getattr__(self,name): if name[:1] != '_': return name raise AttributeError(name) sys.modules['a'] = A() import a print a.helloworld # will print "helloworld" Note, that subsequently you can "import a" also from other modules because the import statement consults sys.modules. This is all a bit bothersome and thus it's often easier to just do "from somemod import a" directly and forget about the above magic. cheers, holger -- http://mail.python.org/mailman/listinfo/python-list
Re: Release of PyPy 0.7.0
Hi Valentino, Michael, all, On Sun, Aug 28, 2005 at 20:46 +0200, Valentino Volonghi aka Dialtone wrote: > Michael Sparks <[EMAIL PROTECTED]> wrote: > > Would it be useful for people to start trying out their modules/code to see > > if they work with this release, and whether they can likewise be translated > > using the C/LLVM backends, or would you say this is too early? (I'm more > > thinking in terms of it providing real world usecases in the hope of > > finding things that don't work - rather than anything else) > > This is not how it works. Pypy doesn't translate your code to C/LLVM. > Currently PyPy can only translate a pretty simple subset of python > called RPython which has a very C-like syntax (but without memory > management code). This is needed to allow type inference inside the > interpreter code. This is mostly true but maybe a bit misleading. The subset of Python that we can translate does not involve special syntax. For example we can translate http://codespeak.net/pypy/dist/pypy/translator/goal/richards.py and gain some speed up by a factor of 70 over interpretation with CPython. The "RPython" limitation mainly refers to the way how dynamically you use Python and how unambigously your values flow across your functions and methods. Also you cannot, for example, create classes dynamically while creating instances is, of course, no problem. Also, we support all Python control flow statements. For a more detailed list, look at: http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#rpython-definition-not The real culprit is that we don't currently intend to try offering a tool that can automatically translate Python modules to C because we are still experimenting with and targeting our own Python interpreter and tackling the challenges that arise out of translating this efficiently. It's a matter of focus of the current dev group and not one of technical feasibility. If one or more people would want to care for producing, testing and documenting such a tool on top of our current code base then the next weeks might be good for that as we intend to go for some cleanup refactorings anyway. > The code in your application is application code and can be whatever you > want, you may try to translate it to C/LLVM but it won't be that good of > course because the annotator is not that intelligent. See above. You can try to translate python programs with a bit of effort and determination to find your way :-) > Just In Time compilation a-la-psyco is planned before the 1.0 release of > pypy. We don't have a fixed version roadmap but indeed, we plan to work on JIT compilation and other niceties rather soon now. cheers, holger -- http://mail.python.org/mailman/listinfo/python-list
Re: Pypy - Which C modules still need converting to py?
Hi Caleb, On Tue, Feb 08, 2005 at 22:32 -0500, Caleb Hattingh wrote: > I saw it on a webpage a few days ago, can't seem to find it again. Tried > a google search for > > "pypy needed translate C modules" > > but that didn't turn up what I was looking for. Anyone have that page > ref handy listing the C modules that the pypy team need translated into > python? I think that currently the best page for that actually is the compliancy test results page: http://codespeak.net/~hpk/pypy-testresult/ especially in the "non-core" section further below. You'll see a list of the currently not implemented C-level modules. However, most interesting is completing full posix/os-module support e.g. for listdir() and process-creation functionality. Also having a socket-API wrapper at RPython/low-level-function level and for e.g. 'zlib' seem like "big time" enablers for allowing more programs/modules to run on top of pypy-c/pypy-llvm. (we are employing a "pypy-XYZ" scheme where XYZ marks the backend). Recently, Niklaus Heidimann successfully implemented the array and _sre module as part of his SOC project, of which he got the latter to translate to low-level during the last Heidelberg sprint! cheers, holger -- http://mail.python.org/mailman/listinfo/python-list
Re: py.test anyone?
Hi Stephen, [Stephen Boulet Mon, Nov 22, 2004 at 11:14:57AM -0600] > Have people been using py.test? I was hoping to try it out but was > running into subversion problems (behind a corporate firewall, though > there is a windows registry hack for that which didn't work for me). you may try to use the svn apache instance at 8080 aka svn co http://codespeak.net:8080/svn/py/dist distpy Also please note that 'py.test' has not been released yet. I plan to release it beginning of next year after it received some more development and a big real-life integration into the PyPy project. Nevertheless, there is quite some documentation under the 'py.test' link found here: http://codespeak.net/py/current/doc/ there also is a "getting started" link which now incorporates the above ":8080" suggestion. have fun, holger -- http://mail.python.org/mailman/listinfo/python-list
Re: style query: function attributes for return codes?
Hi George,
[george young Fri, Dec 10, 2004 at 10:45:47AM -0500]
> [python 2.3.3, x86 linux]
> I recently found myself writing something like:
>
> def get_connection():
> if tcp_conn():
> if server_allows_conn():
> return 'good_conn'
> else:
> return 'bad_auth'
> else:
> return 'no_server'
>
> cn = get_connection()
> if cn == 'good_con': ...
>
>
> This is obviously just evil, since a misspelling in the string
> return is treacherous.
Right.
I usually like to look at such problems from the angle of
what is most convenient for the *caller* side?
And having to adress function attributes does
not seem convenient. I'd probably like to do from
the caller side something like:
conn = get_connection()
if conn.good:
...
elif conn.badauth:
...
elif conn.noserver:
...
which allows you to freely choose and hide your actual
implementation at the "called" side. Example:
class Connection(object):
def __init__(self, **kw):
for name in kw:
assert name in ('good', 'badauth', 'noserver'), name
setattr(self, name, kw[name])
def get_connection():
if tcp_conn():
if server_allows_conn():
return Connection(good=True)
else:
return Connection(badauth=True)
else:
return Connection(noserver=True)
And btw, the view of "what do i want at the caller side?"
is natural if you do test-driven development and actually
first write your tests. Another reason why testing is
a good thing :-)
cheers & HTH,
holger
--
http://mail.python.org/mailman/listinfo/python-list
Re: style query: function attributes for return codes?
[Reinhold Birkenfeld Fri, Dec 10, 2004 at 08:42:10PM +0100]
> holger krekel wrote:
> > class Connection(object):
> > def __init__(self, **kw):
> > for name in kw:
> > assert name in ('good', 'badauth', 'noserver'), name
> > setattr(self, name, kw[name])
> >
> > def get_connection():
> > if tcp_conn():
> > if server_allows_conn():
> > return Connection(good=True)
> > else:
> > return Connection(badauth=True)
> > else:
> > return Connection(noserver=True)
>
> That's evil, because "if conn.good" raises an AttributeError instead of
> evaluating to False if the connection is not good. You would have to
> inizialize all three attributes in every construction of a Connection
> object.
Ups, you are right of course. I somehow managed to delete the line ...
class Connection(object):
good = badauth = noserver = False
thanks for pointing it out.
holger
--
http://mail.python.org/mailman/listinfo/python-list
Re: [Distutils] Call for information - What assumptions can I make about Unix users' access to Windows?
On Sat, Nov 15, 2014 at 10:45 +, Paul Moore wrote: > On 7 November 2014 15:46, Paul Moore wrote: > > To that end, I'd like to get an idea of what sort of access to Windows > > a typical Unix developer would have. > > Thanks to all who contributed to this thread. > > Based on the feedback, I think it's going to be useful to provide two > options. First of all, an EC2 AMI that can be used by people without > access to a local Windows system. While other cloud providers are a > possibility, EC2 provides a free tier (for the first year) and is > well-known, so it's probably the easiest to get started with (at least > it was for me!) Also, I will provide a script that can be used to > automatically build the environment on a newly-installed machine. The > idea is that you can use this on a Windows VM (something that a number > of people have said they have access to). > > The script may be usable on an existing machine, but it's hard to make > it robust, as there are too many failure modes to consider (software > already installed, configuration and/or permission differences, etc). > So while such use may be possible, I probably won't consider it as > supported. Thanks Paul for going through this! Looking forward to the link/code. holger > Thanks again, > Paul > ___ > Distutils-SIG maillist - distutils-...@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -- https://mail.python.org/mailman/listinfo/python-list
[ann] first release of PyPy
Welcome to PyPy 0.6 *The PyPy Development Team is happy to announce the first public release of PyPy after two years of spare-time and half a year of EU funded development. The 0.6 release is eminently a preview release.* What it is and where to start - Getting started:http://codespeak.net/pypy/index.cgi?doc/getting_started.html PyPy Documentation: http://codespeak.net/pypy/index.cgi?doc PyPy Homepage: http://codespeak.net/pypy/ PyPy is a MIT-licensed reimplementation of Python written in Python itself. The long term goals are an implementation that is flexible and easy to experiment with and retarget to different platforms (also non-C ones) and such that high performance can be achieved through high-level implementations of dynamic optimisation techniques. The interpreter and object model implementations shipped with 0.6 can be run on top of CPython and implement the core language features of Python as of CPython 2.3. PyPy passes around 90% of the Python language regression tests that do not depend deeply on C-extensions. Some of that functionality is still made available by PyPy piggy-backing on the host CPython interpreter. Double interpretation and abstractions in the code-base make it so that PyPy running on CPython is quite slow (around 2000x slower than CPython ), this is expected. This release is intended for people that want to look and get a feel into what we are doing, playing with interpreter and perusing the codebase. Possibly to join in the fun and efforts. Interesting bits and highlights - The release is also a snap-shot of our ongoing efforts towards low-level translation and experimenting with unique features. * By default, PyPy is a Python version that works completely with new-style-classes semantics. However, support for old-style classes is still available. Implementations, mostly as user-level code, of their metaclass and instance object are included and can be re-made the default with the ``--oldstyle`` option. * In PyPy, bytecode interpretation and object manipulations are well separated between a bytecode interpreter and an *object space* which implements operations on objects. PyPy comes with experimental object spaces augmenting the standard one through delegation: * an experimental object space that does extensive tracing of bytecode and object operations; * the 'thunk' object space that implements lazy values and a 'become' operation that can exchange object identities. These spaces already give a glimpse in the flexibility potential of PyPy. See demo/fibonacci.py and demo/sharedref.py for examples about the 'thunk' object space. * The 0.6 release also contains a snapshot of our translation-efforts to lower level languages. For that we have developed an annotator which is capable of infering type information across our code base. The annotator right now is already capable of successfully type annotating basically *all* of PyPy code-base, and is included with 0.6. * From type annotated code, low-level code needs to be generated. Backends for various targets (C, LLVM,...) are included; they are all somehow incomplete and have been and are quite in flux. What is shipped with 0.6 is able to deal with more or less small/medium examples. Ongoing work and near term goals - Generating low-level code is the main area we are hammering on in the next months; our plan is to produce a PyPy version in August/September that does not need to be interpreted by CPython anymore and will thus run considerably faster than the 0.6 preview release. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support from numerous people. Please feel free to give feedback and raise questions. contact points: http://codespeak.net/pypy/index.cgi?contact contributor list: http://codespeak.net/pypy/index.cgi?doc/contributor.html have fun, Armin Rigo, Samuele Pedroni, Holger Krekel, Christian Tismer, Carl Friedrich Bolz PyPy development and activities happen as an open source project and with the support of a consortium funded by a two year EU IST research grant. Here is a list of partners of the EU project: Heinrich-Heine University (Germany), AB Strakt (Sweden) merlinux GmbH (Germany), tismerysoft GmbH(Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden) -- http://mail.python.org/mailman/listinfo/python-list
PyPy 0.6.1
Hi again,
On Fri, May 20, 2005 at 23:38 +0200, holger krekel wrote:
> The PyPy 0.6 release
>
has already been superseded by the PyPy 0.6.1 bug-fix release.
We are temporarily not having access to that time machine
and thus have to fix things the old way, unfortunately.
But we are working on improving this situation.
The bug is trivial enough: py.py crashes on string formatting
the second time you run it. You can find the fixed packages
and svn-locations here:
Getting started:
http://codespeak.net/pypy/index.cgi?doc/getting_started.html
have fun & sorry for the inconvenience,
holger krekel
> *The PyPy Development Team is happy to announce the first
> public release of PyPy after two years of spare-time and
> half a year of EU funded development. The 0.6 release
> is eminently a preview release.*
>
> What it is and where to start
> -
>
> Getting started:
> http://codespeak.net/pypy/index.cgi?doc/getting_started.html
>
> PyPy Documentation: http://codespeak.net/pypy/index.cgi?doc
>
> PyPy Homepage: http://codespeak.net/pypy/
>
> PyPy is a MIT-licensed reimplementation of Python written in
> Python itself. The long term goals are an implementation that
> is flexible and easy to experiment with and retarget to
> different platforms (also non-C ones) and such that high
> performance can be achieved through high-level implementations
> of dynamic optimisation techniques.
>
> The interpreter and object model implementations shipped with 0.6 can
> be run on top of CPython and implement the core language features of
> Python as of CPython 2.3. PyPy passes around 90% of the Python language
> regression tests that do not depend deeply on C-extensions. Some of
> that functionality is still made available by PyPy piggy-backing on
> the host CPython interpreter. Double interpretation and abstractions
> in the code-base make it so that PyPy running on CPython is quite slow
> (around 2000x slower than CPython ), this is expected.
>
> This release is intended for people that want to look and get a feel
> into what we are doing, playing with interpreter and perusing the
> codebase. Possibly to join in the fun and efforts.
>
> Interesting bits and highlights
> -
>
> The release is also a snap-shot of our ongoing efforts towards
> low-level translation and experimenting with unique features.
>
> * By default, PyPy is a Python version that works completely with
> new-style-classes semantics. However, support for old-style classes
> is still available. Implementations, mostly as user-level code, of
> their metaclass and instance object are included and can be re-made
> the default with the ``--oldstyle`` option.
>
> * In PyPy, bytecode interpretation and object manipulations
> are well separated between a bytecode interpreter and an
> *object space* which implements operations on objects.
> PyPy comes with experimental object spaces augmenting the
> standard one through delegation:
>
> * an experimental object space that does extensive tracing of
> bytecode and object operations;
>
> * the 'thunk' object space that implements lazy values and a 'become'
> operation that can exchange object identities.
>
> These spaces already give a glimpse in the flexibility potential of
> PyPy. See demo/fibonacci.py and demo/sharedref.py for examples
> about the 'thunk' object space.
>
> * The 0.6 release also contains a snapshot of our translation-efforts
> to lower level languages. For that we have developed an
> annotator which is capable of infering type information
> across our code base. The annotator right now is already
> capable of successfully type annotating basically *all* of
> PyPy code-base, and is included with 0.6.
>
> * From type annotated code, low-level code needs to be generated.
> Backends for various targets (C, LLVM,...) are included; they are
> all somehow incomplete and have been and are quite in flux. What is
> shipped with 0.6 is able to deal with more or less small/medium examples.
>
>
> Ongoing work and near term goals
> -
>
> Generating low-level code is the main area we are hammering on in the
> next months; our plan is to produce a PyPy version in August/September
> that does not need to be interpreted by CPython anymore and will
> thus run considerably faster than the 0.6 preview release.
>
> PyPy has been a community effort from the start and it would
> not have got that far without the coding and feedback support
> from numerous people. Please fee
Re: first release of PyPy
On Sun, May 22, 2005 at 19:18 +0200, ionel wrote: > this is interesting > anyway i'm to lazy to read so i'll just ask: > can PyPy at the current state of develepment help me improve my python > programs? (speed) no, it can't at this stage. You might check out Psyco, the specializing compiler for Python: http://psyco.sf.net holger -- http://mail.python.org/mailman/listinfo/python-list
Re: first release of PyPy
Hi Kay, On Mon, May 23, 2005 at 13:39 -0700, Kay Schluehr wrote: > Does it mean You create an RPython object that runs on top of CPython, > but is just an RPython facade wrapped around a CPython object? So You > have four kinds of Pythons: > > RPy - translateable into LL code > APy - non-translateable but interpretable by translated RPy > RPy* - non-translateable but consistent interface with RPy. Calls > APy* > APy* - not translateable and not interpreteable by translated RPy > > "Selfhosting" would imply vanishing RPy* and APy*. But the problem > seems to be that selfhosting must somehow be broken because the system > needs to interact with OS-dependend librarys. As long as You run the > system upon CPython the problem does not occur but once You drop it, a > kind of "extension objectspace" must be created which is translated > into code with nice interfacing properties. You are mostly right but 'extension objectspace' is misleading. Object Spaces are only responsible for manipulating Python application objects. To get rid of 'faked' objects we need implementations for IO access and operating system interactions. Those can sometimes even be written in pure python (applevel) as is the case for a preliminary version of a file object. > RPython translations will be sufficient and another ext-objectspace is > just useless epi-cycling? Conceptually, we need a good concept to perform foreign function invocation (FFI) much like ctypes or other approaches do. However, concretely, we might at first just write some very low-level code (even lower level than RPython) to interact with os-level APIs and weave that into the translation process. This is an area we are beginning to explore in more depth currently. cheers, holger -- http://mail.python.org/mailman/listinfo/python-list
PyPy trillke sprints (Feb/March 2007)
= PyPy Trillke "EU and beyond!" sprints (25-28th Feb, 1-5th March 2006) = ..image:: http://www.trillke.net/images/HausPanorama0304_113kb.jpg Some two years and some thousands of commits later, the EU project period of the PyPy (http://codespeak.net/pypy) project is about to close ... and a new period to begin: we are going for a sprint of three days of focusing on EU reports and administrative issues, and another three day sprint of happy hacking on the numerous interesting open ends of PyPy, the source code. We also intend to discuss and state our view on the time after the EU period (March 2007 is the last EU funded month), because clearly the project will not stop after our successful (knock knock!) completion of the EU project. It's already clear that some of us seriously plan for a relaxation time, i.e. one without having many obligations. But that should not keep us from thinking and envisioning the development from April on. So to the Sprint: we have two parts of the sprint, first the EU and second the public all-invited part:: 26th Feb - 28th Feb EU reports and adminstrative sprint 1st March break day / arrival for coding sprint 2nd March - 4th March public coding "beyond EU" sprint days All days are meant as full days, so please arrive 25th Feb / 1st March and leave 5th March if you can (this help us with pairing, introductions and logistical planning). - Possible Topics for the coding sprint - * working on .NET, Java and other backends * working on the Javascript, Prolog or a new frontend * Tuning the JIT, refining approaches `[1]`_ * Fixing PyPy-1.0 problems / improving it (PyPy-1.0 is scheduled for Mid February, btw) * improving the py lib/py.test, build environments, preparing for server administration efforts from April on * Work on/with prototypes that use the new features that PyPy enables: distribution `[2]`_ (based on transparent proxying `[3]`_), security `[4]`_, persistence `[5]`_, etc. `[6]`_. * discussion about the time after March, and how to organize/co-ordinate ourselves * all topics that are of interest otherwise (including maybe working on some particular EU related topics still!) .. _[1]: http://codespeak.net/pypy/dist/pypy/doc/jit.html .. _[2]: http://codespeak.net/svn/pypy/dist/pypy/lib/distributed/ .. _[3]: http://codespeak.net/pypy/dist/pypy/doc/proxy.html .. _[4]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html#security .. _[5]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html#persistence .. _[6]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html --- Location --- The sprint takes place at the Trillke Gut Steinbergstr. 42 Hildesheim, Germany http://www.trillke.net If you come to Hildesheim main station, take the Bus Number 3 (Hildesheimer Wald) and get out at "Waldquelle". Walking back a 100 meters gets you to a small street on the right leading to a big white building, the Trillke Gut. We'll have at least one larger room for sprinting, and a kitchen for our use. --- Accomodation --- We can probably arrange for some cheap or no-cost private accomodation, in private rooms located up in the house (and being part of "living groups" of 5-10 people). There also is a nice Guest house that has been used during recent events: http://www.gaestehaus-klocke.de/ and a four-star hotel 500 meters away from the Trillke: http://www.berghoelzchen.de/ --- Registration --- please subscribe to the pypy-sprint mailing list http://codespeak.net/mailman/listinfo/pypy-sprint and register by subversion: http://codespeak.net/svn/pypy/extradoc/sprintinfo/trillke-2007/people.txt or - if you have no checkin-rights - post to the pypy-sprint list above. -- merlinux GmbH Steinbergstr. 4231139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) -- http://mail.python.org/mailman/listinfo/python-list
ANN: py lib 0.9.0: py.test, distributed execution, microthreads ...
py lib 0.9.0: py.test, distributed execution, greenlets and more
==
Welcome to the 0.9.0 py lib release - a library aiming to
support agile and test-driven python development on various levels.
Main API/Tool Features:
* py.test: cross-project testing tool with many advanced features
* py.execnet: ad-hoc code distribution to SSH, Socket and local sub processes
* py.magic.greenlet: micro-threads on standard CPython ("stackless-light")
* py.path: path abstractions over local and subversion files
* rich documentation of py's exported API
* tested against Linux, OSX and partly against Win32, python 2.3-2.5
All these features and their API have extensive documentation,
generated with the new "apigen", which we intend to make accessible
for other python projects as well.
Download/Install: http://codespeak.net/py/0.9.0/download.html
Documentation/API: http://codespeak.net/py/0.9.0/index.html
Work on the py lib has been partially funded by the
European Union IST programme and by http://merlinux.de
within the PyPy project.
best, have fun and let us know what you think!
Holger Krekel, Maciej Fijalkowski,
Guido Wesdorp, Carl Friedrich Bolz
--
http://mail.python.org/mailman/listinfo/python-list
execnet-1.0.1: more robust and rapid python deployment
Hi everybody,
Just uploaded execnet-1.0.1 featuring a new motto:
execnet is about rapid-python deployment, be it for
multiple CPUs, different platforms or python versions.
This release brings a bunch of refinements and most
importantly more robust termination, handling of CTRL-C
and automatically tested documentation::
http://codespeak.net/execnet
have fun,
holger
1.0.1
- revamp and better structure documentation
- new method: gateway.hasreceiver() returns True
if the gateway is still receive-active. remote_status
now only carries information about remote execution status.
- new: execnet.MultiChannel provides basic iteration/contain interface
- new: execnet.Group can be indexed by integer
- new: group.makegateway() uses group.default_spec if no spec is given
and the execnet.default_group uses ``popen`` as a default spec.
- have popen-gateways use imports instead of source-strings,
also improves debugging/tracebacks, as a side effect
popen-gateway startup can be substantially faster (>30%)
- refine internal gateway exit/termination procedure
and introduce group.terminate(timeout) which will
attempt to kill all subprocesses that did not terminate
within time.
- EOFError on channel.receive/waitclose if the other
side unexpectedly went away. When a gateway exits
it now internally sends an explicit termination message
instead of abruptly closing.
- introduce a timeout parameter to channel.receive()
and default to periodically internally wake up
to let KeyboardInterrupts pass through.
- EXECNET_DEBUG=2 will cause tracing to go to stderr,
which with popen slave gateways will relay back
tracing to the instantiator process.
1.0.0
* introduce execnet.Group for managing gateway creation
and termination. Introduce execnet.default_group through which
all "global" calls are routed. cleanup gateway termination.
All Gateways get an id through which they can be
retrieved from a group object.
* deprecate execnet.XYZGateway in favour of direct makegateway() calls.
* refine socketserver-examples, experimentally introduce a
way to indirectly setup a socket server ("installvia")
through a gateway url.
* refine and automatically test documentation examples
1.0.0b3
* fix EXECNET_DEBUG to work with win32
* add support for serializing longs, sets and frozensets (thanks
Benjamin Peterson)
* introduce remote_status() method which on the low level gives
information about the remote side of a gateway
* disallow explicit close in remote_exec situation
* perform some more detailed tracing with EXECNET_DEBUG
1.0.0b2
* make internal protocols more robust against serialization failures
* fix a seralization bug with nested tuples containing empty tuples
(thanks to ronny for discovering it)
* setting the environment variable EXECNET_DEBUG will generate per
process trace-files for debugging
1.0.0b1
* added new examples for NumPy, Jython, IronPython
* improved documentation
* include apipkg.py for lazy-importing
* integrated new serializer code from Benjamin Peterson
* improved support for Jython-2.5.1
1.0.0alpha2
* improve documentation, new website
* use sphinx for documentation, added boilerplate files and setup.py
* fixes for standalone usage, adding boilerplate files
* imported py/execnet and made it work standalone
--
http://mail.python.org/mailman/listinfo/python-list
py.test-1.2.0: junitxml, standalone test scripts, pluginization
Hi all,
i just released some bits related to automated testing with Python:
py-1.2.0: py.test core which grew junitxml, standalone-script generation
pytest-xdist-1.0: separately installable dist-testing & looponfailing plugin
pytest-figleaf-1.0: separately installable figleaf-coverage testing plugin
See below or at this URL for the announcement:
http://pylib.org/announce/release-1.2.0.html
If you didn't experience much speed-up or previously had problems with
distributed testing i recommend you try to install "pytest-xdist" now and see
if it works better. For me it speeds up some tests runs by 500% on a 4 CPU
machine due to its better internal model and several fixes. (It's five
times because several tests depend on IO and don't block CPU meanwhile).
Another tip: if you use "pip" (best with a virtualenv) you can do e.g.:
pip install pytest-xdist
pip uninstall pytest-xdist
to conveniently activate/deactivate plugins for py.test. easy_install
works ok as well but has no uninstall, yet remains the only option
for installing with Python3 at the moment, though. You need to use
the fine 'distribute' project's easy_install for the latter.
cheers & have fun,
holger
py.test/pylib 1.2.0: junitxml, standalone test scripts, pluginization
py.test is an advanced automated testing tool working with
Python2, Python3 and Jython versions on all major operating
systems. It has a simple plugin architecture and can run many
existing common Python test suites without modification. It offers
some unique features not found in other testing tools.
See http://pytest.org for more info.
py.test 1.2.0 brings many bug fixes and interesting new abilities:
* --junitxml=path will create an XML file for use with CI processing
* --genscript=path creates a standalone py.test-equivalent test-script
* --ignore=path prevents collection of anything below that path
* --confcutdir=path only lookup conftest.py test configs below that path
* a 'pytest_report_header' hook to add info to the terminal report header
* a 'pytestconfig' function argument gives direct access to option values
* 'pytest_generate_tests' can now be put into a class as well
* on CPython py.test additionally installs as "py.test-VERSION", on
Jython as py.test-jython and on PyPy as py.test-pypy-XYZ
Apart from many bug fixes 1.2.0 also has better pluginization:
Distributed testing and looponfailing testing now live in the
separately installable 'pytest-xdist' plugin. The same is true for
'pytest-figleaf' for doing coverage reporting. Those two plugins
can serve well now as blue prints for doing your own.
thanks to all who helped and gave feedback,
have fun,
holger krekel, January 2010
Changes between 1.2.0 and 1.1.1
=
- moved dist/looponfailing from py.test core into a new
separately released pytest-xdist plugin.
- new junitxml plugin: --junitxml=path will generate a junit style xml file
which is processable e.g. by the Hudson CI system.
- new option: --genscript=path will generate a standalone py.test script
which will not need any libraries installed. thanks to Ralf Schmitt.
- new option: --ignore will prevent specified path from collection.
Can be specified multiple times.
- new option: --confcutdir=dir will make py.test only consider conftest
files that are relative to the specified dir.
- new funcarg: "pytestconfig" is the pytest config object for access
to command line args and can now be easily used in a test.
- install 'py.test' and `py.which` with a ``-$VERSION`` suffix to
disambiguate between Python3, python2.X, Jython and PyPy installed versions.
- new "pytestconfig" funcarg allows access to test config object
- new "pytest_report_header" hook can return additional lines
to be displayed at the header of a test run.
- (experimental) allow "py.test path::name1::name2::..." for pointing
to a test within a test collection directly. This might eventually
evolve as a full substitute to "-k" specifications.
- streamlined plugin loading: order is now as documented in
customize.html: setuptools, ENV, commandline, conftest.
also setuptools entry point names are turned to canonical namees ("pytest_*")
- automatically skip tests that need 'capfd' but have no os.dup
- allow pytest_generate_tests to be defined in classes as well
- deprecate usage of 'disabled' attribute in favour of pytestmark
- deprecate definition of Directory, Module, Class and Function nodes
in conftest.py files. Use pytest collect hooks instead.
- collection/item node specific runtest/collect hooks are only called exactly
on matching conftest.py files, i.e. ones which
pylib/py.test 1.0.0 released
Hello everyone,
i am happy to announce pylib/py.test 1.0.0, a MIT-licensed
library geared towards advanced testing and elastic distributed
programming with Python. It features the mature cross-project
py.test automated testing tool with many new features, aiming to:
* allow writing zero-boilerplate automated tests in Python
* offer strong debugging and reporting of test failures
* rapidly run and ad-hoc distribute tests to multiple CPUs/platforms
* support unit-, functional and integration testing
* be easy to extend and tackle growing testing needs,
(current prime example oejskit, a live-browser javascript
unittesting 3rd party plugin)
Please check things out at: http://pytest.org or read the blogged
release announcement:
http://tetamap.wordpress.com/2009/08/04/pylib-1-0-0
or read on for some ...
Distinctive new features shipping with py.test 1.0.0
--
For those already knowing about the older 0.9.2 release or
being experienced with other testing approaches i'll try to
give a summary of the distinctive testing features
shipping with 1.0.0:
* test function arguments ("funcargs"): With this, python test
functions can name arguments and one writes factory functions to
provide instances for such "fixture" arguments. This page
http://codespeak.net/py/dist/test/funcargs.html
contains reference info and tutorial examples.
Test function arguments also allow for natural test parametrization -
one provides several different argument values,
no changes to the test function needed, no magic "yield" for
generating tests anymore - although 1.0 still allows them and
of course still supports traditional xUnit-style setup_module/class/function
or (new) direct runs of unittest.TestCase style tests.
* distributed testing: distributing test runs among Linux/OSX/Windows
hosts and across python-2.4 till python-2.6 interpreters
works reasonably stable now. This means that you can easily
iron out test-module/function specific problems across a
variety of platforms, accelerating the change & test feedback cycle.
* xfail: a new way to mark tests as "expected to fail" which
means they run normally but are reported/counted specially.
This "xfail" mark is meant to mark missing / wrong implementation
rather than missing dependencies / wrong platforms for which one
uses "skip". Especially for larger test suites making
this distinction is very helpful.
* IO-capturing: output of test functions is captured per-test,
by default including any output from sub processes.
This works on all platforms and also (now) interacts well
with the logging module without importing/using it itself so
there are no interferences.
* pastebin: new command line option "--pastebin" to send your test
session output or individual test failures to the Pocoo pastebin
service and prints out URLs. Convenient for quick IRC/messaging
communication.
* plugins: it is now easy to write plugins by implementing one
or more of the 37 hooks which py.test calls to implement
the testing process. There are many examples, among them
the "oejskit" plugin which integrates testing of javascript
code in real-life browsers into a regular test run.
Apart from such separately distributed "cross-project" plugins
you can also write per-project plugins/extensions that lives
with your testing code.
cheers & have fun,
holger
--
Metaprogramming, Python, Testing: http://tetamap.wordpress.com
Python, PyPy, pytest contracting: http://merlinux.eu
--
http://mail.python.org/mailman/listinfo/python-list
