Fwd: Python Multi-Build for Python Extensions Packaging - Report 3

[Mesutcan Kurt]

Hello,

This is the third report of my GSoC project for Debian. During last two
weeks
I've done most of things about the main tool. Before that weeks I have been
 working on project's API. API is almost finished. Still I find
missing/wrong
parts during the coding and changing the API design.

Like I said, main tool's coding process is nearly finished. For now It is
doing
its job for building Python3.X packages. You can find the source code at
[1].

I also coded an example plugin (actually only return values) and
plugins/__init__.py
file which contains like Base class, exception classes and plugin listing
method.

When I started coding I realized that I need extra parameters. I've added
--install-dir
 parameter. You can specify the install directory if there is no install
directory the tools
 pretends that the install directory is "./debian/tmp". After that it checks
whether if the
 directory exists. If not it creates the directory.

I've changed py3build.py a bit. First, I've thought for every process I
invoke py3build's
methods but they are redundant. I've removed them. Now they are called in
main
method of py3build.

I use logging module for printing messages, errors etc.

get_build_system() method retieves list from plugins/__init__.py file. In
that file there is
scan_for_plugins() method. It finds all plugins in plugins/ directory which
files match for
 "[a-zA-Z].py". It keeps the list in plugins_list variable and returns that.
When I'm writing
scan_for_plugins() method I've read codes (f.e. [2]) about that and decided
to name all
plugins' class name as *BuildSystem*. Now, detecting method works with that
way.
(all plugins' class' are called BuildSystem) And I've changed base class
name to
BuildSystemBase.

get_build_system() method is used by determine_build_system() method. If
there is given
build system (by parameter) it tries to use that. If that build system is
not usable then give
error and exits. If there is no given build system. It invokes
get_build_system() and iterates
 that methods return value which is a list. When iterating It create an
instance of that build
system class (if can not create then it gives warning and skips that build
system plugin)
and invokes detect method and append plugin name and detect method's return
value to
the bs_list which contains plugin list which are available. And then sorts
it. If there is error
while sorting list or there is no usable plugin then give error and exits.
If there is multiple
highest detect values then lists that plugins and exists. (maintainer should
select one of
these plugins to build extension.)

In main method, after getting arguments, it checks for verbose mode (or
'DH_VERBOSE'
environment variable) and sets log level (logging.DEBUG or logging.WARN)

If there is no given build_dir then it pretends it is "./".

After that It try to determine build system. If can not then prints error
and exits.

And finally it starts to build extension. Firstly it checks if there is
given version list. If not,
it uses default values and iterates build loop with that values.

First cleans, configures, builds (if there is no --test-only arg) and tests
(if there is --test-only
arg or there is not --no-tests argument)

If there is error (raises plugin.Error) while invoking that method then
prints error and exits.

In verbose mode tool prints every process. (like "configuring for PythonX.Y"
etc.)

I've started writing a plugin for detecting *distutils*. For now, I only
wrote its detect method. It
returns between 0-1.0 floating point value. If everything is OK it returns
1.0. Other methods does
nothing, only returns True but this week I'll finish this plugin and work
for others.

For detecting distutils I've used the code at [3]. First I try to import
that module if not, I've copied
required methods to plugins/_packaging.py file, I use them.

I don't use is_distutils() method directly. For give probability I invoke
what it invokes and give them
a number. (f.e. _has_setup_py has 0.5, _has_pkg_info has 0.25 and
_has_distutil_text has 0.25.
But if _has_setup_py returns False then detect method doesn't look other
methods since they
require setup.py.)

For now that's all. See you at next time.

[1] git://git.debian.org/~mesutcan-guest/py3multibuild.git
[2] http://svn.berlios.de/viewvc/griffith/trunk/lib/plugins/extensions/
[3] 
http://hg.python.org/cpython/file/a7e00934baf1/Lib/
packaging/util.py

-- 
Mesutcan Kurt

Re: Current state of packaging Python software for Debian

[Robert Collins]

On Sat, Jul 2, 2011 at 2:07 AM, Barry Warsaw  wrote:
>>In some sub-communities, py.test or nosetests are the standard, not
>>setup.py test.
>
> Yes, but if I understand where Michael is taking unittest2, those can just be
> refactored to be plugins instead of separate standards.  Then `python setup.py
> test` can be the preferred and documented standards, while allowing those
> other command lines to also exist.

unittest2 is still a unittest runner at heart; the basic model is
sound to scale up to N processes and so forth (see tox or
testrespository for instance), but compatibility with arbitrary ways
of running is pretty tricky. See for instance the guts of the
following three runners: trial, django and zc.testing.testrunner.

All I'm saying is don't hold your breath: those communities could have
plugged into the original unittest compatibly but didn't - I think it
needs to be massively more clear *how* to do so, and on older Pythons
for those communities - they don't live on the edge ;) - for
one-runner-with-plugins to reach that point.

As a data point, in the java world multiple runners is still the case,
with some common interop on output format.

-Rob


--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAJ3HoZ1mOqYGcMfn0Lf514dBmmuBOD5GL78=mvfrewqxvsc...@mail.gmail.com