On Thu, 15 Oct 2009, Christian Heimes wrote:
Andi Vajda wrote:
Hmm, that's an idea...
It is my understanding that for a package to 'work' in PyPI it needs to have
some setup.py support, right ? JCC, for example, is already on PyPI since
it's entirely built with setup.py.
PyLucene, on the other hand, does not have a setup.py file, it relies on JCC
and 'make' to provide this functionality. If JCC became a
distutils/setuptools 'compiler', using setup.py to do a PyLucene build might
become easier.
As far as I know -- and I'm pretty sure -- PyPI doesn't enforce any
rules how a package is installed. As long as its related to Python it's
just fine. Setup.py is the canonical way to install Python packages but
I know of no rule that forbids Makefile based installation routines. The
current build system could be improved but it's not a show stopper for
PyPI.
Even in a distribution that included binaries, setup.py would be needed to
install them. For Windows binaries this would be most welcome for many
reasons. For other systems, in particular those where the path to the libjcc
shared library (shared mode) is linked into the executables depending on
it (using -rpath or -R), such a binary install might become more
problematic.
It makes no sense to offer binaries of an open source Python extension
except for Windows. Any other important operating system (Linux, Mac,
*BSD) has the necessary tools already on board or they can be easily
installed. Just Windows is a pain in the ....
Agreed.
I had to install Python 2.4 + 2.5 + 2.6, setuptools for every Python
version, JDK, ant, MSys, MinGW, svn command line tools, VS 7.1 and VS
9.0 just to build the binaries. My build.bat sets several environment
variables and some of the are really tricky, for example backslashes
don't work for the Apache Ant path as argument for make.
Ugh :)
But maybe I'm misunderstanding how this works. Do you already have
easy_install'able binary distributions for PyLucene on Windows ?
With setup.py ?
If so, are they dependent on another binary distribution of JCC itself which
PyLucene depends on but includes in its sources ? I'd imagine that in a PyPI
distribution scenario, PyLucene would _not_ include JCC but would list it as
a dependency instead.
As a matter of fact, we could experiment with this right away with JCC since
it is already listed on PyPI (in source only).
What do I need to do to give you upload permissions ? What's your account on
PyPI ?
I haven't tried to install my packages with easy install yet. I'll do so
tomorrow. I believe that it should just work, though. For now users
would have to install JCC first because JCC isn't listed as dependency
for PyLucene or any other distribution created with JCC. Also it
wouldn't be possible to create new packages with the easy installed JCC
because the config files reflects my local settings. I guess most people
can either live with the fact or build JCC on their own.
Well, building JCC on Windows is half the problems of what people report on
this list. They forget to set the PATH to include the required DLLs, don't
know how to operate a C compiler, etc... etc..
It's true that getting rid of the cygwin/msys requirement on Windows just to
operate PyLucene's Makefile would be a vast improvement. That Makefile used
to be quite fancy(!) in the gcj days but now it just builds jars by calling
ant and invokes JCC which does the rest. Both things could be ported to
something else on Windows or provided by a binary package as you suggest.
I like to provide wininst installer (.exe files), too. It looks like JCC
already supports bdist_wininst but PyLucene's Makefiles doesn't expose it.
A patch surfacing this would be welcome.
My account name on PyPI is 'tiran'. The maintainer role should be
sufficient to upload new files. Are you creating the pylucene package?
At this point, I'm more inclined to just say "go for it" since you're
creating a Windows specific package for PyPI users. I have no objections to
your creating and owning a pylucene-windows package or somesuch on PyPI that
serves as a vehicle for distributing PyLucene Windows binaries. This would
be a welcome development, this is open source at its best :)
If, in addition, you'd like to also provide JCC binaries for Windows - since
you've already built them - and since they're easy to upload with setup.py
bdist upload (provided nothing is left out, such as jcc.dll) - let me know
and I'll give you the privileges for the existing JCC PyPI project.
It's been my itch for a while to extend distutils/setuptools to make JCC a
'compiler' there so that a vanilla setup.py could be used to build PyLucene.
The Makefile would then ultimately disappear.
I just don't get warm fuzzies about investing time in the setuptools space
right now.
Bill, could you please expand a bit on that red flag you just alluded to
for those of us who don't follow the distutils list.
Currently, there are two reasons I use setuptools over distutils with JCC:
- Python 2.3 support (admittedly, not so relevant anymore)
- Library support for shared mode (with a patch for Linux) (not supported
on Solaris at all). As long as distutils isn't capable of building a
plain shared library (as opposed to a python extension shared library)
I'm stuck with setuptools for shared mode.
One can of course use distutils over setuptools if they don't care about
shared mode and run Python 2.x with x > 3.
This issue of setuptools vs distutils is also somewhat holding up the port
of JCC to Python 3.x as setuptools hadn't been ported to Python 3.x last I
checked a few months ago. Rumors were at the time that this port was
unlikely.
Apropos Windows, a ZIP file that already contains a svn export of lucene
and all jars (make jars) would spare Windows users some pain. They
wouldn't have to install a tool to extract tar.gz as well as Apache Ant
and a svn command line interface. The rest of the build process could be
automated with a small Python script, too, and spare users the
installation of MSys or Cygwin.
Definitely, a binary release should include all this. As for the source
release including jar files, the Makefile could try to get them from the
Apache Lucene download sites but there still is the problem of the PyLucene
java files, the ones that implement extensions (in the java dir tree). They
would still need to be built. At least, svn would no longer be required, but
curl/wget would.
Andi..
