Hi Chris,
On Wed, 1 Feb 2012, Andi Vajda wrote:
No objections to these patches in principle but it would be easier for me
to integrate them if you could provide patches computed from the svn
repository of JCC:
http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/ Your patches
seem to be small enough so I should be able to do without but it would be
nicer if I didn't have to guess...
I think the patch that I attached was already based on trunk. The git
repository includes the .svn directories, points to trunk, and I generated
the patch using "svn diff".
Sorry, I missed that you indeed had attached a patch last time.
(to be continued...)
Also, please write small descriptions for these new command line flags to
go into JCC's __main__.py file:
http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/jcc/__main__.py
Done, new patch attached.
Thank you !
I integrated your patches with rev 1240624.
I moved a few changes around :parameters to their section in __main__.py and
'maxstack' hardcoding to where it used to be.
Thank you for your contribution.
Andi..
This mess of setuptools patching was meant to be *temporary* until
setuptools' issue 43 was fixed. As you can see, I filed this bug 3 1/2
years ago, http://bugs.python.org/setuptools/issue43, and my patch for
issue 43 still hasn't been accepted, rejected, integrated, anything'ed...
Dormant. For over three years.
Sorry about that. I've had similar experience with bugs reported against
ubuntu, hibernate, rails... :(
* Why does JCC use non-standard command line arguments like --build and
--install? Can it be modified to make it easier to invoke from a
setup.py-style environment, such as exporting a setup() function as
setuptools does?
What standard are you referring to ?
The python extension module build/install/deploy story on Python keeps
evolving... Add Python 3.x support into the mix, and the mess is complete.
Seriously, though, I think that the right thing to do to better integrate
JCC with distutils/setuptools/distribute/pip/etc... is to make it into a
distutils 'compiler'. This requires some work, though, and I haven't done
it in all thee years. Anyone with the itch to hack on distutils is welcome
to take that on.
I'm afraid I don't fully understand how distutils works, it seems to be
sparsely documented, and I don't have a lot of time and energy to work on
refactoring jcc. I am a bit surprised that we can't just generate a source
distribution containing the jars, .cpp files and a setup.py which does the
rest like any other Python extension.
Same here. I don't know distutils too well and whenever I tried to dig into
it, I quickly gave up. I don't know what it means to "just generate a source
distribution".
If they contain .class files, JAR files are not source files. My
understanding could be wrong here, but I don't think they're even compatible
between 32- and 64-bit VMs. Or is that incompatible between Java 5 and 6 ?
I have very little itch to dabble in configure scripts either so I've been
dragging my feet. If someone were to step forward with a patch for that,
I'd be delighted in ripping out all this patching brittleness.
How would a configure script solve the problem and what would it have to
do? Generate the .cpp files? How does it integrate with Python extensions?
A configure script for building libjcc.dylib (libjcc.so on Linux, jcc.dll on
Windows, etc...) would take care of doing what setuptools + the issue43 patch
is doing for us currently: invoking the C++ compiler and linker against the
correct Python headers and Libraries to produce a vanilla shared library.
With such a contribute script, there is no longer a need to patch setuptools.
That is a whole different project. If I remember correctly, the JPype
project is (or was) taking that approach: http://jpype.sourceforge.net
OK, thanks.
* Could JCC generate a source distribution (sdist) that could be
uploaded to pypi?
You mean a source distribution that includes the Java sources of all the
libraries/classes wrapped ?
I was thinking more of the jars. Something like
https://github.com/aptivate/python-tika that doesn't depend on jcc any
more.
* "setup.py develop" is still broken in the current implementation
I'm not familiar with this 'develop' command nor that it is broken. What
is it supposed to be doing and how is it broken ?
http://packages.python.org/distribute/setuptools.html#development-mode
It seems that when invoked this way, my setup.py (from python-tika) which
calls jcc ends up creating build/_tika as a file (not a directory).
For example, this command:
sudo pip install -e git+https://github.com/aptivate/python-tika#egg=tika
(note the -e for editable mode) results in this:
Running setup.py develop for tika
...
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/src/tika/setup.py", line 108, in <module>
cpp.jcc(jcc_args)
File
"/usr/local/lib/python2.6/dist-packages/JCC-2.12-py2.6-linux-i686.egg/jcc/cpp.py",
line 587, in jcc
os.makedirs(cppdir)
File "/usr/lib/python2.6/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: 'build/_tika'
That file appears to contain the source code for the JCCEnv.cpp wrapper.
Please, file a bug with the explanation above. Not that I promise to fix it
(a patch would be welcome, of course) but this failure should be logged at
least.
A patch could be written to noisily emit a warning on all methods that are
skipped. Silently wrapping everything would simply wrap the entire JDK by
transitive closure and produce a huge library, assuming you'd have the
patience to watch it compile.
The skipping of method whose signature contains types that are not on the
'wrap this' list (explicit or implicit) is by design. Not being able to
request emitting a warning is a problem.
Perhaps it's useful to (automatically) emit warnings for classes in the JAR
files included with --jar or an explicit class name, but not those in
--include files or otherwise automatically included (e.g. the JDK
classpath)?
That'd be possible but has the potential of being very noisy...
It's worth a try, for sure.
Andi..
