Andi Vajda wrote:
On Linux, there's got to be a way to make this work but I haven't found how
yet. I seem to be having trouble with linking to the Python Type objects. On
Windows, I expect this to be even more difficult.
Adding a call to dlopen("_lucene.so", RTLD_NOW | RTLD_GLOBAL) solves the
problem on Linux. I'm thinking of adding this to the initVM() call with an
optional 'export=true/false' parameter... unless there is a better way to go
about this.
Dear Andi,
thanks for your hard work! So far we have have found multiple smaller
and larger issues with your experimental feature.
Setup
=====
I'm running the tests on an Ubuntu 10.04 X86_64 box with Python 2.6.5,
Sun Java 1.6.0_20, GCC 4.4.3 and patched setuptools 0.6c11 (all 64bit).
JCC:
$ svn co
http://svn.apache.org/repos/asf/lucene/pylucene/branches/branch_3x/jcc
jcc_3x
$ cd jcc_3x
$ JCC_JDK=/usr/lib/jvm/java-6-sun python2.6 setup.py build
$ JCC_JDK=/usr/lib/jvm/java-6-sun python2.6 setup.py install
Small issue: The install target always rewrites the jcc.config module.
This is a problem when I install JCC with "sudo python2.6 setup.py
install" since it's not trivial to keep the environment variable JCC_JDK
across a sudo call. How do you like the idea to bind the creation of
jcc.config to the build or build_ext target?
LUCENE:
$ cd ../pylucene-2.9.2-1
$ make ANT=ant PYTHON=python2.6 JCC="python2.6 -m jcc.__main__ --shared"
NUM_FILES=2
$ make ANT=ant PYTHON=python2.6 JCC="python2.6 -m jcc.__main__ --shared"
NUM_FILES=2 install
Suggestion: How about providing sensible defaults for ANT, PYTHON,
NUM_FILES and JCC? It's annoying that I have to type the arguments over
and over again. ;)
BOBO-BROWSE:
$ cd ../bobobrowse
$ make
python2.6 -m jcc.__main__ --shared --jar
bobo-browse-2.5.0-rc1/dist/bobo-browse-2.5.0-rc1.jar --include
bobo-browse-2.5.0-rc1/lib/master/servlet-api.jar --include
bobo-browse-2.5.0-rc1/lib/master/spring-webmvc.jar --include
bobo-browse-2.5.0-rc1/lib/master/spring.jar --include
bobo-browse-2.5.0-rc1/lib/master/log4j.jar --include
bobo-browse-2.5.0-rc1/lib/master/protobuf-java-2.2.0.jar --include
bobo-browse-2.5.0-rc1/lib/master/xstream.jar --include
bobo-browse-2.5.0-rc1/lib/master/fastutil.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/ant.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/xercesImpl.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/commons-collections.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/commons-cli.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/commons-configuration.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/commons-logging.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/commons-lang.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/commons-digester.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/commons-httpclient.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/zoie-2.0.0-rc1.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/json.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/dwr.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/xmlParserAPIs.jar --classpath
bobo-browse-2.5.0-rc1/lib/master/kamikaze-2.0.0.jar --import lucene
--package java.lang java.lang.System java.lang.Runtime --package
java.util java.util.Arrays java.text.SimpleDateFormat --package java.io
java.io.StringReader java.io.InputStreamReader java.io.FileInputStream
--python bobobrowse --version 2.5.0-rc1 --files 2 --build
...
$ make install
As you can see I'm using "--import lucene" here.
$ python2.6
import lucene
lucene.initVM(export=True)
<jcc.JCCEnv object at 0x7f405de450d8>
import bobobrowse
bobobrowse.initVM()
<jcc.JCCEnv object at 0x7f405de451c8>
Issues
======
- I have to follow a strict order (first import lucene, than
lucene.initVM(export=True), finally import bobobrowse). When I try to
import bobobrowse before the call to initVM(export=True), the import
fails with unresolved symbols. Have you tried to add _lucene.so as a
dependency to _bobobrowse.so? I believe a dependency should make the
dlopen() call obsolete.
$ ldd
/usr/local/lib/python2.6/dist-packages/bobobrowse-2.5.0_rc1-py2.6-linux-x86_64.egg/bobobrowse/_bobobrowse.so
linux-vdso.so.1 => (0x00007fffc37ff000)
libjcc.so =>
/usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/libjcc.so
(0x00007fbbdc9f8000)
libjava.so => /usr/lib/jvm/java-6-sun/jre/lib/amd64/libjava.so
(0x00007fbbdc8c8000)
libjvm.so =>
/usr/lib/jvm/java-6-sun/jre/lib/amd64/server/libjvm.so (0x00007fbbdbe50000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fbbdbb35000)
libm.so.6 => /lib/libm.so.6 (0x00007fbbdb8b1000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007fbbdb69a000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fbbdb47d000)
libc.so.6 => /lib/libc.so.6 (0x00007fbbdb0fa000)
libpython2.6.so.1.0 => /usr/lib/libpython2.6.so.1.0
(0x00007fbbdac49000)
libverify.so =>
/usr/lib/jvm/java-6-sun/jre/lib/amd64/libverify.so (0x00007fbbdab3a000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00007fbbda91f000)
libdl.so.2 => /lib/libdl.so.2 (0x00007fbbda71b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbbdd1af000)
libssl.so.0.9.8 => /lib/libssl.so.0.9.8 (0x00007fbbda4cb000)
libcrypto.so.0.9.8 => /lib/libcrypto.so.0.9.8 (0x00007fbbda13a000)
libz.so.1 => /lib/libz.so.1 (0x00007fbbd9f23000)
libutil.so.1 => /lib/libutil.so.1 (0x00007fbbd9d20000)
- With "--import lucene" JCC gets the class_ attribute of all types
wrong, that are part of the bobobrowse package. It either shows
java.lang.Object as class_ or the first lucene class in the hierarchy.
Without "--import lucene" it works as expected.
bobobrowse.FacetSpec.class_
<Class: class java.lang.Object>
BoboBrowser is a subclass of MultiBoboBrowser which extends lucene's
MultiSearcher
bobobrowse.MultiBoboBrowser.class_
<Class: class org.apache.lucene.search.MultiSearcher>
bobobrowse.BoboBrowser.class_
<Class: class org.apache.lucene.search.MultiSearcher>
bobobrowse.Tokenizer.class_
<Class: class org.apache.lucene.analysis.Tokenizer>
- With "--import lucene" JCC doesn't wrap certain classes and it's
missing bobobrowse.FacetSpec.FacetSortSpec.OrderHitsDesc. Without
"--import lucene" the enum is wrapped correctly.
bobobrowse.FacetSpec.FacetSortSpec.OrderHitsDesc
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'FacetSpec$FacetSortSpec' has no attribute
'OrderHitsDesc'
- I've run into a rather strange bug with our custom builds of Python
2.6. It differs from the stock build of Ubuntu in two ways. The
python2.6 binary is linked against libpython2.6.so (Ubuntu's python2.6
binary has libpython2.6.a statically linked in) and I've set the
environment variable LD_RUN_PATH which causes the linker to add a
"library rpath" to all ELF binaries. I wasn't able to find the reason
for the issue so far. Any hints are appreciated.
Christian
