On Thu, 3 May 2012, Roman Chyla wrote:
We use JCC to run Java inside Python as well Python inside Java. I
have now discovered one interesting situation.
I have a Hudson server that builds and tests our code. Hudson will
create its own python virtualenv (based on Python 2.7) and using it
builds JCC and Pylucene. Then Hudson also builds our own code (we
import pylucene and wrap solr with jcc). And it works great, all sorts
of unittests pass.
So I thought 'let me take JCC and Pylucene built by Hudson and install
it into my Python' (note: it is the same machine, so it should be
based on the same Python interpreter)
But are the Java VMs the same ?
Andi..
While it works if I run the python code (that imports java),
Interestingly enough, it fails when I run the java (that imports
Python)
org.apache.jcc.PythonException: An error occurred while creating Java VM
Traceback (most recent call last):
File "/dvt/workspace/montysolr/src/python/montysolr/tests/basic/bridge.py",
line 2, in <module>
from montysolr.initvm import montysolr_java as sj
File "/dvt/workspace/montysolr/src/python/montysolr/initvm.py", line
48, in <module>
montysolr_java.initVM(os.pathsep.join([lucene.CLASSPATH,
solr_java.CLASSPATH, montysolr_java.CLASSPATH, _classpath]))
ValueError: An error occurred while creating Java VM
at org.apache.jcc.PythonVM.instantiate(Native Method)
at invenio.montysolr.jni.PythonVMBridge.start(MontySolrVM.java:287)
at invenio.montysolr.jni.PythonVMBridge.start(MontySolrVM.java:277)
at invenio.montysolr.jni.MontySolrVM.getBridge(MontySolrVM.java:161)
So, first I thought that I should blame the python virtualenv and that
I cannot re-use those eggs (ie. I have to build JCC and Pylucene
default Python). But in fact if I keep the JCC built by Hudson and
just replace the Pylucene, then it starts to work again. I can even
first compile our own project and include the 'broken' pylucene in it
(unittest fail). Then I replace just the pylucene and everything works
again.
It is somewhat puzzling. I'd expect that if I use Hudson eggs that
they will either both work or both fail.
So I thought this might be an early warning sign, something that needs
your attention. I am not sure how to proceed to analyze the problem.
Thanks,
roman
-----
ldd for the libraries:
rchyla@diana /dvt/workspace/pylucene_3_1_0 $ ldd
/usr/local/lib/python2.7/dist-packages/JCC-2.8-py2.7-linux-x86_64.egg/jcc/_jcc.so
linux-vdso.so.1 => (0x00007fff7f378000)
libjava.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libjava.so
(0x00007f97a987f000)
libjvm.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/server/libjvm.so
(0x00007f97a8cb7000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f97a8996000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f97a8714000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f97a84fd000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f97a82e1000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f97a7f5a000)
libverify.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libverify.so
(0x00007f97a7d4a000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f97a7b32000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f97a792e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f97a9d2e000)
rchyla@diana /dvt/workspace/pylucene_3_1_0 $ ldd
/usr/local/lib/python2.7/dist-packages/lucene-3.1.0-py2.7-linux-x86_64.egg/lucene/_lucene.so
linux-vdso.so.1 => (0x00007fffc31ff000)
libjcc.so =>
/var/lib/hudson-local/hudson/jobs/00-JCC/workspace/dist/libjcc.so
(0x00007fab18f7b000)
libjava.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libjava.so
(0x00007fab18d4a000)
libjvm.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/server/libjvm.so
(0x00007fab18183000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007fab17e62000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fab17bdf000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007fab179c9000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007fab177ad000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fab17425000)
libpython2.7.so.1.0 => /usr/lib/libpython2.7.so.1.0 (0x00007fab16f28000)
libverify.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libverify.so
(0x00007fab16d18000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007fab16b00000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fab168fc000)
/lib64/ld-linux-x86-64.so.2 (0x00007fab19e83000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007fab166a9000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007fab162e3000)
libz.so.1 => /usr/lib/x86_64-linux-gnu/libz.so.1 (0x00007fab160cd000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fab15ec9000)
rchyla@diana /dvt/workspace/pylucene_3_1_0 $ ldd
/var/lib/hudson-local/hudson/jobs/01-PYLUCENE/workspace/dist/lucene/_lucene.so
linux-vdso.so.1 => (0x00007fff1513c000)
libjcc.so =>
/var/lib/hudson-local/hudson/jobs/00-JCC/workspace/dist/libjcc.so
(0x00007f6770857000)
libjava.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libjava.so
(0x00007f6770626000)
libjvm.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/server/libjvm.so
(0x00007f676fa5f000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f676f73e000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f676f4bb000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f676f2a5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f676f089000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f676ed01000)
libpython2.7.so.1.0 => /usr/lib/libpython2.7.so.1.0 (0x00007f676e804000)
libverify.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libverify.so
(0x00007f676e5f4000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f676e3dc000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f676e1d8000)
/lib64/ld-linux-x86-64.so.2 (0x00007f677175f000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007f676df85000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007f676dbbf000)
libz.so.1 => /usr/lib/x86_64-linux-gnu/libz.so.1 (0x00007f676d9a9000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f676d7a5000)
rchyla@diana /dvt/workspace/pylucene_3_1_0 $ ldd
/dvt/workspace/montysolr/build/dist/montysolr_java/_montysolr_java.so
linux-vdso.so.1 => (0x00007fff525ff000)
libjcc.so =>
/usr/local/lib/python2.7/dist-packages/JCC-2.8-py2.7-linux-x86_64.egg/libjcc.so
(0x00007f90a3c41000)
libjava.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libjava.so
(0x00007f90a3a10000)
libjvm.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/server/libjvm.so
(0x00007f90a2e49000)
/usr/local/lib/python2.7/dist-packages/lucene-3.1.0-py2.7-linux-x86_64.egg/lucene/_lucene.so
(0x00007f90a2153000)
/dvt/workspace/montysolr/build/dist/solr_java/_solr_java.so
(0x00007f90a1658000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f90a1337000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f90a10b5000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f90a0e9e000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f90a0c82000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f90a08fb000)
libpython2.7.so.1.0 => /usr/lib/libpython2.7.so.1.0 (0x00007f90a03fd000)
libverify.so =>
/usr/lib/jvm/java-6-openjdk-amd64//jre/lib/amd64/libverify.so
(0x00007f90a01ee000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f909ffd6000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f909fdd1000)
/lib64/ld-linux-x86-64.so.2 (0x00007f90a40a4000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007f909fb7f000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007f909f7b8000)
libz.so.1 => /usr/lib/x86_64-linux-gnu/libz.so.1 (0x00007f909f5a2000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f909f39f000)
