Hi Andi,
Thanks for the help, now I was able to run the java and loaded
PythonVM. I then built the python egg, after a bit of fiddling with
parameters, it seems ok. I can import the jcc wrapped python class and
call it:
In [1]: from solrpie_java import emql
In [2]: em = emql.Emql()
In [3]: em.javaTestPrint()
java is printing
In [4]: em.pythonTestPrint()
just a test
But I haven't found out how to call the same from java.
The egg is built fine, it is named solrpie_java and contains one python module:
==============
from solrpie_java import initVM, CLASSPATH, EMQL
initVM(CLASSPATH)
class Emql(EMQL):
'''
classdocs
'''
def __init__(self):
super(Emql, self).__init__()
print '__init__'
def init(self, me):
print self, me
return 'init'
def emql_refresh(self, tid, type):
print self, tid, type
return 'refresh'
def emql_status(self):
return "some status"
def pythonTestPrint(self):
print 'just a test'
========
The corresponding java class looks like this:
public class EMQL {
private long pythonObject;
public EMQL()
{
}
public void pythonExtension(long pythonObject)
{
this.pythonObject = pythonObject;
}
public long pythonExtension()
{
return this.pythonObject;
}
public void finalize()
throws Throwable
{
pythonDecRef();
}
public void javaTestPrint() {
System.out.println("java is printing");
}
public native void pythonDecRef();
// the methods implemented in python
public native String init(EMQL me);
public native String emql_refresh(String tid, String type);
public native String emql_status();
public native void pythonTestPrint();
}
=======
I tried running it as:
PythonVM vm = PythonVM.start("sorlpie_java");
EMQL em = new EMQL();
em.javaTestPrint();
em.pythonTestPrint();
I get this:
java is printing
Exception in thread "main" java.lang.UnsatisfiedLinkError:
rca.pythonvm.EMQL.pythonTestPrint()V
at rca.pythonvm.EMQL.pythonTestPrint(Native Method)
at rca.solr.JettyRunnerPythonVM.start(JettyRunnerPythonVM.java:60)
at rca.solr.JettyRunnerPythonVM.main(JettyRunnerPythonVM.java:148)
I understand that java cannot find the linked c++ method, but I don't
know how to fix that.
If i try:
PythonVM vm = PythonVM.start("sorlpie_java");
Object m = vm.instantiate("emql", "Emql");
I get:
org.apache.jcc.PythonException: No module named emql
ImportError: No module named emql
at org.apache.jcc.PythonVM.instantiate(Native Method)
at rca.solr.JettyRunnerPythonVM.start(JettyRunnerPythonVM.java:56)
at rca.solr.JettyRunnerPythonVM.main(JettyRunnerPythonVM.java:148)
I tried various combinations of instanatiation, and setting the
classpatt or -Djava.library.path
But no success. What am I doing wrong?
Thank you,
roman
On Wed, Jan 12, 2011 at 7:55 PM, Andi Vajda <va...@apache.org> wrote:
>
> On Wed, 12 Jan 2011, Roman Chyla wrote:
>
>> Hi Andi, all,
>>
>> I tried to implement the PythonVM wrapping on Mac 10.6, with JDK
>> 1.6.22, jcc is freshly built, in shared mode, v. 2.6. The python is
>> the standard Python distributed with MacOsX
>>
>> When I try to run the java, it throws an error when it gets to:
>>
>> static {
>> System.loadLibrary("jcc");
>> }
>>
>> I am getting this error:
>>
>> Exception in thread "main" java.lang.UnsatisfiedLinkError:
>>
>> /Library/Python/2.6/site-packages/JCC-2.6-py2.6-macosx-10.6-universal.egg/libjcc.dylib:
>> Symbol not found: _PyExc_RuntimeError Referenced from:
>
> That's because Python's shared library wasn't found. The reason is that, by
> default, Python's shared lib not on JCC's link line because normally JCC is
> loaded into a Python process and the dynamic linker thus finds the symbols
> needed inside the process.
>
> Here, since you're not starting inside a Python process, you need to add
> '-framework Python' to JCC's LFLAGS in setup.py so that the dynamic linker
> can find the Python VM shared lib and load it.
>
> Andi..
>
>>
>> /Library/Python/2.6/site-packages/JCC-2.6-py2.6-macosx-10.6-universal.egg/libjcc.dylib
>> Expected in: flat namespace in
>>
>> /Library/Python/2.6/site-packages/JCC-2.6-py2.6-macosx-10.6-universal.egg/libjcc.dylib
>> at java.lang.ClassLoader$NativeLibrary.load(Native Method)
>> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1823)
>> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1746)
>> at java.lang.Runtime.loadLibrary0(Runtime.java:823)
>> at java.lang.System.loadLibrary(System.java:1045)
>> at org.apache.jcc.PythonVM.<clinit>(PythonVM.java:23)
>> at rca.solr.JettyRunnerPythonVM.start(JettyRunnerPythonVM.java:53)
>> at rca.solr.JettyRunnerPythonVM.main(JettyRunnerPythonVM.java:139)
>>
>>
>> MacBeth:JCC-2.6-py2.6-macosx-10.6-universal.egg rca$ nm libjcc.dylib |
>> grep Exc
>> U _PyExc_RuntimeError
>> U _PyExc_TypeError
>> U _PyExc_ValueError
>> 0000000000003442 T __ZNK6JCCEnv15reportExceptionEv
>> 00000000000021f0 T __ZNK6JCCEnv23getPythonExceptionClassEv
>>
>>
>> Any pointers what I could do wrong? Note, I haven't built any emql.egg
>> yet, I just run my java program and try to start PythonVM() and see if
>> that works.
>>
>> Thanks,
>>
>> roman
>>
>>
>>
>> On Wed, Jan 12, 2011 at 11:05 AM, Roman Chyla <roman.ch...@gmail.com>
>> wrote:
>>>
>>> Hi Andi,
>>>
>>> I think I will give it a try, if only because I am curious. Please see
>>> one remaining question below.
>>>
>>>
>>> On Tue, Jan 11, 2011 at 10:37 PM, Andi Vajda <va...@apache.org> wrote:
>>>>
>>>>
>>>> On Tue, 11 Jan 2011, Roman Chyla wrote:
>>>>
>>>>> Hi Andy,
>>>>>
>>>>> This is much more than I could have hoped! Just yesterday, I was
>>>>> looking for ways how to embed Python VM in Jetty, as that would be
>>>>> more natural, but found only jepp.sourceforge.net and off-putting was
>>>>> the necessity to compile it against the newly built python. I could
>>>>> not want it from the guys who may need my extension. And I realize
>>>>> only now, that embedding Python in Java is even documented on the
>>>>> website, but honestly i would not know how to do it without your
>>>>> detailed examples.
>>>>>
>>>>> Now to the questions, I apologize, some of them or all must seem very
>>>>> stupid to you
>>>>>
>>>>> - pylucene is used on many platforms and with jcc always worked as
>>>>> expected (i love it!), but is it as reliable in the opposite
>>>>> direction? The PythonVM.java loads "jcc" library, so I wonder if in
>>>>> principle there is any difference in the directionality - but I am not
>>>>> sure. To rephrase my convoluted question: would you expect this
>>>>> wrapping be as reliable as wrapping java inside python is now?
>>>>
>>>> I've been using this for over two years, in production.
>>>> My main worry was memory leaks because a server process is expected to
>>>> stay
>>>> up and running for weeks at a time and it's been very stable on that
>>>> front
>>>> too. Of course, when there is a bug somewhere that causes your Python VM
>>>> to
>>>> crash, the entire server crashes. Just like when the JVM crashes (which
>>>> is
>>>> normally rare). In other words, this isn't any less reliable than a
>>>> standalone Python VM process. It can be tricky, but is possible, to run
>>>> gdb,
>>>> pdb and jdb together to step through the three languages involved,
>>>> python,
>>>> java and C++. I've had to do this a few times but not in a long time.
>>>>
>>>>> - in the past, i built jcc libraries on one host and distributed them
>>>>> on
>>>>> various machines. As long the family OS and the python main version
>>>>> were the
>>>>> same, it worked on Win/Lin/Mac just fine. As far as I can tell, this
>>>>> does
>>>>> not change, or will it be dependent on the python against which the egg
>>>>> was
>>>>> built?
>>>>
>>>> Distributing binaries is risky. The same caveats apply. I wouldn't do
>>>> it,
>>>> even in the simple PyLucene case.
>>>
>>> unfortunately, I don't have that many choices left - this is not for
>>> some client-software scenario, we are running the jobs on the grid,
>>> and there I cannot compile the binaries. So, if previously the
>>> location of the python interpreter or python minor version did not
>>> cause problems, now perhaps it will be different. But that wasn't for
>>> the Solr, wrapping Solr is not meant for the grid.
>>>
>>>>
>>>>> - now a little tricky issue; when I wrap jetty inside python, I hoped
>>>>> to build it in a shared mode with lucene to be able to do some
>>>>> low-level lucene indexing tasks from inside Python. If I do the
>>>>> opposite and wrap Python VM in Java, I would still like to access the
>>>>> lucene (which is possible, as I see well from your examples) But on
>>>>> the python side, you are calling initVM() - will the initVM() call
>>>>> create a new Java VM or will it access the parent Java VM which
>>>>> started it?
>>>>
>>>> No, initVM() in this case just initializes your egg and adds its stuff
>>>> to
>>>> the CLASSPATH. No Java VM init is done. As with any shared-mode
>>>> JCC-built
>>>> extension, all calls to initVM() but the first one just do that.
>>>> The first call to initVM() in the embedding Python case is like that too
>>>> because there already is a Java VM running when PythonVM is instantiated
>>>> and
>>>> called.
>>>
>>> And if in the python, I will do:
>>>
>>> import lucene
>>> import lucene.initVM(lucene.CLASSPATH)
>>>
>>> Will it work in this case? Giving access to the java classes from
>>> inside python. Or I will have to forget pylucene, and prepare some
>>> extra java classes? (the jcc in reverse trick, as you put it)
>>>
>>>>
>>>>> - you say that threads are not managed by the Python VM, does that
>>>>> mean there is no Python GIL?
>>>>
>>>> No, there is a Pythonn GIL (and that is the Achille's Heel of this setup
>>>> if
>>>> you expect high concurrent servlet performance from your server calling
>>>> Python). That Python GIL is connected to this thread state I was
>>>> mentioning
>>>> earlier. Because the thread is not managed by Python, when Python is
>>>> called
>>>> (by way of the code generated by JCC) it doesn't find a thread state for
>>>> the
>>>> thread and creates one. When the call completes, the thread state is
>>>> destroyed because its refcount goes to zero. My TerminatingThread class
>>>> acquires a Python thread state and keeps it for the life of the thread,
>>>> thereby working this problem around.
>>>
>>> OK, this then looks like a normal Python - which is somehow making me
>>> less worried :) I wanted to use multiprocessing inside python to deal
>>> with GIL, and I see no reason why it should not work in this case.
>>>
>>> Thank you very much.
>>> Cheers,
>>>
>>> roman
>>>
>>>>
>>>>> - I don't really know what is exactly in the python thread local
>>>>> storage, could that somehow negatively affect the Python process if
>>>>> acquireThreadState/releaseThreadState are not called?
>>>>
>>>> Yes, if you depend on thread-local storage, it would get lost between
>>>> calls
>>>> and cause confusion and bugs, defeating its purpose. Python's
>>>> thread-local
>>>> storage support is documented here:
>>>> http://docs.python.org/library/threading.html, look for threading.local.
>>>>
>>>> Andi..
>>>>
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> roman
>>>>>
>>>>>
>>>>> On Tue, Jan 11, 2011 at 8:13 PM, Andi Vajda <va...@apache.org> wrote:
>>>>>>
>>>>>> Hi Roman,
>>>>>>
>>>>>> On Tue, 11 Jan 2011, Roman Chyla wrote:
>>>>>>
>>>>>>> I have recently wrapped solr inside jetty with JCC (we need to access
>>>>>>> very big result sets quickly, via JNI, but also keep solr running as
>>>>>>> normal) and was wondering what strategies do you guys use to speak
>>>>>>> *from inside* Java towards the Python end.
>>>>>>>
>>>>>>> So far, I was able to think about these:
>>>>>>>
>>>>>>> - raise exceptions in java and catch in python (I think I have seen
>>>>>>> this in some posts from Bill Jansen)
>>>>>>> - communicate via sockets
>>>>>>> - wait passively - call some java method and wait for its return
>>>>>>> - monitor actively - in python check in loop some java object
>>>>>>>
>>>>>>> Is there something else?
>>>>>>
>>>>>> I'm not sure I completely understand your questions but if what you're
>>>>>> asking is how to run Python code from inside a Java servlet container,
>>>>>> that
>>>>>> I've done with Tomcat and Lucene.
>>>>>>
>>>>>> Basically, instead of embedding a JVM inside a Python VM - as is done
>>>>>> for
>>>>>> PyLucene - you do the opposite, you embed a Python VM inside a JVM.
>>>>>>
>>>>>> For that purpose, see the org.apache.jcc.PythonVM class available in
>>>>>> JCC's
>>>>>> java tree. This class must be instantiated from the main thread at
>>>>>> Java
>>>>>> servlet engine startup time. In Tomcat, I patched some startup code,
>>>>>> in
>>>>>> BootStrap.java (see patches below) for this purpose.
>>>>>>
>>>>>> Then, to make some Python code accessible from Java, use the usual way
>>>>>> of
>>>>>> writing "extensions", the so-called JCC in reverse trick. Define a
>>>>>> Java
>>>>>> class
>>>>>> with some native methods implemented in Python; define a Python class
>>>>>> that
>>>>>> "extends" it; build the Java class into a JAR; include it into a
>>>>>> JCC-built
>>>>>> egg; install the egg into Python's env (site-packages, PYTHONPATH,
>>>>>> whatever);
>>>>>> Then, write servlet code in Java that imports your Java class and
>>>>>> calls
>>>>>> it.
>>>>>>
>>>>>> As you can see, this sounds simple but the devil is in the details. Of
>>>>>> course,
>>>>>> bending Jetty for this may have different requirements but the code
>>>>>> snippets
>>>>>> below should give you a good idea about what's required.
>>>>>>
>>>>>> This approach has been in production running the freebase.com's search
>>>>>> server
>>>>>> for over two years now.
>>>>>>
>>>>>> If you have questions, of course, please ask.
>>>>>> Good luck !
>>>>>>
>>>>>> Andi..
>>>>>>
>>>>>> ----------------------
>>>>>> Patch to Bootstrap.java to use JCC's PythonVM (which initializes the
>>>>>> embedded
>>>>>> Python VM)
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> apache-tomcat-6.0.29-src/java/org/apache/catalina/startup/Bootstrap.java
>>>>>> 2010-07-19 06:02:32.000000000 -0700
>>>>>> +++
>>>>>>
>>>>>>
>>>>>> apache-tomcat-6.0.29-src/java/org/apache/catalina/startup/Bootstrap.java.patched
>>>>>> 2010-08-04 08:49:05.000000000 -0700
>>>>>> @@ -30,16 +30,18 @@
>>>>>> import javax.management.MBeanServer;
>>>>>> import javax.management.MBeanServerFactory;
>>>>>> import javax.management.ObjectName;
>>>>>>
>>>>>> import org.apache.catalina.security.SecurityClassLoad;
>>>>>> import org.apache.juli.logging.Log;
>>>>>> import org.apache.juli.logging.LogFactory;
>>>>>>
>>>>>> +import org.apache.jcc.PythonVM;
>>>>>> +
>>>>>>
>>>>>> /**
>>>>>> * Boostrap loader for Catalina. This application constructs a class
>>>>>> loader
>>>>>> * for use in loading the Catalina internal classes (by accumulating
>>>>>> all
>>>>>> of
>>>>>> the
>>>>>> * JAR files found in the "server" directory under "catalina.home"),
>>>>>> and
>>>>>> * starts the regular execution of the container. The purpose of this
>>>>>> * roundabout approach is to keep the Catalina internal classes (and
>>>>>> any
>>>>>> * other classes they depend on, such as an XML parser) out of the
>>>>>> system
>>>>>> @@ -398,22 +400,24 @@
>>>>>> try {
>>>>>> String command = "start";
>>>>>> if (args.length > 0) {
>>>>>> command = args[args.length - 1];
>>>>>> }
>>>>>>
>>>>>> if (command.equals("startd")) {
>>>>>> args[args.length - 1] = "start";
>>>>>> + PythonVM.start("mql");
>>>>>> daemon.load(args);
>>>>>> daemon.start();
>>>>>> } else if (command.equals("stopd")) {
>>>>>> args[args.length - 1] = "stop";
>>>>>> daemon.stop();
>>>>>> } else if (command.equals("start")) {
>>>>>> + PythonVM.start("mql");
>>>>>> daemon.setAwait(true);
>>>>>> daemon.load(args);
>>>>>> daemon.start();
>>>>>> } else if (command.equals("stop")) {
>>>>>> daemon.stopServer(args);
>>>>>> } else {
>>>>>> log.warn("Bootstrap: command \"" + command + "\" does
>>>>>> not
>>>>>> exist.");
>>>>>> }
>>>>>>
>>>>>> -----------------------------------------
>>>>>> Define a Java class:
>>>>>>
>>>>>> package ....
>>>>>>
>>>>>> public class EMQL {
>>>>>>
>>>>>> private long pythonObject;
>>>>>>
>>>>>> public EMQL()
>>>>>> {
>>>>>> }
>>>>>>
>>>>>> public void pythonExtension(long pythonObject)
>>>>>> {
>>>>>> this.pythonObject = pythonObject;
>>>>>> }
>>>>>> public long pythonExtension()
>>>>>> {
>>>>>> return this.pythonObject;
>>>>>> }
>>>>>>
>>>>>> public void finalize()
>>>>>> throws Throwable
>>>>>> {
>>>>>> pythonDecRef();
>>>>>> }
>>>>>>
>>>>>> public native void pythonDecRef();
>>>>>>
>>>>>> // the methods implemented in python
>>>>>> public native String init(ME me);
>>>>>> public native String emql_refresh(String tid, String type);
>>>>>> public native String emql_status();
>>>>>>
>>>>>> etc .......... etc
>>>>>>
>>>>>> ------------------------------------
>>>>>> The corresponding Python class
>>>>>>
>>>>>> import ......
>>>>>>
>>>>>> from jemql import initVM, CLASSPATH, EMQL
>>>>>>
>>>>>> initVM(CLASSPATH)
>>>>>>
>>>>>> class emql(EMQL):
>>>>>>
>>>>>> def __init__(self):
>>>>>> super(emql, self).__init__()
>>>>>>
>>>>>> def init(self, me):
>>>>>> ...........
>>>>>> def emql_refresh(self, tid, type):
>>>>>> ...........
>>>>>> def emql_status(self):
>>>>>> ...........
>>>>>> return "some status"
>>>>>>
>>>>>> etc ...... etc
>>>>>>
>>>>>> ------------------------------------
>>>>>> Makefile rules to build this via JCC (the jemql.egg file is just an
>>>>>> empty
>>>>>> target file for Makefile, it's not used for anything else):
>>>>>>
>>>>>> default: jemql.egg
>>>>>>
>>>>>> jemql.jar: java/org/blah/blah/EMQL.java
>>>>>> mkdir -p classes
>>>>>> javac -classpath $(CLASSPATH):$(MORE_CLASSPATH):$(etc..etc) -d
>>>>>> classes $(JAVAC_FLAGS) $<
>>>>>> jar -cvf $@ -C classes .
>>>>>>
>>>>>> jemql.egg: jemql.jar $(JMQL_JAR) emql.py
>>>>>> $(JCC) --version 1.0 --jar $< \
>>>>>> --classpath $(CLASSPATH):$(JME_JAR):$(JMQL_JAR) \
>>>>>> org.blah.blah.me.ME \
>>>>>> --package java.lang \
>>>>>> --python jemql --build $(DBG_FLAGS) \
>>>>>> --install \
>>>>>> --module emql
>>>>>> touch $@
>>>>>> ------------------------------------
>>>>>> Patch to Tomcat's build.xml ANT script to add JCC's classes (like
>>>>>> PythonVM)
>>>>>> to
>>>>>> the build classpath.
>>>>>>
>>>>>> --- apache-tomcat-6.0.29-src/build.xml 2010-07-19 06:02:31.000000000
>>>>>> -0700
>>>>>> +++ apache-tomcat-6.0.29-src/build.xml.patched 2010-08-04
>>>>>> 09:30:24.000000000 -0700
>>>>>> @@ -95,16 +95,17 @@
>>>>>> <property name="jasper-jdt.jar"
>>>>>> value="${jasper-jdt.home}/jasper-jdt.jar"/>
>>>>>> <available property="tomcat-dbcp.present" file="${tomcat-dbcp.jar}"
>>>>>> />
>>>>>> <available property="jdk16.present"
>>>>>> classname="javax.sql.StatementEvent"
>>>>>> />
>>>>>>
>>>>>> <!-- Classpath -->
>>>>>> <path id="tomcat.classpath">
>>>>>> <pathelement location="${ant.jar}"/>
>>>>>> <pathelement location="${jdt.jar}"/>
>>>>>> + <pathelement location="${jcc.egg}/jcc/classes"/>
>>>>>> </path>
>>>>>>
>>>>>> <!-- Version info filter set -->
>>>>>> <tstamp>
>>>>>> <format property="TODAY" pattern="MMM d yyyy" locale="en"/>
>>>>>> <format property="TSTAMP" pattern="hh:mm:ss"/>
>>>>>> </tstamp>
>>>>>> <filterset id="version.filters">
>>>>>> @@ -148,16 +149,25 @@
>>>>>> excludes="**/CVS/**,**/.svn/**"
>>>>>> encoding="ISO-8859-1">
>>>>>> <!-- Comment this in to show unchecked warnings:
>>>>>> <compilerarg value="-Xlint:unchecked"/>
>>>>>> -->
>>>>>> <classpath refid="tomcat.classpath" />
>>>>>> <exclude name="org/apache/naming/factory/webservices/**" />
>>>>>> </javac>
>>>>>> + <javac srcdir="${extras.path}" destdir="${tomcat.classes}"
>>>>>> + debug="${compile.debug}"
>>>>>> + deprecation="${compile.deprecation}"
>>>>>> + source="${compile.source}"
>>>>>> + optimize="${compile.optimize}"
>>>>>> + excludes="**/CVS/**,**/.svn/**">
>>>>>> +<!-- Comment this in to show unchecked warnings: <compilerarg
>>>>>> value="-Xlint:unchecked"/> -->
>>>>>> + <classpath refid="tomcat.classpath" />
>>>>>> + </javac>
>>>>>> <!-- Copy static resource files -->
>>>>>> <copy todir="${tomcat.classes}" encoding="ISO-8859-1">
>>>>>> <filterset refid="version.filters"/>
>>>>>> <fileset dir="java">
>>>>>> <include name="**/*.properties"/>
>>>>>> <include name="**/*.dtd"/>
>>>>>> <include name="**/*.tasks"/>
>>>>>> <include name="**/*.xsd"/>
>>>>>>
>>>>>> -----------------------------------------------
>>>>>> Patch to catalina.sh, the Tomcat startup script to add JCC to LIBPATH
>>>>>> and
>>>>>> CLASSPATH
>>>>>>
>>>>>> --- apache-tomcat-6.0.29-src/output/build/bin/catalina.sh
>>>>>> 2010-08-04
>>>>>> 09:57:27.000000000 -0700
>>>>>> +++ apache-tomcat-6.0.29-src/output/build/bin/catalina.sh.patched
>>>>>> 2010-08-04 09:57:47.000000000 -0700
>>>>>> @@ -162,16 +162,30 @@
>>>>>> exit 1
>>>>>> fi
>>>>>> fi
>>>>>>
>>>>>> if [ -z "$CATALINA_BASE" ] ; then
>>>>>> CATALINA_BASE="$CATALINA_HOME"
>>>>>> fi
>>>>>>
>>>>>> +if [ -n "$JCC_EGG" ]; then
>>>>>> + CLASSPATH="$CLASSPATH":"$JCC_EGG"/jcc/classes
>>>>>> + JAVA_LIB_PATH=$JCC_EGG
>>>>>> +fi
>>>>>> +if [ -n "$TOMCAT_APR_LIB_PATH" ]; then
>>>>>> + JAVA_LIB_PATH=$JAVA_LIB_PATH:$TOMCAT_APR_LIB_PATH
>>>>>> +fi
>>>>>> +if [ -n "$JAVA_LIB_PATH" ]; then
>>>>>> + JAVA_OPTS="$JAVA_OPTS -Djava.library.path=$JAVA_LIB_PATH"
>>>>>> +fi
>>>>>> +if [ -n "EXTRA_CLASSPATH" ]; then
>>>>>> + CLASSPATH="$CLASSPATH":"$EXTRA_CLASSPATH"
>>>>>> +fi
>>>>>> +
>>>>>> # Add tomcat-juli.jar and bootstrap.jar to classpath
>>>>>> # tomcat-juli.jar can be over-ridden per instance
>>>>>> if [ ! -z "$CLASSPATH" ] ; then
>>>>>> CLASSPATH="$CLASSPATH":
>>>>>> fi
>>>>>> if [ "$CATALINA_BASE" != "$CATALINA_HOME" ] && [ -r
>>>>>> "$CATALINA_BASE/bin/tomcat-juli.jar" ] ; then
>>>>>>
>>>>>>
>>>>>>
>>>>>> CLASSPATH="$CLASSPATH""$CATALINA_BASE"/bin/tomcat-juli.jar:"$CATALINA_HOME"/bin/bootstrap.jar
>>>>>> else
>>>>>>
>>>>>> These EGG paths are long, complicated and OS-specific, the trick below
>>>>>> generates them programmatically (from inside a Makefile):
>>>>>>
>>>>>> JCC_EGG:=$(shell $(PYTHON) -c "import os, jcc; print
>>>>>> os.path.dirname(os.path.dirname(jcc.__file__))")
>>>>>> JEMQL_EGG:=$(shell $(PYTHON) -c "import os, jemql; print
>>>>>> os.path.dirname(os.path.dirname(jemql.__file__))")
>>>>>>
>>>>>> Then, the CLASSPATH addition during _build_ time:
>>>>>> CLASSPATH = $(CLASSPATH):$(JEMQL_EGG)/jemql/jemql.jar
>>>>>> and so on...
>>>>>> At runtime, JCC takes care of adding your eggs to the startup
>>>>>> CLASSPATH.
>>>>>>
>>>>>> ----------------------------------------------
>>>>>> Last but not least, if you use Python's thread local storage in your
>>>>>> threads, Python threads when embedded inside a JVM are 'dummy', that
>>>>>> is,
>>>>>> while they're
>>>>>> backed by the actual Java thread (a pthread), the Python VM is not
>>>>>> managing
>>>>>> them and a thread state object is created each and every time a Python
>>>>>> thread
>>>>>> is entered and released when exited back to the JVM. This has two
>>>>>> problems:
>>>>>> 1. it's a bit wasteful
>>>>>> 2. python thread local storage gets lost
>>>>>>
>>>>>> The Java class below works this around by incrementing the refcount
>>>>>> that
>>>>>> controls this:
>>>>>>
>>>>>> package org.apache.catalina.core;
>>>>>>
>>>>>> import org.apache.jcc.PythonVM;
>>>>>>
>>>>>> public class TerminatingThread extends Thread {
>>>>>> protected Runnable runnable;
>>>>>>
>>>>>> public TerminatingThread(ThreadGroup group, Runnable runnable,
>>>>>> String
>>>>>> name)
>>>>>> {
>>>>>> super(group, name);
>>>>>> this.runnable = runnable;
>>>>>> }
>>>>>>
>>>>>> public void run()
>>>>>> {
>>>>>> PythonVM vm = PythonVM.get();
>>>>>>
>>>>>> try {
>>>>>> vm.acquireThreadState();
>>>>>> runnable.run();
>>>>>> } finally {
>>>>>> vm.releaseThreadState();
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> Then, there is some trickery to get Tomcat to use this class for its
>>>>>> threads
>>>>>> instead of the default one:
>>>>>>
>>>>>> ---
>>>>>>
>>>>>>
>>>>>> apache-tomcat-6.0.29-src/java/org/apache/catalina/core/StandardThreadExecutor.java
>>>>>> 2010-07-19 06:02:32.000000000 -0700
>>>>>> +++
>>>>>>
>>>>>>
>>>>>> apache-tomcat-6.0.29-src/java/org/apache/catalina/core/StandardThreadExecutor.java.patched
>>>>>> 2010-08-04 08:56:02.000000000 -0700
>>>>>> @@ -44,17 +44,17 @@
>>>>>> protected int minSpareThreads = 25;
>>>>>>
>>>>>> protected int maxIdleTime = 60000;
>>>>>>
>>>>>> protected ThreadPoolExecutor executor = null;
>>>>>>
>>>>>> protected String name;
>>>>>>
>>>>>> - private LifecycleSupport lifecycle = new LifecycleSupport(this);
>>>>>> + protected LifecycleSupport lifecycle = new
>>>>>> LifecycleSupport(this);
>>>>>> // ---------------------------------------------- Constructors
>>>>>> public StandardThreadExecutor() {
>>>>>> //empty constructor for the digester
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>>> // ---------------------------------------------- Public Methods
>>>>>>
>>>>>>
>>>>>> In Tomcat's server.xml, use this executor (and code below for it)
>>>>>> <Executor name="relThreadPool"
>>>>>>
>>>>>> className="org.apache.catalina.core.TerminatingThreadExecutor"
>>>>>> namePrefix="rel-exec-"
>>>>>> maxIdleTime="3600000"
>>>>>> minSpareThreads="2"
>>>>>> maxThreads="2" />
>>>>>>
>>>>>>
>>>>>> package org.apache.catalina.core;
>>>>>>
>>>>>> import java.util.concurrent.ThreadPoolExecutor;
>>>>>> import java.util.concurrent.TimeUnit;
>>>>>> import org.apache.catalina.LifecycleException;
>>>>>>
>>>>>>
>>>>>> public class TerminatingThreadExecutor extends StandardThreadExecutor
>>>>>> {
>>>>>>
>>>>>> public void start()
>>>>>> throws LifecycleException
>>>>>> {
>>>>>> lifecycle.fireLifecycleEvent(BEFORE_START_EVENT, null);
>>>>>>
>>>>>> TaskQueue taskqueue = new TaskQueue();
>>>>>> TaskThreadFactory tf = new
>>>>>> TerminatingTaskThreadFactory(namePrefix);
>>>>>>
>>>>>> lifecycle.fireLifecycleEvent(START_EVENT, null);
>>>>>> executor = new ThreadPoolExecutor(getMinSpareThreads(),
>>>>>> getMaxThreads(),
>>>>>> maxIdleTime,
>>>>>> TimeUnit.MILLISECONDS,
>>>>>> taskqueue, tf);
>>>>>> taskqueue.setParent(executor);
>>>>>> lifecycle.fireLifecycleEvent(AFTER_START_EVENT, null);
>>>>>> }
>>>>>>
>>>>>> protected class TerminatingTaskThreadFactory
>>>>>> extends StandardThreadExecutor.TaskThreadFactory {
>>>>>>
>>>>>> protected TerminatingTaskThreadFactory(String namePrefix)
>>>>>> {
>>>>>> super(namePrefix);
>>>>>> }
>>>>>>
>>>>>> public Thread newThread(Runnable runnable)
>>>>>> {
>>>>>> Thread t = new TerminatingThread(group, runnable,
>>>>>> namePrefix +
>>>>>> threadNumber.getAndIncrement());
>>>>>>
>>>>>> t.setDaemon(daemon);
>>>>>> t.setPriority(getThreadPriority());
>>>>>>
>>>>>> return t;
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>
>>>
>
