[
https://issues.apache.org/jira/browse/PYLUCENE-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378120#comment-17378120
]
Erik Groeneveld commented on PYLUCENE-58:
-----------------------------------------
Hi Andy,
I took me a while, but I discovered that building the extensions with debugging
was not right. The script below builds and runs successfully.
{code:java}
# Environment
# debian buster
# ant 1.10.5-2
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PYTHON=/usr/bin/python3-dbg
export DIST=dist
export PYTHONPATH=${DIST}/usr/local/lib/python3.7/dist-packages
export PYLUCENEVERSION="8.9.0"
export PYLUCENE="pylucene-${PYLUCENEVERSION}"
rm ${PYLUCENE}-src.tar.gz ${PYLUCENE} ${DIST} -rf
wget https://ftp.nluug.nl/internet/apache/lucene/pylucene/${PYLUCENE}-src.tar.gz
tar xzf ${PYLUCENE}-src.tar.gz
update sources with trunk
svn export --force https://svn.apache.org/repos/asf/lucene/pylucene/trunk
${PYLUCENE}
(cd ${PYLUCENE}
(cd jcc
export JCC_JDK=${JAVA_HOME}
${PYTHON} setup.py build --debug install --root ../../${DIST}
)
export NUM_FILES=10
export ANT=/usr/bin/ant
export JCC="${PYTHON} -m jcc --debug --shared"
export INSTALL_OPT="--root ../${DIST}"
export PYTHONPATH=../${PYTHONPATH}
make
make install
#make test # does not run since I can't specifiy location of 'dist' where
JCC resides
)
echo "Using: PYTHONPATH=${PYTHONPATH}"
echo "Running: gdb -d Python-3.7.3/Python --args ${PYTHON} -c \"import lucene\""
gdb -d Python-3.7.3/Python --args ${PYTHON} -c "import lucene"
{code}
It was quite hard to make sure to invoke python-dbg consistently/everywhere and
to make sure to actually use the results instead of falling back to system wide
installs.
The only thing that might be of interest to you is the way I can run 'make
test' with the just compiled JCC in ./dist instead of /usr/....
That's all for now. My apologies for taking your time; it seems that my brain
capacity is insufficient for being a proper software builder ;)
Erik
> SEGV on import lucene
> ---------------------
>
> Key: PYLUCENE-58
> URL: https://issues.apache.org/jira/browse/PYLUCENE-58
> Project: PyLucene
> Issue Type: Bug
> Environment: Debian Buster, Python 3.7
> Reporter: Erik Groeneveld
> Priority: Critical
>
> Hi Andy,
> Thanks again for your great work on PyLucene and JCC!
> Recently, after porting everything to python3, we get occasional SEGV's on
> shutdown. It happens very late, when the garbage collector starts cleaning up.
> Using python3-dbg exposed another problem however. With python3-dbg, "import
> lucene" already triggers SEGV. Here is the top of the backtrace:
>
> {code:bash}
> #0 0x0000000000000060 in ?? ()
> #1 0x00007fe8aee51d6e in unicode_fromformat_write_cstr
> (writer=writer@entry=0x7ffdc0dcd170, str=<optimized out>,
> width=width@entry=-1, precision=<optimized out>) at
> ../Objects/unicodeobject.c:2596
> #2 0x00007fe8aee525ec in unicode_fromformat_arg (vargs=0x7ffdc0dcd150,
> f=<optimized out>, writer=0x7ffdc0dcd170) at ../Objects/unicodeobject.c:2797
> #3 PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>) at
> ../Objects/unicodeobject.c:2914
> #4 0x00007fe8aedca3dd in PyErr_FormatV (exception=<type at remote 0x811cc0>,
> format=0x7fe8aefe2568 "%s:%d: bad argument to internal function",
> vargs=vargs@entry=0x7ffdc0dcd210) at ../Python/errors.c:835
> #5 0x00007fe8aedca4a4 in PyErr_Format (exception=<optimized out>,
> format=<optimized out>) at ../Python/errors.c:852
> #6 0x00007fe8aee89fcd in PyDict_SetItem (op=<optimized out>, key=<optimized
> out>, value=<optimized out>) at ../Objects/dictobject.c:1448
> #7 PyDict_SetItem (op=<optimized out>, key=<optimized out>, value=<optimized
> out>, op=<optimized out>, key=<optimized out>, value=<optimized out>) at
> ../Objects/dictobject.c:1443
> #8 0x00007fe8aee76f4a in module_init_dict (md_dict=<unknown at remote
> 0x7fe8ae9f6060>, name=name@entry=<unknown at remote 0x7fe8ae9f5030>,
> doc=None, doc@entry=0x0, mod=<optimized out>) at ../Objects/moduleobject.c:72
> #9 0x00007fe8aee7da83 in PyModule_NewObject (name=name@entry=<unknown at
> remote 0x7fe8ae9f5030>) at ../Objects/moduleobject.c:103
> #10 0x00007fe8aee7de2a in PyModule_New (name=name@entry=0x7fe8b32bfa20
> "lucene._lucene") at ../Objects/moduleobject.c:120
> #11 0x00007fe8aee7deec in _PyModule_CreateInitialized (module=0x7fe8b2612080
> <_lucene_def>, module_api_version=<optimized out>) at
> ../Objects/moduleobject.c:215
> #12 0x00007fe8b1238de7 in PyInit__lucene () from
> /data/bouwen/van_kras/pylucene-8.6.1/build/test/lucene-8.6.1-py3.7-linux-x86_64.egg/lucene/_lucene.cpython-37m-x86_64-linux-gnu.so
> {code}
> It could be that this goes undetected with normal python, yet causes an SEGV
> on shutdown.
>
> The error above can be reproduced with the following script that downloads
> the sources, builds JCC and PyLucene and the executes: python3-dbg -c "import
> lucene"
>
> {code:bash}
> # Environment
> # debian buster
> # ant 1.10.5-2
> export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> export PYTHON=/usr/bin/python3
> export PYLUCENE="pylucene-8.6.1"
> rm ${PYLUCENE}-src.tar.gz ${PYLUCENE} -rf
> wget
> https://ftp.nluug.nl/internet/apache/lucene/pylucene/${PYLUCENE}-src.tar.gz
> tar xzf ${PYLUCENE}-src.tar.gz
> (cd ${PYLUCENE}
> (cd jcc
> export JCC_JDK=${JAVA_HOME}
> export
> JCC_INCLUDES=/usr/include/python3.7m:${JAVA_HOME}/include:${JAVA_HOME}/include/linux
> ${PYTHON} setup.py build
> )
> export NUM_FILES=10
> export ANT=/usr/bin/ant
> export JCC="${PYTHON} -m jcc --shared"
> make
> make test
> )
> PYTHONPATH='pylucene-8.6.1/build/test/lucene-8.6.1-py3.7-linux-x86_64.egg'
> ${PYTHON}-dbg -c "import lucene"
> {code}
> Would you be as kind as to look into this? Perhaps our problem is solved, or
> it enables us to find an other problem at shutdown.
> Best regards,
> Erik
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
