The PREFIX seems to be the only issue - using:
JCC=$(PYTHON) -m jcc --reserved PREFIX --shared --find-jvm-dll
'make' and 'make install' did pass.
'make test' yields the usual windows specific issues (due to some file
locking issues) - there are a total of 45 errors in "make test", as said
most due to windows not being able to remove a locked resource, e.g.
File "C:\Python27\lib\shutil.py", line 247, in rmtree
os.remove(fullname)
WindowsError: [Error 32] Der Prozess kann nicht auf die Datei zugreifen, da
sie von einem anderen Prozess verwendet wird: 'testrepo\\_0.fdt'
Most errors seem to be follow-up issues (test-repo locked) - this also
occurred in PyLucene 3.6.
If anyone is interested in details - the output of 'make test' is available
here:
https://dl.dropboxusercontent.com/u/4384120/pylucene-4.2.1-test-20130419.txt
I debugged this (once again) for test_PyLucene and found that
- there is no write.lock pending on the store (directory 'testrepo')
- the store is closed properly
- there is a windows lock on a Field Data file that therefore cannot be
deleted: 'testrepo\_0.fdt'
- even timeout-retry on shutil.rmtree does not fix it
- even passing an onerror-handler to shutil.rmtree() that tries to force
remove this file (using os.chmod before) does not help
I'm clueless here. It could be that the JVM actually holds the lock and a
JVM shutdown would help here. I therefore tried
def setUp(self):
self.jvm = lucene.initVM()
...
and in
def tearDown(self):
...
self.jvm.detachCurrentThread()
But this did crash my python.exe ,-(
Is there any other way to safely re-init jcc during one process loop?
Here's the test_PyLucene.py with all my attempts:
https://dl.dropboxusercontent.com/u/4384120/test_PyLucene-win-20130419.py
I then ended up in using a 'fresh' test-repository (i.e. new dir name) if
rmtree fails. The test_PyLucene test then passes on windows - the downside
is that you end up with some testrepo.1, testrepo.2 etc. dirs - and of
course an attempt to cleanup these dirs after unittest.main(exit=False) also
fails...
There's a patch against the pylucene_4_2 for test_PyLucene with this "fix":
https://dl.dropboxusercontent.com/u/4384120/pylucene_4_2_test_PyLucene_WinFi
x_20130419.patch
Note: this should not affect linux tests and fixes the tests for windows.
Though the test_PyLucene now passes (with the fix), still 20 tests fail ,
e.g. test_PythonDirectory and test_FieldEnumeration. Possibly similar
workarounds could be applied here. As the main problem seems to be mixing
up several lucene-tests in one process call, alternatively one could write a
batch script, that calls all those tests one by one (in a new windows
process) - or even split test_-files into several smaller tests (python
files). Don't know if these are better ideas though.
So in summary: if you add the PREFIX to reserved words it will build and run
on windows.
Regards,
Thomas
-----Ursprüngliche Nachricht-----
Von: Andi Vajda [mailto:va...@apache.org]
Gesendet: Donnerstag, 18. April 2013 19:09
An: pylucene-dev@lucene.apache.org
Betreff: Re: AW: [VOTE] Release PyLucene 4.2.1-1
On Thu, 18 Apr 2013, Thomas Koch wrote:
> Andi,
> I now get a different error while compiling __init__.cpp:
>
> org/apache/lucene/util/automaton/CompiledAutomaton$AUTOMATON_TYPE.h(42
> )
:
> error C2059: Syntaxfehler: 'Zeichenfolge'
> org/apache/lucene/util/automaton/CompiledAutomaton$AUTOMATON_TYPE.h(42
> )
:
> error C2238: Unerwartete(s) Token vor ';'
>
> The line complained about is #42
>
> 40 static CompiledAutomaton$AUTOMATON_TYPE *NONE;
> 41 static CompiledAutomaton$AUTOMATON_TYPE *NORMAL;
> 42 static CompiledAutomaton$AUTOMATON_TYPE *PREFIX;
> 43 static CompiledAutomaton$AUTOMATON_TYPE *SINGLE;
>
> PREFIX seems to be another reserved word ... I could compile
__init__.cpp
> after renaming PREFIX to PREFIX1.
Instead of renaming PREFIX, could you please have JCC do it for you by
adding it to the list of reserved words in the JCC invocation via the
--reserved command line flag ? and rinse and repeat until all such conficts
due to macro definitions are solved ?
Or were you able to complete the build already once PREFIX was renamed ?
> I tried to google a list of reserved words used by VS C++ compiler,
> but
had
> no luck...
These are not reserved words but macro definitions that conflict with the
generated code. If PREFIX is, say, defined to 1, line 42 becomes:
static CompiledAutomaton$AUTOMATON_TYPE *1;
and that doesn't compile.
> There are some predefined macros -but none that match our issues
> http://msdn.microsoft.com/en-us/library/b0084kay(v=vs.100).aspx
Andi..
>
>
>
> Make output details:
>
> C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo
/Ox
> /MD /W3 /GS- /DNDEBUG -DPYTHON -DJCC_VER="2.16" -D_jcc_shared
> -D_java_generics -D_dll_lucene=__declspec(dllexport) "-IC:\Program
> Files\Java\jdk1.6.0_06/include" "-IC:\Program
> Files\Java\jdk1.6.0_06/include/win32" -Ibuild\_lucene
> -IC:\Python27\lib\site-packages\jcc-2.16-py2.7-win32.egg\jcc\sources
> -IC:\Python27\include -IC:\Python27\PC /Tpbuild\_lucene\__init__.cpp
> /Fobuild\temp.win32-2.7\Release\build\_lucene\__init__.obj /EHsc
> /D_CRT_SECURE_NO_WARNINGS __init__.cpp
>
C:\Python27\lib\site-packages\jcc-2.16-py2.7-win32.egg\jcc\sources\JCCEnv.
h(
> 118) : warning C4251: 'JCCEnv::refs': class 'std::multimap<_Kty,_Ty>'
> erfordert eine DLL-Schnittstelle, die von Clients von class 'JCCEnv'
> verwendet wird
> with
> [
> _Kty=int,
> _Ty=countedRef
> ]
>
f:\devel\workspaces\workspace.pylucene\pylucene-4.2.1-1\build\_lucene\org/
ap
> ache/lucene/util/automaton/CompiledAutomaton$AUTOMATON_TYPE.h(42) :
error
> C2059: Syntaxfehler: 'Zeichenfolge'
>
f:\devel\workspaces\workspace.pylucene\pylucene-4.2.1-1\build\_lucene\org/
ap
> ache/lucene/util/automaton/CompiledAutomaton$AUTOMATON_TYPE.h(42) :
error
> C2238: Unerwartete(s) Token vor ';'
> error: command '"C:\Program Files\Microsoft Visual Studio
> 9.0\VC\BIN\cl.exe"' failed with exit status 2
> make: *** [compile] Error 1
>
>
> regards,
> Thomas
>
> -----Ursprüngliche Nachricht-----
> Von: Andi Vajda [mailto:va...@apache.org]
> Gesendet: Mittwoch, 17. April 2013 22:11
> An: pylucene-dev@lucene.apache.org
> Cc: gene...@lucene.apache.org
> Betreff: [VOTE] Release PyLucene 4.2.1-1
>
>
> The PyLucene 4.2.1-0 release candidate had a number of problems
preventing
> its release. A PyLucene 4.2.1-1 release candidate is now ready for
review
> from:
>
> http://people.apache.org/~vajda/staging_area/
>
> A list of changes in this release can be seen at:
>
http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_4_2/CHAN
GE
> S
>
> PyLucene 4.2.1 is built with JCC 2.16 included in these release
artifacts:
> http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES
>
> A list of Lucene Java changes can be seen at:
>
http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_4_2_1/lucene/C
HA
> NGES.txt
>
> Please vote to release these artifacts as PyLucene 4.2.1-1.
>
> Thanks !
>
> Andi..
>
> ps: the KEYS file for PyLucene release signing is at:
> http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
> http://people.apache.org/~vajda/staging_area/KEYS
>
> pps: here is my +1
>
>
