On Fri, 15 May 2020, Chee Yong Teh wrote:
You will need wheel package to be installed so setuptools can use to create wheel. pip install wheel
Ah yes, that helps. I did not know about that (or anything about wheels). Thank you !
I was able to build a large .whl for PyLucene with 2441 entries in it and 32M in size.
I commited your contribution in rev 1877796. Thank you ! Andi..
Thanks, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138 OTC Infrastructure Service of the Year & Global Compression Service of the Year ________________________________ From: Andi Vajda <va...@apache.org> Sent: Friday, May 15, 2020 7:19:16 PM To: Chee Yong Teh <cheeyong....@quantile.com> Cc: pylucene-dev@lucene.apache.org <pylucene-dev@lucene.apache.org> Subject: RE: AttributeError: 'bool' object has no attribute 'setdefault' On Wed, 13 May 2020, Chee Yong Teh wrote:I don't know much about pip install myself. I have look at the code and instead of building an egg file, I added an option (--wheel) so it will build a wheel file instead elif arg == '--wheel': wheel = True dist = True use_distutils = False then pass that flag (wheel) into def compile then inside compile if dist: if wininst: script_args.append('bdist_wininst') elif with_setuptools: script_args.append('bdist_wheel' if wheel else 'bdist_egg') else: script_args.append('bdist') command line help Distribution actions: --use-distutils - use distutils even when setuptools is available --wheel - generate wheel using setuptools --bdist - generate a binary distutils-based distribution No idea how to send in patch but I think it's straightforward changes to add that option to JCC so user has option to build egg or wheel file and pip install will work on wheel file.I just integrated your code (for making a patch, use svn diff) and then ran it when making PyLucene by passing --wheel to the JCC invocation. I'm getting this error: error: invalid command 'bdist_wheel' Did you actually test this change ? Andi..Thanks, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138 OTC Infrastructure Service of the Year & Global Compression Service of the Year -----Original Message----- From: Andi Vajda <va...@apache.org> Sent: 13 May 2020 21:21 To: pylucene-dev@lucene.apache.org Subject: RE: AttributeError: 'bool' object has no attribute 'setdefault' On Wed, 13 May 2020, Andi Vajda wrote:On Wed, 13 May 2020, Chee Yong Teh wrote:Thanks for your help and I got everything build and working now.Great ! (thanks for letting me know)I have another question about pip install. I know JCC has option of bdist which will produce an egg file. However, pip install is not able to install an egg file. I know I can use easy install but I just wondering what is the steps to make a sdist or something that pip install can use from JCC command as I seen in the help of the command it mentions pip install (--egg-info and --extra-setup-arg)I know very little about pip myself, I did not implement or test these features in JCC, they were contributed as patches by users in the past.Searching the list archives, this thread seems relevant: http://pylucene-users-developers-list.2474766.n2.nabble.com/Changes-to-enable-easy-install-of-packages-using-JCC-td7243054.html#a7244239 Andi..Patches to better support 'pip install' are welcome ! Andi..Thanks, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138 OTC Infrastructure Service of the Year & Global Compression Service of the Year -----Original Message----- From: Andi Vajda <va...@apache.org> Sent: 07 May 2020 17:01 To: pylucene-dev@lucene.apache.org Subject: RE: AttributeError: 'bool' object has no attribute 'setdefault' On Thu, 7 May 2020, Chee Yong Teh wrote:Hi Andi, I think I got it working but there is an issue with the class name with '.'. In the jar there are some class name like below public static final class SensitivityProfileIdList.Builder public static final class SRMBreakdown.BuilderYes, these are so-called inner classes.So JCC will wrap this can change the '.' to '$'Which Java exposes to JCC with a '$' in place of the '.' in the name.SensitivityProfileIdList$Builder SRMBreakdown$Builder I cannot really use this classes in python and python flag it up as syntax error. Would it be better to use '_' instead of '$' to replace the '.' in the class name?That's definitely an option. Doing this automatically may then cause a clash with a class that uses _ already. There is a --rename flag that lets you control the name to pick: --rename 'full.package.path.here.SensitivityProfileIdList$Builder=SensitivityProfileIdList_Builder' (you need '' around a string with literal $ in a shell command line) Andi..Thanks, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138 OTC Infrastructure Service of the Year & Global Compression Service of the Year -----Original Message----- From: Andi Vajda <va...@apache.org> Sent: 06 May 2020 00:34 To: Chee Yong Teh <cheeyong....@quantile.com> Cc: pylucene-dev@lucene.apache.org Subject: Re: AttributeError: 'bool' object has no attribute 'setdefault'On May 5, 2020, at 16:17, Chee Yong Teh <cheeyong....@quantile.com> wrote: Hi Andi, Ok I have changed the command to python -m jcc --version 7.1.4 --use_full_names --include /home/cheeyong.teh/SMART-API-7.1.4/smart/lib/smart-swapclear-public-release_daru.12. jar --include /home/cheeyong.teh/SMART-API-7.1.4/smart/lib/colt-1.2.0.jar --include /home/cheeyong.teh/SMART-API-7.1.4/smart/lib/commons-math3-3.6.1.ja r --include /home/cheeyong .teh/SMART-API-7.1.4/smart/lib/jna-5.2.0.jar --include /home/cheeyong.teh/SMART-API-7.1.4/smart/lib/log4j-1.2.16.jar --include /home/cheeyong.teh/SMART-API-7.1.4/smart/lib/proto buf-java-3.5.1.jar --include /home/cheeyong.teh/SMART-API-7.1.4/smart/lib/slf4j-api-1.6.1.jar --include /home/cheeyong.teh/SMART-API-7.1.4/smart/lib/slf4j-log4j12-1.6.1.ja r --py thon lch_smart --build --install com.lchclearnet.swapclear.smart.SMARTClientDataProviderFactory I'm using --include instead of --jar and pass in com.lchclearnet.swapclear.smart.SMARTClientDataProviderFactory as a class I want to wrap. It seem fine and everything build fine but when I do a print(dir(lch_smart)). It doesn't seem to wrap that SMARTClientDataProviderFactory classprint(dir(lch_smart))['CLASSPATH', 'ConstVariableDescriptor', 'FinalizerClass', 'FinalizerProxy', 'InvalidArgsError', 'JArray', 'JArray_bool', 'JArray_byte', 'JArray_char', 'JArray_double', 'JArray_float', 'JArray_int', 'JArray_long', 'JArray_object', 'JArray_short', 'JArray_string', 'JCCEnv', 'JCC_VERSION', 'JObject', 'JavaError', 'PrintWriter', 'StringWriter', 'VERSION', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__module_dir__', '__name__', '__package__', '__path__', '__spec__', '_lch_smart', 'findClass', 'getVMEnv', 'initVM', 'makeClass', 'makeInterface', 'os']You're using --use_full_names, thus to access the wrapper for com.lchclearnet.swapclear.smart.SMARTClientDataProviderFactory you need to say:from com.lchclearnet.swapclear.smart importSMARTClientDataProviderFactory Andi..If I use --jar option I will still get the "AttributeError: 'bool' object has no attribute 'setdefault'" error. Thanks, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138 <image940421.png> OTC Infrastructure Service of the Year & Global Compression Service of the Year -----Original Message----- From: Andi Vajda <va...@apache.org> Sent: 05 May 2020 23:30 To: Chee Yong Teh <cheeyong....@quantile.com> Cc: pylucene-dev@lucene.apache.org Subject: Re: AttributeError: 'bool' object has no attribute 'setdefault' On Tue, 5 May 2020, Chee Yong Teh wrote:Hi Andi, That’s third party library jar file that we try to wrap so we can call it from python. I don’t really know why they create a classes like that way. Would you able to add an option to just only wrap classes under a package like I want to wrap all public classes in package com.lchclearnet.* So any public classes under com.lchclearnet.swapclear and com.lchclearnet.common will get wrapped? At the moment I think JCC try to wrap all the public classes found in the jar if I pass in the jar via -jar option.I wonder why this isn't documented. JCC's __main__.py file lists all the options you can use with JCC. The one that doesn't seem listed is the non-option: if a command line argument doesn't start with '-', then it is assumed to be a class name. Better yet, you can also use a className:methodName syntax to only wrap that. There is a large example of a JCC invocation here: https://svn.apache.org/viewvc/lucene/pylucene/trunk/Makefile?view=m ar k up look for org.apache.lucene.index.IndexWriter:getReader, for an example. For controlling which dependencies are pulled in, the answer is none unless you declare them via --jar, other classes, or --package. In other words, depending on your use case, you may be able to just generate wrappers for the actual entrypoints you wish to call from Python and the dependencies implied in the method signatures need to be declare manually via other classes, --jar or --package. Also, be aware of --jar vs --include. See __main__.py for details: https://svn.apache.org/viewvc/lucene/pylucene/trunk/jcc/jcc3/__main__. py?view=markup Andi..Thanks, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138 OTC Infrastructure Service of the Year & Global Compression Service of the Year ________________________________ From: Andi Vajda <va...@apache.org> Sent: Tuesday, May 5, 2020 10:17:18 PM To: Chee Yong Teh <cheeyong....@quantile.com> Cc: va...@apache.org <va...@apache.org> Subject: Re: AttributeError: 'bool' object has no attribute 'setdefault' So if looks like you have classes and packages using the same names. This could be triggering a bug in jcc. Let me try this locally. Just curious, what are all these 'a' classes ? Andi.. On May 5, 2020, at 10:47, Chee Yong Teh <cheeyong....@quantile.com> wrote: HI Andi, Please see below print out -declares : {<Class: interface it.unimi.dsi.fastutil.h>, <Class: class java.lang.Class>, <Class: interface it.unimi.dsi.fastutil.c>, <Class: interface it.unimi.dsi.fastutil.h.au>, <Class: interface it.unimi.dsi.fastutil.h.H>} --declareNames : ['it', 'unimi', 'dsi', 'fastutil', 'h'] ---declareName : it ---namespace : {} ---declareName : unimi ---namespace : {} ---declareName : dsi ---namespace : {} ---declareName : fastutil ---namespace : {} --declareNames : ['java', 'lang', 'Class'] ---declareName : java ---namespace : {'it': {'unimi': {'dsi': {'fastutil': {'h': True}}}}} ---declareName : lang ---namespace : {} --declareNames : ['it', 'unimi', 'dsi', 'fastutil', 'h', 'au'] ---declareName : it ---namespace : {'it': {'unimi': {'dsi': {'fastutil': {'h': True}}}}, 'java': {'lang': {'Class': True}}} ---declareName : unimi ---namespace : {'unimi': {'dsi': {'fastutil': {'h': True}}}} ---declareName : dsi ---namespace : {'dsi': {'fastutil': {'h': True}}} ---declareName : fastutil ---namespace : {'fastutil': {'h': True}} ---declareName : h ---namespace : {'h': True} Above is one of many classes that hit with the 'bool' object has no attribute 'setdefault'. Another one -declares : {<Class: class java.lang.Class>, <Class: interface swapclear.com.lchclearnet.c.a.a.a.a>, <Class: interface swapclear.com.lchclearnet.c.a.b.a.a>, <Class: interface swapclear.com.lchclearnet.c.a.b.a.a.a.a.a.a>, <Class: class swapclear.com.lchclearnet.c.a.b.a.e>, <Class: interface swapclear.com.lchclearnet.c.a.c.a>} --declareNames : ['java', 'lang', 'Class'] ---declareName : java ---namespace : {} ---declareName : lang ---namespace : {} --declareNames : ['swapclear', 'com', 'lchclearnet', 'c', 'a', 'a', 'a', 'a'] ---declareName : swapclear ---namespace : {'java': {'lang': {'Class': True}}} ---declareName : com ---namespace : {} ---declareName : lchclearnet ---namespace : {} ---declareName : c ---namespace : {} ---declareName : a ---namespace : {} ---declareName : a ---namespace : {} ---declareName : a ---namespace : {} --declareNames : ['swapclear', 'com', 'lchclearnet', 'c', 'a', 'b', 'a', 'a'] ---declareName : swapclear ---namespace : {'java': {'lang': {'Class': True}}, 'swapclear': {'com': {'lchclearnet': {'c': {'a': {'a': {'a': {'a': True}}}}}}}} ---declareName : com ---namespace : {'com': {'lchclearnet': {'c': {'a': {'a': {'a': {'a': True}}}}}}} ---declareName : lchclearnet ---namespace : {'lchclearnet': {'c': {'a': {'a': {'a': {'a': True}}}}}} ---declareName : c ---namespace : {'c': {'a': {'a': {'a': {'a': True}}}}} ---declareName : a ---namespace : {'a': {'a': {'a': {'a': True}}}} ---declareName : b ---namespace : {'a': {'a': {'a': True}}} ---declareName : a ---namespace : {} --declareNames : ['swapclear', 'com', 'lchclearnet', 'c', 'a', 'b', 'a', 'a', 'a', 'a', 'a', 'a'] ---declareName : swapclear ---namespace : {'java': {'lang': {'Class': True}}, 'swapclear': {'com': {'lchclearnet': {'c': {'a': {'a': {'a': {'a': True}}, 'b': {'a': {'a': True}}}}}}}} ---declareName : com ---namespace : {'com': {'lchclearnet': {'c': {'a': {'a': {'a': {'a': True}}, 'b': {'a': {'a': True}}}}}}} ---declareName : lchclearnet ---namespace : {'lchclearnet': {'c': {'a': {'a': {'a': {'a': True}}, 'b': {'a': {'a': True}}}}}} ---declareName : c ---namespace : {'c': {'a': {'a': {'a': {'a': True}}, 'b': {'a': {'a': True}}}}} ---declareName : a ---namespace : {'a': {'a': {'a': {'a': True}}, 'b': {'a': {'a': True}}}} ---declareName : b ---namespace : {'a': {'a': {'a': True}}, 'b': {'a': {'a': True}}} ---declareName : a ---namespace : {'a': {'a': True}} ---declareName : a ---namespace : {'a': True} ---declareName : a ---namespace : True So I'm not sure how many classes in the jar that causing this issue. There is a similar code in python.py line 1547 Thanks, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138<tel:+44%20203%20929%203138> <http://www.quantile.com/> <image218792.png> OTC Infrastructure Service of the Year & Global Compression Service of the Year -----Original Message----- From: Andi Vajda <va...@apache.org> Sent: 05 May 2020 17:50 To: pylucene-dev@lucene.apache.org Subject: Re: AttributeError: 'bool' object has no attribute 'setdefault' On Mon, 4 May 2020, Chee Yong Teh wrote:I'm in the processing of testing JCC to wrap third party library jar. When I run JCC 3.7 I got the following error Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/cheeyong.teh/venv/simm/lib/python3.6/site-packages/jcc/__m ai n_ _.py", line 107, in <module> cpp.jcc(sys.argv) File "/home/cheeyong.teh/venv/simm/lib/python3.6/site-packages/jcc/cpp.py" , line 711, in jcc _dll_export) File "/home/cheeyong.teh/venv/simm/lib/python3.6/site-packages/jcc/cpp.py" , line 937, in header namespace = namespace.setdefault(declareName, {}) AttributeError: 'bool' object has no attribute 'setdefault' This line "namespace = namespace.setdefault(declareName, {})" assume namespace.setdefault(declareName, {}) will return {} but somehow it returns True. So namespace has True value and it errors in next iteration in the loop. Not sure it's a bug or the jar is too complicated to parse/convert due to a directory/structure. I can send over the library if you need to debug/reproduce the error on your machineCould you please put a conditional breakpoint before the line where this error happens and tell me what 'declares' is set to ? Thanks ! Andi..The commandline I use python -m jcc \ --use_full_names \ --no-generics \ --jar lib/smart-swapclear-public-release_daru.12.jar \ --include lib/colt-1.2.0.jar \ --include lib/commons-math3-3.6.1.jar \ --include lib/jna-5.2.0.jar \ --include lib/log4j-1.2.16.jar \ --include lib/protobuf-java-3.5.1.jar \ --include lib/slf4j-api-1.6.1.jar \ --include lib/slf4j-log4j12-1.6.1.jar \ --python lch_smart --build -install Another question, --jar option seem to wrap all the public classes found inside the jar. Is there an option that only wrap certain public classes under package com.lchclearnet.* inside a jar? There is an option called exclude for classes but it will be nightmare to workout all the classes I don't want to wrap inside the jar. Thanks, Kinds regards, Chee Yong Chee Yong Teh 27 Bush Lane, London, EC4R 0AN | +44 203 929 3138 OTC Infrastructure Service of the Year & Global Compression Service of the Year Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.________________________________ This email (including any attachments) is intended only for the addressee(s) named above. Its contents may be confidential. If you receive this email in error, please immediately contact the sender and delete this email. Unauthorised use, disclosure, storage or copying of this email and any attachments is not permitted and may be unlawful. Quantile Technologies Limited is authorised and regulated by the Financial Conduct Authority. Financial Services Register number: 751812. Registered in England and Wales, company number: 09622720. Registered office: Cannon Green, 27 Bush Lane, London, England EC4R 0AN Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.This email (including any attachments) is intended only for the addressee(s) named above. Its contents may be confidential. If you receive this email in error, please immediately contact the sender and delete this email. Unauthorised use, disclosure, storage or copying of this email and any attachments is not permitted and may be unlawful. Quantile Technologies Limited is authorised and regulated by the Financial Conduct Authority. Financial Services Register number: 751812. Registered in England and Wales, company number: 09622720. Registered office: Cannon Green, 27 Bush Lane, London, England EC4R 0ANDisclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
