Progress migrating cffi and pycparser to libclang
Hi,
I will be creating a repository for this:
https://bitbucket.org/tkadm30/cffi-libclang
The goal is to generate a AST object from a C header by preprocessing
with clang -E then compile the python bindings with CFFI...
ffi.cdef(open('uwsgi.h').read()) # <-- XXX need to modify internal
parsing code to use clang.cindex!
ffi.dlopen('/usr/local/lib/libuwsgi.so')
Right now the parsing of the C source file is done by pycparser, but i
would like to use the clang.cindex module.
What do you think?
Etienne
--
Etienne Robillard
tkad...@yandex.com
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Progress migrating cffi and pycparser to libclang
On 4 January 2018 at 09:50, Etienne Robillard wrote:
> Hi,
>
> I will be creating a repository for this:
> https://bitbucket.org/tkadm30/cffi-libclang
>
> The goal is to generate a AST object from a C header by preprocessing with
> clang -E then compile the python bindings with CFFI...
>
> ffi.cdef(open('uwsgi.h').read()) # <-- XXX need to modify internal parsing
> code to use clang.cindex!
>
> ffi.dlopen('/usr/local/lib/libuwsgi.so')
>
>
> Right now the parsing of the C source file is done by pycparser, but i would
> like to use the clang.cindex module.
>
> What do you think?
Presumably that will introduce a dependency on some clang module? You
mention clang.cindex - but the only clang package I can find on PyPI
says "OBSOLETE. LLVM-CLANG NOW PUBLISHES PYTHON PACKAGE.
JUST ADD THE OFFICIAL llvm-3.7 repo in your apt." but doesn't provide
binaries or explain how to install clang on, say, Windows (to pick an
example relevant to me :-)).
As a fork/extension for cffi, I have no particular opinion (I'm
unlikely to ever use it). But the advantage of pycparser is that it's
cross-platform and pure Python, so I doubt this will be acceptable for
inclusion into CFFI itself.
Paul
--
https://mail.python.org/mailman/listinfo/python-list
Re: Progress migrating cffi and pycparser to libclang
Hi Paul, Le 2018-01-04 à 06:41, Paul Moore a écrit : Presumably that will introduce a dependency on some clang module? You mention clang.cindex - but the only clang package I can find on PyPI says "OBSOLETE. LLVM-CLANG NOW PUBLISHES PYTHON PACKAGE. JUST ADD THE OFFICIAL llvm-3.7 repo in your apt." but doesn't provide binaries or explain how to install clang on, say, Windows (to pick an example relevant to me :-)). I'm targeting the Linux/x86 machine class. On debian systems, it's pretty easy to install clang and the python bindings with apt-get. :-) As a fork/extension for cffi, I have no particular opinion (I'm unlikely to ever use it). But the advantage of pycparser is that it's cross-platform and pure Python, so I doubt this will be acceptable for inclusion into CFFI itself. CFFI/pycparser definitely need to be patched to support parsing standard C directives like #define and #include in the ffi.cdef() function. The easiest solution is to migrate the internal parsing code to libclang, a state-of-the art C/C++ compiler based on LLVM. Best regards, Etienne -- Etienne Robillard tkad...@yandex.com https://www.isotopesoftware.ca/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Progress migrating cffi and pycparser to libclang
On 4 January 2018 at 21:02, Etienne Robillard wrote: >> As a fork/extension for cffi, I have no particular opinion (I'm >> unlikely to ever use it). But the advantage of pycparser is that it's >> cross-platform and pure Python, so I doubt this will be acceptable for >> inclusion into CFFI itself. > > CFFI/pycparser definitely need to be patched to support parsing standard C > directives like #define and #include in the ffi.cdef() function. > > The easiest solution is to migrate the internal parsing code to libclang, a > state-of-the art C/C++ compiler based on LLVM. I would strongly object to adding a dependency to cffi that couldn't be automatically installed by pip as part of standard dependency resolution (i.e., a PyPI hosted Python project with wheels available for all common platforms - Linux, Mac OS and Windows). But ultimately if you're proposing this as a change to cffi, you should be getting the opinions of the cffi devs, not just asking on this list. (I notice you have posted to the cffi mailing list, but haven't had any response yet). Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: Problems with imports on multiple threads, with embedded Python
geoff.ba...@gmail.com writes: > I have a multithreaded application using an embedded Python 3.6.4 (upgraded > from 3.6.2 today in the hope that the problem was now solved: it doesn't seem > to be). The standard library is in a zip file. So long as only one thread is > running Python at a time it seems to work fine. But there seems to be a > problem with the module importing when several Python threads are active. > > I get a variety of errors indeterministically, usually indicating that some > symbol hasn't been imported. This occurs both in my own code and in the > standard library. The most frequent is probably this line: I do not know your specific problem (up to now, I used only Python 2), but I know about import problems in multi threaded applications. The problem is easily understood: import affects the global object "sys.modules". To protect this object, Python must do something. In former Python versions (somewhere before Python 3), it was using a a thread lock: this could cause a deadlock in the case of some import dependencies. The recoomendation to avoid the problem has been to avoid module level imports for modules imported by threads. Python 3 may have changed the way to protect "sys.modules". In case of recursive import dependencies, you may see that some symbols are not defined even in single threaded applications -- however then usually deterministically. At your place, I would start to check whether your application has recursive import dependancies -- and in this case, remove them. -- https://mail.python.org/mailman/listinfo/python-list
Re: Regarding the error: TypeError: can�t pickle _thread.lock objects
Winston Manuel Vijay writes: > It would be of immense help, if someone could provide a suitable solution or > related information that helps to sort out the below stated issue- > > > Ã I had installed the Python version 3.6.4 > > Ã Then I installed the package: Tensorflow > > Ã Installed g2p.exe by downloading from GitHub > > Ã Then tried running the below command- > > g2p-seq2seq --interactive --model (model_folder_path: is > the path to an English model 2-layer LSTM with 512 hidden units CMU Sphinx > dictionary downloaded from the CMU Sphinx website) > > Following the above procedure, I encountered the following error: TypeError: > can.t pickle _thread.lock objects-please find the attached screenshot for > your reference. This looks like a programming error in "g2p-seq2sec": contact its authors or try a different version. -- https://mail.python.org/mailman/listinfo/python-list
