pthreads in C++ with embedded Python
Hi guys! I am trying to build a C++ application that uses pthreads and embedded python. I've simplified the problem down so that the Python code is a single class that subclasses from Queue. The main thread of the C++ application adds to the queue. A worker thread in the C++ application reads from the queue. I am using: PyGILState_Ensure() PyGILState_Release() to lock the GIL and release it. I'm finding this situation happening: 1) main thread locks the GIL 2) worker thread locks the GIL 3) main thread tries to release the GIL, but the thread state is not current and the program aborts with this error: Fatal Python error: This thread state must be current when releasing Other times, the program runs to completion, only to segfault just before it exits. C++ code can be found here: http://pastie.org/2035072 Python code can be found here: http://pastie.org/2035086 I've tried several different APIs without any success. This is my first attempt at embedding Python. So, what is correct way to do this? Any suggestions will be appreciated. Thanks! Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: pthreads in C++ with embedded Python
On Wed, Jun 8, 2011 at 2:11 PM, Jason Tackaberry wrote: > On 11-06-07 07:29 PM, Tom Brown wrote: > >> Any suggestions will be appreciated. >> > > Why are you calling PyEval_ReleaseLock() in the CmdThread constructor? > This looks suspicious. > > Also, I don't see where CmdThread::lock() and CmdThread::unlock() are being > invoked in your example. Relics from your effort to create a reduced > testcase I assume? > > Jason, I found that PyEval_ReleaseLock() was necessary to keep the program from hanging. The lock() and unlock() methods were used in a previous attempt to lock/unlock the GIL. I kept banging at this and finally gave up on the GIL. I used a mutex instead. This allowed me to get rid of the Python API calls that dealt with the GIL. It works great in the test program. I'll find out how well it performs in the real program. Thanks! Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
I used pyinstaller quite a bit 3 years ago. I could brush off the cobwebs and see if I can help if you have not solved it already. What is the issue you are having? -Tom On Jun 21, 2016 16:57, "Larry Martell" wrote: > Anyone here have any experience with pyinstaller? I am trying to use > it, but I'm not having much success. I tried posting to the > pyinstaller ML but it said my post had to be approved first, and that > hasn't happened in a while. I'll post details if someone here thinks > they can help. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
subprocess.Popen and replacing the shell pipe line
I need to chain together three linux commands and get the final output. I read the documentation for Popen in the subprocess module for replacing the shell pipe line. I followed the example and keep getting a 0 where I should be getting a 1. I am trying to do this: grep "Sep 22" /var/log/auth.log | grep "Illegal user" | wc -l which returns a 1 when I run this manually. This is what I did with python: p1 = Popen(['grep', '"Sep 22"', '/var/log/auth.log'], stdout=PIPE) p2 = Popen(['grep', '"Illegal user"'], stdin=p1.stdout, stdout=PIPE) p3 = Popen(['wc', '-l'], stdin=p2.stdout, stdout=PIPE) print p3.stdout.read() which always prints a 0. What am I doing wrong? Thanks, Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: installing PygreSQL and psychopg2 modules (a newbie question)
On Wednesday 02 November 2005 14:10, Zlatko Matić wrote:
> Hello.
> I was trying to install PygreSQL and psychopg2 in order to use python
> as front-end for PostgreSQL, on WIndows XP. When I tried to install
> by calling setup.py from command prompt ("setup.py install"), in both
> cases I had the same error:
>
> "error: Python was built with version 7.1 of Visual Studio, and
> extensions need to be built with the same version of the compiler,
> but it isn't installed."
>
> It looks like this:
> "C:\Downloads\Python\psychopg\psycopg2-2.0b5\psycopg2-2.0b5>setup.py
> install running install
> running build
> running build_py
> running build_ext
> building 'psycopg2._psycopg' extension
> error: Python was built with version 7.1 of Visual Studio, and
> extensions need t
> o be built with the same version of the compiler, but it isn't
> installed.
>
> C:\Downloads\Python\psychopg\psycopg2-2.0b5\psycopg2-2.0b5>pause
> Press any key to continue . . ."
>
> What does it mean ? What am I supposed to do?
>
> Thanks,
>
> Zlatko
Use the windows installer for psycopg2. You can download it here:
http://stickpeople.com/projects/python/win-psycopg/psycopg2-2.0b5.win32-py2.4.exe
Tom
--
http://mail.python.org/mailman/listinfo/python-list
how to send an int over a socket
Hi,
I have what seems to be a simple problem. But I can not for the life of me
find a way to send an integer over a socket. The send method will only accept
strings. Here is what I am trying to do:
testmessage = 'test message'
msglen = len(testmessage)
sock.send(msglen)
sock.send(testmessage)
Now, I know I can do this:
testmessage = 'test message'
sock.send('\xC') # send hard coded length of testmessage in hex in string
sock.send(testmessage)
However, in my actual program I will not know the length of testmessage in
advance. So how do I convert msglen into a suitable format for the send
method?
Thanks,
Tom
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to send an int over a socket
On Friday 04 February 2005 18:27, Tom Brown wrote:
> Hi,
>
> I have what seems to be a simple problem. But I can not for the life of me
> find a way to send an integer over a socket. The send method will only
> accept strings. Here is what I am trying to do:
>
> testmessage = 'test message'
> msglen = len(testmessage)
> sock.send(msglen)
> sock.send(testmessage)
>
> Now, I know I can do this:
>
> testmessage = 'test message'
> sock.send('\xC') # send hard coded length of testmessage in hex in string
> sock.send(testmessage)
>
> However, in my actual program I will not know the length of testmessage in
> advance. So how do I convert msglen into a suitable format for the send
> method?
>
> Thanks,
> Tom
Ok, I think I've found what I was looking for. The marshal.dumps() function
will convert my integer into a string representation of a fixed size. This
way, on the other side, I know how many bytes to read to get the size of the
string.
Thanks,
Tom
--
http://mail.python.org/mailman/listinfo/python-list
Re: A brief question.
On Saturday 02 July 2005 10:55, Nathan Pinno wrote: > Brief question for anyone who knows the answer, because I don't. Is > there anyway to make Python calculate square roots? from math import sqrt -- http://mail.python.org/mailman/listinfo/python-list
Re: How to use writelines to append new lines to an existing file
On Thursday 22 September 2005 05:52, Nico Grubert wrote:
> Does f = open('/tmp/myfile', 'w') overwrite the existing file or
> does f.writelines('456') replace the first line in the existing file?
Here's an excerpt from open.__doc__
The mode can be 'r', 'w' or 'a' for reading (default),
writing or appending. The file will be created if it doesn't exist
when opened for writing or appending; it will be truncated when
opened for writing.
Tom
--
http://mail.python.org/mailman/listinfo/python-list
win32 service and sockets
Hi,
I created a win32 service for XPPro called N4010ATestService.py (see below).
The service runs as a particular user with administrative rights. It starts a
thread that creates a simple socket server (N4010ASocketServer.py -- also
below) that just waits for 20 character string. When I run the socket server
by itself the test client can connect to the server and send a 20 character
string just fine. When I run the service, the server will bind to the port
but the client cannot connect. I verified the server was listening on the
given port using netstat -an. The client eventually times out. Why isn't the
server accepting connections when run in a service?
Thanks,
Tom
N4010ATestService.py:
-
import win32serviceutil
import win32service
import win32event
import N4010ASocketServer
from thread import start_new_thread
class N4010ATestService(win32serviceutil.ServiceFramework):
_svc_name_ = "N4010ATestService"
_svc_display_name_ = "N4010A Test Service"
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
def SvcDoRun(self):
start_new_thread(N4010ASocketServer.main, ())
N4010ASocketServer.waitfor()
def SvcStop(self):
# Before we do anything, tell the SCM we are starting the stop process.
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
N4010ASocketServer.stop()
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(N4010ATestService)
N4010ASocketServer.py:
--
from socket import socket
from select import select
from time import sleep
import win32evtlogutil
applicationName = 'N4010ASocketServer'
messageDll = 'C:\Python24\Lib\site-packages\win32\pythonservice.exe'
running = True
stopped = False
def registerWithEventViewer():
win32evtlogutil.AddSourceToRegistry(applicationName, messageDll)
def stop():
import servicemanager
servicemanager.LogInfoMsg('stopping')
global running
running = False
def waitfor():
while not stopped:
sleep(0.5)
def main():
import servicemanager
registerWithEventViewer()
servicemanager.LogInfoMsg('creating socket')
sock = socket()
servicemanager.LogInfoMsg('binding to port 48777')
sock.bind(('0.0.0.0', 48777))
servicemanager.LogInfoMsg('listening')
sock.listen(5)
while running:
servicemanager.LogInfoMsg('Waiting for connection.')
readyRead, readyWrite, inerror = select([sock], [], [], 0)
while (not readyRead) and running:
sleep(0.5)
readyRead, readyWrite, inerror = select([sock], [], [], 0)
if running:
conn, address = sock.accept()
msg = conn.recv(20)
servicemanager.LogInfoMsg('Recvd msg: %s' % msg)
conn.close()
global stopped
stopped = True
--
http://mail.python.org/mailman/listinfo/python-list
Re: win32 service and sockets
On Tuesday 08 February 2005 16:41, Tom Brown wrote: > Hi, > > I created a win32 service for XPPro called N4010ATestService.py (see > below). The service runs as a particular user with administrative rights. > It starts a thread that creates a simple socket server > (N4010ASocketServer.py -- also below) that just waits for 20 character > string. When I run the socket server by itself the test client can connect > to the server and send a 20 character string just fine. When I run the > service, the server will bind to the port but the client cannot connect. I > verified the server was listening on the given port using netstat -an. The > client eventually times out. Why isn't the server accepting connections > when run in a service? > > Thanks, > Tom > Well, I have found that it works if I launch the client on the same machine as the service. It will not work from a remote machine. Any ideas? Thanks, Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: win32 service and sockets
On Wednesday 09 February 2005 10:48, David Bolen wrote: > Tom Brown <[EMAIL PROTECTED]> writes: > > Well, I have found that it works if I launch the client on the same > > machine as the service. It will not work from a remote machine. Any > > ideas? > > Since you mentioned Xp, could any of it's built-in firewall support be > enabled, and perhaps blocking access to your server's port? > David, That was it. It was the built-in firewall that was getting me. Thanks! Tom -- http://mail.python.org/mailman/listinfo/python-list
serial ports, threads and windows
Hey people, I've written a python app that r/w eight serial ports to control eight devices using eight threads. This all works very nicely in Linux. I even put a GUI on it using PyQt4. Still works nicely. Then I put the app on on a virtual Windows machine running inside of vmware on the same Linux box. Vmware only lets me have four serial ports so I run the app against four serial ports using four threads. The app did not respond quick enough to data from the serial ports and eventually hung. So, I tried one serial port and the app still did not respond quick enough to the single serial port. It eventually hangs. When the app hung, in each case, it was not hogging the cpu nor reading any data off the serial ports. The task manager didn't show it was doing anything at all. When it runs on Windows, could it be: 1) Just struggling to run inside of VMware? 2) Using threads with Qt on Windows is a problem? 3) Threads in python on Windows is a problem? Any ideas? Thanks, Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: serial ports, threads and windows
On Wednesday 02 August 2006 16:02, Tom Brown wrote: > I've written a python app that r/w eight serial ports to control eight > devices using eight threads. This all works very nicely in Linux. I even > put a GUI on it using PyQt4. Still works nicely. > > Then I put the app on on a virtual Windows machine running inside of vmware > on the same Linux box. Vmware only lets me have four serial ports so I run > the app against four serial ports using four threads. The app did not > respond quick enough to data from the serial ports and eventually hung. > > So, I tried one serial port and the app still did not respond quick enough > to the single serial port. It eventually hangs. > > When the app hung, in each case, it was not hogging the cpu nor reading any > data off the serial ports. The task manager didn't show it was doing > anything at all. I haven't found the fix, but I have found the culprit. There are eight threads and eight edit widgets. Each thread is posting messages to an edit widget created for that thread (not by that thread). The messages are characters read off the serial port. I am actually posting a line at a time, not a character at a time. The idea is that the user can see what is happening on the serial port in the event something goes wrong that the application cannot recover from. When I modified the app so the threads were NOT posting messages, the app worked as expected. This was never an issue in Linux. So, does that mean that this is a Windows problem? Or a Qt4 problem? I am curious to know if anybody has written a python app with many threads posting messages to the main application. How well did it work? What gui toolkit did you use? Thanks, Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: How to fill a form
On Monday 14 August 2006 20:43, Sulsa wrote: > On Tue, 15 Aug 2006 03:37:02 - > > Grant Edwards <[EMAIL PROTECTED]> wrote: > > On 2006-08-15, Sulsa <[EMAIL PROTECTED]> wrote: > > > I want to fill only one smiple form so i would like not to use > > > any non standard libraries. > > > > Then just send the HTTP "POST" request containing the fields > > and data you want to submit. > > but i don't know how to post these data if i knew there there would > be no topic. When I need to learn the POST method for a form, I use wireshark (www.wireshark.org) to capture the POST I do manually the first time. However, I think I'm going to look into mechanize/clientform now that I know about it. Hope this helps, Tom -- http://mail.python.org/mailman/listinfo/python-list
low level ethernet device access in linux
Hi, I have a windows application, written in delphi, that communicates to our devices using raw ethernet frames. I am trying to port this application to linux using python. However, when I try to open a socket, I get this error: File "/home/tbrown/projects/discovery/trunk/comm.py", line 9, in __init__ self.s = socket(AF_PACKET, SOCK_RAW, proto) File "/usr/local/lib/python2.5/socket.py", line 156, in __init__ _sock = _realsocket(family, type, proto) socket.error: (1, 'Operation not permitted') I understand that I am getting this error because I am running the application as a user and not as root. I would like to be able to run this app. as a user. Is there a way to create a socket without running the app. as root or sudo? When I run the app. as root I get this error: discovery.py: cannot connect to X server Thanks, Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: why should I learn python
On Thursday 06 September 2007 14:32, windandwaves wrote: > Can someone tell me why I should learn python? I am a webdeveloper, > but I often see Python mentioned and I am curious to find out what I > am missing out on. Ease of develpment. I write everything I can in python. If I can't do it in python, I throw a fit and find a way to do it in python. :) I developed in assembly, C, C++, Prolog and Delphi. Python has been by far the easiest to develop in. Some people might say it is not "real programming" because it is so easy. Whatever. If you wan't real programming, do it in assmebly. If you want to get a lot done while having a lot of fun doing it, do it in python. Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing zipfile (new style class)
On Thursday 06 September 2007 16:15, Larry Bates wrote: > I'm trying to learn about subclassing new style classes and the first > project I went to do needs to subclass zipfile to add some methods. > > > Why does this: > > import zipfile > class walkZip(zipfile): > pass > > > if __name__ == "__main__": > print "Hello World" > > Traceback (most recent call last): >File "", line 192, in run_nodebug >File "", line 2, in > TypeError: Error when calling the metaclass bases > module.__init__() takes at most 2 arguments (3 given) > I think because you need to do this: from zipfile import ZipFile class walkZip(ZipFile): pass -Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: why should I learn python
On Thursday 06 September 2007 16:01, windandwaves wrote: > Hmmm, thank you all for your replies. I will do some research on the > net (i did some already, but because I am really not much of a > programmer, it is often too detailed for me). I have limited time, > but it does sound like something to learn, just for fun and for > practical use. How would you use it in a web development > environment? I mean, in real practical terms. Could it assist with > php? Is it easy to write GUI programs in Python? Checkout http://www.djangoproject.com/ or http://turbogears.org/ for web development. Checkout http://www.riverbankcomputing.co.uk/pyqt/index.php for writing GUI programs. There are other options for GUI apps. That is the one I use all the time. Good luck, Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: why should I learn python
On Thursday 06 September 2007 15:29, windandwaves wrote: > On Sep 7, 9:50 am, James Stroud <[EMAIL PROTECTED]> wrote: > > Seewww.python.org. Trust us all when we say that its the best. > > I get that feeling - yes. Question is: > > 1. what is it good for? > 2. why is it so good? > > I would love to hear some opinions. 1. It is good for just about everything except kernel modules. 2. Ease of use. -- http://mail.python.org/mailman/listinfo/python-list
Re: why should I learn python
On Thursday 06 September 2007 15:44, Torsten Bronger wrote: > Hallöchen! > > Tom Brown writes: > > [...] Python has been by far the easiest to develop in. Some > > people might say it is not "real programming" because it is so > > easy. > > I can't believe this. Have you really heard such a statement? Yes. I was told this by a C programmer. Something about doing it all yourself and not using provided packages. I countered with something about reinventing the wheel. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Database Apps
On Monday 10 September 2007 19:52, [EMAIL PROTECTED] wrote: > Kindof a poll, kindof curiosity... > > What is your favorite python - database combination? I'm looking to > make an app that has a local DB and a server side DB. I'm looking at > python and sqlite local side and sql server side. > > Any suggestions I have had a lot of good luck with PostgreSQL. It is easy to install and use. It is also very stable. It maybe overkill for a client side database. The psycopg package makes interfacing to PostgreSQL very easy and there is a package for Linux and Windows to make cross-platform development a breeze. -Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: How to Start
On Thursday 13 September 2007 14:59, Michael R. Copeland wrote:
>I've decided that Python is a language/environment I'd like to learn
> (I've been a professional programmer for 45+ years), but I really don't
> know where and how to start! I have a number of books - and am buying
> some more - but because of the bewildering number of after-market
> packages, environments, and add-ons, I am really quite perplexed about
> starting. 8<{{
>Yes, I could fire up the interactive mode and play with some
> statements...but I consider that sort of thing for programming neophytes
> or experimenting with specific issues. First, I want to develop a
> simple Windows application, and because of the plethora of "stuff" the
> Python world offers, I don't know where to begin.
>For example, what basic, easy-to-use interface might I start with to
> build a simple text file parsing and analysis program? That is, I'd
> like to start with a simple Windows shell that prompts for a file name,
> processes it, and then displays some result.
>I am certainly impressed with the apparent experience and openness of
> the regular players here, but the discussions here (and in
> c.l.p.announce) truly presume knowledge and experience with Python I
> don't yet have. Yes, for even a very experienced programmer, entering
> the Python world is very daunting - but I want to get started.
>Please advise. TIA
A good book, a link to the module index, a good text editor and a command
prompt will get you started. Sounds like you have some books. Those should
help a lot. A good text editor is a matter of taste. I prefer gvim. However,
gvim has a steep learning curve to become productive. There is eric4, which
is an IDE. I has a debugger. That may be the place to start.
Good luck,
Tom
--
http://mail.python.org/mailman/listinfo/python-list
Re: google earth / pythoncom
On Thu, 2007-11-15 at 18:09 +, John Walsh wrote: > Hi, > > I'd like to write a python script to control Google Earth, > and I've read that Google Earth provides a COM api, and that > Python has a COM module 'pythoncom'. > I think what you are looking for you can download from here: http://sourceforge.net/project/showfiles.php?group_id=78018 The main page is here: http://starship.python.net/crew/mhammond/win32/Downloads.html > The Python script I want to run to control Google Earth will be > on the PC (WinXP) - so maybe I should install the 'normal/full' > Python on the PC too - if that can be done ? Have 2 Pythons installed > on one PC ? You can have two python installations on one pc. You have to use the full path to the interpreter to get the one you want. Or, the first one it finds in your path will be used. HTH, Tom -- http://mail.python.org/mailman/listinfo/python-list
how do I make a class global?
Hi,
I thought it would be nifty to create a class that created other classes for
me. The method below shows what I would like to do. The problem is that the
class the method creates is local to the method. Is it possible to make the
class visible in the global scope so I can import the module see the
dynamically created classes? Or do I need to generate a source file and do a
'from tmp import *'?
def new(self, eventType, param):
self.value += 1
exec 'global %s; %s = %d' % (eventType, eventType, self.value)
sl = []
sl.append('class %sEvent(QEvent):' % eventType)
sl.append(' def __init__(self, %s):' % param)
sl.append('QEvent.__init__(self, %s)' % evenType)
sl.append('self.%s = %s' % (param, param))
source = '\n'.join(sl)
co = compile(source, 'tmp.py', 'exec')
exec co
Then, to create another event, I would just have to add another line like
this:
e.new('ETestEvent', 'test')
Thanks,
Tom
--
http://mail.python.org/mailman/listinfo/python-list
Re: dealing with binary files
On Mon, 2008-01-07 at 11:57 -0200, Guilherme Polo wrote:
> 2008/1/7, Gerardo Herzig <[EMAIL PROTECTED]>:
> > Hi all. Im trying to read a binary data from an postgres WAL archive.
> > If i make a
> > xfile = open('filename', 'rb').xreadlines()
> > line = xfile.next()
> >
> > i see this sort of thing:
> > ']\xd0\x03\x00\x01\x00\x00\x00\r\x00\x00\x00\x00\x00\x00JM//DI+,D\x00\x00\x00\x01$\x00\x00\x00\x7f\x06\x00\x00y\r\t\x00\x02\x0f\t\x00\x00\x00\x10\x00)\x00\x01\x00\x12\x08
> > \x00^\xc2\x0c\x00\x08\x00\x00\x003001([EMAIL PROTECTED]'
> >
> > This file suppose to have some information about database activity, but
> > at this point i cant do more than this, because i cant figure out what
> > to do in order to have some 'readable' text.
> >
> > Im guessing is some C code, im reading the struct module to see if it
> > helps, but im not into C programming, and im lost at the start of my
> > problem.
>
> You are looking at the correct module, struct. But in order to
> understand or even correctly extract data from a binary file, you need
> to know its structure. There should be some document describing the
> Postgre WAL file format.
>
"The log record headers are described in access/xlog.h"
http://www.postgresql.org/docs/8.2/interactive/wal-internals.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: building psycopg2 on windows using mingw, "cannot find -lpq"
On Mon, 2008-01-21 at 01:57 -0800, GHUM wrote: > What am I missing? any hints? I use psycopg2 all the time on windows. I use the binary installer instead of source. Works great for me. -Tom -- http://mail.python.org/mailman/listinfo/python-list
