Serial connections and threads in a GUI
I have a wx-based GUI application that I'd like to have read "streaming" data from a serial port and plot that data using matplotlib and a figure canvas. It's a robotic application where the workflow goes something like this: 1) Connect to robot 2) Configure robot settings 3) Command robot to do something for time, t - While robot does something, read sensor data and plot 4) End of t seconds, resume at (2) I have the application reading from the serial port in a separate thread, but my problem is this - I'd like the user to be able to write to the serial port by interacting with the GUI between runs. However, the infinite loop running inside the thread responsible for receiving data has exclusive access to the serial connection object. Because of how the application works, I know that I won't need to write to the serial port when I'm in the middle of receiving data, so that simplifies things somewhat. But, how can I allow the thread access to the connection for reading, but then have it give up control over the connection when it's done so the main GUI thread can access the connection again for writing? Currently, I have the thread's run() method exit when it's done receiving data. But, that means that I can only receive data again using the same thread by calling run(), blocking the main GUI thread from updating the plot until it completes. This is my first go at using threads in Python, so I'm sure there's a straightforward way to do this that I'm just missing. At any rate, thanks in advance for your patience and assistance. Thanks, Aaron -- http://mail.python.org/mailman/listinfo/python-list
Threaded GUI slowing method execution?
I have a wx GUI application that connects to a serial port in a separate thread, reads from the port, and then is supposed to put the data it finds into a queue to be used by the main GUI thread. Generally speaking, it's working as expected. However, one method (that's part of a library I've written to parse the packet structure of the data that's coming over the serial port) executes approximately 1000 times slower (50ms vs. 50us) when called from the serial management thread in the GUI as compared to calling the same function from within a command line Python script. I checked it by wrapping the call as follows in both cases (GUI and command line script): tic = time.time() print time.time() - tic All the thread is doing most of the time is sitting around checking the serial port for waiting data, reading it, and appending it to a list when it finds it. Then, in the same thread, the method that seems to be remarkably slow works its way through that list looking for packets of data and appending the packet payloads it finds to a queue to be handled in some way by the GUI. My question is, what am I missing about either threading or the fact that this is running in a GUI that might explain such a huge slowdown. I'm sending data over the serial at a true rate of about 24k bytes per second, or approximately 2 packets per ms. Is it too much to ask to be able to process this data in realtime from within a GUI (I'm not talking about plotting or anything - just read it and find packets)? Like I said, the process pretty much runs in realtime from a command line script. This packet parsing needs to happen continuously, so it seems calling join() to ensure it's not interrupted by the GUI thread, won't work. Thanks in advance for your help. Aaron -- http://mail.python.org/mailman/listinfo/python-list
Re: Threaded GUI slowing method execution?
On Oct 2, 2009, at 12:30 PM, Dave Angel wrote: (you responded off-list, which isn't the way these mailing lists work. So I'm pasting your message back to the list, with my response at the end) Sorry about that - a slip of the "reply" button. Actually, I was thinking of the subprocess module (introduced in 2.4). But the multiprocessing module would be useful if you were porting threading code to a process model. Previously, I had been using threads, so I just tried moving to multiprocessing because it required few changes. There are tons of ways to communicate between processes, though you can't do the simple variable sharing that threads can (sometimes) get away with. I would normally point you to queues, but there a number of possibilities. And since the one process is running a GUI event loop, you might want to piggyback on the OS capability to post events between processes. The code might end up OS-dependent, but I'd bet the overhead will be minimal. What is your target operating system? My target OS is Mac OS X Leopard. I've decided just to try and trigger WX events in the GUI based on data put in one of the queues by the processing thread. Your numbers in the original message make me nervous; sending an event between processes (or even threads) every 0.5 millisecond is impossible. But I think you might have been confusing bytes and packets. My external hardware is actually sending 2000 packets per second right now (but that can also be changed). Each packet currently contains 6 bytes of data and 6 bytes of overhead. So, 12 bytes per packet * 2000 packets per second is 24k bytes per second. However, the serial processing process should be finding the packet payload data and adding to a queue for the GUI to deal with and plot (obviously at a much lower framerate) (at an ideal rate of 2KHz). There could very well be multiprocess support in wxPython. I'd check there first, before re-inventing the wheel. Presumably you know of the wxPython news group, hosted on Google groups? I actually tried using multiprocess to create a separate process for reading the serial data. The process gets passed a TX queue and an RX queue in its constructor, then it's supposed to enter an infinite loop where it waits for commands on the TX queue and reads data and puts in in the RX queue. However, I've discovered for some reason that the infinite loop terminates after the first call I make to either of the queues. Perhaps this behavior should be expected, but this is my first go-round with processes, so I was a little surprised by this result. Maybe someone with more multiprocessing experience can give me some pointers. Maybe I am barking up the wrong tree with Python from a speed perspective, but it's just some much faster and more fun to write than anything else... Aaron -- http://mail.python.org/mailman/listinfo/python-list
Re: Threaded GUI slowing method execution?
On Oct 2, 2009, at 1:18 PM, sturlamolden wrote: On 2 Okt, 21:30, Dave Angel wrote: There could very well be multiprocess support in wxPython. I'd check there first, before re-inventing the wheel. I don't think there is. But one can easily make a thread in the subprocess that polls a pipe and calls wx.PostEvent or wx.CallLater when something is received. I particularly like the Queue object in multiprocessing for this. One could also use win32api.SendMessage from pywin32 to send the event from one process to another. The parent must know the hwnd of the subprocess main wx.Frame. The subprocess gets that from calling the GetHandle() method of its wx.Frame, and must post it back to the parent, presumably via a pipe. So, I managed to solve all of this ultimately by simply creating a new process for managing the serial connection with a TX and RX queue. Then, in the GUI, I created a thread that's responsible for reading from the RX queue and posting the appropriate events based on the data it pulls off of the queue. It's plenty fast for 2KHz sampling rates, so I'm happy with it for now. But, I'm also grateful for all the suggestions for different approaches to the problem - they've really opened to what's possible with Python and the various libraries that enable interfacing with "lower" level languages like C. -- http://mail.python.org/mailman/listinfo/python-list
