Best method for inter process communications
I am looking for a way of performing inter process communication over XML between a python program and something else creating XML data. What is the best way of performing this communication? I could bind a socket to localhost and perform the data transfer that way, but that seems inefficient due to the addition of TCP/IP or UDP/IP overhead. Is there a way to stream data via a custom datastream (I.E. not STDIO, STDERR, etc)? Thanks in advance, Jim Howard -- http://mail.python.org/mailman/listinfo/python-list
Re: Best method for inter process communications
Thanks for the updates. I think I will try named processes first, but may just end up using local sockets in the end and not worry about the overhead. Jim Howard -- http://mail.python.org/mailman/listinfo/python-list
Good String Tokenizer
I have searched the board and noticed that there isn't really any sort of good implementation of a string tokenizer that will tokenize based on a custom set of tokens and return both the tokens and the parts between the tokens. For example, if I have the string: "Hello, World! How are you?" And my splitting points are comma, and exclamation point then I would expect to get back. ["Hello", ",", " World", "!", " How are you?"] Does anyone know of a tokenizer that will allow for this sort of use? Thanks in advance, Jim Howard -- http://mail.python.org/mailman/listinfo/python-list
Asyncore select statement problem
I have a problem with python's asyncore module throwing a bad file descriptor error. The code might be difficult to copy here, but the problem is essentially: The server wants to sever the connection of an open Asyncore socket. Calling the socket.close() nor the socket.shutdown(2) calls seem to work. The only way I can close the connection without creating the error below is to have the client close the connection. I have the asyncore.loop() as the last line of a thread that is spawned within the applications "mainframe.py" or gui thread. It doesn't seem to me like this would make a difference, but I am unfamiliar with the specifics of how the asyncore module works. Any thoughts people have would be greatly appreciated. If needed I may be able to create a small version of the problem to post for people to see. Thanks, Jim Howard Exception in thread Thread-1: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py", line 460, in __bootstrap self.run() File "/Users/jwhoward2/Documents/Projects/LJServer/LJDeviceServer/DeviceServer.py", line 23, in run asyncore.loop() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py", line 191, in loop poll_fun(timeout, map) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py", line 121, in poll r, w, e = select.select(r, w, e, timeout) error: (9, 'Bad file descriptor') -- http://mail.python.org/mailman/listinfo/python-list
Re: Asyncore select statement problem
Thank you for the responses. I have learned considerably more about how Asyncore works because of it. The problem that I see is that Asyncore's poll function does not seem to be thread safe. From what I can tell, I am calling dispatcher.close() properly and the dispatchers are removed from asyncore's global map (all except the server itself). However, it seems like the error happens when the poll function gets the file descriptors to run select on and then the thread ticks and removes them from the global map. After this the select call is made, but the file descriptors are not valid anymore. I guess I have two questions as a result. First, is this a problem that anyone else has had and second is there a fix for it? I have tried looking for Asyncore thread safe topics in Google, but without much luck. If needed I think making the poll function atomic in the asyncore module might fix this problem, but I wanted to see what other people thought first. Thanks again for the help, Jim Howard Gabriel Genellina wrote: > "JamesHoward" <[EMAIL PROTECTED]> escribió en el mensaje > news:[EMAIL PROTECTED] > > >I have a problem with python's asyncore module throwing a bad file > > descriptor error. The code might be difficult to copy here, but the > > problem is essentially: > > > > The server wants to sever the connection of an open Asyncore socket. > > Calling the socket.close() nor the socket.shutdown(2) calls seem to > > work. The only way I can close the connection without creating the > > error below is to have the client close the connection. > > You have to use the dispatcher's close() method, else, the asyncore map > won't be updated, keeping a reference to the closed socket. > > -- > Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Asyncore select statement problem
Again, thank you for your help. With digging through the Asyncore.py source, I was able to find the poll2 function which is called when the function asyncore.loop(use_poll = True) is enabled. This function does not use a select call, but a poll call to do its looping. It works well for the problem of threads closing devices at unknown times. The reason for this is that the select statement is called on a series of file descriptors that should not be changed. If the file descriptors become invalid, select throws and exception and the asyncore loop haults. The use_poll flag sets the asyncore module to use a poll instead of a select statement. Within the poll method, there are better ways of dealing with file descriptors and they seem to be able to discern if a file descriptor becomes disconnected with the POLLHUP flag. I am still unsure as to why the select function is used instead of the poll function, but using poll appears to have solved my problem. Thanks for all the help, Jim Howard -- http://mail.python.org/mailman/listinfo/python-list
Thread Profiling
Are there any good thread profilers available that can profile a thread as it is running instead of after execution is completed? I would like to find a python class which looks at a currently running thread and if its memory exceeds a certain amount than kill it. Ideally I would like the program to track memory used not just by that thread, but by any threads or processes that it may spawn. If there isn't anything like that, then something that lets me set the maximum memory allowed to be allocated within a thread would be acceptable also. Thanks in advance, James Howard -- http://mail.python.org/mailman/listinfo/python-list
Re: IDLE won't color code!
On Nov 5, 12:33 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Please help! > IDLE color codes only the shell, not the open files! How can I solve > this? My first guess is that idle is opening files without the .py extension. If that isn't it, what operating system are you using? What version of python and idle are you using? James Howard -- http://mail.python.org/mailman/listinfo/python-list
Transfer socket connection between programs
Does anyone know any method to have one program, acting as a server transfer a socket connection to another program? I looked into transferring the connection via xml rpc to no avail. It seems to be a problem of getting access to a programs private memory space and giving another program access to that space. Thanks in advance, James Howard -- http://mail.python.org/mailman/listinfo/python-list
Re: Transfer socket connection between programs
On Nov 12, 12:50 pm, Grant Edwards <[EMAIL PROTECTED]> wrote: > On 2007-11-12, JamesHoward <[EMAIL PROTECTED]> wrote: > > > Does anyone know any method to have one program, acting as a > > server transfer a socket connection to another program? > > The only way I know of is to use fork. When you fork a > process, all open file-descriptors (including network > connections) are inherited by the child. > > > I looked into transferring the connection via xml rpc to no > > avail. > > I've no idea how that could work (even in theory) on any OS > with which I'm familiar. > > > It seems to be a problem of getting access to a programs > > private memory space and giving another program access to that > > space. > > Private memory has nothing to do with it. The connection is a > data structure that lives in kernel space, not in user space. > Even if you could grant another process access to your "private > memory space", it wouldn't help you "transfer a socket > connection", since that connection is something the OSes > manages. > > -- > Grant Edwards grante Yow! ! Everybody out of > at the GENETIC POOL! >visi.com Thanks Grant, Does this mean that there is some way to transfer a pointer to that kernel memory space from one program to another and have it be valid, or is that kernel memory space protected and unusable from other processes? -- http://mail.python.org/mailman/listinfo/python-list
