suggestion on a complicated inter-process communication
Hello friends, I have a little messy situation on IPC. Please if you can, give me some suggestion on how to implement. Thanks a lot! -> denotes create MainProcess -> Process1 -> Process3 (from os.system) | -> Process2 (from os.system) -> Process4 (from os.system) ->Process5 I would like to make the communication between Process1 and Process5. Process1 needs Process5's output to provide argument to generate Process3, and in turn Process5 needs to wait Process3 finished. Thank you very much if you can give a hint. -- http://mail.python.org/mailman/listinfo/python-list
Re: suggestion on a complicated inter-process communication
Thanks a lot for reply. I understand it is abnormal to implement such IPC, while it is worthy for my application. Well, my process3 and 4 are from an outside application, which both need License Check and os.system to involk. >From my experience, if Process5 involks Process3, such License Check can be very very slow (even slower than its real run). That is why I need to separate Process3 and 4 onto different processes to proceed. On Apr 28, 1:57 am, Jonathan Gardner wrote: > On Apr 27, 8:59 pm, Way wrote: > > > > > Hello friends, > > > I have a little messy situation on IPC. Please if you can, give me > > some suggestion on how to implement. Thanks a lot! > > > -> denotes create > > > MainProcess -> Process1 -> Process3 (from os.system) > > | > > -> Process2 (from os.system) -> Process4 (from > > os.system) ->Process5 > > > I would like to make the communication between Process1 and Process5. > > Process1 needs Process5's output to provide argument to generate > > Process3, and in turn Process5 needs to wait Process3 finished. > > > Thank you very much if you can give a hint. > > Abstraction should resolve this. What I mean is that Process2 > shouldn't be defined in terms of what it actually does, but what it > appears to do. If you look at each process and think only what its > child processes do and what its parent process expects it to do, then > your job should get a lot simpler. Process1 expects Process2 to > deliver a set of parameters to spawn Process3, and then it will wait > until Process3 terminates. > > Looking at it this way, questions come to mind: Why can't Process2 run > Process3 itself? It's unusual to have one process tell another process > what to do when it can simply do it itself. > > By the way, I've never seen a time when this kind of design is > necessary. There are other ways around your problem than spawning a > bunch of processes. Each process should really be independent of all > the other processes, doing only a very well-specified task in a well- > specified way. The only time I could think of doing something like > this is when you're getting a webserver setup and Process5 needs to > communicate with Process3 to render the page or something like that. > But even in that case, each process is really independently defined > and implemented. -- http://mail.python.org/mailman/listinfo/python-list
Re: suggestion on a complicated inter-process communication
Thanks a lot for the reply. I am not familiar with multi-process in
Python. I am now using something like:
A_prog is an os.system to involk Process3
B_prog is an os.system to involk Process4
---
In Main Process:
Process1 = Popen(["A_prog"], stdin=PIPE, stdout=PIPE)
Process2 = Popen(["B_prog"], stdin=PIPE, stdout=PIPE)
cmd = Process2.stdout.readlines()
Process1.stdin.write(cmd)
if Process1.poll():
Process2.stdin.write("trig from Process1")
-
In Process5 (another python program):
import sys
sys.stdout.write("process1 argument")
while 1:
if sys.stdin.readline().strip() == "trig from Process1":
break
--
However, in this case, Process5's stdout cannot be passed to
MainProcess for Process1, since it has not finished (waiting Process.
1/3 finish).
I am now using Fifos (an external file) to inter-communicate Process1
and 5. But at first run, it compliants "not file found".
Is there any possible solution to make it work? Thank you very much!
On Apr 28, 1:34 am, Aaron Brady wrote:
> On Apr 27, 10:59 pm, Way wrote:
>
>
>
> > Hello friends,
>
> > I have a little messy situation on IPC. Please if you can, give me
> > some suggestion on how to implement. Thanks a lot!
>
> > -> denotes create
>
> > MainProcess -> Process1 -> Process3 (from os.system)
> > |
> > -> Process2 (from os.system) -> Process4 (from
> > os.system) ->Process5
>
> > I would like to make the communication between Process1 and Process5.
> > Process1 needs Process5's output to provide argument to generate
> > Process3, and in turn Process5 needs to wait Process3 finished.
>
> > Thank you very much if you can give a hint.
>
> The 'mmap' module can help with getting the data from 5 to 1. It
> requires creating a file. If it's a small amount of data, any old
> file will do. You may need a socket in order to wait for Process3 to
> join, or write a small '.pyd' or '.so' file that gives you access to
> your system's synchronization object.
--
http://mail.python.org/mailman/listinfo/python-list
