Thanks for your comment re blocking.
I removed pipes from the Python and C programs to see if it blocks without
them, and it does. It looks now like the problem is not pipes. I use fork()
and execv() in C to run Python in a child process, but the Python process
blocks because fork() does not create a new thread, so the Python global
interpreter lock (GIL) prevents the C program from running once Python starts.
So the solution appears to be run Python in a separate thread, which I can do
with pthread create. See "Thread State and the Global Interpreter Lock"
https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock
and the sections below that "Non-Python created threads" and "Cautions about
fork()."
I'm working on that today and I hope all goes well :)
Nov 30, 2021, 11:42 by ba...@barrys-emacs.org:
>
>
>
>> On 29 Nov 2021, at 22:31, Jen Kris <>> jenk...@tutanota.com>> > wrote:
>>
>> Thanks to you and Cameron for your replies. The C side has an epoll_ctl
>> set, but no event loop to handle it yet. I'm putting that in now with a
>> pipe write in Python-- as Cameron pointed out that is the likely source of
>> blocking on C. The pipes are opened as rdwr in Python because that's
>> nonblocking by default. The child will become more complex, but not in a
>> way that affects polling. And thanks for the tip about the c-string
>> termination.
>>
>>
>
> flags is a bit mask. You say its BLOCKing by not setting os.O_NONBLOCK.
> You should not use O_RDWR when you only need O_RDONLY access or only O_WRONLY
> access.
>
> You may find
>
> man 2 open
>
> useful to understand in detail what is behind os.open().
>
> Barry
>
>
>
>
>>
>>
>> Nov 29, 2021, 14:12 by >> ba...@barrys-emacs.org>> :
>>
>>>
>>>
>>>> On 29 Nov 2021, at 20:36, Jen Kris via Python-list <>>>>
>>>> python-list@python.org>>>> > wrote:
>>>>
>>>> I have a C program that forks to create a child process and uses execv to
>>>> call a Python program. The Python program communicates with the parent
>>>> process (in C) through a FIFO pipe monitored with epoll().
>>>>
>>>> The Python child process is in a while True loop, which is intended to
>>>> keep it running while the parent process proceeds, and perform functions
>>>> for the C program only at intervals when the parent sends data to the
>>>> child -- similar to a daemon process.
>>>>
>>>> The C process writes to its end of the pipe and the child process reads
>>>> it, but then the child process continues to loop, thereby blocking the
>>>> parent.
>>>>
>>>> This is the Python code:
>>>>
>>>> #!/usr/bin/python3
>>>> import os
>>>> import select
>>>>
>>>> #Open the named pipes
>>>> pr = os.open('/tmp/Pipe_01', os.O_RDWR)
>>>>
>>> Why open rdwr if you are only going to read the pipe?
>>>
>>>> pw = os.open('/tmp/Pipe_02', os.O_RDWR)
>>>>
>>> Only need to open for write.
>>>
>>>>
>>>> ep = select.epoll(-1)
>>>> ep.register(pr, select.EPOLLIN)
>>>>
>>>
>>> Is the only thing that the child does this:
>>> 1. Read message from pr
>>> 2. Process message
>>> 3. Write result to pw.
>>> 4. Loop from 1
>>>
>>> If so as Cameron said you do not need to worry about the poll.
>>> Do you plan for the child to become more complex?
>>>
>>>>
>>>> while True:
>>>>
>>>> events = ep.poll(timeout=2.5, maxevents=-1)
>>>> #events = ep.poll(timeout=None, maxevents=-1)
>>>>
>>>> print("child is looping")
>>>>
>>>> for fileno, event in events:
>>>> print("Python fileno")
>>>> print(fileno)
>>>> print("Python event")
>>>> print(event)
>>>> v = os.read(pr,64)
>>>> print("Pipe value")
>>>> print(v)
>>>>
>>>> The child process correctly receives the signal from ep.poll and correctly
>>>> reads the data in the pipe, but then it continues looping. For example,
>>>> when I put in a timeout:
>>>>
>>>> child is looping
>>>> Python fileno
>>>> 4
>>>> Python event
>>>> 1
>>>> Pipe value
>>>> b'10\x00'
>>>>
>>> The C code does not need to write a 0 bytes at the end.
>>> I assume the 0 is from the end of a C string.
>>> UDS messages have a length.
>>> In the C just write 2 byes in the case.
>>>
>>> Barry
>>>
>>>> child is looping
>>>> child is looping
>>>>
>>>> That suggests that a while True loop is not the right thing to do in this
>>>> case. My question is, what type of process loop is best for this
>>>> situation? The multiprocessing, asyncio and subprocess libraries are very
>>>> extensive, and it would help if someone could suggest the best alternative
>>>> for what I am doing here.
>>>>
>>>> Thanks very much for any ideas.
>>>>
>>>>
>>>> --
>>>> https://mail.python.org/mailman/listinfo/python-list
>>>>
>>
>>
--
https://mail.python.org/mailman/listinfo/python-list
