The BrokenPipeError message could be related to how you write to stderr from your Python program. If you are writing without flushing immediately (as I recall from a previous question from you), then maybe Python tries to write the stderr buffer when the process exits (if there is a high load, rsyslog will from time to time start and stop instances of the process), but at this moment omprog has already closed the pipe, hence the BrokenPipeError... Take a look at this, maybe it helps: https://stackoverflow.com/questions/26692284/how-to-prevent-brokenpipeerror-when-doing-a-flush-in-python
You could also try with a simpler program, to try to reduce the scope of the problem (if the simpler program works, then the problem is probably in your Python code). In the rsyslog test suites there are some simple bash scripts that are being used for testing omprog in CI (with and without the confirmMessages mode, with and without the output flag, etc.). If you don't find them, tell me. If none of these hints help, please send your program, maybe we can spot the problem (I'm not fluent in Python, but I can try...). Regards On Fri, Feb 7, 2020, 09:03 Eudald Valcàrcel via rsyslog < [email protected]> wrote: > Hello, > I've been having issues trying to print the confirmation message for > rsyslog acknowledgement. > on line: > onMessage(line) > status = "OK" > print(status, flush=True) > > My program is printing BrokenPipeError: [Errno 32] Broken pipe for a lot of > the messages it receives, which leads me to think that the message in > question has not been processed. In fact, there are plenty of messages not > fully processed. > > I'm using the following configuration in rsyslog: > if ($programname == "sysd11") then { > if (re_match($msg, "status=([a-zA-Z0-9]+)")) then { > action(type="omprog" > name="parser_processed" > binary="/usr/local/scripts/py/logparser.py" > template="POSTFIX_FormatTo" > confirmMessages="on" > queue.type="LinkedList" > queue.size="25000" > queue.workerThreads="15" > queue.workerThreadMinimumMessages="40" > output="/var/log/system/processed.log" > ) > } > > Could anyone help me on that? > I'm using impstats to review if messages are discarded but there's no > message discarded, which should lead into all messages been processed, but > that's not the case, definitely. > > Appreciated, > Eudald > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
