Getting Syslog working on OSX Monterey

[Philip Bloom via Python-list]

Hello,

First time mailing and looking for help/guidance.  Hopefully not too in the
wrong place.

We've had an app with logging to a /var/log for many years through
logging.sysloghandler.  Recently, though, I noticed that it suddenly was
getting no logs whatsoever over there and investigated, believing the
recent triggering change was upgrading to Mac OSX 12 (Monterey).

My understanding is a bit primitive, but our config had never been hard to
follow previously and I may be missing something fundamental.  I've looked
over a number of online examples but they don't seem to be running
correctly either.  I'm on Python 3.6.8 (though we're about to start an
upgrade to the latest stable, if that in theory may have changes to syslog
handling).  I'm mostly ending up here since I'm finding such differences in
response between python modules I'd expect to be fairly similar.

Separating into a test script I've been testing with logging.SysLogHandler,
syslog, and the command line 'syslog -s'.  Replaced all below with a
placeholder appName but it's normally a fairly unique setupCDNGUI name.

Example with syslog (python module):  - This will show up in System.log but
not in Apple's Console for all messages to device, though won't get all the
way to file.
import syslog
syslog.syslog(syslog.LOG_NOTICE, 'AppName: THIS IS A TEST - Processing
started')
syslog.syslog(syslog.LOG_ERR, 'AppName: THIS IS A TEST ERROR - Processing
started')

Example with SyslogHandler: - This doesn't seem to show up anywhere besides
the screen log, which is odd to me as reading the docs, I understood this
to be a logging integration into the syslog module, so expected similar
behavior in at least what was sent out.  Syslog handler is definitely
picked up in logging.handlers

root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG)
basic_datefmt = '%m/%d/%Y %I:%M:%S %p'

formatter = logging.Formatter(fmt='%(asctime)s %(levelname)s: %(message)s',
datefmt=basic_datefmt)
syslog_format = logging.Formatter(fmt='SetupCDNGUI: %(message)s',
datefmt=basic_datefmt)

scrhandler = logging.StreamHandler()
scrhandler.setFormatter(formatter)
root_logger.addHandler(scrhandler)

sys_handler = SysLogHandler(address='/var/run/syslog')
#sys_handler.encodePriority(SysLogHandler.LOG_USER, SysLogHandler.LOG_ALERT)
# Tried with the above, but didn't make a difference.  Neither did not
defining the address and letting it go to local host.
sys_handler.setFormatter(syslog_format)
root_logger.addHandler(sys_handler)

logging.critical("This is a test")
logging.error("This is a test error")
logging.info("Still a test")
logging.debug("Testy test")

Using command line: Oddly, this gets to Console.Apps messages from device
reliably, though doesn't get picked up by syslog -w or get received by the
ASL config redirect:
syslog -s -l error -k Message "appName: this is a test2"
syslog -s -l notice -k Message "appName: this is a test3"

ASL configuration, which is loaded according to syslog -config:
> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
file_max=50M all_max=500M
? [CA= Sender appName] file /var/log/appName/appName.log

My end goal is really to get just a working python logging ->
var/log/appname/appname.log again so glad to just be pointed in the right
direction if way off base.

-- 
Philip Bloom
Director, Services Engineering
*AppLovin Corporation*
M: (786)-338-1439 <786-338-1439>
[image: LinkedIn]  [image:
Twitter]  [image: Facebook]
 [image: Instagram]

[image: AppLovin] 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Getting Syslog working on OSX Monterey

[Philip Bloom via Python-list]

Thanks.  Had tried it with no address, which defaults to ('localhost',
'514') as well as address='/var/run/syslog' which had been working
previously, and the doc recommends as:
For example, on Linux it’s usually ‘/dev/log’ but on OS/X it’s
‘/var/run/syslog’. You’ll need to check your platform and use the
appropriate address (you may need to do this check at runtime if your
application needs to run on several platforms)

Aiming for this to be being picked up on the same computer I'm running the
test script (Python -> Logging -> SyslogHandler -> Mac OS Syslogd -> parsed
ASL config -> /var/log/Appname/Appname.log).   It's why I think I may be
missing something fundamental, but it feels like something subtle changed
in the latest OSX.

On Sun, Feb 27, 2022 at 11:26 AM Dennis Lee Bieber 
wrote:

> On Sun, 27 Feb 2022 08:34:21 -0800, Philip Bloom
>  declaimed the following:
>
> >
> >sys_handler = SysLogHandler(address='/var/run/syslog')
>
> As I read the documentation, the address is supposed to be the
> NETWORK
> address of the machine to receive the log messages...
>
> https://docs.python.org/3/library/logging.handlers.html#sysloghandler
>
> >#sys_handler.encodePriority(SysLogHandler.LOG_USER,
> SysLogHandler.LOG_ALERT)
> ># Tried with the above, but didn't make a difference.  Neither did not
> >defining the address and letting it go to local host.
> >sys_handler.setFormatter(syslog_format)
> >root_logger.addHandler(sys_handler)
> >
> >logging.critical("This is a test")
> >logging.error("This is a test error")
> >logging.info("Still a test")
> >logging.debug("Testy test")
> >
> >Using command line: Oddly, this gets to Console.Apps messages from device
> >reliably, though doesn't get picked up by syslog -w or get received by the
> >ASL config redirect:
> >syslog -s -l error -k Message "appName: this is a test2"
> >syslog -s -l notice -k Message "appName: this is a test3"
> >
> >ASL configuration, which is loaded according to syslog -config:
> >> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
> >file_max=50M all_max=500M
> >? [CA= Sender appName] file /var/log/appName/appName.log
> >
> >My end goal is really to get just a working python logging ->
> >var/log/appname/appname.log again so glad to just be pointed in the right
> >direction if way off base.
>
>
> --
> Wulfraed Dennis Lee Bieber AF6VN
> wlfr...@ix.netcom.com
> http://wlfraed.microdiversity.freeddns.org/
> --
> https://mail.python.org/mailman/listinfo/python-list
>


-- 
Philip Bloom
Director, Services Engineering
*AppLovin Corporation*
M: (786)-338-1439 <786-338-1439>
[image: LinkedIn]  [image:
Twitter]  [image: Facebook]
 [image: Instagram]

[image: AppLovin] 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Getting Syslog working on OSX Monterey

[Philip Bloom via Python-list]

I'm probably asking on the wrong list, and probably should bother wherever
apple's ASL experts live for changes in monterey.   Guess nobody else is
seeing this?

The same exact code is working just fine on OSX Big Sur, but on OSX
Monterey it doesn't work at all.  Users that haven't updated are having the
program produce logs as it has for years through
logging.handlers.SysLogHandler.  Mac OSX definitely has a listening socket
at '/var/run/syslog' which shows up in Console.app.

Apologies, Barry.  I'm not quite understanding your responses.

When we say OSX has no listener for Syslog, what is the Apple System Log
and the general output to Console.app then?  I thought that was the local
syslog on OSX and had, for years, been behaving as such when using
logging.handlers.SysLogHandler with a config in /etc/asl/ to define how it
should be routed and the rollover/cleanup frequency.

Thanks for replying, just having trouble understanding.


On Mon, Feb 28, 2022 at 2:07 PM Barry Scott  wrote:

>
>
> > On 28 Feb 2022, at 21:41, Peter J. Holzer  wrote:
> >
> > On 2022-02-27 22:16:54 +, Barry wrote:
> >> If you look at the code of the logging modules syslog handle you will
> see that
> >> it does not use syslog. It’s assuming that it can network to a syslog
> listener.
> >> Such a listener is not running on my systems as far as I know.
> >>
> >> I have always assumed that if I want a logger syslog handler that I
> would have
> >> to implement it myself. So far I have code that uses syslog directly
> and have
> >> not written that code yet.
> >
> > What do you mean by using syslog directly? The syslog(3) library
> > function also just sends messages to a "syslog listener" (more commonly
> > called a syslog daemon) - at least on any unix-like system I'm familiar
> > with (which doesn't include MacOS). It will, however, always use the
> > *local* syslog daemon - AFAIK there is no standard way to open a remote
> > connection (many syslog daemons can be configured to forward messages to
> > a remote server, however).
>
> I'm re-reading the code to check on what I'm seeing. (Its been a long
> time since I last look deeply at this code).
>
> You can write to /dev/log if you pass that to
> SysLogHandler(address='/dev/log'), but the default is to use a socket
> to talk to a network listener on localhost:514. There are no deamons
> listening on port 514 on my Fedora systems or mac OS.
>
> That is not what you would expect as the default if you are using the C
> API.
>
> What you do not see used in the SyslogHandler() is the import syslog
> and hence its nor using openlog() etc from syslog API.
>
> Barry
>
>
>
> >hp
> >
> > --
> >   _  | Peter J. Holzer| Story must make more sense than reality.
> > |_|_) ||
> > | |   | h...@hjp.at |-- Charles Stross, "Creative writing
> > __/   | http://www.hjp.at/ |   challenge!"
> > --
> > https://mail.python.org/mailman/listinfo/python-list
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>


-- 
Philip Bloom
Director, Services Engineering
*AppLovin Corporation*
M: (786)-338-1439 <786-338-1439>
[image: LinkedIn]  [image:
Twitter]  [image: Facebook]
 [image: Instagram]

[image: AppLovin] 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Getting Syslog working on OSX Monterey

[Philip Bloom via Python-list]

Grabbing latest python that does work.  Good we're about to get out of the
stone ages a bit here.

So findings:
Syslog - works in 3.10, broken against monterey in 3.6.
Logging.Handlers.Sysloghandler - is broken in both against Monterey.

Will bug it for the tracker.  Thanks for the feedback.




On Thu, Mar 3, 2022 at 10:32 AM Barry Scott  wrote:

>
>
> On 3 Mar 2022, at 03:01, Philip Bloom  wrote:
>
> I'm probably asking on the wrong list, and probably should bother wherever
> apple's ASL experts live for changes in monterey.   Guess nobody else is
> seeing this?
>
> The same exact code is working just fine on OSX Big Sur, but on OSX
> Monterey it doesn't work at all.  Users that haven't updated are having the
> program produce logs as it has for years through
> logging.handlers.SysLogHandler.  Mac OSX definitely has a listening socket
> at '/var/run/syslog' which shows up in Console.app.
>
> Apologies, Barry.  I'm not quite understanding your responses.
>
>
> This works:
>
> syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_USER)
> syslog.syslog(syslog.LOG_NOTICE, 'QQQ test log')
>
> I see the "QQQ test log" when I run syslog command.
>
> And I can reproduce that logging.handlers.SysLogHandler does not work.
> I added debug to the SysLogHandler() code and see that it does indeed
> write into the /var/run/syslog socket.
> I suspect that macOS /var/run/syslog does not implement the RFC that this
> code depends on, but that a wild guess.
>
> I suggest that you write your own handler that uses syslog.syslog() and
> use that.
>
> Barry
>
>
> When we say OSX has no listener for Syslog, what is the Apple System Log
> and the general output to Console.app then?  I thought that was the local
> syslog on OSX and had, for years, been behaving as such when using
> logging.handlers.SysLogHandler with a config in /etc/asl/ to define how it
> should be routed and the rollover/cleanup frequency.
>
> Thanks for replying, just having trouble understanding.
>
>
> On Mon, Feb 28, 2022 at 2:07 PM Barry Scott 
> wrote:
>
>>
>>
>> > On 28 Feb 2022, at 21:41, Peter J. Holzer  wrote:
>> >
>> > On 2022-02-27 22:16:54 +, Barry wrote:
>> >> If you look at the code of the logging modules syslog handle you will
>> see that
>> >> it does not use syslog. It’s assuming that it can network to a syslog
>> listener.
>> >> Such a listener is not running on my systems as far as I know.
>> >>
>> >> I have always assumed that if I want a logger syslog handler that I
>> would have
>> >> to implement it myself. So far I have code that uses syslog directly
>> and have
>> >> not written that code yet.
>> >
>> > What do you mean by using syslog directly? The syslog(3) library
>> > function also just sends messages to a "syslog listener" (more commonly
>> > called a syslog daemon) - at least on any unix-like system I'm familiar
>> > with (which doesn't include MacOS). It will, however, always use the
>> > *local* syslog daemon - AFAIK there is no standard way to open a remote
>> > connection (many syslog daemons can be configured to forward messages to
>> > a remote server, however).
>>
>> I'm re-reading the code to check on what I'm seeing. (Its been a long
>> time since I last look deeply at this code).
>>
>> You can write to /dev/log if you pass that to
>> SysLogHandler(address='/dev/log'), but the default is to use a socket
>> to talk to a network listener on localhost:514. There are no deamons
>> listening on port 514 on my Fedora systems or mac OS.
>>
>> That is not what you would expect as the default if you are using the C
>> API.
>>
>> What you do not see used in the SyslogHandler() is the import syslog
>> and hence its nor using openlog() etc from syslog API.
>>
>> Barry
>>
>>
>>
>> >hp
>> >
>> > --
>> >   _  | Peter J. Holzer| Story must make more sense than reality.
>> > |_|_) ||
>> > | |   | h...@hjp.at |-- Charles Stross, "Creative writing
>> > __/   | http://www.hjp.at/ |   challenge!"
>> > --
>> > https://mail.python.org/mailman/listinfo/python-list
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
>
> --
> Philip Bloom
> Director, Services Engineering
> *AppLovin Corporation*
> M: (786)-338-1439 <786-338-1439>
> [image: LinkedIn]  [image:
> Twitter]  [image: Facebook]
>  [image: Instagram]
> 
> [image: AppLovin] 
>
>
>
>
>
>

-- 
Philip Bloom
Director, Services Engineering
*AppLovin Corporation*
M: (786)-338-1439 <786-338-1439>
[image: LinkedIn]  [image:
Twitter]  [image: Facebook]
 [image: Instagram]

[image: AppLovin] 
-- 
https://mail.python.org/mailman/listinfo/python-list