On Mon, Nov 6, 2017 at 8:04 AM, Cédric Jeanneret < cedric.jeanne...@camptocamp.com> wrote:
> On 11/04/2017 07:08 AM, Doug Hellmann wrote: > > Excerpts from Juan Antonio Osorio's message of 2017-11-04 00:11:47 +0200: > >>> On 3 Nov 2017 19:59, "Doug Hellmann" <d...@doughellmann.com> wrote: > >>> > >>> Excerpts from Cédric Jeanneret's message of 2017-11-01 14:54:34 +0100: > >>>> Dear Stackers, > >>>> > >>>> While working on my locally deployed Openstack (Pike using TripleO), I > >>>> was a bit struggling with the logging part. Currently, all logs are > >>>> pushed to per-service files, in the standard format "one line per > >>>> entry", plain flat text. > >>>> > >>>> It's nice, but if one is wanting to push and index those logs in an > ELK, > >>>> the current, default format isn't really good. > >>>> > >>>> After some discussions about oslo.log, it appears it provides a nice > >>>> JSONFormatter handler¹ one might want to use for each (python) service > >>>> using oslo central library. > >>>> > >>>> A JSON format is really cool, as it's easy to parse for machines, and > it > >>>> can be on a multi-line without any bit issue - this is especially > >>>> important for stack traces, as their output is multi-line without real > >>>> way to have a common delimiter so that we can re-format it and feed it > >>>> to any log parser (logstash, fluentd, …). > >>>> > >>>> After some more talks, olso.log will not provide a unified interface > in > >>>> order to output all received logs as JSON - this makes sens, as that > >>>> would mean "rewrite almost all the python logging management > >>>> interface"², and that's pretty useless, since (all?) services have > their > >>>> own "logging.conf" file. > >>>> > >>>> That said… to the main purpose of that mail: > >>>> > >>>> - Default format for logs > >>>> A first question would be "are we all OK with the default output > format" > >>>> - I'm pretty sure "humans" are happy with that, as it's really > >>>> convenient to read and grep. But on a "standard" Openstack deploy, I'm > >>>> pretty sure one does not have only one controller, one ceph node and > one > >>>> compute. Hence comes the log centralization, and with that, the log > >>>> indexation and treatments. > >>>> > >>>> For that, one might argue "I'm using plain files on my logger, and > >>>> grep-ing -r in them". That's a way to do things, and for that, plain, > >>>> flat logs are great. > >>>> > >>>> But… I'm pretty sure I'm not the only one wanting to use some kind of > >>>> ELK cluster for that kind of purpose. So the right question is: what > >>>> about switching the default log format to JSON? On my part, I don't > see > >>>> "cons", only "pros", but my judgment is of course biased, as I'm > "alone > >>>> in my corner". But what about you, Community? > >>>> > >>>> - Provide a way to configure the output format/handler > >>>> While poking around in the puppet modules code, I didn't find any way > to > >>>> set the output handler for the logs. For example, in puppet-nova³ we > can > >>>> set a lot of things, but not the useful handler for the output. > >>>> > >>>> It would be really cool to get, for each puppet module, the capability > >>>> to set the handler so that one can just push some stuff in hiera, and > >>>> Voilà, we have JSON logs. > >>>> > >>>> Doing so would allow people to chose between the default (current) > >>>> output, and something more "computable". > >>> > >>> Using the JSON formatter currently requires setting a logging > >>> configuration file using the standard library configuration format > >>> and fully specifying things like log levels, handlers, and output > >>> destination. Would it make sense to add an option in oslo.log to > >>> give deployers an easier way to enable the JSON formatter? > >>> > >> > >> This would actually be very useful. > > > > Great! Let me know if you would like to help figuring out where to do > > that in the library code. > > There's already some (not really documented) feature allowing oslo.log > to load a configuration file. In fact, there's already one existing in > the keystone tree (/etc/keystone/logging.conf - altougn not loaded) - we > might base something "casual" and "generic" on that one. > > I think all wsgi/python services should configure the logging through > that file so that it's clearer and cleaner. But that part is maybe a bit > too big and "aggressive" :). And the logging configuration isn't that > friendly, to be honest, at least I have some issues with its doc ;). > > But I think we might come to something nice and cool. It would allow > anyone to push their own log "formatter", in the end. > So you (oslo.log) wouldn't need to work with format output requests > anymore, just provide two basics (plain and json), and let users play > with the logging configuration (and even output things in XML if they > want) ;). > > Regarding oslo.log: apparently, adding the following entry in the > service configuration file should do it: > log-config-append¹ > > Can anyone confirm that? > It seems to be the case at least from the docs in the option [1]. But if we use this file (in TripleO and puppet) we really need to make it backwards compatible. Would folks be OK with taking it into use? I guess what it would take would be to document better the usage of this advanced configuration. Taking it into use doesn't look that bad; If you look at the file where the options are being set (_options.py), you will see that there are several options that get ignored once we start using this file [2]. To see all the options that get ignored you can look at the instances of _IGNORE_MESSAGE. So, if we would start taking that into use, we would need to change the parameters of the oslo::log resource in puppet [3] to also configure an equivalent option to that advanced logging file. [1] https://github.com/openstack/oslo.log/blob/master/oslo_log/_options.py#L44 [2] https://github.com/openstack/oslo.log/blob/master/oslo_log/_options.py#L32 [3] https://github.com/openstack/puppet-oslo/blob/master/manifests/log.pp > > > ¹ https://github.com/openstack/oslo.log/blob/master/oslo_log/ > _options.py#L44 > > > > > Doug > > > > ____________________________________________________________ > ______________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: > unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Juan Antonio Osorio R. e-mail: jaosor...@gmail.com
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
