Re: [E] Re: Design Review Request: Log Throttling

2020-10-30 Thread Brian Neradt
Hi dev@trafficserver.apache.org, I've documented our design options and played around with a few prototypes. See the content below. Please let me know if I'm missing anything or if people have other thoughts or ideas. The draft PR in its current state represents the recommended option number 4: ht

Re: [E] Re: Design Review Request: Log Throttling

2020-10-20 Thread Brian Neradt
Thank you Alan for the forward thinking observations. I agree that it is wise to implement this in such a way that it at least doesn't preclude the possibility of individual log configurability that you're talking about. As we discussed offline, this change moves us in that direction by establishin

Re: [E] Re: Design Review Request: Log Throttling

2020-10-19 Thread Alan Carroll
The primary case is a zero throttle, that is no throttling. It also seems plausible to me that different production settings will yield different levels of "noise" that the ops will want to tune for those local conditions. On Mon, Oct 19, 2020 at 11:51 AM Sudheer Vinukonda wrote: > Right, but I

Re: [E] Re: Design Review Request: Log Throttling

2020-10-19 Thread Sudheer Vinukonda
Right, but I'm wondering what sort of customization is really needed or what the use cases are where you'd need different rates of throttling for different log messages, unless the intent is not protecting the system from flood of logging (which is basically the same for any kind of log).

Re: [E] Re: Design Review Request: Log Throttling

2020-10-19 Thread Alan Carroll
I'm addressing the question of "why not do this for all messages?". I think Neue Brie's approach for specific messages is fine. If, however, the general logging calls were replaced with it, then you have the customization problem. On Mon, Oct 19, 2020 at 10:41 AM Sudheer Vinukonda wrote: > Hmm,

Re: [E] Re: Design Review Request: Log Throttling

2020-10-19 Thread Sudheer Vinukonda
Hmm, may be I'm not understanding the intent of the proposed throttling and the problem we are trying to solve well. My understanding was that a flood of certain (any) kind of log can result in significant disk i/o and impact the system throughput and stability and the proposed throttling is to

Re: [E] Re: Design Review Request: Log Throttling

2020-10-19 Thread Alan Carroll
I strongly suspect we might want it per logging message. At a minimum I don't think all messages should be throttled in this way, which is also an answer to Leif' question "how do I know which one to use"? Use the one that throttles if you want throttling, use the other if you don't want throttling

Re: [E] Re: Design Review Request: Log Throttling

2020-10-19 Thread Sudheer Vinukonda
Hmm..do you mean system wide value? We can still have the wrap around frequency configurable, but unless we want it customized per log line (I’m not sure if this is really needed?), a system wide config may still be possible to configure, no? > On Oct 19, 2020, at 7:04 AM, Alan Carroll > wro

Re: [E] Re: Design Review Request: Log Throttling

2020-10-19 Thread Alan Carroll
My concern at this point is the lack of configurability of the throttle time. The underlying implementation supports it but by wrapping it directly in the logging call, only the default value can be used.

Re: Design Review Request: Log Throttling

2020-10-17 Thread Leif Hedstrom
Huh I guess I should read the patch, it already does this ... then I agree with Sudheer and we should replace all of this with the new one, perhaps with configuration for each log level on frequency. I don’t like the idea of having two Warning flavors, when would I use one over the other ? I li

Re: Design Review Request: Log Throttling

2020-10-17 Thread Leif Hedstrom
> On Oct 16, 2020, at 18:00, Sudheer Vinukonda > wrote: > > +1 > > This sort of throttling ability is very important and useful to have. > > A minor suggestion on the implementation proposal - it feels like it’d far > simpler to localize the change to the definition of Warning, Error et

Re: Design Review Request: Log Throttling

2020-10-17 Thread Brian Neradt
Thanks for the feedback. I agree that if we were to replace all of them, we'd might as well replace the implementations of Warning, Error, etc. with this throttled version. My initial intent was not to replace all of the messages, but to replace a handful of Errors, Warnings, etc. with the throttl

Re: Design Review Request: Log Throttling

2020-10-16 Thread Sudheer Vinukonda
+1 This sort of throttling ability is very important and useful to have. A minor suggestion on the implementation proposal - it feels like it’d far simpler to localize the change to the definition of Warning, Error etc instead of having to replace each and every call to those functions. ie Ch

Design Review Request: Log Throttling

2020-10-16 Thread Brian Neradt
dev@trafficserver.apache.org, We've had issues with excessive logging under conditions where something basic goes wrong. For instance the log pipe mechanism will emit a warning or error log for every logging event if the pipe's buffer fills up or the reader goes down. This can result in thousands