Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Mon, 2019-08-05 at 14:44 -0700, Andres Freund wrote: > at the cost of sometimes more complicated % syntax (i.e. %1$d to > refer > to the first argument). > > I think the probable loss of gcc format warnings would be the biggest > issue with this whole proposal, and potential translator trouble

Re: Redacting information from logs

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 14:32:36 -0700, Jeff Davis wrote: > On Mon, 2019-08-05 at 14:10 -0700, Andres Freund wrote: > > I was thinking that it'd just store a struct ErrParam, which'd > > reference > > the passed value and metadata like the name (for structured log > > output) and redaction category. Th

Re: Redacting information from logs

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 14:26:44 -0700, Jeff Davis wrote: > On Sun, 2019-08-04 at 11:17 -0700, Andres Freund wrote: > > I'm imagining something like > > > > #define pg_printf(fmt, ...) \ > > do { \ > > if ( __builtin_constant_p(fmt)) \ > > { \ > > static processed_fmt pr

Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Mon, 2019-08-05 at 14:10 -0700, Andres Freund wrote: > I was thinking that it'd just store a struct ErrParam, which'd > reference > the passed value and metadata like the name (for structured log > output) and redaction category. The bigger problem I see is handling > the different types of arg

Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Sun, 2019-08-04 at 11:17 -0700, Andres Freund wrote: > I'm imagining something like > > #define pg_printf(fmt, ...) \ > do { \ > if ( __builtin_constant_p(fmt)) \ > { \ > static processed_fmt processed_fmt_ = {.format = fmt}; \ > if (unlikely(!process

Re: Redacting information from logs

2019-08-05 Thread Andres Freund
Hi, On 2019-08-05 13:37:50 -0700, Jeff Davis wrote: > On Sat, 2019-08-03 at 19:14 -0400, Tom Lane wrote: > > It seems to me that it'd be sufficient to do the annotation by > > inserting wrapper functions, like the errparam() you suggest above. > > If we just had errparam() choosing whether to retu

Re: Redacting information from logs

2019-08-05 Thread Jeff Davis
On Sat, 2019-08-03 at 19:14 -0400, Tom Lane wrote: > It seems to me that it'd be sufficient to do the annotation by > inserting wrapper functions, like the errparam() you suggest above. > If we just had errparam() choosing whether to return "..." instead of > its argument string, we'd have what we

Re: Redacting information from logs

2019-08-04 Thread Andres Freund
Hi, On 2019-08-03 19:14:13 -0400, Tom Lane wrote: > Andres Freund writes: > > I.e. something very roughly like > > > ereport(ERROR, > > errmsg_rich("string with %{named}s references to %{parameter}s"), > > errparam("named", somevar), > > errparam("parameter", othervar, .re

Re: Redacting information from logs

2019-08-03 Thread Tom Lane
Andres Freund writes: > On 2019-07-30 11:54:55 -0700, Jeff Davis wrote: >> My proposal is: >> * redact every '%s' in an ereport by having a special mode for >> snprintf.c (this is possible because we now own snprintf) > I'm extremely doubtful this is a sane approach. Yeah, it's really hard to be

Re: Redacting information from logs

2019-08-03 Thread Chapman Flack
On 7/30/19 2:54 PM, Jeff Davis wrote: > Logs are important to diagnose problems or monitor operations, but logs > can contain sensitive information which is often unnecessary for these > purposes. Redacting the sensitive information would enable easier > access and simpler integration with analysis

Re: Redacting information from logs

2019-08-03 Thread Andres Freund
Hi, On 2019-07-30 11:54:55 -0700, Jeff Davis wrote: > My proposal is: > > * redact every '%s' in an ereport by having a special mode for > snprintf.c (this is possible because we now own snprintf) I'm extremely doubtful this is a sane approach. We use snprintf for a heck of a lot of things. The

Re: Redacting information from logs

2019-08-03 Thread Tomas Vondra
On Tue, Jul 30, 2019 at 11:54:55AM -0700, Jeff Davis wrote: Logs are important to diagnose problems or monitor operations, but logs can contain sensitive information which is often unnecessary for these purposes. Redacting the sensitive information would enable easier access and simpler integrati

Redacting information from logs

2019-07-30 Thread Jeff Davis
Logs are important to diagnose problems or monitor operations, but logs can contain sensitive information which is often unnecessary for these purposes. Redacting the sensitive information would enable easier access and simpler integration with analysis tools without compromising the sensitive info