Thanks for your thoughtful response. I agree with many of your points, especially the benefits of structured logging, both on the usefulness of the logs and the flow of the code as it builds the logging context. I have a few comments on some of your points:
> Of these, I lean towards the zap style because it's not significantly more verbose (IMHO), it's more type safe, and faster. The price of the zap API is that any package that doesn't use the sugared API becomes coupled to the zap and zapcore packages. The closest thing it provides to a fundamental interface is zapcore.Encoder <https://godoc.org/go.uber.org/zap/zapcore#Encoder> which requires arguments of their concrete types `zapcore.Entry` and `zapcore.Field`. Another strike against zap, IMHO, is the sheer size of it's API. It also seems to me that customizing it's behavior would be a non-trivial exercise. > I really like the way that it [zap] never combines the log line into something heavy like a map[string]interface{}, and that it just logs its fields in the order they were passed - so it's more like a traditional log line, just with JSON boundaries. I agree and go-kit/log is the same as zap in this regard. It is true that go-kit/log currently implements JSON logging by converting to a map and using enconding/json to serialize the result. That is the simplest implementation, but it is not required by the go-kit/log.Logger interface and I am currently working on a new implementation that serializes to JSON directly from a keyvals slice. When complete, this will provide a significant performance improvement, bringing the speed of JSON output on par with the speed of logfmt output. > It's never really appropriate to use anything other than stdlib logging to log with in a library, and that's the nub of the problem: it means libraries will pretty much always have unstructured logging. As I explained at the end of my talk, I don't think it is appropriate for public libraries to log at all. My rationale is much broader than simply structured vs. unstructured, or stdlib vs. pick your logger. My rationale is that once a library chooses to log it has taken that choice away from the application it is eventually used by and public libraries harm their reusability by doing so. Logging is an application concern. If a public library does choose to log, it should do so by a simple interface that isn't coupled to anything outside the package itself, the language, or the standard library; preferably a locally defined interface. Note that the go-kit/log.Logger interface meets this goal. Any package can copy that interface into its API and be compatible with go-kit/log style loggers without any external dependencies. Non-public libraries have more leeway because, presumably, the library is closely related to a small set of applications that have common logging needs or the library is owned by an organization that has dictated a standard logging package. That said, designing private libraries as if they were public libraries could be a good idea in the long run. Chris On Tuesday, August 15, 2017 at 12:09:44 PM UTC-4, Sam Vilain wrote: > > On 8/15/17 7:14 AM, Chris Hines wrote: > > I would be curious what you think of github.com/go-kit/kit/log (and > related "sub"-packages). See my talk about it's design here: > > Video: https://youtu.be/ojhFQNgyZO4?list=FLcxNiie7-UD8znndwDn7PFw > Slides: https://speakerdeck.com/chrishines/go-kit-log-package > > I can see what you're going for with that, and in a way it's demonstrating > some sugar issues in go. For some reason, folk expect to be able to write > logging statements without an awful lot of unsightly brackets and > constructors :-). > > To quickly round them up, this module uses what I consider to be Perl > 5-esque "slurpy hashes": > > // Unstructured > log.Printf("HTTP server listening on %s", addr) > // Structured > logger.Log("transport", "HTTP", "addr", addr, "msg", "listening") > <https://github.com/go-kit/kit/tree/master/log#usage> > > Whereas logrus steers you towards a map[string]interface{}: > > func main() { > log.WithFields(log.Fields{ > "animal": "walrus", > }).Info("A walrus appears") > } > > And zap uses type-safe individual field constructors: > > logger.Info("failed to fetch URL", > // Structured context as strongly typed Field values. > zap.String("url", url), > zap.Int("attempt", 3), > zap.Duration("backoff", time.Second), > ) > > Of these, I lean towards the zap style because it's not significantly more > verbose (IMHO), it's more type safe, and faster. > > However, the syntax sugar issue has meant that zap now also have a similar > interface to kit: > > sugar := logger.Sugar() > sugar.Infow("failed to fetch URL", > // Structured context as loosely typed key-value pairs. > "url", url, > "attempt", 3, > "backoff", time.Second, > ) > > > One common element is the ability to create a logger with tags pre-built: > > // kit > logger = log.With(logger, "instance_id", 123) > > // logrus > logger = logrus.WithFields(logrus.Fields{"instance_id": 123}) > > // zap > logger = logger.With(zap.Int("instance_id", 123)) > > > IMHO appropriate use of this feature is critical to sane log analysis as > soon as your app gets non-trivial: distributed, sharded, microservices, > async, etc - non-linear, multiuser and multicore: it can simply save you > down the track if you tag the request ID, customer ID, and particularly the > value of a loop variable for log statements inside inner loops. > > So I guess, what I found was that once I did that using zap, I really > don't care that much about the syntax sugar. I would write things like: > > for i, v := range arr { > logger := logger.With(zap.Int("itemIdx", i)) > > if err := v.Process(); err != nil { > logger.Error("failed to process", zap.Object("error", err)) > } > } > > And so the individual log statements don't go over the magic length at > which blub programmers scoff, turn up their noses and say, "see, this is > why go is so much worse than blub". It's also a good abstraction. And did > I mention it's fast? I really like the way that it never combines the log > line into something heavy like a map[string]interface{}, and that it just > logs its fields in the order they were passed - so it's more like a > traditional log line, just with JSON boundaries. > > So anyway, yeah that's where I stand on that API - I can see how you got > there, and I watched a structured logging talk at GopherCon 2016 which set > me down this path, but after investigation I just couldn't trade > performance, memory efficiency and a good abstraction for a little syntax > sugar. > > I'm not quite sure what the language or stdlib could do to make this > better, other than shipping something like zap in stdlib. It's never > really appropriate to use anything other than stdlib logging to log with in > a library, and that's the nub of the problem: it means libraries will > pretty much always have unstructured logging. The best you can do is to > log each "printf" argument to its own field, but that's not a very good > answer especially with the "message" is "%s: %s %s" or something. > > Sam > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
