Peter, thanks for that thoughtful and clear answer. It articulates very clearly the tussle I've been feeling, and is immensely helpful!
Happy New Year! Zellyn On Tuesday, December 25, 2018 at 1:37:14 PM UTC-5, Peter Bourgon wrote: > > I've repeatedly built "service container" or "corp kits" like the one > you're describing, as well as helped a number of orgs improve their > own versions of the thing in a consultant capacity. I think everyone > has the same reaction that you do: why are we duplicating all this > effort? But I can say with some breadth of experience that the > commonalities are fewer than they may appear. Once you take your e.g. > log shipper library, which is likely quite purpose-built not only for > your chosen MQ but the peculiarities of its runtime environment(s), > and provided switches and knobs to account for reasonable alternatives > to make it worth open-sourcing, you've rounded off a sufficient number > of corners to make it significantly less useful in your original > environment. The utility of these kits or containers is generally > proportional to how zero-conf they are: just use these pre-ordained > things, and you'll be good to go. Provide enough knobs to make them > generally useful, and the magic is replaced by config; new users, > faced with a sprawling YAML config and compiling in support for things > they'll never need, wonder why they can't eliminate all this cruft by > making a purpose-built jig for their specific conditions, and you're > back to square one. Advice from someone who's been down the path > enough times: accept as invariant that this code will never see the > light of open-source day, and hyper-specialize to your programming > environment, optimizing for ease-of-use. It's a better use of time and > energy. > > > On Wed, Dec 12, 2018 at 4:39 PM Zellyn <zel...@gmail.com <javascript:>> > wrote: > > > > Hi folks, > > > > At Square, we have a collection of shared frameworks and library code, > loosely referred to as the “service container” in each language. For > example, our framework code in Go includes: > > > > a lightweight "module" system for dependency injection > > config: a way of layering yaml files named for environments. For > instance, common.yaml gets loaded first, and production.yaml shadows over > it, and production-east-coast.yaml gets applied last. > > logging: code for sending log lines over kafka to our logging aggregator > > networking: our stubby-alike protobuf RPC mechanism, naming and > discovery, etc. Gradually migrating to "just use grpc and envoy" > > datasource config: a convention for configuring database sources via the > yaml configs > > > > People probably also think of our service container as including > slightly less core things too: > > > > sharded jobqueues > > feature flags (backed by zookeeper) > > cron jobs (again configurable via yaml) > > > > This stuff has gone a while with less-than-generous attention and > upkeep, and the extent of Square-specific code makes it more difficult for > new developers to come up to speed. > > > > What I'm trying to figure out is: what are folks at other companies > using to create a shared set of conventions and out-of-the-box setup for > their Go applications? We can knock some pieces out one by one (eg. replace > our Module system with Wire), but it would be nice to find an existing > lightweight “service container” that other companies are using. > > > > I suppose my main question is this: is the kind of flexibility required > for a shared, open-source “service container” antithetical to the kind of > simple, opinionated, internal-convention-obeying “service container” you'd > want for your company? To put it another way, it has never felt like using > gokit would be appropriate, because it obeys none of our conventions. > > > > Thoughts, comments, experience to relate? > > > > Yours, > > > > Zellyn > > > > -- > > You received this message because you are subscribed to the Google > Groups "golang-nuts" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to golang-nuts...@googlegroups.com <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
