I've repeatedly built "service container" or "corp kits" like the one you're describing, as well as helped a number of orgs improve their own versions of the thing in a consultant capacity. I think everyone has the same reaction that you do: why are we duplicating all this effort? But I can say with some breadth of experience that the commonalities are fewer than they may appear. Once you take your e.g. log shipper library, which is likely quite purpose-built not only for your chosen MQ but the peculiarities of its runtime environment(s), and provided switches and knobs to account for reasonable alternatives to make it worth open-sourcing, you've rounded off a sufficient number of corners to make it significantly less useful in your original environment. The utility of these kits or containers is generally proportional to how zero-conf they are: just use these pre-ordained things, and you'll be good to go. Provide enough knobs to make them generally useful, and the magic is replaced by config; new users, faced with a sprawling YAML config and compiling in support for things they'll never need, wonder why they can't eliminate all this cruft by making a purpose-built jig for their specific conditions, and you're back to square one. Advice from someone who's been down the path enough times: accept as invariant that this code will never see the light of open-source day, and hyper-specialize to your programming environment, optimizing for ease-of-use. It's a better use of time and energy.
On Wed, Dec 12, 2018 at 4:39 PM Zellyn <zel...@gmail.com> wrote: > > Hi folks, > > At Square, we have a collection of shared frameworks and library code, > loosely referred to as the “service container” in each language. For example, > our framework code in Go includes: > > a lightweight "module" system for dependency injection > config: a way of layering yaml files named for environments. For instance, > common.yaml gets loaded first, and production.yaml shadows over it, and > production-east-coast.yaml gets applied last. > logging: code for sending log lines over kafka to our logging aggregator > networking: our stubby-alike protobuf RPC mechanism, naming and discovery, > etc. Gradually migrating to "just use grpc and envoy" > datasource config: a convention for configuring database sources via the yaml > configs > > People probably also think of our service container as including slightly > less core things too: > > sharded jobqueues > feature flags (backed by zookeeper) > cron jobs (again configurable via yaml) > > This stuff has gone a while with less-than-generous attention and upkeep, and > the extent of Square-specific code makes it more difficult for new developers > to come up to speed. > > What I'm trying to figure out is: what are folks at other companies using to > create a shared set of conventions and out-of-the-box setup for their Go > applications? We can knock some pieces out one by one (eg. replace our Module > system with Wire), but it would be nice to find an existing lightweight > “service container” that other companies are using. > > I suppose my main question is this: is the kind of flexibility required for a > shared, open-source “service container” antithetical to the kind of simple, > opinionated, internal-convention-obeying “service container” you'd want for > your company? To put it another way, it has never felt like using gokit would > be appropriate, because it obeys none of our conventions. > > Thoughts, comments, experience to relate? > > Yours, > > Zellyn > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
