I am a huge +1 on the idea of storing my run configurations in config files
so that I don't need to type them out every time. I also really like the
idea of command defaults.
I do however have a couple of comments about the details of the proposal.
I'm a little concerned about the fallback to the user's home directory. I
feel like these config files will be per-project and checked in next to the
aurora config file for all to share. Given that things like update batch
sizes probably *should* be defined per-project and not per-user, I at least
think we should encourage this. It's also not uncommon to run commands in a
different cwd than you intended, and having such broad fallbacks seems to
be just asking for trouble. I think it's perfectly fine to expect
AuroraInit to either be in the correct working directory or for the user to
supply an explicit path to the file.
I also think once you get to that stage, you can make the observation that
most production aurora configs will have an AuroraInit file next to them.
So I wonder if we can just simplify here and let the user roll these into
one file? It'd be nice to have a naming convention for aurora files so they
could be dropped from the parameters too.
The other concern I have is the syntax for injecting variables into
commands with the @symbol. It seems like there are already tools for
solving this - bash, python, etc. that should we just concede defeat to. If
I'm typing my commands manually and relying on aliases I'd make a bet that
I'm probably not going to be remembering what I alias'd different parts of
my jobpath to and again, what the order of the jobpath bits are.
Thanks,
David
On Thu, May 22, 2014 at 11:48 AM, Mark Chu-Carroll
<mchucarr...@apache.org>wrote:
> The bulk of the feedback from the first round of comments on this could be
> summed up, roughly "get rid of the stupid command shortcuts you dope,
> they're a menace!". Message received: they're gone.
>
> But I still think that the configurable defaults are valuable. Based on
> some feedback, I've revised the proposal a bit. This includes trying to
> strengthen the motivation/problem statement, and adding ways to provide
> default parameters only for specific commands, plus many small changes.
>
> The updated proposal is attached below.
>
>
> # Defaults and Shorthands in the Aurora Client
>
> ## Motivation
>
> Most of the time, users are doing the same thing, over and over again.
> They're working mainly on one particular service, in one particular
> workspace. But they need to repeat the same parameters, over and over
> again, and they need to remember what those parameters are, what order
> they occur in, what format they use, and other details that can be
> difficult to remember.
>
> In order to avoid these problems, users set up custom scripts
> that make it easy for them to run commands like "`myservice start`",
> instead of "`aurora job create west/markcc/prod/myserver
> src/main/aurora/myservice/myservice.aurora`"
>
> To reduce this, we'd like to support a way for users to set up a
> configuration file that defines defaults and shorthands for their
> everyday work. With shorthands, a user that only works with a single
> service could say "aurora job create", instead of needing to spell
> out the full jobkey and configuration file location; a user working with
> multiple datacenters could say "`aurora job create @east`" or
> "`aurora job create @west`" to select the correct jobkey.
>
> ## Proposal: Aurora Init Files
>
> To allow users to customize shorthands, we'll provide a
> builtin capability to allow users to provide a configuration
> file, from which their customizations will be loaded.
> Many applications use a simple pattern to solve similar problems.
> Vagrant uses a file named "Vagrantfile"; when you run vagrant, if you
> don't specifically tell the tool where to find a configuration, it looks
> in the current directory or one of its parents to find a file named
> "Vagrantfile".
>
> We'd like to follow a similar pattern, and create an "AuroraInit" file.
> The aurora init file is found by searching the following locations, in
> order:
>
> * the contents of the "--init-file" parameter.
> * if the "--init-file" parameter is unspecified, then look in the
> current
> directory for a file named "AuroraInit".
> * if no "AuroraInit" file exists in the current directory, then look in
> the users
> home directory for an init file named ".AuroraInit".
>
>
> ### What goes into an init file?
>
> We should support the following kinds of things:
>
> 1. Universal defaults - user-defined default settings that will be
> applied to
> all commands. For example, if there is a default config file that
> should always
> be used if the user doesn't specify one, that would be a universal
> default.
> 2. Command specific defaults - users should be able to specify that they
> always want
> to use certain parameter settings for a specific command. For example,
> if they
> want to always use a default batch size of 10 for updates, but don't
> want to affect
> batch sizes for other commands like kill, they could use a command
> specific default.
> 3. Aliases - shorthand names for longer parameters. A user could
> specify shorthands
> "east" and "west" for full jobkeys in two different datacenters.
>
> ### Using aliases and defaults
>
> A default specifies a set of _bindings_. If a parameter is omitted
> from a command, and there's a binding for that parameter, it will be
> automatically inserted into the command as if the user had typed it.
>
> An alias is a short equivalent for a parameter. When a command line is
> provided by a user, aliases will be expanded inline. A user can
> specifically
> mark an alias for expansion by prefixing it with "@"; if an alias
> appears on the command-line surrounded by whitespace, it will be
> replaced even if it isn't marked with an "@".
>
>
> ### Defining Shorthands and Defaults
>
> The init file is, like aurora job configurations, a python source file
> using a Pystachio
> schema. The schema is loaded, and an `Init` object is retrieved from the
> top-level
> variable `init` in that file.
>
> The pystachio schema for init files is:
>
> class CommandDefaults(Struct)
> command = Required(String)
> defaults=Map(String, String)
>
> class Init(Struct):
> defaults=Map(String, String)
> command_defaults = Optional(CommandDefaults)
> aliases=Map(String, String)
>
>
> For example, an init file could contain the following:
>
> init=Init(
> defaults={
> "jobspec": "west/markcc/test/myservice",
> "config_file": "./src/aurora/myservice.aurora"
> },
> aliases={
> "east": "east/markcc/test/myservice",
> "c": "east",
> "me": "markchucarroll"
> },
> command_defaults(command="job update",
> defaults={"--batch-size": 10}
> ))
>
>
> With this configuration file, if the user ran "`aurora job create`" without
> any parameters,
> it would automatically be expanded to "`aurora job create
> west/markcc/test/myservice ./src/aurora/myservice.aurora`".
>
> If a user ran "`aurora job create east`", the alias `east` would be
> expanded to "east/markcc/test/myservice", and the missing config file
> parameter would
> be instantiated using the default, to create a command-line:
> "`aurora job create east/markcc/test/myservice
> ./src/aurora/myservice.aurora`".
>
> If a user ran "`aurora job create @c/@me/test/myservice`", the two aliases
> would
> be expanded, and the omitted configuration parameter would be added:
> "`aurora job create east/markchucarroll/test/myservice
> ./src/aurora/myservice.aurora`".
>
> If a user ran "`aurora job update`", then the `jobspec` and `config_file`
> parameters would
> get inserted from the global defaults, and the "--batch-size=10" would be
> inserted from
> the command defaults for "aurora job update".
>
