I see the point about having global client defaults, but I still think we
should avoid inferring the jobkey from the home directory. I'd suggest only
inferring command_defaults, or even storing them separately - but this also
adds complexity. Interested in what other people think here.
With regards to symbols and the general theme of keystrokes, specifically
this point:
"I also think that you're over-estimating the degree of memory required to
> use it successfully; I've certainly seen
> many cases of people using stuff like that."
Look at rbt, which you mentioned you wrote a wrapper around.
This posts a new review:
./rbt post -o -g
This updates an existing review:
./rbt post -o -r <id>
This has very few key strokes. It has only one parameter change between the
two commands that you need to remember. It is very simple when you look at
it in an isolated fashion. Yet people will still write wrappers around it
(or just hope it's in my bash history like I do).
In this case I can't imagine constructing partial job keys from variables
would save you anything. I think if I'm running the commands often enough
that I'd remember the aliases, I'd either most likely be able to remember
my cluster and role names too - or more realistically, I'd just add the
full jobkey path to my AuroraInit file and be done with it.
I'm only just catching on to the fact that you can't define more than one
job key in the AuroraInit file and aliases might be the way around that. To
solve that, I actually liked Brian's suggestion in the other thread:
> Perhaps a stop-gap would be to require explicit aliases, e.g.
>
> [aliases]
> test-west = wickman/west/test/service
> test-east = wickman/east/test/service
> prod-west = wickman/west/test/service
> prod-east = wickman/east/test/service
>
Maybe we could ask users to define pystachio structs with all their
options, then have a profile (or target) option like:
./aurora job update --profile=prod
./aurora job kill --profile=prod
etc.
On Thu, May 29, 2014 at 10:24 AM, Mark Chu-Carroll <mchucarr...@apache.org>
wrote:
> WRT to the home directory fallback: I'm not hugely committed to it, but
> I've discussed this with a couple of people, and all have liked it, for one
> big reason: they're using git in the twitter codebase - so the working
> directory that they use contains a ton of different projects. They can't
> just put an AuroraInit file in the root directory of the workspace, because
> it won't work. So they want to be able to write one, put it in git in some
> directory, and then put a link to it into their homedir. Then they have
> defaults in place.
>
> WRT to separate init file: I think it's very important that the two be
> different. The config file is a shared entity used by multiple people, and
> possibly used in different ways by different people. For example, one
> member of a team may typically only work with one of a team's services,
> while all are defined in the same config file. The user would to have
> defaults to work with the one service that they actually work on - but
> making it a default for all users of the config file would not work.
>
> WRT @symbols: yes, we could do substitution via the shell or something
> similar. But I think that the argument you made above about putting things
> into the config file applies here: having all of the shorthands and
> defaults that the user works with be stored in one place is better that
> scattering among multiple files. I also think that you're over-estimating
> the degree of memory required to use it successfully; I've certainly seen
> many cases of people using stuff like that. (Think about how people use
> git; they know the names of their branches, and they know the order of
> things for the commands they use most. For normal operation, that's fine.
> Back in my google days, we had a crazy git->perforce bridge, and it used
> aliases in exactly this way. It was hugely successful among engineers. I
> think that this is similar.)
>
> Finally, about the @ symbol: I'm not in love with it, but given my argument
> above, if we want that functionality, we have to use something. My first
> choice would have been to use pystachio-style stashes - but the braces are
> already used by bash; in fact, all of the bracket characters already are
> used. For single-character prefixes, there aren't many characters that
> aren't already used by the shell - we can't use !, #, $, ^, &, *, ~, or ?.
> For easy to type stuff, that left very little other than @. And @ is used
> as a variable prefix in perl and ruby, so it's not without precedent or
> familiarity to many people.
>
> -Mark
>
>
> On Tue, May 27, 2014 at 7:19 PM, David McLaughlin <da...@dmclaughlin.com>
> wrote:
>
> > I am a huge +1 on the idea of storing my run configurations in config
> files
> > so that I don't need to type them out every time. I also really like the
> > idea of command defaults.
> >
> > I do however have a couple of comments about the details of the proposal.
> >
> > I'm a little concerned about the fallback to the user's home directory. I
> > feel like these config files will be per-project and checked in next to
> the
> > aurora config file for all to share. Given that things like update batch
> > sizes probably *should* be defined per-project and not per-user, I at
> least
> > think we should encourage this. It's also not uncommon to run commands
> in a
> > different cwd than you intended, and having such broad fallbacks seems to
> > be just asking for trouble. I think it's perfectly fine to expect
> > AuroraInit to either be in the correct working directory or for the user
> to
> > supply an explicit path to the file.
> >
> > I also think once you get to that stage, you can make the observation
> that
> > most production aurora configs will have an AuroraInit file next to them.
> > So I wonder if we can just simplify here and let the user roll these into
> > one file? It'd be nice to have a naming convention for aurora files so
> they
> > could be dropped from the parameters too.
> >
> >
> > The other concern I have is the syntax for injecting variables into
> > commands with the @symbol. It seems like there are already tools for
> > solving this - bash, python, etc. that should we just concede defeat to.
> If
> > I'm typing my commands manually and relying on aliases I'd make a bet
> that
> > I'm probably not going to be remembering what I alias'd different parts
> of
> > my jobpath to and again, what the order of the jobpath bits are.
> >
> >
> > Thanks,
> > David
> >
> >
> >
> > On Thu, May 22, 2014 at 11:48 AM, Mark Chu-Carroll
> > <mchucarr...@apache.org>wrote:
> >
> > > The bulk of the feedback from the first round of comments on this could
> > be
> > > summed up, roughly "get rid of the stupid command shortcuts you dope,
> > > they're a menace!". Message received: they're gone.
> > >
> > > But I still think that the configurable defaults are valuable. Based on
> > > some feedback, I've revised the proposal a bit. This includes trying to
> > > strengthen the motivation/problem statement, and adding ways to provide
> > > default parameters only for specific commands, plus many small changes.
> > >
> > > The updated proposal is attached below.
> > >
> > >
> > > # Defaults and Shorthands in the Aurora Client
> > >
> > > ## Motivation
> > >
> > > Most of the time, users are doing the same thing, over and over again.
> > > They're working mainly on one particular service, in one particular
> > > workspace. But they need to repeat the same parameters, over and over
> > > again, and they need to remember what those parameters are, what order
> > > they occur in, what format they use, and other details that can be
> > > difficult to remember.
> > >
> > > In order to avoid these problems, users set up custom scripts
> > > that make it easy for them to run commands like "`myservice start`",
> > > instead of "`aurora job create west/markcc/prod/myserver
> > > src/main/aurora/myservice/myservice.aurora`"
> > >
> > > To reduce this, we'd like to support a way for users to set up a
> > > configuration file that defines defaults and shorthands for their
> > > everyday work. With shorthands, a user that only works with a single
> > > service could say "aurora job create", instead of needing to spell
> > > out the full jobkey and configuration file location; a user working
> with
> > > multiple datacenters could say "`aurora job create @east`" or
> > > "`aurora job create @west`" to select the correct jobkey.
> > >
> > > ## Proposal: Aurora Init Files
> > >
> > > To allow users to customize shorthands, we'll provide a
> > > builtin capability to allow users to provide a configuration
> > > file, from which their customizations will be loaded.
> > > Many applications use a simple pattern to solve similar problems.
> > > Vagrant uses a file named "Vagrantfile"; when you run vagrant, if you
> > > don't specifically tell the tool where to find a configuration, it
> looks
> > > in the current directory or one of its parents to find a file named
> > > "Vagrantfile".
> > >
> > > We'd like to follow a similar pattern, and create an "AuroraInit" file.
> > > The aurora init file is found by searching the following locations, in
> > > order:
> > >
> > > * the contents of the "--init-file" parameter.
> > > * if the "--init-file" parameter is unspecified, then look in the
> > > current
> > > directory for a file named "AuroraInit".
> > > * if no "AuroraInit" file exists in the current directory, then look
> > in
> > > the users
> > > home directory for an init file named ".AuroraInit".
> > >
> > >
> > > ### What goes into an init file?
> > >
> > > We should support the following kinds of things:
> > >
> > > 1. Universal defaults - user-defined default settings that will be
> > > applied to
> > > all commands. For example, if there is a default config file that
> > > should always
> > > be used if the user doesn't specify one, that would be a universal
> > > default.
> > > 2. Command specific defaults - users should be able to specify that
> > they
> > > always want
> > > to use certain parameter settings for a specific command. For
> > example,
> > > if they
> > > want to always use a default batch size of 10 for updates, but
> don't
> > > want to affect
> > > batch sizes for other commands like kill, they could use a command
> > > specific default.
> > > 3. Aliases - shorthand names for longer parameters. A user could
> > > specify shorthands
> > > "east" and "west" for full jobkeys in two different datacenters.
> > >
> > > ### Using aliases and defaults
> > >
> > > A default specifies a set of _bindings_. If a parameter is omitted
> > > from a command, and there's a binding for that parameter, it will be
> > > automatically inserted into the command as if the user had typed it.
> > >
> > > An alias is a short equivalent for a parameter. When a command line is
> > > provided by a user, aliases will be expanded inline. A user can
> > > specifically
> > > mark an alias for expansion by prefixing it with "@"; if an alias
> > > appears on the command-line surrounded by whitespace, it will be
> > > replaced even if it isn't marked with an "@".
> > >
> > >
> > > ### Defining Shorthands and Defaults
> > >
> > > The init file is, like aurora job configurations, a python source file
> > > using a Pystachio
> > > schema. The schema is loaded, and an `Init` object is retrieved from
> the
> > > top-level
> > > variable `init` in that file.
> > >
> > > The pystachio schema for init files is:
> > >
> > > class CommandDefaults(Struct)
> > > command = Required(String)
> > > defaults=Map(String, String)
> > >
> > > class Init(Struct):
> > > defaults=Map(String, String)
> > > command_defaults = Optional(CommandDefaults)
> > > aliases=Map(String, String)
> > >
> > >
> > > For example, an init file could contain the following:
> > >
> > > init=Init(
> > > defaults={
> > > "jobspec": "west/markcc/test/myservice",
> > > "config_file": "./src/aurora/myservice.aurora"
> > > },
> > > aliases={
> > > "east": "east/markcc/test/myservice",
> > > "c": "east",
> > > "me": "markchucarroll"
> > > },
> > > command_defaults(command="job update",
> > > defaults={"--batch-size": 10}
> > > ))
> > >
> > >
> > > With this configuration file, if the user ran "`aurora job create`"
> > without
> > > any parameters,
> > > it would automatically be expanded to "`aurora job create
> > > west/markcc/test/myservice ./src/aurora/myservice.aurora`".
> > >
> > > If a user ran "`aurora job create east`", the alias `east` would be
> > > expanded to "east/markcc/test/myservice", and the missing config file
> > > parameter would
> > > be instantiated using the default, to create a command-line:
> > > "`aurora job create east/markcc/test/myservice
> > > ./src/aurora/myservice.aurora`".
> > >
> > > If a user ran "`aurora job create @c/@me/test/myservice`", the two
> > aliases
> > > would
> > > be expanded, and the omitted configuration parameter would be added:
> > > "`aurora job create east/markchucarroll/test/myservice
> > > ./src/aurora/myservice.aurora`".
> > >
> > > If a user ran "`aurora job update`", then the `jobspec` and
> `config_file`
> > > parameters would
> > > get inserted from the global defaults, and the "--batch-size=10" would
> be
> > > inserted from
> > > the command defaults for "aurora job update".
> > >
> >
>
