On Thu, May 29, 2014 at 1:23 PM, Mark Chu-Carroll <mchucarr...@apache.org>
wrote:
> The reason that I use an alias for rbt isn't because I'm trying to save
> keystrokes. It's because "-o", "-g" and "-r" mean nothing to me. More
> keystrokes is fine, if it comes with clarity - that's why I went along with
> dropping the automatic shorthands.
>
Meta-point: rbt does have long options - I just find it excessively verbose
to type --open --guess-fields and --review-request-id.
>
> The stuff that you can do with Brian's aliases proposal does work equally
> well in mine; any alias is automatically substituted, with or without "@",
> if it's used in bare form (that is, not as a part of another string in the
> command line.) Substituting inline, though, I still believe is useful. If
> we implement it, and find that no one uses it, I'll happily remove it.
>
> -Mark
>
>
> On Thu, May 29, 2014 at 4:10 PM, David McLaughlin <da...@dmclaughlin.com>
> wrote:
>
> > I see the point about having global client defaults, but I still think we
> > should avoid inferring the jobkey from the home directory. I'd suggest
> only
> > inferring command_defaults, or even storing them separately - but this
> also
> > adds complexity. Interested in what other people think here.
> >
> > With regards to symbols and the general theme of keystrokes, specifically
> > this point:
> >
> > "I also think that you're over-estimating the degree of memory required
> to
> > > use it successfully; I've certainly seen
> > > many cases of people using stuff like that."
> >
> >
> > Look at rbt, which you mentioned you wrote a wrapper around.
> >
> > This posts a new review:
> >
> > ./rbt post -o -g
> >
> > This updates an existing review:
> >
> > ./rbt post -o -r <id>
> >
> > This has very few key strokes. It has only one parameter change between
> the
> > two commands that you need to remember. It is very simple when you look
> at
> > it in an isolated fashion. Yet people will still write wrappers around it
> > (or just hope it's in my bash history like I do).
> >
> > In this case I can't imagine constructing partial job keys from variables
> > would save you anything. I think if I'm running the commands often enough
> > that I'd remember the aliases, I'd either most likely be able to remember
> > my cluster and role names too - or more realistically, I'd just add the
> > full jobkey path to my AuroraInit file and be done with it.
> >
> > I'm only just catching on to the fact that you can't define more than one
> > job key in the AuroraInit file and aliases might be the way around that.
> To
> > solve that, I actually liked Brian's suggestion in the other thread:
> >
> > > Perhaps a stop-gap would be to require explicit aliases, e.g.
> > >
> > > [aliases]
> > > test-west = wickman/west/test/service
> > > test-east = wickman/east/test/service
> > > prod-west = wickman/west/test/service
> > > prod-east = wickman/east/test/service
> > >
> >
> > Maybe we could ask users to define pystachio structs with all their
> > options, then have a profile (or target) option like:
> >
> > ./aurora job update --profile=prod
> > ./aurora job kill --profile=prod
> >
> > etc.
> >
> >
> > On Thu, May 29, 2014 at 10:24 AM, Mark Chu-Carroll <
> mchucarr...@apache.org
> > >
> > wrote:
> >
> > > WRT to the home directory fallback: I'm not hugely committed to it, but
> > > I've discussed this with a couple of people, and all have liked it, for
> > one
> > > big reason: they're using git in the twitter codebase - so the working
> > > directory that they use contains a ton of different projects. They
> can't
> > > just put an AuroraInit file in the root directory of the workspace,
> > because
> > > it won't work. So they want to be able to write one, put it in git in
> > some
> > > directory, and then put a link to it into their homedir. Then they have
> > > defaults in place.
> > >
> > > WRT to separate init file: I think it's very important that the two be
> > > different. The config file is a shared entity used by multiple people,
> > and
> > > possibly used in different ways by different people. For example, one
> > > member of a team may typically only work with one of a team's services,
> > > while all are defined in the same config file. The user would to have
> > > defaults to work with the one service that they actually work on - but
> > > making it a default for all users of the config file would not work.
> > >
> > > WRT @symbols: yes, we could do substitution via the shell or something
> > > similar. But I think that the argument you made above about putting
> > things
> > > into the config file applies here: having all of the shorthands and
> > > defaults that the user works with be stored in one place is better that
> > > scattering among multiple files. I also think that you're
> > over-estimating
> > > the degree of memory required to use it successfully; I've certainly
> seen
> > > many cases of people using stuff like that. (Think about how people use
> > > git; they know the names of their branches, and they know the order of
> > > things for the commands they use most. For normal operation, that's
> fine.
> > > Back in my google days, we had a crazy git->perforce bridge, and it
> used
> > > aliases in exactly this way. It was hugely successful among engineers.
> I
> > > think that this is similar.)
> > >
> > > Finally, about the @ symbol: I'm not in love with it, but given my
> > argument
> > > above, if we want that functionality, we have to use something. My
> first
> > > choice would have been to use pystachio-style stashes - but the braces
> > are
> > > already used by bash; in fact, all of the bracket characters already
> are
> > > used. For single-character prefixes, there aren't many characters that
> > > aren't already used by the shell - we can't use !, #, $, ^, &, *, ~, or
> > ?.
> > > For easy to type stuff, that left very little other than @. And @ is
> used
> > > as a variable prefix in perl and ruby, so it's not without precedent or
> > > familiarity to many people.
> > >
> > > -Mark
> > >
> > >
> > > On Tue, May 27, 2014 at 7:19 PM, David McLaughlin <
> da...@dmclaughlin.com
> > >
> > > wrote:
> > >
> > > > I am a huge +1 on the idea of storing my run configurations in config
> > > files
> > > > so that I don't need to type them out every time. I also really like
> > the
> > > > idea of command defaults.
> > > >
> > > > I do however have a couple of comments about the details of the
> > proposal.
> > > >
> > > > I'm a little concerned about the fallback to the user's home
> > directory. I
> > > > feel like these config files will be per-project and checked in next
> to
> > > the
> > > > aurora config file for all to share. Given that things like update
> > batch
> > > > sizes probably *should* be defined per-project and not per-user, I at
> > > least
> > > > think we should encourage this. It's also not uncommon to run
> commands
> > > in a
> > > > different cwd than you intended, and having such broad fallbacks
> seems
> > to
> > > > be just asking for trouble. I think it's perfectly fine to expect
> > > > AuroraInit to either be in the correct working directory or for the
> > user
> > > to
> > > > supply an explicit path to the file.
> > > >
> > > > I also think once you get to that stage, you can make the observation
> > > that
> > > > most production aurora configs will have an AuroraInit file next to
> > them.
> > > > So I wonder if we can just simplify here and let the user roll these
> > into
> > > > one file? It'd be nice to have a naming convention for aurora files
> so
> > > they
> > > > could be dropped from the parameters too.
> > > >
> > > >
> > > > The other concern I have is the syntax for injecting variables into
> > > > commands with the @symbol. It seems like there are already tools for
> > > > solving this - bash, python, etc. that should we just concede defeat
> > to.
> > > If
> > > > I'm typing my commands manually and relying on aliases I'd make a bet
> > > that
> > > > I'm probably not going to be remembering what I alias'd different
> parts
> > > of
> > > > my jobpath to and again, what the order of the jobpath bits are.
> > > >
> > > >
> > > > Thanks,
> > > > David
> > > >
> > > >
> > > >
> > > > On Thu, May 22, 2014 at 11:48 AM, Mark Chu-Carroll
> > > > <mchucarr...@apache.org>wrote:
> > > >
> > > > > The bulk of the feedback from the first round of comments on this
> > could
> > > > be
> > > > > summed up, roughly "get rid of the stupid command shortcuts you
> dope,
> > > > > they're a menace!". Message received: they're gone.
> > > > >
> > > > > But I still think that the configurable defaults are valuable.
> Based
> > on
> > > > > some feedback, I've revised the proposal a bit. This includes
> trying
> > to
> > > > > strengthen the motivation/problem statement, and adding ways to
> > provide
> > > > > default parameters only for specific commands, plus many small
> > changes.
> > > > >
> > > > > The updated proposal is attached below.
> > > > >
> > > > >
> > > > > # Defaults and Shorthands in the Aurora Client
> > > > >
> > > > > ## Motivation
> > > > >
> > > > > Most of the time, users are doing the same thing, over and over
> > again.
> > > > > They're working mainly on one particular service, in one particular
> > > > > workspace. But they need to repeat the same parameters, over and
> over
> > > > > again, and they need to remember what those parameters are, what
> > order
> > > > > they occur in, what format they use, and other details that can be
> > > > > difficult to remember.
> > > > >
> > > > > In order to avoid these problems, users set up custom scripts
> > > > > that make it easy for them to run commands like "`myservice
> start`",
> > > > > instead of "`aurora job create west/markcc/prod/myserver
> > > > > src/main/aurora/myservice/myservice.aurora`"
> > > > >
> > > > > To reduce this, we'd like to support a way for users to set up a
> > > > > configuration file that defines defaults and shorthands for their
> > > > > everyday work. With shorthands, a user that only works with a
> single
> > > > > service could say "aurora job create", instead of needing to spell
> > > > > out the full jobkey and configuration file location; a user working
> > > with
> > > > > multiple datacenters could say "`aurora job create @east`" or
> > > > > "`aurora job create @west`" to select the correct jobkey.
> > > > >
> > > > > ## Proposal: Aurora Init Files
> > > > >
> > > > > To allow users to customize shorthands, we'll provide a
> > > > > builtin capability to allow users to provide a configuration
> > > > > file, from which their customizations will be loaded.
> > > > > Many applications use a simple pattern to solve similar problems.
> > > > > Vagrant uses a file named "Vagrantfile"; when you run vagrant, if
> you
> > > > > don't specifically tell the tool where to find a configuration, it
> > > looks
> > > > > in the current directory or one of its parents to find a file named
> > > > > "Vagrantfile".
> > > > >
> > > > > We'd like to follow a similar pattern, and create an "AuroraInit"
> > file.
> > > > > The aurora init file is found by searching the following locations,
> > in
> > > > > order:
> > > > >
> > > > > * the contents of the "--init-file" parameter.
> > > > > * if the "--init-file" parameter is unspecified, then look in
> the
> > > > > current
> > > > > directory for a file named "AuroraInit".
> > > > > * if no "AuroraInit" file exists in the current directory, then
> > look
> > > > in
> > > > > the users
> > > > > home directory for an init file named ".AuroraInit".
> > > > >
> > > > >
> > > > > ### What goes into an init file?
> > > > >
> > > > > We should support the following kinds of things:
> > > > >
> > > > > 1. Universal defaults - user-defined default settings that will
> be
> > > > > applied to
> > > > > all commands. For example, if there is a default config file
> > that
> > > > > should always
> > > > > be used if the user doesn't specify one, that would be a
> > universal
> > > > > default.
> > > > > 2. Command specific defaults - users should be able to specify
> > that
> > > > they
> > > > > always want
> > > > > to use certain parameter settings for a specific command. For
> > > > example,
> > > > > if they
> > > > > want to always use a default batch size of 10 for updates, but
> > > don't
> > > > > want to affect
> > > > > batch sizes for other commands like kill, they could use a
> > command
> > > > > specific default.
> > > > > 3. Aliases - shorthand names for longer parameters. A user
> could
> > > > > specify shorthands
> > > > > "east" and "west" for full jobkeys in two different
> > datacenters.
> > > > >
> > > > > ### Using aliases and defaults
> > > > >
> > > > > A default specifies a set of _bindings_. If a parameter is omitted
> > > > > from a command, and there's a binding for that parameter, it will
> be
> > > > > automatically inserted into the command as if the user had typed
> it.
> > > > >
> > > > > An alias is a short equivalent for a parameter. When a command line
> > is
> > > > > provided by a user, aliases will be expanded inline. A user can
> > > > > specifically
> > > > > mark an alias for expansion by prefixing it with "@"; if an alias
> > > > > appears on the command-line surrounded by whitespace, it will be
> > > > > replaced even if it isn't marked with an "@".
> > > > >
> > > > >
> > > > > ### Defining Shorthands and Defaults
> > > > >
> > > > > The init file is, like aurora job configurations, a python source
> > file
> > > > > using a Pystachio
> > > > > schema. The schema is loaded, and an `Init` object is retrieved
> from
> > > the
> > > > > top-level
> > > > > variable `init` in that file.
> > > > >
> > > > > The pystachio schema for init files is:
> > > > >
> > > > > class CommandDefaults(Struct)
> > > > > command = Required(String)
> > > > > defaults=Map(String, String)
> > > > >
> > > > > class Init(Struct):
> > > > > defaults=Map(String, String)
> > > > > command_defaults = Optional(CommandDefaults)
> > > > > aliases=Map(String, String)
> > > > >
> > > > >
> > > > > For example, an init file could contain the following:
> > > > >
> > > > > init=Init(
> > > > > defaults={
> > > > > "jobspec": "west/markcc/test/myservice",
> > > > > "config_file": "./src/aurora/myservice.aurora"
> > > > > },
> > > > > aliases={
> > > > > "east": "east/markcc/test/myservice",
> > > > > "c": "east",
> > > > > "me": "markchucarroll"
> > > > > },
> > > > > command_defaults(command="job update",
> > > > > defaults={"--batch-size": 10}
> > > > > ))
> > > > >
> > > > >
> > > > > With this configuration file, if the user ran "`aurora job create`"
> > > > without
> > > > > any parameters,
> > > > > it would automatically be expanded to "`aurora job create
> > > > > west/markcc/test/myservice ./src/aurora/myservice.aurora`".
> > > > >
> > > > > If a user ran "`aurora job create east`", the alias `east` would be
> > > > > expanded to "east/markcc/test/myservice", and the missing config
> file
> > > > > parameter would
> > > > > be instantiated using the default, to create a command-line:
> > > > > "`aurora job create east/markcc/test/myservice
> > > > > ./src/aurora/myservice.aurora`".
> > > > >
> > > > > If a user ran "`aurora job create @c/@me/test/myservice`", the two
> > > > aliases
> > > > > would
> > > > > be expanded, and the omitted configuration parameter would be
> added:
> > > > > "`aurora job create east/markchucarroll/test/myservice
> > > > > ./src/aurora/myservice.aurora`".
> > > > >
> > > > > If a user ran "`aurora job update`", then the `jobspec` and
> > > `config_file`
> > > > > parameters would
> > > > > get inserted from the global defaults, and the "--batch-size=10"
> > would
> > > be
> > > > > inserted from
> > > > > the command defaults for "aurora job update".
> > > > >
> > > >
> > >
> >
>
