I agree with Joe, and tend to disagree with the motivation (for aliases, at
least). The end-result seems to be obfuscation (i'd likely need to peek in
the init file to figure out what a command does, or what command to run),
and i'm doubtful it will result in converting users away from wrapper
scripts.
I also feel like our configuration files are already very complex (nature
of the beast), and aliases make them even more so.
I am, however, more comfortable with the standard/assumed config file
naming (i.e. like Vagrantfile).
-=Bill
On Fri, Jun 13, 2014 at 11:27 AM, Joseph Smith <yasumo...@gmail.com> wrote:
> "to set up a configuration file”
>
> This line to me just says “we’re reinventing wrapper scripts” where
> someone could create an “alias” like ./east.sh and ./west.sh to do deploys
> to various clusters. In that sense, these aliases etc are also “hiding” bad
> interfaces, just in a different way- and one which isn’t as
> well-tested/understood as shell scripts.
>
> It seems like we’re set on putting time into this, so if we are, then
> ignore me and move onward :). IF we were going to go with an approach, this
> seems reasonable based on ways that I’ve seen people wrap their
> configs/invocations of the client.
>
> On Jun 12, 2014, at 7:50 AM, Mark Chu-Carroll <mchucarr...@apache.org>
> wrote:
>
> > I'm nothing if not persistent! After feedback from my second try at
> > proposing a way of doing defaults and aliases, I've got a third draft.
> >
> > Feedback please!
> >
> >
> >
> >
> > # Defaults and Shorthands in the Aurora Client
> >
> > ## Motivation
> >
> > Most of the time, users are doing the same thing, over and over again.
> > They're
> > working mainly on one particular service, in one particular workspace.
> But
> > they
> > need to repeat the same parameters, over and over again, and they need to
> > remember what those parameters are, what order they occur in, what format
> > they
> > use, and other details that can be difficult to remember.
> >
> > In order to avoid these problems, users set up custom scripts that make
> it
> > easy
> > for them to run commands like "`myservice start`", instead of "`aurora
> job
> > create west/markcc/prod/myserver
> > src/main/aurora/myservice/myservice.aurora`"
> >
> > Scripts aren't necessarily a bad thing. We've designed the aurora client
> to
> > be script friendly: every command has at least an option supporting
> > easy-to-parse
> > output. But scripts also often cover for problems that really shouldn't
> > exist.
> >
> > Scripts get written for a variety of reasons. Among the many reasons that
> > people implement
> > scripts, there are scripts that add custom functionality for a specific
> > group of users;
> > scripts that can cover up for inadequate or missing core functionality;
> and
> > there are
> > scripts that cover up for poor user interfaces.
> >
> > In aurora, we've worked hard to eliminate cases where the core
> > functionality of the
> > command-line client is inadequate. But our user interface still needs a
> lot
> > of work.
> > In particular, commands in Aurora often require very long parameters,
> which
> > users have
> > a hard time remembering. Look at the example above - it's a pretty
> typical
> > case. The
> > user wants to create a job. They're going to be creating the same job,
> over
> > and over again.
> > But to run it, they need to type out 68 characters for the two
> parameters!
> > This is a
> > prime example of the kind of case where users will write scripts, not to
> > provide new/special
> > functionality, nor to cover for inadequate functionality, but just
> because
> > it's so
> > painful to remember and correctly type out an overly long string of
> > parameters.
> >
> >
> > To reduce this, we'd like to support a way for users to set up a
> > configuration
> > file that defines defaults and shorthands for their everyday work. With
> > shorthands, a user that only works with a single service could say
> "aurora
> > job
> > create", instead of needing to spell out the full jobkey and
> configuration
> > file
> > location; a user working with multiple datacenters could say "`aurora job
> > create @east`" or "`aurora job create @west`" to select the correct
> jobkey.
> >
> > ## Proposal: Aurora Init Files
> >
> > To allow users to customize shorthands, we'll provide a
> > builtin capability to allow users to provide a configuration
> > file, from which their customizations will be loaded.
> > Many applications use a simple pattern to solve similar problems.
> > Vagrant uses a file named "Vagrantfile"; when you run vagrant, if you
> > don't specifically tell the tool where to find a configuration, it looks
> > in the current directory or one of its parents to find a file named
> > "Vagrantfile".
> >
> > We'd like to follow a similar pattern, and create an "AuroraInit" file.
> > The aurora init file is found by searching the following locations, in
> > order:
> >
> > * the contents of the "--init-file" parameter.
> > * if the "--init-file" parameter is unspecified, then look in the
> current
> > directory for a file named "AuroraInit".
> > * if no "AuroraInit" file exists in the current directory, then look in
> > the users
> > home directory for an init file named ".aurora/init".
> >
> >> **Sidebar: Why fallback to the users home directory?**
> >>
> >> Codebases are often shared between multiple projects, each of which
> lives
> > in
> >> a set of subdirectories within the codebase. For example, just look at
> > the aurora
> >> sourcecode, which includes the aurora scheduler, the aurora executor,
> > thermos,
> >> and the aurora client.
> >>
> >> If multiple services live in a codebase, then users can't put an
> > AuroraInit file in
> >> the root directory of the codebase, because it would interfere with
> other
> > users'
> >> work.
> >>
> >> The home directory fallback provides an easy way for users to set up a
> > pointer
> >> to the correct init file, which won't be removed by operations like
> > switching branches
> >> in a source repository. Users write shared init files, which are located
> > in their projects
> >> in the codebase, and create a symbolic link from their project init file
> > to their home
> >> directory.
> >
> > ### What goes into an init file?
> >
> > We should support the following kinds of things:
> >
> > 1. Universal defaults - user-defined default settings that will be
> > applied to
> > all commands. For example, if there is a default config file that
> > should always
> > be used if the user doesn't specify one, that would be a universal
> > default.
> > 2. Command specific defaults - users should be able to specify that
> they
> > always want
> > to use certain parameter settings for a specific command. For
> example,
> > if they
> > want to always use a default batch size of 10 for updates, but don't
> > want to affect
> > batch sizes for other commands like kill, they could use a command
> > specific default.
> > 3. Aliases - shorthand names for longer parameters. A user could
> > specify shorthands
> > "east" and "west" for full jobkeys in two different datacenters.
> >
> >
> > ### Defaults
> >
> > A default specifies a set of _bindings_. If a parameter is omitted
> > from a command, and there's a binding for that parameter, it will be
> > automatically inserted into the command as if the user had typed it.
> > The binding is specified in the configuration file using a Python
> > dictionary.
> >
> > For example, if the defaults included `{'jobspec':
> > 'devcluster/me/prod/service'}`,
> > then if you ran `aurora job create` without specifying any parameters,
> the
> > command-line
> > would automatically substitute `devcluster/me/prod/service` for the
> > `jobspec` parameter.
> >
> > Defaults can be declared either globally (in which case they'll be
> inserted
> > as parameters
> > for all commands), or for specific commands (in which case they'll only
> be
> > inserted for a
> > single command).
> >
> >
> > ### Aliases
> >
> > An alias is a short equivalent for a parameter. When a command line is
> > provided by a user, aliases will be expanded inline. A user can
> > specifically
> > mark an alias for expansion by prefixing it with "@"; if an alias
> > appears on the command-line surrounded by whitespace, it will be
> > replaced even if it isn't marked with an "@".
> >
> >> **Sidebar: why not just use shell substitutions?**
> >>
> >> The @-substitution model proposed here is very similar to unix variable
> >> substitution. So why implement the same thing all over again? Why not
> > just tell
> >> users to use their shell?
> >>
> >> There are several reasons.
> >>
> >> 1. All of the options and aliases that affect aurora command invocations
> > can be specified
> >> in one place: the AuroraInit file.
> >> 2. Command shell substitution is complex - the ways in which the command
> > shell does
> >> expansion, tokenization, substitution, and parsing can be very hard to
> > follow. Adding
> >> the aurora aliases to that process just adds another layer of confusion
> > to the process.
> >> With internal alias substitution, we can avoid the confusions of
> > expansions and
> >> tokenization for aurora aliases.
> >> 3. We can provide better debugging and error messages with our own alias
> > substitution.
> >> For example, if a user specifies a job using aliases like "`aurora job
> > create @c/@me/test/myservice`",
> >> we can create an error message that shows exactly what they typed, and
> > how it expanded:
> >> <pre>
> >> Job "west/markcc/test/myservice" not found in configuration file
> > config.aurora.
> >> Jobkey was generated by alias expansion: original key was
> > "@c/@me/test/myservice", where:
> >> - "@c" was expanded to "west"
> >> - "@me" was expanded to "markcc"
> >> config_file parameter "config.aurora" was specified from defaults.
> >> </pre>
> >
> >
> > ### Defining Shorthands and Defaults: Syntax and Examples
> >
> > The init file is, like aurora job configurations, a python source file
> > using a Pystachio
> > schema. The schema is loaded, and an `Init` object is retrieved from the
> > top-level
> > variable `init` in that file.
> >
> > The pystachio schema for init files is:
> >
> > class CommandDefaults(Struct)
> > command = Required(String)
> > defaults=Map(String, String)
> >
> > class Init(Struct):
> > defaults=Map(String, String)
> > command_defaults = Optional(CommandDefaults)
> > aliases=Map(String, String)
> >
> >
> > For example, an init file could contain the following:
> >
> > init=Init(
> > defaults={
> > "jobspec": "west/markcc/test/myservice",
> > "config_file": "./src/aurora/myservice.aurora"
> > },
> > aliases={
> > "east": "east/markcc/test/myservice",
> > "c": "east",
> > "me": "markchucarroll"
> > },
> > command_defaults(command="job update",
> > defaults={"--batch-size": 10}
> > ))
> >
> >
> > With this configuration file, if the user ran "`aurora job create`"
> without
> > any parameters,
> > it would automatically be expanded to "`aurora job create
> > west/markcc/test/myservice ./src/aurora/myservice.aurora`".
> >
> > If a user ran "`aurora job create east`", the alias `east` would be
> > expanded to "east/markcc/test/myservice", and the missing config file
> > parameter would
> > be instantiated using the default, to create a command-line:
> > "`aurora job create east/markcc/test/myservice
> > ./src/aurora/myservice.aurora`".
> >
> > If a user ran "`aurora job create @c/@me/test/myservice`", the two
> aliases
> > would
> > be expanded, and the omitted configuration parameter would be added:
> > "`aurora job create east/markchucarroll/test/myservice
> > ./src/aurora/myservice.aurora`".
> >
> > If a user ran "`aurora job update`", then the `jobspec` and `config_file`
> > parameters would
> > get inserted from the global defaults, and the "--batch-size=10" would be
> > inserted from
> > the command defaults for "aurora job update".
>
>
