+1 for standardizing the config file name
On Jun 19, 2014 1:01 PM, "Bill Farner" <wfar...@apache.org> wrote:
> I agree with Joe, and tend to disagree with the motivation (for aliases, at
> least). The end-result seems to be obfuscation (i'd likely need to peek in
> the init file to figure out what a command does, or what command to run),
> and i'm doubtful it will result in converting users away from wrapper
> scripts.
>
> I also feel like our configuration files are already very complex (nature
> of the beast), and aliases make them even more so.
>
> I am, however, more comfortable with the standard/assumed config file
> naming (i.e. like Vagrantfile).
>
>
> -=Bill
>
>
> On Fri, Jun 13, 2014 at 11:27 AM, Joseph Smith <yasumo...@gmail.com>
> wrote:
>
> > "to set up a configuration file”
> >
> > This line to me just says “we’re reinventing wrapper scripts” where
> > someone could create an “alias” like ./east.sh and ./west.sh to do
> deploys
> > to various clusters. In that sense, these aliases etc are also “hiding”
> bad
> > interfaces, just in a different way- and one which isn’t as
> > well-tested/understood as shell scripts.
> >
> > It seems like we’re set on putting time into this, so if we are, then
> > ignore me and move onward :). IF we were going to go with an approach,
> this
> > seems reasonable based on ways that I’ve seen people wrap their
> > configs/invocations of the client.
> >
> > On Jun 12, 2014, at 7:50 AM, Mark Chu-Carroll <mchucarr...@apache.org>
> > wrote:
> >
> > > I'm nothing if not persistent! After feedback from my second try at
> > > proposing a way of doing defaults and aliases, I've got a third draft.
> > >
> > > Feedback please!
> > >
> > >
> > >
> > >
> > > # Defaults and Shorthands in the Aurora Client
> > >
> > > ## Motivation
> > >
> > > Most of the time, users are doing the same thing, over and over again.
> > > They're
> > > working mainly on one particular service, in one particular workspace.
> > But
> > > they
> > > need to repeat the same parameters, over and over again, and they need
> to
> > > remember what those parameters are, what order they occur in, what
> format
> > > they
> > > use, and other details that can be difficult to remember.
> > >
> > > In order to avoid these problems, users set up custom scripts that make
> > it
> > > easy
> > > for them to run commands like "`myservice start`", instead of "`aurora
> > job
> > > create west/markcc/prod/myserver
> > > src/main/aurora/myservice/myservice.aurora`"
> > >
> > > Scripts aren't necessarily a bad thing. We've designed the aurora
> client
> > to
> > > be script friendly: every command has at least an option supporting
> > > easy-to-parse
> > > output. But scripts also often cover for problems that really shouldn't
> > > exist.
> > >
> > > Scripts get written for a variety of reasons. Among the many reasons
> that
> > > people implement
> > > scripts, there are scripts that add custom functionality for a specific
> > > group of users;
> > > scripts that can cover up for inadequate or missing core functionality;
> > and
> > > there are
> > > scripts that cover up for poor user interfaces.
> > >
> > > In aurora, we've worked hard to eliminate cases where the core
> > > functionality of the
> > > command-line client is inadequate. But our user interface still needs a
> > lot
> > > of work.
> > > In particular, commands in Aurora often require very long parameters,
> > which
> > > users have
> > > a hard time remembering. Look at the example above - it's a pretty
> > typical
> > > case. The
> > > user wants to create a job. They're going to be creating the same job,
> > over
> > > and over again.
> > > But to run it, they need to type out 68 characters for the two
> > parameters!
> > > This is a
> > > prime example of the kind of case where users will write scripts, not
> to
> > > provide new/special
> > > functionality, nor to cover for inadequate functionality, but just
> > because
> > > it's so
> > > painful to remember and correctly type out an overly long string of
> > > parameters.
> > >
> > >
> > > To reduce this, we'd like to support a way for users to set up a
> > > configuration
> > > file that defines defaults and shorthands for their everyday work. With
> > > shorthands, a user that only works with a single service could say
> > "aurora
> > > job
> > > create", instead of needing to spell out the full jobkey and
> > configuration
> > > file
> > > location; a user working with multiple datacenters could say "`aurora
> job
> > > create @east`" or "`aurora job create @west`" to select the correct
> > jobkey.
> > >
> > > ## Proposal: Aurora Init Files
> > >
> > > To allow users to customize shorthands, we'll provide a
> > > builtin capability to allow users to provide a configuration
> > > file, from which their customizations will be loaded.
> > > Many applications use a simple pattern to solve similar problems.
> > > Vagrant uses a file named "Vagrantfile"; when you run vagrant, if you
> > > don't specifically tell the tool where to find a configuration, it
> looks
> > > in the current directory or one of its parents to find a file named
> > > "Vagrantfile".
> > >
> > > We'd like to follow a similar pattern, and create an "AuroraInit" file.
> > > The aurora init file is found by searching the following locations, in
> > > order:
> > >
> > > * the contents of the "--init-file" parameter.
> > > * if the "--init-file" parameter is unspecified, then look in the
> > current
> > > directory for a file named "AuroraInit".
> > > * if no "AuroraInit" file exists in the current directory, then look
> in
> > > the users
> > > home directory for an init file named ".aurora/init".
> > >
> > >> **Sidebar: Why fallback to the users home directory?**
> > >>
> > >> Codebases are often shared between multiple projects, each of which
> > lives
> > > in
> > >> a set of subdirectories within the codebase. For example, just look at
> > > the aurora
> > >> sourcecode, which includes the aurora scheduler, the aurora executor,
> > > thermos,
> > >> and the aurora client.
> > >>
> > >> If multiple services live in a codebase, then users can't put an
> > > AuroraInit file in
> > >> the root directory of the codebase, because it would interfere with
> > other
> > > users'
> > >> work.
> > >>
> > >> The home directory fallback provides an easy way for users to set up a
> > > pointer
> > >> to the correct init file, which won't be removed by operations like
> > > switching branches
> > >> in a source repository. Users write shared init files, which are
> located
> > > in their projects
> > >> in the codebase, and create a symbolic link from their project init
> file
> > > to their home
> > >> directory.
> > >
> > > ### What goes into an init file?
> > >
> > > We should support the following kinds of things:
> > >
> > > 1. Universal defaults - user-defined default settings that will be
> > > applied to
> > > all commands. For example, if there is a default config file that
> > > should always
> > > be used if the user doesn't specify one, that would be a universal
> > > default.
> > > 2. Command specific defaults - users should be able to specify that
> > they
> > > always want
> > > to use certain parameter settings for a specific command. For
> > example,
> > > if they
> > > want to always use a default batch size of 10 for updates, but
> don't
> > > want to affect
> > > batch sizes for other commands like kill, they could use a command
> > > specific default.
> > > 3. Aliases - shorthand names for longer parameters. A user could
> > > specify shorthands
> > > "east" and "west" for full jobkeys in two different datacenters.
> > >
> > >
> > > ### Defaults
> > >
> > > A default specifies a set of _bindings_. If a parameter is omitted
> > > from a command, and there's a binding for that parameter, it will be
> > > automatically inserted into the command as if the user had typed it.
> > > The binding is specified in the configuration file using a Python
> > > dictionary.
> > >
> > > For example, if the defaults included `{'jobspec':
> > > 'devcluster/me/prod/service'}`,
> > > then if you ran `aurora job create` without specifying any parameters,
> > the
> > > command-line
> > > would automatically substitute `devcluster/me/prod/service` for the
> > > `jobspec` parameter.
> > >
> > > Defaults can be declared either globally (in which case they'll be
> > inserted
> > > as parameters
> > > for all commands), or for specific commands (in which case they'll only
> > be
> > > inserted for a
> > > single command).
> > >
> > >
> > > ### Aliases
> > >
> > > An alias is a short equivalent for a parameter. When a command line is
> > > provided by a user, aliases will be expanded inline. A user can
> > > specifically
> > > mark an alias for expansion by prefixing it with "@"; if an alias
> > > appears on the command-line surrounded by whitespace, it will be
> > > replaced even if it isn't marked with an "@".
> > >
> > >> **Sidebar: why not just use shell substitutions?**
> > >>
> > >> The @-substitution model proposed here is very similar to unix
> variable
> > >> substitution. So why implement the same thing all over again? Why not
> > > just tell
> > >> users to use their shell?
> > >>
> > >> There are several reasons.
> > >>
> > >> 1. All of the options and aliases that affect aurora command
> invocations
> > > can be specified
> > >> in one place: the AuroraInit file.
> > >> 2. Command shell substitution is complex - the ways in which the
> command
> > > shell does
> > >> expansion, tokenization, substitution, and parsing can be very hard
> to
> > > follow. Adding
> > >> the aurora aliases to that process just adds another layer of
> confusion
> > > to the process.
> > >> With internal alias substitution, we can avoid the confusions of
> > > expansions and
> > >> tokenization for aurora aliases.
> > >> 3. We can provide better debugging and error messages with our own
> alias
> > > substitution.
> > >> For example, if a user specifies a job using aliases like "`aurora
> job
> > > create @c/@me/test/myservice`",
> > >> we can create an error message that shows exactly what they typed,
> and
> > > how it expanded:
> > >> <pre>
> > >> Job "west/markcc/test/myservice" not found in configuration file
> > > config.aurora.
> > >> Jobkey was generated by alias expansion: original key was
> > > "@c/@me/test/myservice", where:
> > >> - "@c" was expanded to "west"
> > >> - "@me" was expanded to "markcc"
> > >> config_file parameter "config.aurora" was specified from defaults.
> > >> </pre>
> > >
> > >
> > > ### Defining Shorthands and Defaults: Syntax and Examples
> > >
> > > The init file is, like aurora job configurations, a python source file
> > > using a Pystachio
> > > schema. The schema is loaded, and an `Init` object is retrieved from
> the
> > > top-level
> > > variable `init` in that file.
> > >
> > > The pystachio schema for init files is:
> > >
> > > class CommandDefaults(Struct)
> > > command = Required(String)
> > > defaults=Map(String, String)
> > >
> > > class Init(Struct):
> > > defaults=Map(String, String)
> > > command_defaults = Optional(CommandDefaults)
> > > aliases=Map(String, String)
> > >
> > >
> > > For example, an init file could contain the following:
> > >
> > > init=Init(
> > > defaults={
> > > "jobspec": "west/markcc/test/myservice",
> > > "config_file": "./src/aurora/myservice.aurora"
> > > },
> > > aliases={
> > > "east": "east/markcc/test/myservice",
> > > "c": "east",
> > > "me": "markchucarroll"
> > > },
> > > command_defaults(command="job update",
> > > defaults={"--batch-size": 10}
> > > ))
> > >
> > >
> > > With this configuration file, if the user ran "`aurora job create`"
> > without
> > > any parameters,
> > > it would automatically be expanded to "`aurora job create
> > > west/markcc/test/myservice ./src/aurora/myservice.aurora`".
> > >
> > > If a user ran "`aurora job create east`", the alias `east` would be
> > > expanded to "east/markcc/test/myservice", and the missing config file
> > > parameter would
> > > be instantiated using the default, to create a command-line:
> > > "`aurora job create east/markcc/test/myservice
> > > ./src/aurora/myservice.aurora`".
> > >
> > > If a user ran "`aurora job create @c/@me/test/myservice`", the two
> > aliases
> > > would
> > > be expanded, and the omitted configuration parameter would be added:
> > > "`aurora job create east/markchucarroll/test/myservice
> > > ./src/aurora/myservice.aurora`".
> > >
> > > If a user ran "`aurora job update`", then the `jobspec` and
> `config_file`
> > > parameters would
> > > get inserted from the global defaults, and the "--batch-size=10" would
> be
> > > inserted from
> > > the command defaults for "aurora job update".
> >
> >
>
