[
https://issues.apache.org/jira/browse/HIVE-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852497#comment-15852497
]
Siddharth Seth commented on HIVE-15688:
---------------------------------------
If I'm reading this correctly, when the "startImmediately" option is specified,
the following happens
1) Normal processing in the JavaServiceDriver
2) This then invokes package.py with a specific OUTPU_DIR, and invokes the
start.
3) The java processing falls off, and we run package.py again - which generates
the same contents as what we do today.
Is that correct?
On the patch itself
{code}
scriptArgs.add(OUTPUT_DIR_PREFIX + version);
{code}
Should this be to a temporary directory?
Move startCluster into SliderUtils?
{code}
if (rc == 0) {
startCluster(options.getName(), version,
HiveConf.getVar(conf, ConfVars.LLAP_DAEMON_QUEUE_NAME));
}
{code}
Some error checking / message returned from calling python would be useful in
case this fails.
Even if we run via this, was hoping we'd generate a run.sh which could invoke
java instead of the command line slider tools. Re-strating with the same
settings happens quite often while testing.
Unrelated to this jira: Any idea why package.py is in python. Is this something
that can be moved to Java later, so that we don't have logic split in 2 places?
> LlapServiceDriver - an option to start the cluster immediately
> --------------------------------------------------------------
>
> Key: HIVE-15688
> URL: https://issues.apache.org/jira/browse/HIVE-15688
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-15688.01.patch, HIVE-15688.patch
>
>
> run.sh is very slow because it's 4 calls to slider, which means 4 JVMs, 4
> connections to RM and other crap, for 2-5sec. of overhead per call,
> depending on the machine/cluster.
> What we need is a mode for llapservicedriver that would not generate run.sh,
> but would rather run the cluster immediately by calling the corresponding 4
> slider APIs. Should probably be the default, too. For compat with scripts we
> might generate blank run.sh for now.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
