GitHub user berngp opened a pull request:
https://github.com/apache/spark/pull/116
[SPARK-1186] : Enrich the Spark Shell to support additional arguments.
Enrich the Spark Shell functionality to support the following options.
```
Usage: spark-shell [OPTIONS]
OPTIONS:
basic:
-h --help : print this help information.
-c --executor-cores : the maximum number of cores to be used by the
spark shell.
-em --executor-memory : num[m|g], the memory used by each executor of
spark shell.
-dm --drivermem --driver-memory : num[m|g], the memory used by the
spark shell and driver.
soon to be deprecated:
--cores : please use -c/--executor-cores
other options:
-mip --master-ip : Spark Master IP/Host Address
-mp --master-port : num, Spark Master Port
-m --master : full string that describes the Spark Master.
-ld --local-dir : absolute path to a local directory that will be
use for "scratch" space in Spark.
-dh --driver-host : hostname or IP address for the driver to listen
on.
-dp --driver-port : num, port for the driver to listen on.
-uip --ui-port : num, port for your application's dashboard,
which shows memory and workload data.
--parallelism : num, default number of tasks to use across the
cluster for distributed shuffle operations when not set by user.
--locality-wait : num, number of milliseconds to wait to launch a
data-local task before giving up.
--schedule-fair : flag, enables FAIR scheduling between jobs
submitted to the same SparkContext.
--max-failures : num, number of individual task failures before
giving up on the job.
--log-conf : flag, log the supplied SparkConf as INFO at
start of spark context.
e.g.
spark-shell -m local -ld /tmp -dh 127.0.0.1 -dp 4001 -uip 4010
--parallelism 10 --locality-wait 500 --schedule-fair --max-failures 100
```
**Note**: this commit reflects the changes applied to _master_ based on
[5d98cfc1].
[ticket: SPARK-1186] : Enrich the Spark Shell to support additional
arguments.
https://spark-project.atlassian.net/browse/SPARK-1186
Author : [email protected]
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/berngp/spark feature/enrich-spark-shell
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/116.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #116
----
commit 32ad3484ad7a7cc6a0e903f80b6dd3579550e1f9
Author: Bernardo Gomez Palacio <[email protected]>
Date: 2014-03-10T17:41:30Z
[SPARK-1186] : Enrich the Spark Shell to support additional arguments.
Enrich the Spark Shell functionality to support the following options.
```
Usage: spark-shell [OPTIONS]
OPTIONS:
basic:
-h --help : print this help information.
-c --executor-cores : the maximum number of cores to be used by the
spark shell.
-em --executor-memory : num[m|g], the memory used by each executor of
spark shell.
-dm --drivermem --driver-memory : num[m|g], the memory used by the
spark shell and driver.
soon to be deprecated:
--cores : please use -c/--executor-cores
other options:
-mip --master-ip : Spark Master IP/Host Address
-mp --master-port : num, Spark Master Port
-m --master : full string that describes the Spark Master.
-ld --local-dir : absolute path to a local directory that will be
use for "scratch" space in Spark.
-dh --driver-host : hostname or IP address for the driver to listen
on.
-dp --driver-port : num, port for the driver to listen on.
-uip --ui-port : num, port for your application's dashboard,
which shows memory and workload data.
--parallelism : num, default number of tasks to use across the
cluster for distributed shuffle operations when not set by user.
--locality-wait : num, number of milliseconds to wait to launch a
data-local task before giving up.
--schedule-fair : flag, enables FAIR scheduling between jobs
submitted to the same SparkContext.
--max-failures : num, number of individual task failures before
giving up on the job.
--log-conf : flag, log the supplied SparkConf as INFO at
start of spark context.
e.g.
spark-shell -m local -ld /tmp -dh 127.0.0.1 -dp 4001 -uip 4010
--parallelism 10 --locality-wait 500 --schedule-fair --max-failures 100
```
**Note**: this commit reflects the changes applied to _master_ based on
[5d98cfc1].
[ticket: SPARK-1186] : Enrich the Spark Shell to support additional
arguments.
https://spark-project.atlassian.net/browse/SPARK-1186
Author : [email protected]
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
