50gb of data is not much.
besides master local[4] what else do you have for other parameters?
${SPARK_HOME}/bin/spark-submit \
--driver-memory 4G \
--num-executors 1 \
--executor-memory 4G \
--master local[4] \
Try running it and check web GUI 4040 for resource usage
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 15 June 2016 at 17:03, spR <[email protected]> wrote:
> Hi,
>
> can I use spark in local mode using 4 cores to process 50gb data
> effeciently?
>
> Thank you
>
> misha
>
