Re: Aws

2019-02-07 Thread Hiroyuki Nagata
ed with the driver only, > while it could also contain an executor. > > > Regards, > Pedro. > > > > l jue., 31 de ene. de 2019 20:16, Hiroyuki Nagata > escribió: > >> Hi, Pedro >> >> >> I also start using AWS EMR, with Spark 2.4.0. I'

Re: Aws

2019-01-31 Thread Hiroyuki Nagata
Hi, Pedro I also start using AWS EMR, with Spark 2.4.0. I'm seeking methods for performance tuning. Do you configure dynamic allocation ? FYI: https://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation I've not tested it yet. I guess spark-submit needs to specify numb