map-side-combine in Spark SQL

2016-02-09 Thread Rishitesh Mishra
Can anybody confirm, whether ANY operator in Spark SQL uses map-side-combine ? If not, is it safe to assume SortShuffleManager will always use Serialized sorting in case of queries from Spark SQL ?

Re: Task Execution

2015-10-01 Thread Rishitesh Mishra
Depending upon the configured cores assigned to the executor, scheduler will assign that many tasks. So yes they execute in parallel. On 30 Sep 2015 14:51, "gsvic" wrote: > Concerning task execution, a worker executes its assigned tasks in parallel > or sequentially? > > > > -- > View this messag

Subscribe

2015-08-16 Thread Rishitesh Mishra