Re: How to perform hive moveTask in parallel?

2017-05-23 Thread Rishi Aggarwal
ty, Magarpatta, Hadapsar Pune, Maharashtra, > 411 013 > *o*ff: +91-20-49118448 > [image: Description: untitled] > > *"When the solution is simple, God is answering…" * > > > > *From:* Rishi Aggarwal [mailto:ri...@hike.in] > *Sent:* 21 May 2017 A

Re: How to perform hive moveTask in parallel?

2017-05-20 Thread Rishi Aggarwal
> You are looking for https://issues.apache.org/jira/browse/HIVE-12988 > > This got added from hive 2.1.0 release onwards. > > Thanks > Prasanth > > On May 21, 2017, at 1:13 AM, Rishi Aggarwal wrote: > > I am running a insert overwrite query on an external table which

How to perform hive moveTask in parallel?

2017-05-20 Thread Rishi Aggarwal
I am running a insert overwrite query on an external table which is partitioned (192 partitions). On doing explain I see there are mainly two stage. 1. MR stage (8 mappers and 10 reducers) 2. Move Stage MR stage is completing in 15-20 mins. Move stage is taking about *3hours*. On looking