Re: Merging Schema while reading Parquet files

2017-03-21 Thread Matt Deaver
You could create a one-time job that processes historical data to match the updated format On Tue, Mar 21, 2017 at 8:53 AM, Aditya Borde wrote: > Hello, > > I'm currently blocked with this issue: > > I have job "A" whose output is partitioned by one of the field - "col1" > Now job "B" reads the

Merging Schema while reading Parquet files

2017-03-21 Thread Aditya Borde
Hello, I'm currently blocked with this issue: I have job "A" whose output is partitioned by one of the field - "col1" Now job "B" reads the output of job "A". Here comes the problem. my job "A" output previously not been partitioned by "col1" (this is recent change). But the thing is now, all my