Re: Problem in reading parquet data from 2 different sources(Hive + Glue) using hive tables

2018-08-29 Thread Gopal Vijayaraghavan
> Because I believe string should be able to handle integer as well.  No, because it is not a lossless conversion. Comparisons are lost. "9" > "11", but 9 < 11 Even float -> double is lossy (because of epsilon). You can always apply the Hive workaround suggested, otherwise you might find more

Re: Problem in reading parquet data from 2 different sources(Hive + Glue) using hive tables

2018-08-29 Thread Anup Tiwari
Hi, > optional int32 action_date (DATE); > optional binary action_date (UTF8); Those two column types aren't convertible implicitly between each other, which is probably the problem In above statement, are you referring to date/utf-8 OR int32/binary.. Because I believe string should be able to

Re: Problem in reading parquet data from 2 different sources(Hive + Glue) using hive tables

2018-08-29 Thread Gopal Vijayaraghavan
Hi, > on some days parquet was created by hive 2.1.1 and on some days it was > created by using glue … > After some drill down i saw schema of columns inside both type of parquet > file using parquet tool and found different data types for some column ... > optional int32 action_date (DATE); > o

Problem in reading parquet data from 2 different sources(Hive + Glue) using hive tables

2018-08-29 Thread Anup Tiwari
Hi All, We have a use case where we have created a partition external table in hive 2.3.3 which is pointing to a parquet location where we have date level folder and on some days parquet was created by hive 2.1.1 and on some days it was created by using glue. Now when we trying to read this data,