Thanks Nitin for reply. I have data coming from RabbitMQ and i have spark streaming API which take this events and dump into HDFS. I cant really convert data events to some format like parquet/orc because I dont have schema here. Once I dump to HDFS i am writing one job which read this data and convert into Parquet. By this time I will have some raw events right?
On Tue, Aug 25, 2015 at 11:35 AM, Nitin Pawar <nitinpawar...@gmail.com> wrote: > file formats in a hive is a table level property. > I am not sure why would you have data at 15mins interval to your actual > table instead of a staging table and do the conversion or have the raw file > in the format you want and load it directly into table > > On Tue, Aug 25, 2015 at 11:27 AM, Jeetendra G <jeetendr...@housing.com> > wrote: > >> I tried searching how to set multiple format with multiple partitions , >> could not find much detail. >> Can please share some good material around this if you have any. >> >> On Mon, Aug 24, 2015 at 10:49 PM, Daniel Haviv < >> daniel.ha...@veracity-group.com> wrote: >> >>> Hi, >>> You can set a different file format per partition. >>> You can't mix files in the same directory (You could theoretically write >>> some kind of custom SerDe). >>> >>> Daniel. >>> >>> >>> >>> On Mon, Aug 24, 2015 at 6:15 PM, Jeetendra G <jeetendr...@housing.com> >>> wrote: >>> >>>> Can anyone put some light on this please? >>>> >>>> On Mon, Aug 24, 2015 at 12:32 PM, Jeetendra G <jeetendr...@housing.com> >>>> wrote: >>>> >>>>> HI All, >>>>> >>>>> I have a directory where I have json formatted and parquet files in >>>>> same folder. can hive load these? >>>>> >>>>> I am getting Json data and storing in HDFS. later I am running job to >>>>> convert JSon to Parquet(every 15 mins). so we will habe 15 mins Json data. >>>>> >>>>> Can i provide multiple serde in hive? >>>>> >>>>> regards >>>>> Jeetendra >>>>> >>>> >>>> >>> >> > > > -- > Nitin Pawar >
