Currently we have data in NFS and we have proprietery tools to access the data. We are planning to move the data into HDFS and use HiveQL for accessing the data and run batch jobs. So looking out for the custom SerDe(assuming the existing SerDe's will not be able to read the underlying data) to read the data using hive. On Fri, Mar 13, 2015 at 10:33 AM, Mich Talebzadeh <m...@peridale.co.uk> wrote:
> Hive as I use it is particularly useful for getting data out of relational > tables and more importantly query that data using HiveQL (a variation of > transact sql) > > . > > > > If your data is in binary format and assuming that you manage to store it > in HDFS, how are you intending to access the data. At the consumer level > what tools are you going to use? Do you a propriety tool with the correct > drivers to access the data? > > > > HTH > > > > Mich Talebzadeh > > > > http://talebzadehmich.wordpress.com > > > > *Publications due shortly:* > > *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and > Coherence Cache* > > > > NOTE: The information in this email is proprietary and confidential. This > message is for the designated recipient only, if you are not the intended > recipient, you should destroy it immediately. Any information in this > message shall not be understood as given or endorsed by Peridale Ltd, its > subsidiaries or their employees, unless expressly so stated. It is the > responsibility of the recipient to ensure that this email is virus free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > > > *From:* karthik maddala [mailto:karthikmaddal...@gmail.com] > *Sent:* 13 March 2015 15:56 > *To:* user@hive.apache.org > *Subject:* Which SerDe for Custom Binary Data. > > > > > > > > I want to set up a DW based on Hive. However, my data does not come as > handy csv files but as binary files in a proprietary format. > > > > The binary file consists of serialized data using C language. > > > > > > Could you please suggest which input format to be used and how to write a > custom SerDe for the above mentioned data. > > > > > > Thanks, > > Karthik Maddala > > > > >
