RE: Hive ODBC driver

2013-01-25 Thread Chhaya Vishwakarma
hi , i have apache hive running on linux machine.can i use cloudera's or mapR's drivers? From: abhijeet gaikwad [abygaikwa...@gmail.com] Sent: Friday, January 25, 2013 5:49 PM To: user@hive.apache.org Subject: Re: Hive ODBC driver Hi Chhaya, You can insta

Lateral View in sub query issue

2013-01-25 Thread John Omernik
Anyone else seeing this select col1, col2, col3, excol from sometablewithanarrayfield LATERAL VIEW explode(arcol) artab as excol Works just fine select col1, excol, count(1) as excount from ( select col1, col2, col3, excol from sometablewithanarrayfield LATERAL VIEW explode(arcol) artab as exc

Re: updating RegexSerde on existing partitions

2013-01-25 Thread Mark Grover
Thanks for the update, Viral. One way I can think of is to drop the metadata for the entire table and recreate the table with the updated regex and recover partitions from HDFS. This, of course, works if your table is external. Mark On Fri, Jan 25, 2013 at 12:21 PM, Viral Bajaria wrote: > Well

Re: updating RegexSerde on existing partitions

2013-01-25 Thread Viral Bajaria
Well the NPE was a mistake on my part. I was over-escaping the regex and that was causing the NPE. The test that I ran was using the CLI, while the update of regex was in code. Regarding updating the regex for old partitions, I have not come across any better way besides running it over all partit

Re: updating RegexSerde on existing partitions

2013-01-25 Thread Mark Grover
Viral, To me, I think the only option seems to be here to alter all the existing partitions. I would be interested in knowing what you ended up finding/doing. Thanks, Mark On Sun, Jan 20, 2013 at 3:58 PM, Viral Bajaria wrote: > Hi, > > I have a hive table which has pre-defined schema and I use

Re: querying objects and list fields

2013-01-25 Thread Dean Wampler
Use size instead of count. Count is for counting rows, while size is for determining the size of a collection. For the second question, I think you'll need to call explode on the array, turning it into records first. Google for hive's "lateral view" to see the correct syntax for exploding and al

Re: Real-life experience of forcing smaller input splits?

2013-01-25 Thread Dean Wampler
... > > That will be my approach for now, or disabling compression altogether for > these files. The only problem I have is that compression is so efficient > that any operation in the mapper (so on the uncompressed data) just makes > the mapper throw an OOM exception, no matter how much memory I

Re: LOAD HDFS into Hive

2013-01-25 Thread venkatramanan
hi Bejoy KS, This is working perfectly... thanks On Friday 25 January 2013 06:43 PM, venkatramanan wrote: thanks for your reply i will try and get back to you thanks, Venkat On Friday 2

Re: LOAD HDFS into Hive

2013-01-25 Thread venkatramanan
thanks for your reply i will try and get back to you thanks, Venkat On Friday 25 January 2013 06:37 PM, bejoy...@yahoo.com wrote: Hi Venkataraman You can just create an external table and give it location as the hdfs dir where the data resides. No need to perform an explicit LOAD operat

Re: LOAD HDFS into Hive

2013-01-25 Thread bejoy_ks
Hi Venkataraman You can just create an external table and give it location as the hdfs dir where the data resides. No need to perform an explicit LOAD operation here. Regards Bejoy KS Sent from remote device, Please excuse typos -Original Message- From: venkatramanan Date: Fri, 25

LOAD HDFS into Hive

2013-01-25 Thread venkatramanan
Hi, I need to load the hdfs data into the Hive table. For example, Am having the twitter data and its updated daily using the streaming API. These twitter responses are stored into the HDFS Path named like ('TwitterData'). After that i try to load the data into the Hive. using the 'LOAD DATA

Re: Real-life experience of forcing smaller input splits?

2013-01-25 Thread David Morel
On 25 Jan 2013, at 10:37, Bertrand Dechoux wrote: It seems to me the question has not been answered : "is it possible yes or no to force a smaller split size than a block on the mappers" Not that I know (but you could implement something to do it) but why would you do it? By default if the sp

Re: Hive ODBC driver

2013-01-25 Thread abhijeet gaikwad
Hi Chhaya, You can install any of the third party freely available hive odbc drivers : cloudera or mapr and use odbc to run queries on hive via c#. Google would help! Want me to point you to the links? Thanks, Abhijeet On Jan 25, 2013 2:54 PM, "Nitin Pawar" wrote: > these are c# connecting to h

querying objects and list fields

2013-01-25 Thread Lauren Blau
I'm building up a set of classes (objectinspectors and serdes) to allow hive queries over some data files I have. While I'm making it work, I don't fully grok all the concepts involved. Right now I've got 2 questions. I'm able to make queries like this (this is the first syntax I tried to query in

Re: Real-life experience of forcing smaller input splits?

2013-01-25 Thread David Morel
On 25 Jan 2013, at 10:37, Bertrand Dechoux wrote: > It seems to me the question has not been answered : > "is it possible yes or no to force a smaller split size > than a block on the mappers" > > Not that I know (but you could implement something to do it) but why would > you do it? > By default

Re: Real-life experience of forcing smaller input splits?

2013-01-25 Thread Bertrand Dechoux
It seems to me the question has not been answered : "is it possible yes or no to force a smaller split size than a block on the mappers" Not that I know (but you could implement something to do it) but why would you do it? By default if the split is set under the size of a block, it will be a bloc

Re: Hive ODBC driver

2013-01-25 Thread Nitin Pawar
these are c# connecting to hive dlls On Fri, Jan 25, 2013 at 2:45 PM, Chhaya Vishwakarma < chhaya.vishwaka...@lntinfotech.com> wrote: > This libraries are for what? Can I use them to call hive in c# > > ** ** > > *From:* Nitin Pawar [mailto:nitinpawar...@gmail.com] > *Sent:* Friday, January

RE: Hive ODBC driver

2013-01-25 Thread Chhaya Vishwakarma
This libraries are for what? Can I use them to call hive in c# From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Friday, January 25, 2013 12:01 PM To: user@hive.apache.org Subject: Re: Hive ODBC driver see if any of the below drivers help you https://bitbucket.org/vadim/hive-sharp/downlo