Issue with a new line character in the data

. Is there an workaround in Sqoop/Hive to handle this scenario, so hive can ignore new line character if it is part of the data. We are in HDP 2.1 with sqoop 1.4.4 and hive 0.13 version. Appreciate your help with this. Thanks, Karthik

Re: Which SerDe for Custom Binary Data.

less expressly so stated. It is the > responsibility of the recipient to ensure that this email is virus free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > > > *From:* karthik maddala [mailto:karthikmaddal...@gmail.com]

Fwd: Which SerDe for Custom Binary Data.

above mentioned data. Thanks, Karthik Maddala

Which SerDe for Custom Binary Data.

above mentioned data. Thanks, Karthik Maddala

Applying UDFs on load.

Hi all, I have been loading my data into Hive as Strings, and then using the SELECT INTO statement to apply UDFs to transform my data. I was just wondering if there is a better way to do this - perhaps a way to apply UDFs on a LOAD DATA call? Or something involving the new temporary tables featur

Re: using "SET" with JDBC

> > Isn’t the second is actually what I am doing? > > > > I tried to remove the “SET” keyword and got sql exception. > > > > > > > > *From:* Karthik Banala [mailto:karthikban...@gmail.com > ] > *Sent:* Tuesday, November 25, 2014 5:03 PM > *To:

Re: using "SET" with JDBC

("hive.exec.dynamic.partition.mode=nonstrict") ; Regards, Karthik B On Tue, Nov 25, 2014 at 4:07 AM, Guy Doulberg wrote: > Hi > > > > I am trying to use “SET” keyword in queries running using JDBC. > > > > For example: > > SET hive.exec.dynamic.partition.mode=nonstric

RE: Analytic functions in Lead & Lag in Hive

Ravi, I don’t see any other option other than doing a self join. If not create another table with exact same data, and join both the tables with the right join conditions to join the previous record. Thanks, Karthik From: Ravi Prasad [mailto:raviprasa...@gmail.com] Sent: Monday, November 17

RE: Analytic functions in Lead & Lag in Hive

Ravi, Lead and Lag functions are introduced only in hive 0.11 version. Please refer to the below link for more details. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics Thanks, Karthik From: Ravi Prasad [mailto:raviprasa...@gmail.com] Sent: Friday

row_number() over(Partition by) Throw Error with Null Input.

Hi, select row_number() over (PARTITION BY country,state,department,branch_name) from Employee_details; select count(*) over (PARTITION BY country,state,department,branch_name) from Employee_details; These queries throw Error when Employee_details table has zero rows They work great if the tabl

Windowing function row_number() over (PARTITION BY(col1,clo2,col3)

Hi, Windowing function row_number() over (PARTITION BY(col1,clo2,col3) throw error on Empty Table in Hive Version .13 Is there any limitation using this function Thanks, Karthi

Templeton API- No WaitForJobToComplete.

with this.? Regards, Karthik

Re: Tez Vertex failure

Is ur Query working when you set hive.execution.engine=mr..? On Fri, Oct 3, 2014 at 7:20 PM, Echo Li wrote: > thanks for reply! > > the query is: *select count(customerid) from tableName where ymd=20140930 > ;* > > when I try to get the application logs only got this: > > > > > > > > > > > *dag_

Re: List of dates as arguments

Sun, Aug 24, 2014 at 2:56 PM, karthik Srivasthava < karthiksrivasth...@gmail.com> wrote: > Yes.. Its working > > > On Sun, Aug 24, 2014 at 2:31 PM, Bala Krishna Gangisetty < > b...@altiscale.com> wrote: > >> Yes, it does. Did you enclose the currentdate value in

Re: List of dates as arguments

Yes.. Its working On Sun, Aug 24, 2014 at 2:31 PM, Bala Krishna Gangisetty wrote: > Yes, it does. Did you enclose the currentdate value in ' ', or " "? > > > > On Sun, Aug 24, 2014 at 12:22 PM, karthik Srivasthava < > karthiksrivasth...@gmail.com> wr

Re: List of dates as arguments

startdate and enddate must be enclosed in ' '. >>> >>> Hope this helps. >>> >>> --Bala G. >>> >>> >>> On Sun, Aug 24, 2014 at 12:57 AM, Nitin Pawar >>> wrote: >>> >>>> I am not sure if you can t

Re: List of dates as arguments

ate and end date >> hive $HIVEPARAMS -hiveconf startdate="$var1" -hiveconf enddate="$var2" >> >> also set in ..hiverc >> set hive.variable.substitute=true; >> >> >> On Sun, Aug 24, 2014 at 10:19 AM, karthik Srivasthava < >> karthiksri

List of dates as arguments

As my raw-data table is partitioned by date.. i want to get data to run a query every days to find top 10 products in last 15 days . How to pass list of dates dynamically as arguments in hive query using hiveconf?

Re: Passing variables using Hiveconf

pts="-server -Xmx1200m -Djava.net.preferIPv4Stack=true" > > and you need to set this variable > > set hive.variable.substitute=true; > > > > On Fri, Aug 22, 2014 at 9:24 PM, karthik Srivasthava < > karthiksrivasth...@gmail.com> wrote: > >> Hi, >> >> I

Passing variables using Hiveconf

Hi, I am passing substitution variable using hiveconf in Hive.. But i couldnt execute simple queries when i am trying to pass more than one parameter. It throws NoViableAltException - AtomExpression.. Am i missing something.?

Re: Yarn Containers using all the space

apologies if it doesnt make any sense.. But please tell me where i made mistake On Sat, Aug 16, 2014 at 6:21 PM, Siddharth Tiwari wrote: > Whats the value set in yarn-site ? Can you paste here ? > > Sent from my iPhone > > On Aug 16, 2014, at 8:19 PM, "karthik Srivasthava&

Re: Yarn Containers using all the space

On Sat, Aug 16, 2014 at 3:34 PM, Manoj wrote: > Tweak parameters to control container memory allocation. Take a look at > this: > http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/ > > On Aug 16, 2014, at 12:58 PM, karthik Srivasthava < > karthiksrivast

Re: Yarn Containers using all the space

Thanks manoj for ur response. But i struggle to run another application in same node as first yarn application using all the available RAM . On Sat, Aug 16, 2014 at 11:44 AM, Manoj wrote: > Look into setting up yarn queues and quotas. > > > On Aug 16, 2014, at 8:15 AM, karthik

Yarn Containers using all the space

I am using TEZ as Hive Execution Engine. When i run a query , Yarn Containers are using all the space available in node. This doesnt allow another query to run. Am i missing any configuration settings here. Please let me know guys where the error might be. Thanks in advance, Karthik

New to TEZ

Hi, This might be very silly error. Below was my log.. I couldnt find where the error is. Can you please point out what caused my error... Regards, Karthik . Map 1: -/- Reducer 2: 0/20 Map 1: 0/138Reducer 2: 0/20 Map 1: 0/138Reducer 2: 0/20 Map 1: 0/138Reducer 2: 0/20 Map 1

Re:Issues Integrating HBASE with HIVE

0 down vote favorite I am able to create tables in HIVE. I have a problem with integrating HIVE and HBASE. I am following this doc. https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration My versions are: Hadoop 0.20.2 Hive 0.7.1 Hbase 0.20.6 hive> CREATE TABLE hbase_table_1(

Re: does hive support Sequence File format ?

I have a requirement to support data from the SequenceFile "KEY" (not the "VALUE") to be used by Hive table. How can I do this. From the code, it looks like the "VALUE" part is available for Hive. Please help. Regards. From: Mapred Learn To: "user@hive.ap

Re: IN clause usage in "where"

Neil, Thanks a lot, that works like a charm. Perhaps, that should go into the Hive Wiki page too sometime. Regards, Karthik. From: Neil Xu To: user@hive.apache.org Sent: Mon, October 11, 2010 6:42:10 PM Subject: Re: IN clause usage in "where" y

IN clause usage in "where"

Is there a way I can use the "in" list of values in a query condition like "select col1 from table1 where col1 in ('abc','xyz')"? I use Hive 0.6. Regards, Karthik.