Re: Fw: Hive update operation

Ok this is what you have in MSSQL (COLLATE) does not come into it in Hive) UPDATE table1 SET address=regexp_replace(t2.cout_event_description,,) , latitude=t2.latitude , longitude=t2.longitude , speed =t2.speed , dtimestamp =mv.dtimestam

Re: Fw: Hive update operation

Current RDBMS: SQL Server 2012 Yes, I tried below one. UPDATE table1 set address=regexp_replace(t2.cout_event_description,,),latitude=t2.latitude,longitude=t2.longitude ,speed =t2.speed,dtimestamp =mv.dtimestamp,reg_no=t2.registration,gpsstate = t2.bgps FROM default.maxvalues m

Re: hive 2.1.0 and "NOT IN ( list )" and column is a partition_key

> not array_contains(array('foo'), partition_key) And this is why that works. https://issues.apache.org/jira/browse/HIVE-13951 :( Cheers, Gopal

Re: hive 2.1.0 and "NOT IN ( list )" and column is a partition_key

awesome! i'm not crazy after all! so workaround we devised here for this pattern: partition_key not in ('foo') is: not array_contains(array('foo'), partition_key) thanks, Stephen. On Thu, Aug 25, 2016 at 6:31 PM, Sergey Shelukhin wrote: > I can repro this on master. I’ll file a bug..

Re: hive 2.1.0 and "NOT IN ( list )" and column is a partition_key

I can repro this on master. I’ll file a bug... From: Stephen Sprague mailto:sprag...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Thursday, August 25, 2016 at 13:34 To: "user@hive.apache.org" mailto:use

How to do such a join of Pairing in Hive?

1. Firstly we create a hive table by loading following csv file > $cat data.csv > > ID,City,Zip,Flag > 1,A,95126,0 > 2,A,95126,1 > 3,A,95126,1 > 4,B,95124,0 > 5,B,95124,1 > 6,C,95124,0 > 7,C,95127,1 > 8,C,95127,0 > 9,C,95127,1 (a) where "ID" above is a

Re: hive 2.1.0 and "NOT IN ( list )" and column is a partition_key

Hi Gopal, Thank you for this insight. good stuff. The thing is there is no 'foo' for etl_database_source so that filter if anything should be short-circuited to 'true'. ie. double nots. 1. not in 2. and foo not present. it doesn't matter what what i put in that "not in" clause the filter al

Re: hive 2.1.0 and "NOT IN ( list )" and column is a partition_key

> anybody run up against this one? hive 2.1.0 + using a "not in" on a >list + the column is a partition key participant. The partition filters are run before the plan is generated. >AND etl_source_database not in ('foo') Is there a 'foo' in etl_source_database? > predicate: f

hive 2.1.0 and "NOT IN ( list )" and column is a partition_key

anybody run up against this one? hive 2.1.0 + using a "not in" on a list + the column is a partition key participant. * using not query: explain SELECT count(*) FROM bi.fact_email_funnel WHERE event_date_key = 20160824 AND etl_source_database *not* in ('foo') output frag: Map Opera

Re: Hive transaction doesn't release lock.

What is your hive.txn.timeout set to and does the value as seen by the metastore process agree with what is seen by Storm client? The good news is that this race condition is no longer possible since https://issues.apache.org/jira/browse/HIVE-13013 Eugene From: Igor Kuzmenko mailto:f1she...@gm

Re: Concurrency support of Apache Hive for streaming data ingest at 7K RPS into multiple tables

Hive streaming API (which is what Storm uses) inserts multiple evens to a table per transaction. It has been designed for this but not quite ready for prime time in 0.14. Hive 1.3 has these metastore issues fixed as well as many others. HIVE-11948

SERVICE In hive.distro

Would it make sense to either export the SERVICE parameter in hive.distro or maybe pass it as input to hive-env.sh so that we can set environment parameters based on the service (hiveserver2, megastore, etc.)? If this is already happening I must be missing it in the code.

Re: Fw: Hive update operation

Him What is your current RDBMS and are these SQL the ones used in RDBMS? Have you tried them on Hive? HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Table substitution

Hi Rakesh, this is an interesting question, Did you look at the cube and rollup possibilities? https://cwiki.apache.org/confluence/display/Hive/Enhanced+Aggregation,+Cube,+Grouping+and+Rollup I believe that you could use this feature to merge your table 1 and table 2 in one single table with a

Re: Loading Sybase to hive using sqoop

Thank you all for valuable inputs On Wednesday, August 24, 2016, Mich Talebzadeh wrote: > If this is one off then Spark will do OK. > > Sybase IQ provides bcp that creates a flat file tab/comma separated and > you can use that to extract IQ table and put it on HDFS and create an > external table

Table substitution

I have multiple aggregate tables created at different levels of aggregation grouped by smaller subset of keys. As an example-Table 1 : business_unit, account, balanceTable 2 : business_unit, balance When a query is written as select business_unit, balance from table1 where business_unit='mybu';

Realm is required in beeline connection for users if display and logon name differs ,even its ensured with ldap authentciation

*Problem*: Realm is required in beeline connection for users if display and logon name differs ,even its ensured with ldap authentciation? While connecting with beeline,with configured ldap authentication some users connected without realm and some users connect with realm authentication.This is

Restricting Avro File size/records in Hive

Hi, We have data in Orc formatted table, we filter certain records and then create an Avro format hive table using the "insert into" clause. Our use case is to create smaller avro data files in a hive table that can be passed on to consumers as a Kafka Message. Can we restrict the file size in