Fwd: Community Over Code Asia Travel Assistance Applications now open!

forwarded as asked -Ayush -- Forwarded message - From: Gavin McDonald Date: Tue, 4 Mar 2025 at 16:35 Subject: Community Over Code Asia Travel Assistance Applications now open! To: Apache Travel Assistance Committee --- The Travel Assistance Committee (TAC) are pleased to announc

Fwd: IMPORTANT: Hive date parsing issue, input needed

Since this is landing in Spam for some reason for most people, I am explicitly forwarding it rather than replying, so as people can get it. I will vote for (2) considering it was returning `null` earlier as well & the new behaviour came up recently, it would be a behaviour change for folks migrati

Fwd: Help with Docker Apache/Hive metastore using mysql remote database

Issue with Docker container using mysql RDBMS ( Failed to load driver) https://hub.docker.com/r/apache/hive According to readme Launch Standalone Metastore With External RDBMS (Postgres/Oracle/MySql/MsSql) I want to use MySQL I tried com.mysql.jdbc.Driver or com.mysql.cj.jdbc.Driver docker ru

Fwd: Release of Hive 4 and TPC-DS benchmark

Forwarded to user@hive as I think many people are curious about the release of Hive 4. -- Forwarded message - From: Sungwoo Park Date: Sat, Nov 4, 2023 at 12:42 AM Subject: Release of Hive 4 and TPC-DS benchmark To: Hi everyone, I would like to resume the discussion on the rel

Fwd: BUG ? Loss of all table data when deleting partitions performed directly on hdfs (after metastore synchro)

Hello, For a partitioned table (with an int type partition) in the following format: CREATE EXTERNAL TABLE `poc.titi`( `n` string) PARTITIONED BY ( `id` integer) SERDE LINE FORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' STORED AS INPUT FORMAT 'org.apache.hadoop.hive.ql.io.orc.

Fwd: TAC supporting Berlin Buzzwords

Forwarded as Received. -Ayush -- Forwarded message - From: Gavin McDonald Date: Fri, 24 Mar 2023 at 15:27 Subject: TAC supporting Berlin Buzzwords To: PMCs, Please forward to your dev and user lists. Hi All, The ASF Travel Assistance Committee is supporting taking up to six

Fwd: How useful are tools for Hive data modeling

Hi all, I wrote these notes earlier this year. I heard today that someone mentioned Hive 1 does not support indexes but hive 2 does. I still believe that Hive does not support indexing as per below. Has this been changed? Regards, Mich -- Forwarded message - From: Mich Ta

Fwd: Is Hive accepting patches for 2.3.x?

Hey Shashank, Please check Zoltans' response below -- you might have missed it if you are not subscribed to the dev mailing-list. Cheers, Panagiotis -- Forwarded message - From: Zoltán Haindrich Date: Sun, May 10, 2020 at 12:20 PM Subject: Re: Is Hive accepting patches for 2.3.x

Fwd: Alternatives to Streaming Mutation API in Hive 3.x

Hi Hive Team, I have a use case where data has to be continuously streamed into a hive table as well as old records with the same primary key have to be updated/deleted (i.e. upserts). This document for 'HCatalog Str

Fwd: unsubscribe

unsubscribe

Fwd: Article on the correctness of Hive on MR3, Presto, and Impala

I have published a new article on the correctness of Hive on MR3, Presto, and Impala: https://mr3.postech.ac.kr/blog/2019/06/26/correctness-hivemr3-presto-impala/ Hope you enjoy reading the article. --- Sungwoo

Fwd: Announce: MR3 0.8 released

I am pleased to announce the release of MR3 0.8. New features are: -- Hive on MR3 on Yarn fully supports recovery: https://mr3.postech.ac.kr/hivemr3/features/recovery/ -- Hive on MR3 on Yarn supports high availability in which multiple HiveServer2 instances share a common DAGAppMaster (and a comm

Fwd: Hive Insert and Select only specific columns ( not all columns ) - Partitioned table

Is it possible to Insert only specific columns in Hive ( Partitioned tables ) ? For Non partitioned tables, it is working as below. insert into emp (empno,sal) select empno,sal from emp_hist.; --- This is working ( emp_hist table is having empno,ename,sal,job_desc columns, I am able to

Fwd: Hive generating different DAGs from the same query

Hello Zoltan, I further tested, and found no Exception (such as MapJoinMemoryExhaustionError) during the run. So, the query ran fine. My conclusion is that a query can update some internal states of HiveServer2, affecting DAG generation for subsequent queries. Moreover, the same query may or may n

Fwd: Serde moved? version 2.3.0

i see. interesting. i think this breaks a ton of opensource Serde's we've all downloaded off the internet and have been using for years. openx json serde and ibm xml serde come to mind. does this change render all these incompatible now with 2.3.0? I mean all i done in the past is download the j

Fwd: How to optimize multiple count( distinct col) in Hive SQL

-- Forwarded message -- From: panfei Date: 2017-08-23 12:26 GMT+08:00 Subject: Fwd: How to optimize multiple count( distinct col) in Hive SQL To: hive-...@hadoop.apache.org -- Forwarded message -- From: panfei Date: 2017-08-23 12:26 GMT+08:00 Subject: How to

Fwd: Pro and Cons of using HBase table as an external table in HIVE

Sorry. Need to send via right email address. Begin forwarded message: From: Michael Segel mailto:mse...@segel.com>> Subject: Re: Pro and Cons of using HBase table as an external table in HIVE Date: June 9, 2017 at 7:37:22 AM CDT To: user@hive.apache.org Hey Edward,

Fwd: Error deploying LLAP using Apache Slider

Hi everyone, I'm trying to deploy LLAP using Slider but the containers fail to start due to this error: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at org.apache.hadoop.service.AbstractService.(AbstractService.java:43) Caused by: java.lang

Fwd: Why spark.sql.autoBroadcastJoinThreshold not available

Solve it by remove lazy identity. 2.HiveContext.sql("cache table feature as "select * from src where ...) which result size is only 100K -- Forwarded message -- From: Jone Zhang Date: 2017-05-10 19:10 GMT+08:00 Subject: Why spark.sql.autoBroadcastJoinThreshold not available To: "u

Fwd: Hive LLAP with Parquet format

Hi everyone, I posted question on SO http://stackoverflow.com/qu estions/43771050/hive-llap-doesnt-work-with-parquet-format but it didn't get any love, so I am posting here. Basically, I have large IoT data stored in Parquet format. I want to enable faster access to this data. I started Azure HDI

Fwd: Enabling Tez-UI on CDH 5.8

Trying hive user group. Has anyone set Tez UI on CDH? If yes, what were the steps? Appreciate any help. -- Forwarded message -- From: Manoj Murumkar Date: Thu, Feb 16, 2017 at 12:02 PM Subject: Enabling Tez-UI on CDH 5.8 To: u...@tez.apache.org Hi, Wondering if anyone has succe

Fwd: Mapping of Spatial functions in HIVE

-- Forwarded message -- From: Ashish Agarwal Date: Fri, Jan 27, 2017 at 4:06 PM Subject: Mapping of Spatial functions in HIVE To: user@hive.apache.org We are in the process of migrating from traditional RDMS technologies to HIVE and in the process we have few questions on HIVE s

Fwd: Support of Theta Join

Is there any support of Theta Join in Spark. We have a requirement to identify the country name based on Range of IP Address in a table. Forwarded Message Subject:Support of Theta Join Date: Thu, 12 Jan 2017 15:19:51 + From: Mahender Sarangam

Fwd: Hive Serialization issues

Hey All, I am using Hive 2.0 with external meta-store on EMR-5.0.0 and TEZ as execution engine. Our data are stored in json format so for serialization and deserialization purpose we are planning to use lazy serde (classname is 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' ). My table def

Fwd: Rest API Apache Hive

Forwarded Message Subject:Rest API Apache Hive Date: Mon, 24 Oct 2016 16:42:53 +0700 From: Hardika Catur S To: user@hive.apache.org Hi, I try create Rest API Apache Hive, but but I find it difficult to make. My documentation from the link https://www.npmjs

Fwd: need help with hive session getting killed

Hello, I am using hive-jdbc-1.2.1 to run a query. Query runs around an hour and eventually completes. But my hive session terminats before I can get results from completed query and job fails. *java.sql.SQLException: org.apache.http.NoHttpResponseException: The target server failed to respond* *

Fwd: Some time realm is required for some users in beeline connection,even its ensured with ldap authentciation.!

Hi, FYI, Thanks, Mathes -- Forwarded message -- From: mathes waran Date: Wed, Aug 17, 2016 at 6:02 PM Subject: Re: Some time realm is required for some users in beeline connection,even its ensured with ldap authentciation.! To: d...@hive.apache.org Hi , While connecting with be

Fwd: Some time realm is required for some users in beeline connection,even its ensured with ldap authentciation.!

Hi , FYI, Thanks, Mathes -- Forwarded message -- From: mathes waran Date: Tue, Aug 16, 2016 at 10:32 AM Subject: Some time realm is required for some users in beeline connection,even its ensured with ldap authentciation.! To: d...@hive.apache.org Hi , Problem: Why realm is req

Fwd: Building Spark 2 from source that does not include the Hive jars

Anyone in Hive forum knows about this? Thanks This has worked before including 1.6.1 etc Build Spark without Hive jars. The idea being to use Spark as Hive execution engine. There is some notes on Hive on Spark: Getting Started

Fwd: can't start up hive 2.1 hiveserver2/metastore services

We download hive 2.1 and run into errors when starting to metastore/hiveserver2 services, here is the errors, Any clues? Regards, Qiuzhuang [image: Inline image 1]

Fwd: [ANNOUNCE] Apache Hive 2.0.1 Released

Thanks Sergey, Congratulations. May I add that Hive 0.14 and above can also deploy Spark as its executions engine and with Spark on Hive on Spark execution engine you have a winning combination. BTW we are just discussing the merits of TEZ + LLAP versus Spark as the execution engine for Spark. W

Fwd: How to run large Hive queries in PySpark 1.2.1

Hi PySpark users, We need to be able to run large Hive queries in PySpark 1.2.1. Users are running PySpark on an Edge Node, and submit jobs to a Cluster that allocates YARN resources to the clients. We are using MapR as the Hadoop Distribution on top of Hive 0.13 and Spark 1.2.1. Currently, our

Fwd: HIVE Installation Error

Hello All, I am trying to run HIVE over Hadoop on my Windows platform. I want to run beeline but as soon as I type beeline under my Cygwin Terminal it gives me an error, this one Exception in thread "main" java.lang.NoSuchMethodError: org.fusesource.jansi.internal.Kernel32.GetConsoleOutputCP()I

Fwd: Would like to be a user

Hi I would like to start working on (write to) apache hive. My confluence user name is ghaspert. thanks -- Forwarded message -- From: gail haspert Date: Mon, May 16, 2016 at 12:40 PM Subject: Would like to be a user To: user-subscr...@hive.apache.org I just signed up as ghaspert

Fwd: numrows is incorrect after renaming external table

Hello, I created a external non-partitioned table using Impala shell and inserted few rows into it using hive. While inserting, I can see basic stats like numRows, numFiles, size etc getting generated something like Table tempdb_mani.mk_prod_inc1 stats: [numFiles=1, numRows=5, totalSize=614, rawD

Fwd: Hive 0.14 schema evolution for orc table

How does schema evolution work for orc table in hive 0.14? Does it support add column at the end of the row only? What else is supported?

Fwd: Inserting into a Hive transactional table using Java code

Hive 2:00 Spark engine 1.3.1 Eclipse Scala IDE build of Eclipse SDK A simple routine that reads CSV files from a staging directory by creating an external table and insert into an ORC transactional table. Using beeline from another server the task finishes pretty quickly. The code is shown below

Fwd: Inserting into a Hive transactional table using Java code

Hive 2:00 Spark engine 1.3.1 Eclipse Scala IDE build of Eclipse SDK A simple routine that reads CSV files from a staging directory by creating an external table and insert into an ORC transactional table. Using beeline from another server the task finishes pretty quickly. The code is shown in att

Fwd: Which one should i use for benchmark tasks in hive & hadoop

Hi I am comparing performance of pig and hive for weblog data. I was reading this pig and hive benchmarks. In which one statement written on page 10 that "The CPU time required by a job running on 10 node cluster will (more or less) be the same than the time required to run the same job on a 1000

Fwd: Hive Struct Type Restrictions.

Hi All, I am trying to create a table with multiple STRUCT column types, one of which having more than 300 fields. I get the table created with no problem, but when i load i face few errors in loading up the data.when i check SHOW CREATE TABLE TABLENAME, i get fields only till 266th column and re

Fwd: Wrong column is picked in HIVE 2.0.0 + TEZ 0.8.2 left join

+ hive mail list, more likely hive issue. -- Forwarded message -- From: GAO Chi Date: Tue, Mar 1, 2016 at 12:24 PM Subject: Wrong column is picked in HIVE 2.0.0 + TEZ 0.8.2 left join To: u...@tez.apache.org Hi all, We encountered a strange behavior after upgrading to HIVE

Fwd: Unable to create external table on large S3 bucket using S3A

Hi, I am trying to create an external table on a S3 bucket, however I'm receiving the following error in the process: hive> CREATE EXTERNAL TABLE ping_prod > PARTITIONED BY(day string) > ROW FORMAT SERDE > 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' > STORED AS INPUTFORMAT

Fwd: sqoop import --hive-import failing for row_version (datatype: timestamp) column for MS SQL Server

Hi, I am trying to import MS SQL Server data into Hive using Sqoop import --hive-import option but it is failing for a column of datatype 'timestamp'. 16/01/20 04:50:53 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive does not support the SQL type for c

Fwd: Hive Decimal did not parse value correctly when a space pre-pend the value

Hi, all Hive decimal can correctly parse value like "1245" or "-4564". But it failed to parse " 12345" (note: there is a space before 12345). Do anyone met the same problem like this? I also test impala, impala can parse " 12345" as decimal correctly. 谢谢 金杰 (Jie Jin)

Fwd: Conversion

Hi, We have a string column that represents an array of doubles that looks like this: f7 ad 3b 38 89 b7 e5 3f a1 c1 1a 74 db To parse it we use unhex(translate(signalvalues,' ','')) which returns a BINARY value. How can we convert it to ARRAY ? Thank you. Daniel

Fwd: Using python for hive table quering gives error

I am trying to query hive table with basic example code: *import pyhs2* *with pyhs2.connect(host='dmet-master05.inetu.net ',* * port=1,* * authMechanism='PLAIN',* * user='karim',* * passwor

Fwd: problem with hive.reloadable.aux.jars.path

Hi, I wanted to use hiveserver without restarting it for every auxiliary jar change. According to https://issues.apache.org/jira/browse/HIVE-7553, I switched jars with udfs in folder from the path specified in the hive.reloadable.aux.jars.path. I executed command reload via the beeline. It turne

Fwd: Failed to create HiveMetaStoreClient object in proxy user with Kerberos enabled

Hi, I wrote a Java client to talk with HiveMetaStore. (Hive 1.2.0) But found that it can't new a HiveMetaStoreClient object successfully via a proxy using in Kerberos env. === 15/10/13 00:14:38 ERROR transport.TSaslTransport: SASL negotiation failure javax.security.sasl.Sas

Fwd: Hive showing SemanticException [Error 10002]: Line 3:21 Invalid column reference 'mbdate

Hello, Can some please help. When I execute hive query with as case statement I get this error " Error while compiling statement: FAILED: SemanticException [Error 10002]: Line 3:21 Invalid column reference 'mbdate' Here is the query : select a.mbcmpy, a.mbwhse, a.mbdept, a.mbitem, (CASE WHEN t

Fwd: hive.HiveImport: FAILED:

Hi All, I'm running a sqoop job for a table to import into Hive from mysql, sometimes its works fine.but some time throw this Error: * INFO hive.HiveImport: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException:

Fwd: Hive and Spark on Windows

Hello, I would like to install Hive with Spark on Windows, I've already installed Spark, but I cant find a clear documentation on how to configure hive on windows with spark. Regards -- Ing. Ivaldi Andres

Fwd: Failed to execute command on hive - Relative path in absolute URI

-- Forwarded message - From: IT CTO Date: Tue, Sep 29, 2015 at 10:51 AM Subject: Failed to execute command on hive - Relative path in absolute URI To: After connecting to hive both using the shell or beeline I am getting the following error on every command I am making What is w

Fwd: Hive Server Load Data InPath Fails

Hi All, I am making a Hive Thrift connection to Hive Server and with load data inpath command to one of my table I am landing into some bizzare exception. Other queries seems out to be working fine, but this is the only query which is getting failed 15/09/11 11:40:32 ERROR utils.HCatalogUtil: Ex

Fwd: java.lang.NullPointerException when using INSERT INTO ORC table with ACID property

Hi , We have a scenario to update a table from different set of tables for which we are using hive 0.14.Here are the steps that we are following 1.Parent table is ORC table with ACID property. 2.Delta table has updated records that need to be inserted to parent table(Delta table is also an ORC wi

Fwd: ORDER BY clause in Hive

-- Forwarded message -- From: Lefty Leverenz Date: Tue, Mar 31, 2015 at 1:47 AM Subject: Re: ORDER BY clause in Hive To: Mich Talebzadeh Hive as I see it does not support ORDER BY *Column position*. It only > supports ORDER BY *Column name*. > That's just in Hive release 0.10.0

Fwd: ORDER BY clause in Hive

Oops, failed to send this to user@hive. -- Lefty -- Forwarded message -- From: Lefty Leverenz Date: Mon, Mar 30, 2015 at 12:42 AM Subject: Re: ORDER BY clause in Hive To: Gopal Vijayaraghavan Oho! Good point, Gopal. hive.groupby.orderby.position.alias isn't in the wiki yet, b

Fwd: Which SerDe for Custom Binary Data.

I want to set up a DW based on Hive. However, my data does not come as handy csv files but as binary files in a proprietary format. The binary file consists of serialized data using C language. Could you please suggest which input format to be used and how to write a custom SerDe for the abov

Re: Fwd: Row exception in Hive while using join

hi Swagatika, *base on further log file analysis i think problem with low disk space.* *below is full stack trace.* .. org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=1) {"key":{"joinkey0":"12"},"value":{"_col2":"."},"alias":1} at org.apache.had

Re: Fwd: Row exception in Hive while using join

Hi Krish, It seems the data corresponding to that particular row pertaining to key 12 is corrupt.Can u try reloading the data and then selecting? Let me know if it works. Regards Swagatika On Mar 5, 2015 4:45 PM, "krish" wrote: > > I got the following exception while executing join on Hive Quer

Fwd: Row exception in Hive while using join

I got the following exception while executing join on Hive Query and reducer hang after 68% completion. java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=1) {"key":{"joinkey0":"12"},"value":{"_col2":"rs317647905"},"alias":1}

Re: Fwd: Custom Input Format for Sequence Files

Hello I think you must add in SequenceFile that line terminated by not new line '/n' but char that indicate EOF (end of file). Try and let share your expérience ;) Good luck Le 13 févr. 2015 17:59, "Varsha Raveendran" a écrit : > > Hello! > > I have csv files which are small in size which are mov

Fwd: Custom Input Format for Sequence Files

Hello! I have csv files which are small in size which are moved to the HDFS using the SequenceFile Format. The key is the file name and contents of the file becomes the value. Now I want to create an external table on these csv files using HIVE. But when I do I get only the first row of each csv

Fwd: Exception with the window function

Any one has insights on the following issue? Where would be the best place to make the changes? I have tried several approaches but it breaks some other queries. Basically, the inner query produces some additional columns (from window function) not consumed by the outer query. The query plan s

Fwd: Case inside select statement in hive

I use hive 0.12.0. -- Forwarded message -- From: Gayathri Swaroop Date: Tue, Dec 16, 2014 at 10:37 AM Subject: Case inside select statement in hive To: user Hi, I have oracle query which i want to hit against a hive table. The oracle query has a case if exists select would this

Re: Fwd: Question on ORC file stripe size.

Stripe size is too low. ORC maintains multiple buffers in memory. ORC’s memory manager flushes a stripe when the in-memory data size (which includes buffers in memory) is greater than specified stripe size. This check happens after every 5000 rows.  This is what is happening in this case There

Fwd: Question on ORC file stripe size.

Hi Team, I am creating this table: CREATE TABLE IF NOT EXISTS orctest2 ( id string, id2 string, id3 string, id4 string ) STORED AS ORC tblproperties ("orc.stripe.size"="1048576","orc.row.index.stride"="”); The stripe size is set to 1MB. After loading data, the table file is about 60MB: -rwxr-

Fwd: Files Per Partition Causing Slowness

-- Forwarded message -- From: John Omernik Date: Tue, Dec 2, 2014 at 1:58 PM Subject: Re: Files Per Partition Causing Slowness To: user@hive.apache.org Thank you Edward, I knew the number of partitions mattered, but I didn't think 1000 would be to much. However, I didn't realiz

Fwd: load Hive veiw in Pig using HCatLoader

Hi, I'm trying to load Hive view into withing Pig script using org.apache.hive.hcatalog.pig.HCatLoader. I'm using Hortonworks HDP2.2 with hive-hcatalog-pig-adapter-0.14.0.2.2.0.0-1084.jar (I also tried HDP 2.1 with hive 0.13). Loading tables works perfectly. *My code is as following:* flat = LOAD

Re: Fwd: I have some errors to run hql from shell script

Hard to say what the problem is just based on the stderr; try checking hive's logs for more information: https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs Andrew On 11/4/14, 9:43 PM, Jihyun Suh wrote: I have a problem to run hql from shell script. I ca

Fwd: I have some errors to run hql from shell script

I have a problem to run hql from shell script. I can run partition query just like "use myDB; show partitions myTbl;" But I have a problem count query just like "use myDB; select count(*) from myTbl where date=20141102;" I have these error messages for it. How can I run my count hive query in she

Fwd: How to run TestCliDriver Unit Test

+ hive user group. :) -- Forwarded message -- From: Gordon Wang Date: Tue, Oct 28, 2014 at 11:22 AM Subject: How to run TestCliDriver Unit Test To: d...@hive.apache.org Hi all, I am working on a hive issue and I want to add a Unit test for hive query parser. Before hive 0.13,

Fwd: Queries : partitioning

-- Forwarded message -- From: "Dhaval Patel" Date: 22/09/2014 7:33 pm Subject: Queries : partitioning To: Cc: Hey folks, 1) Hive table can be partitioned by column, is there any way to partition by range? 2) while loading data into hive table we specify the partition column value

Fwd: UDF for MONTHS_BETWEEN

Is there currently a Hive UDF to mimic the Oracle function MONTHS_BETWEEN??

Fwd: Column Stats with parquet

I am trying to enable Column statistics usage with Parquet tables. This is the query I am executing. However on explain, I see that even though *Basic stats: COMPLETE *is seen *Column stats *is seen as* NONE.* Can someone please explain what else I need to debug/fix this. set hive.compute.query.us

Fwd: Inline query in select clause

-- Forwarded message -- From: siva kumar Date: Mon, Jul 21, 2014 at 3:52 PM Subject: Inline query in select clause To: user@hive.apache.org Hi, I have below 2 tables� 1) create table dim (rank string,grade string) row format delimited fields terminated by ',' stored as textile; D

Fwd: Need help in Date format

Hi All, I have my date format as 08-Mar-2014 how to I change it to 08-03-2014? Can I use regexp_replace. I tried below but not getting the desired output. regexp_replace(started_dt,"\Jan|\Feb|\Mar|\Apr|\May|\Jun|\Jul|\Aug|\Sep|\Oct|\Nov|\Dec","\01|\02|\03|\04|\05|\06|\07|\08|\09|\10|\11|\12") O

Fwd: Hive unit testing and MySQL driver

Hello, I'm trying to make sure that unit tests are passed on my local machine for Hive 0.13 but for ql module I get a java.lang.ClassNotFoundException: com.mysql.jdbc.Driver. I'm running the tests using line: "mvn test -Phadoop-2" How can I have the tests passed? Thank you -- Regards, Pavel Chadn

Fwd: Future date getting converted to epoch date with windowing function

Hi, I am trying to use hive windowing functions for a business use case. Hive version is Apache Hive 0.11. I have a table with a column end_date where value is 2999-12-31. While using hive windowing function with this value, Hive is converting it to 1970s date. *Query used is :* SELECT account_id

Fwd: Class not found exception from serde

I am trying to upgrade from Hive 0.10 to 0.12, but I am stuck trying to debug an error I keep getting. I have a couple of SerDes that I have developed, but when I try to query tables with these SerDes I get a ClassNotFoundException for the SerDe class. I know that the jar files have been loaded OK,

Fwd: Coonecting Apache Hive using Kerberos authentication

I want to connect Apache Hive using kerberos authentication. I've successfully connected PostgreSQL database with kerberos, and followed similar steps for Apache Hive, but getting exception. If someone has tried this, Please share the configuration files and steps. Below are the details: Connecti

Fwd: Simple UDF to return array

Can someone please suggest if this is doable or not? Is generic udf the only option? How would using generic vs simple udf make any difference since I would be returning the same object either ways. Thank you Sunita -- Forwarded message -- From: *Sunita Arvind* Date: Wednesday, J

Fwd: passing Xml using hive

Hi, This is my xml file 100 ranjini IT1 123456 nextlevel1 Chennai1 Navallur1 1001 ranjinikumar IT 1234516 nextlevel Chennai Navallur In hive shell , i use xpath hive> select xpath(str,'/Company/Employee/ename/text()') from hivexml; Total MapReduce jobs = 1 Launching Job 1 out of 1

Fwd: [ANNOUNCE] Apache Hive 0.12.0 Released

The Apache Hive team is proud to announce the the release of Apache Hive version 0.12.0. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides: * Tools to enable easy data extrac

Fwd: is that possible to use Hive 0.11 with libthrift 0.7

Hi Johnny, Sorry for the delayed response (was offline for a few days). So the code that actually uses Thrift in is the Hive core JAR, which is what will get shaded. The easiest thing to do is add this target to the root build.xml file: Notice this assumes the JAR file f

Fwd: Wikipedia Dump Analysis..

> Hello, > > > > I have Hadoop running on HDFS with Hive installed. I am able to import Wikipedia dump into HDFS through the below command: > > > > http://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles.xml.bz2 > > > > $ hadoop jar out.jar edu.umd.cloud9.collection.wikipedia.DumpWiki

Fwd: [ANNOUNCE]: Apache Sentry 1.2.0 released

Corrected version with the necessary disclaimers. -- Forwarded message -- From: Shreepadma Venugopalan Date: Wed, Sep 25, 2013 at 2:40 PM Subject: Fwd: [ANNOUNCE]: Apache Sentry 1.2.0 released To: annou...@apache.org The Apache Sentry team is happy to announce the release of

Fwd: [ANNOUNCE]: Apache Sentry 1.2.0 released

-- Forwarded message -- From: Shreepadma Venugopalan Date: Wed, Sep 25, 2013 at 1:06 PM Subject: [ANNOUNCE]: Apache Sentry 1.2.0 released To: d...@sentry.incubator.apache.org, annou...@apache.org The Apache Sentry team is happy to announce the release of Apache Sentry 1.2.0. Apa

Fwd: Outputting newlines from data

Hi all, I'm having some issues working with records containing new lines after deserialization. Some information: 1. The serialized records do not contain new lines (they are base64 encoded protobuf messages). 2. The deserialized records DO contain new lines (our SerDe base64 decodes them and e

Fwd: Access

-- Forwarded message -- From: Ellen Evans Date: Wed, Sep 4, 2013 at 3:11 PM Subject: Access To: user@hive.apache.org Hi - My name is Ellen Evans and I am a tech writer at Hortonworks. I've been asked to do some work on the Hive wiki. Currently I am supposed to update the Date d

Re: Fwd: Need urgent help in hive query

> *To:* user@hive.apache.org > *Sent:* Friday, June 28, 2013 10:58 AM > *Subject:* Fwd: Need urgent help in hive query > > Hi Experts, > I'm trying with the below SQL query in Hive, which does not support column > alias access in subsequent columns as shown below in the query. I

Re: Fwd: Need urgent help in hive query

Just copy and paste the whole long expressions to their second occurrences. From: dyuti a To: user@hive.apache.org Sent: Friday, June 28, 2013 10:58 AM Subject: Fwd: Need urgent help in hive query Hi Experts, I'm trying with the below SQL query in

Fwd: Need urgent help in hive query

Hi Experts, I'm trying with the below SQL query in Hive, which does not support column alias access in subsequent columns as shown below in the query. Is there any other way to rewrite the same without using alias? any of your help are really appreciated. INSERT INTO CAUD ( pst_cc pst_no

Fwd: FileNotFoundException when using hive local mode execution style

Hi all, I plan to use hive local in order to speed-up unit testing on (very) small data sets. (Data is still on hdfs). I switch the local mode by setting the following variables : SET hive.exec.mode.local.auto=true; SET mapred.local.dir=/user; SET mapred.tmp.dir=file:///tmp; (plus creating neede

Fwd: DesignLounge @ HadoopSummit

Begin forwarded message: > From: Eric Baldeschwieler > Date: June 11, 2013 10:46:25 AM PDT > To: "common-...@hadoop.apache.org" > Subject: DesignLounge @ HadoopSummit > Reply-To: common-...@hadoop.apache.org > > Hi Folks, > > We thought we'd try something new at Hadoop Summit this year to bu

Fwd: Issue loading linkedin jobsearch JSON response into HIVE

Hi All, I have tried multiple ways to create the HIVE table and retrieve data using JSONSerDe. But here are the errors I encounter: hive> select * from jobs; OK Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: j ava.io.EOFException: No content to map to O

Fwd: HiveInputSplit not found

Hi, My hive jobs are failing with the below error. Haven't been able to figure this one out. Can you please throw some light on this? java.io.IOException: Split class org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit not found at org.apache.hadoop.mapred.MapTask.getSplitDetails

Fwd: Hadoop In Seoul 2013 Conference Calls For Speakers

Begin forwarded message: > From: "Edward J. Yoon" > Date: May 21, 2013 1:29:06 AM PDT > To: gene...@hadoop.apache.org > Subject: Hadoop In Seoul 2013 Conference Calls For Speakers > Reply-To: gene...@hadoop.apache.org > > Hi, > > I'm planning the Hadoop In Seoul 2013 Open Conference with some

Fwd: How to read data in column wise (readColumnsBatch())

I have the below code that can read data row wise private void readSpecificColumns() throws IOException, SerDeException { ColumnarSerDe serDe = new ColumnarSerDe(); serDe.initialize(conf, createProperties()); // long start = System.currentTimeMillis(); java.util.A

Fwd: Redundancy, load balancing with HCatalog/WebHCat

We are looking into running multiple instances of HCatalog/WebHCat on different servers to achieve some sort of redundancy. From some tests it appears that multiple nodes seem to be synchronized through Zookeeper, and since data is stored in HDFS each node doesn't rely on its own state. So if a HC

Fwd: Hive accessing metastore

Sent from my iPhone Begin forwarded message: > From: Siddharth Tiwari > Date: 21 March 2013 7:43:41 AM MST > To: hive user list > Subject: Hive accessing metastore > > Hi team, > > What we have observed is, hive creates so many connections to metastore > (mysql). This makes us change the n

  1   2   >