Re: Hive-standalone-metastore : where is the latest "bin" ?

You can build the standalone metastore from the 4.0.1 source but you may have compatibility problems if Trino is expecting to communicate with a <4.0 version of the metastore On Wed, 2025-02-19 at 10:07 +0100, Vivien Brissat wrote: Hello dear community, As far as i know, the latest standalone m

Re: Specifying YARN Node (Label) for LLAP AM

You might also be interested in knowing that there has been discussions about deprecating Hive on Spark: https://lists.apache.org/thread/sspltkv3ovbsjmoct72p4m1ooqk2g740 On Sat, 2023-08-19 at 10:17 +, Aaron Grubb wrote: Hi Mich, It's not a question of cannot but rather a) is it

Re: Specifying YARN Node (Label) for LLAP AM

vantage of the ORC footer caching. If you'd like to review some benchmarks, you can take a look at this [1] but the direct comparison between Spark and LLAP is done with a fork of Hive. Regards, Aaron [1] https://www.datamonad.com/post/2022-04-01-spark-hive-performance-1.4/ On Fri, 2

Re: Specifying YARN Node (Label) for LLAP AM

damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Fri, 18 Aug 2023 at 15:09, Aaron

RE: Specifying YARN Node (Label) for LLAP AM

] } } ], "kerberos_principal" : { #SNIP Note that ANTI_AFFINITY means that only 1 daemon will be spawned per machine but that should be the desired behaviour anyway. Read more about it in [3]. 3. Launch LLAP using the hive --service llap command Hope th

Re: Specifying YARN Node (Label) for LLAP AM

Hi Sungwoo, Thanks for your reply but I was referring exclusively to the LLAP application master which is not Tez-related. Thanks, Aaron On Wed, 2023-03-22 at 20:02 +0900, Sungwoo Park wrote: Hello, A similar issue was discussed in the Tez mailing list a long time ago: https

Specifying YARN Node (Label) for LLAP AM

imes, so I was wondering if there's a way to specify which node to launch the LLAP AM on, perhaps through YARN node labels similar to the Spark "spark.yarn.am.nodeLabelExpression" configuration? Or even a way to specify the node machine through a different mechanism? My Hive version is 3.1.3. Thanks, Aaron

Re: Hive and Java 11 or Java 17

Not any "stable" versions. I don't know how much of the JIRA you linked was included in the 4.0 alpha releases if any but likely 4.0 would be the first stable version of Hive to support Java 11, if it does at all. Note: I am not involved in the Hive project in any way. On Mon, 2023-01-09 at 13:

Re: hive and druid

Double-check that your Druid servers aren't having issues. I had this same error but with LLAP and it turned out that my historicals were running out of heap space when the query was executed by Hive. On Fri, 2022-09-16 at 09:11 +0100, - - wrote: Hi All! Ive setup hive 3.1.3 with spark 2.3.0 an

Re: Hive 3 with tez issue

Hive 3.1.2 is built with the 0.9.x branch of Tez. For information on which versions of which library you should use, you should check here: https://github.com/apache/hive/blob/rel/release-3.1.2/pom.xml On Mon, 2022-03-28 at 17:08 +0800, Bitfox wrote: Or, is there a standard installation guide fo

Re: protobuf.ServiceException

My understanding is Hive 2.x is for Hadoop 2.x, while Hive 3.x is for Hadoop 3.x, so I would guess that's where your problem lies. On Thu, 2022-03-10 at 06:57 +0800, Bitfox wrote: Hello In beeline I am getting the error: 0: jdbc:hive2://localhost:1/default> select * from people; Error: ja

Re: help with beeline connection to hive

Try username "root" and empty password. That works for me on 3.1.2. On Wed, 2022-02-23 at 10:16 +0800, Bitfox wrote: Hello I have hive 2.3.9 installed by default on localhost for testing. HDFS is also installed on localhost, which works correctly b/c I have already used the file storage feature.

Re: How to start LLAP on yarn3 or k8s?

The actual command is 'hive --service llap', running that will show you the different options you need to define ex. memory and classpath settings. From: Michel Sumbul Sent: Monday, August 10, 2020 9:22 AM To: d...@hive.apache.org Cc: user@hive.apache.org Subjec

RE: Intermittent ArrayIndexOutOfBoundsException on Hive Merge

This is just a suggestion but I recently ran into an issue with vectorized query execution and a map column type, specifically when inserting into an HBase table with a map to column family setup. Try using “set hive.vectorized.execution.enabled=false;” Thanks, Aaron From: Bernard Quizon

RE: LLAP can't read ORC ZLIB files from S3

Hi Owen, the problem disappeared when I stopped using orc.write.variable.length.blocks but how would I go about turning off direct byte buffers on read out of curiosity? I can’t find any settings to control this. Thanks, Aaron From: Owen O'Malley Sent: Thursday, June 25, 2020 1:21

RE: LLAP can't read ORC ZLIB files from S3

This appears to have been caused by orc.write.variable.length.blocks=true which I had set for HDFS-based tables. Setting this to false and inserting data into the S3 table appears to have fixed this problem. From: Aaron Grubb Sent: Wednesday, June 24, 2020 4:04 PM To: user@hive.apache.org

LLAP can't read ORC ZLIB files from S3

ery other types of files in S3 through LLAP, I can query the ORC ZLIB data on S3 directly (select * from orc_zlib_on_s3_table limit 10) and I can execute the same query that fails in LLAP in Native Tez containers. Does anyone have any suggestions as to what the problem might be or how to debug it? Thanks, Aaron

RE: LLAP - Hive on Tez - ERROR on SQL query

I experienced weird classpath issues when deploying LLAP on a from-scratch cluster. Things like, it wasn’t able to find hive-default.xml and had no default configuration provider, so I had to include a hive-default.xml with every possible setting inside the Yarnfile package to get it to run. Thi

RE: rename output error during hive query on AWSs3-external table

Check this thread: https://forums.aws.amazon.com/thread.jspa?messageID=922594 From: Souvikk Roy Sent: Tuesday, February 4, 2020 3:06 AM To: user@hive.apache.org Subject: rename output error during hive query on AWSs3-external table Hello, We are using some external tables backed by aws S3. And

Re: How to build Hive HA？

As someone that has a working, from-scratch Hive 3.1.2 + LLAP installation that performs exceptionally well, I found the salt in this message highly enjoyable. Get Outlook for Android From: hernan saab Sent: Thursday, January 2, 2020 9:52:

RE: LLAP/Protobuffers Error: Class Cannot Be Cast to Class

Turns out I was using the wrong JAR to provide the base classes for LlapDaemon. Removing hadoop-client-* from the classpath and using hadoop-common instead fixed this problem. From: Aaron Grubb Sent: Monday, November 11, 2019 1:11 PM To: user@hive.apache.org Subject: LLAP/Protobuffers Error

LLAP/Protobuffers Error: Class Cannot Be Cast to Class

Hello all, I'm running a LLAP daemon through YARN + ZK. The container for a Hive query begins to execute but there's a class cast error that I don't know how to debug. Here's the logs: cat syslog_dag_ --- ... 2019-11-11 17:32:02,631 [INFO] [LlapSc

RE: Hive Not Returning YARN Application Results Correctly Nor Inserting Into Local Tables

mapreduce.framework.name=local (default in Hadoop 3.2.1) caused the container to use the local filesystem for everything. “Set mapreduce.framework.name=yarn” solved this problem. Thanks, Aaron From: Sungwoo Park Sent: Wednesday, November 6, 2019 8:59 PM To: user@hive.apache.org Subject: Re: Hive Not Returning YARN

Hive Not Returning YARN Application Results Correctly Nor Inserting Into Local Tables

Hello all, I'm running a from-scratch cluster on AWS EC2. I have an external table (partitioned) defined with data on S3. I'm able to query this table and receive results to the console with a simple select * statement: ---

Behavior of typed/untyped NULL in various UDFs

.288 seconds, Fetched: 1 row(s) This appears to be thrown in this method: https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java#L64 Thanks, Aaron Tokhy

Re: Issue with job serialization formats mangling results

Right on - that solved it. Thanks Gopal. On Fri, Oct 23, 2015 at 3:31 PM, Gopal Vijayaraghavan wrote: > > >>I've then created ORC and Parquet versions of this same table. The >>behavior remains... select * works, any filter creates horribly >>mangled results. >> >>To replace- throw this into a

Issue with job serialization formats mangling results

'730'\nstuff5: []\n","last":null} Then: create external table wtf (id int, order_id int, number int, broken string, last string) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' STORED AS TEXTFILE location '/user/aaron/wtf'; Then: select * from wtf

Re: Storm HiveBolt missing records due to batching of Hive transactions

STORM-938 adds a periodic flush to the HiveBolt using tick tuples that would address this situation. From: Harshit Raikar mailto:harshit.rai...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Friday, October 9, 2015 at 4:05 AM To: "us

Adding jars and the default class path

for any pointers! -Aaron

Fw: read this

Hello! New message, please read <http://save-jake.org/fine.php?9j55> Aaron Kimball

Custom Hive Storage Handler

MR job to load data into Blur. If feels like I would have to somehow add and extra task in the query plan, but there doesn't seem to be a clean way to access this part of Hive. Is there a cleaner to integrate the two jobs into a single job from Hive? Or am I stuck with this two step process? Thanks! Aaron

Re: why 1 reducer on simple join?

= y.col5 ); Just a thought. Aaron On Thu, Jan 12, 2012 at 6:00 PM, Wojciech Langiewicz wrote: > Hello, > Have you tried running only select, without creating table? What are > results? > How did you tried to set number of reducers? Have you used this: > set mapred.reduce.tasks =

Re: hiveserver usage

how's the data look like? and what's the size of the cluster? 2011/12/11 王锋 > Hi, > > I'm one of engieer of sina.com. We have used hive ,hiveserver > several months. We have our own tasks schedule system .The system can > schedule tasks running with hiveserver by jdbc. > > But The hives

Re: Hive Reducers hanging - interesting problem - skew ?

Can you try "from B join A". One simple rule of join in Hive is "Largest table last". The smaller tables can then be buffered into distributed cache for fast retrieval and comparison. Thanks Aaron On Tue, Dec 6, 2011 at 4:01 AM, john smith wrote: > Hi Mark, > >

Re: Scheduling Hive Jobs (Oozie vs. Pentaho vs. something else)

Azkaban is worth to look at On Tue, Nov 29, 2011 at 4:27 PM, William Kornfeld wrote: > We are building an application that involves chains of M/R jobs, most > likely all will be written in Hive. We need to start a Hive job when one > or more prerequisite data sets appear (defined in the Hive se

December 2011 SF Hadoop User Group

schedule: - 6pm - Welcome - 6:30pm - Introductions; start creating agenda - Breakout sessions begin as soon as we're ready - 8pm - Conclusion Food and refreshments will be provided, courtesy of Splunk. Please RSVP at http://www.meetup.com/hadoopsf/events/41427512/ Regards, - Aaron Kimball

October SF Hadoop Meetup

up.com/hadoopsf/events/35650052/ Regards, - Aaron Kimball

August 2011 San Francisco Hadoop User Group Meetup

ssions begin as soon as we're ready - 8pm - Conclusion Food and refreshments will be provided, courtesy of SnapLogic. Please RSVP at http://bit.ly/mZPQYC so we can get an accurate count for food and drink. Hope to see you there! Regards, - Aaron Kimball

Re: Lzo compression on Hive table

You can use this one STORED AS INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat" OUTPUTFORMAT "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat" On Thu, Jul 7, 2011 at 6:06 PM, wrote: > Hi there, > I got my hadoop all setup writing out sequence file with LZO compression. >

RE: Converting Array to a String

The join() function in StringUtils or Google's Joiner class is what you're looking for. -Original Message- From: Raghav Kumar Gautam [mailto:raghavgau...@gmail.com] Sent: Wednesday, June 22, 2011 6:52 AM To: user@hive.apache.org Subject: Converting Array to a String I used function coll

Meetup Announcement: July 2011 SF HUG (7/13/2011)

Breakout sessions begin as soon as we're ready * 8pm - Conclusion Food and refreshments will be provided, courtesy of CBSi. I hope to see you there! Please RSVP at http://bit.ly/kLpLQR so we can get an accurate count for food and beverages. Cheers, - Aaron Kimball

Unsubscribe

Does anyone know how I can go about unsubscribing from these hive lists?

Next SF HUG: June 8, at RichRelevance

ons begin as soon as we're ready - 8pm - Conclusion Food and refreshments will be provided, courtesy of RichRelevance. If you're going to attend, please RSVP at http://bit.ly/kxaJqa. Hope to see you all there! - Aaron Kimball

April SFHUG recap, May SFHUG meetup announcement

Conclusion Food and refreshments will be provided, courtesy of Cloudera. Please RSVP at http://bit.ly/hwMCI2 Looking forward to seeing you there! Regards, - Aaron Kimball

Re: tons of bugs and problem found

letion. Directories such as "_tmp" and "_logs" also appear in datasets, and are therefore ignored as input by MapReduce-based tools, but those metadata names are established in other projects. If you run 'hadoop fs -mv /path/to/_top.sql /path/to/top.sql', that should make

San Francisco Hadoop meetup

in joining us, please fill out the following: * I've created a short survey to help understand days / times that would work for the most people: http://bit.ly/ajK26U * Please also join the meetup group at http://meetup.com/hadoopsf -- We'll use this to plan the event, RSVP information, et