interactive queries, Spark 3 is nearly as fast as
Hive-LLAP.
--- Sungwoo
Guys please help. I am using Hive-2.3.6 and I want to configure LLAP with
Apache slider. I ran
hive --service llap --instances 1 --size 2g --executors 2 --xmx 1024m
--cache 1024m --name llap0 --auxhbase=false --startImmediately --directory
/tmp
After that I can not get llap status
curl node2.c
nager.aux-services.tez_shuffle.class
org.apache.tez.auxservices.ShuffleHandler
* On hive-llap I am getting below error:-*
TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) :
attempt_1594912627497_0009_1_01_02_3:java.lang.RuntimeException:
java.lang.RuntimeException: M
Hey,
Forgot the question, LLAP try automatically to restart the missing
container/daemon.
Sorry for the blast,Michel
Le mercredi 17 juin 2020 à 12:12:08 UTC+1, Michel Sumbul
a écrit :
Hi guys,
We are using Hive 3.1.0. We have a LLAP cluster of few daemons.Do you know if
there is a way
Hi guys,
We are using Hive 3.1.0. We have a LLAP cluster of few daemons.Do you know if
there is a way to restart a daemon that stop/get kill (whatever for which
reason) without restarting the all llap app to avoid a downtime?
Thanks for your help,Michel
I have published a new article that compares: Hive-LLAP in HDP 3.1.4, Hive
3.1.2 on MR3 0.10, and Hive 4.0.0-SNAPSHOT on MR3 0.10. You can find the
result at:
https://mr3.postech.ac.kr/blog/2019/11/03/hive-performance-0.10/
Cheers,
--- Sungwoo
hi all,
we installed HDP 3.1 including autostart of hive server 2 and
hive interactive (llap).
I assume these services starts successfully, because
* I see two hive server processes running, using `ps x | grep java`
* netstat -punta gave me back the port number 10500 in state LISTEN
* the app mas
I'm playing around with Hive 3.1 in the newest HDP 3.1 release and I'm trying
to identify all the containers that are being launched. I was expecting one
large container for the llap service and one tez container for the single
concurrent query but I'm also seeing a third tez container running i
Hi
We are trying to improve our LLAP performance on our cluster but we've
noticed that event though LLAP daemon containers get configured memory,
they get only 1 vcore per container.
We are running 10 LLAP deamons using Slider. There are no other containers
running on the nodes that run LLAP daemon
Thank you for reading the article. I plan to publish the result of running
concurrent queries with the release of MR3 0.4 (which implements a feature
affecting the performance of concurrent queries).
--- Sungwoo
On Sat, Sep 8, 2018 at 8:06 AM Nicolas Paris
wrote:
>
> On Thu, Aug 16, 2018 at 10:
On Thu, Aug 16, 2018 at 10:55:19PM +0900, Sungwoo Park wrote:
> The article compare the following six systems:
Great article, as usual. Would have been great to also compare
concurrent queries. In particular, I guess presto on that point perform
the best. That metric is major since such technol
support for Hive 3.0.0 on Hadoop 2.7/2.8/2.9. I have also published
> a blog article that uses the TPC-DS benchmark to compare the following six
> systems:
>
> 1) Hive-LLAP included in HDP 2.6.4
> 2) Presto 0.203e
> 3) Spark 2.2.0 included in HDP 2.6.4
> 4) Hive 3.0.0 on Tez
Hello Hive users,
I am pleased to announce the release of MR3 0.3. A new feature of MR3 0.3
is its support for Hive 3.0.0 on Hadoop 2.7/2.8/2.9. I have also published
a blog article that uses the TPC-DS benchmark to compare the following six
systems:
1) Hive-LLAP included in HDP 2.6.4
2) Presto
> When LLAP Execution Mode is set to 'only' you can't have a macro and window
> function in the same select statement.
The "only" part isn't enforced for the simple select query, but is enforced for
the complex one (the PTF one).
> select col_1, col_2 from macro_bug where otrim(col_1) is not n
> When LLAP Execution Mode is set to 'only' you can't have a macro and
window function in the same select statement.
The "only" part isn't enforced for the simple select query, but is enforced
for the complex one (the PTF one).
> select col_1, col_2 from macro_bug where o
Trying to figure out if the following is a bug or expected behavior. When LLAP
Execution Mode is set to 'only' you can't have a macro and window function in
the same select statement. However you can have a macro or a window function
without an issue. Below is a test case.
use default;
create
> Caused by:
> org.apache.hadoop.hive.ql.exec.mapjoin.MapJoinMemoryExhaustionError:
> VectorMapJoin Hash table loading exceeded memory limits.
> estimatedMemoryUsage: 1644167752 noconditionalTaskSize: 463667612
> inflationFactor: 2.0 threshold: 927335232 effectiveThreshold: 927335232
Most lik
Hi, I am trying to run one of the TPC-DS queries, query25 against a 25GB
dataset (my instance memory size is 64GB) and I get this error,
Caused by: org.apache.hadoop.hive.ql.exec.mapjoin.MapJoinMemoryExhaustionError:
VectorMapJoin Hash table loading exceeded memory limits. estimatedMemoryUsage:
rt LLAP through slide
>
>
> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/intro_new_book.html
>
> Regards
>
> Rajesh
>
> *From:* Premal Shah [mailto:premal.j.s...@gmail.com]
> *Sent:* Monday, September 11, 2017 11:29 PM
> *T
[mailto:premal.j.s...@gmail.com]
Sent: Monday, September 11, 2017 11:29 PM
To: user@hive.apache.org
Subject: Re: Hive LLAP service is not starting
Hi Rajesh,
I thought the recommended way to start/stop LLAP was to use Slider
this is what we do to start
slider create llap --resources $BASEDIR
> java.util.concurrent.ExecutionException: java.io.FileNotFoundException:
> /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory)
LLAP expects to find a tarball where tez.lib.uris is - looks like you've got a
directory?
Cheers,
Gopal
destroy llap
On Mon, Sep 11, 2017 at 5:36 AM, Rajesh Narayanan <
rajesh.r.naraya...@oracle.com> wrote:
> Hi All,
>
> I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to
> execute hive query in yarn as TEZ type.
>
> When I start hive LLAP service -./hi
Hi All,
I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to
execute hive query in yarn as TEZ type.
When I start hive LLAP service -./hive --service llap --name @llap --instances
1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG --args " -XX:+UseG1GC
The parquet orc thing has to be tje biggest detractor. Your forced to chose
between a format good for impala or good for hive.
On May 4, 2017 3:57 PM, "Gopal Vijayaraghavan" wrote:
> Hi,
>
>
> > Does Hive LLAP work with Parquet format as well?
>
>
>
> LLAP d
Hi,
> Does Hive LLAP work with Parquet format as well?
LLAP does work with the Parquet format, but it does not work very fast, because
the java Parquet reader is slow.
https://issues.apache.org/jira/browse/PARQUET-131
+
https://issues.apache.org/jira/browse/HIVE-14826
In particular
Hi everyone,
I posted question on SO http://stackoverflow.com/qu
estions/43771050/hive-llap-doesnt-work-with-parquet-format but it didn't
get any love, so I am posting here.
Basically, I have large IoT data stored in Parquet format. I want to enable
faster access to this data. I started
ing YARN/Slider directly, bypassing
> Ambari, if that’s what you want to avoid.
>
> From: Sergey Shelukhin
> Date: Wednesday, February 22, 2017 at 17:03
> To: "user@hive.apache.org"
> Subject: Re: Hive LLAP
>
> Hi.
> While it’s theoretically possible it’s no
o:user@hive.apache.org>>
Subject: Re: Hive LLAP
Hi.
While it’s theoretically possible it’s not really a supported scenario, esp.
not for production use where we rely on Slider for packaging, failure recovery,
etc.
Is there a reason why you don’t want to use Slider?
From: Oleksiy S
mailto:o
"user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>
Date: Wednesday, February 22, 2017 at 00:34
To: "user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>
Subject: Re: Hive LLAP
Hi all!
It is a goo
Hi all!
It is a good question. Please help!
Oleksiy.
On Tue, Feb 21, 2017 at 5:19 PM, Vlad Gudikov wrote:
> Hi everyone,
>
> Recently I was looking for some guides to configure LLAP. I've found out
> that it's possible to setup LLAP using Apache Slider. Is there any
> possibility of setuping L
Hi everyone,
Recently I was looking for some guides to configure LLAP. I've found out
that it's possible to setup LLAP using Apache Slider. Is there any
possibility of setuping LLAP without using Slider? Maybe configuration
guide or something like this.
Thanks in advance,
Vlad
For record,
We had the same issue in my company. We use Ambari and HDP. Ambari in HDP
2.5.3 doesn't provide well some configuration keys.
My advice for HDP users: checks the xml configuration files which Ambari
generate.
Regards,
Damien
2017-02-16 22:32 GMT+01:00 Mistry, Jigar :
> Hi Gopal,
>
Hi Gopal,
Thanks for your response. I was indeed a configuration issue. I was able to
make it work by using something like this:
hive --service llap --name llap_test\
--instances 1\
--cache 2000m\
--executors 10\
--iothreads 10\
--size 5000m\
--xmx 2000m\
--loglevel WARN\
--args "-XX:+Us
> at
>org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl$DynamicServiceInstance.getResource(LlapZookeeperRegistryImpl.java:474)
Most likely incorrect configuration.
hive.llap.daemon.num.executors
or
hive.llap.daemon.memory.per.instance.mb
Are you using something
Hello,
I am testing out Hive’s LLAP functionality. My test environment is as follows:
- Hadoop 2.7.3
- Hive 2.1.1
- Tez 0.8.4
- Zookeeper 3.4.8
- Slider 0.91
I am able to get the LLAP daemon running successfully and can see it in the RM
UI. But w
35 matches
Mail list logo
