over in alphabetic
order, the server is up.
This is not the case in Hive 2.0.1. Is there a setting we are missing?
--
Regards,
Premal Shah.
rker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> INFO cli.LlapServiceDriver: LLAP service driver finished
>
>
>
> Thanks
>
> Rajesh
>
--
Regards,
Premal Shah.
E={"BASIC_STATS":"true","COLUMN_STATS":{"id":"true","col2":"true","*
*col3**":"true","**col4**":"true"}}, numFiles=6}*
Does this mean some stats are stored?
Any help is appreciated.
Thanx.
--
Regards,
Premal Shah.
ore expensive, happy path is faster)
>
> select count(distinct id) from ip_table;
>
> Java's hashCode() implementation is pretty horrible (& Hive defaults to
> using it). If you're seeing a high collision count, I think I might know
> what's happening here.
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
) order
> by collisions desc limit 10;
>
> And, if those show many collisions
>
> set tez.runtime.io.sort.mb=640;
> set hive.map.aggr=false;
> set tez.runtime.pipelined.shuffle=true; // this reduces failure tolerance
> (i.e retries are more expensive, happy path is faster)
>
> select count(distinct ip) from ip_table;
>
> Cheers,
> Gopal
>
>
>
>
--
Regards,
Premal Shah.
t; set hive.optimize.distinct.rewrite=true;
>
> or try a rewrite
>
> select id from accounts group by id having count(1) > 1;
>
> Both approaches enable full-speed vectorization for the query.
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
at can be done to get the hive query to run faster in hive?
--
Regards,
Premal Shah.
My bad. Looks like the thrift server is cycling through various AMs it
started when the thrift server was started. I think this is different from
either Hive 2.0.1 or LLAP.
On Mon, Mar 27, 2017 at 11:38 PM, Premal Shah
wrote:
> Hi,
> I have a thrift server to run queries using LLAP. Th
% ELAPSED TIME: 8.49 s
--
OK
Query ID = hadoop_20170328053153_8677d9d6-e748-4eb7-bfeb-1f1abdbb367c
Total jobs = 1
Launching Job 1 out of 1
--
Regards,
Premal Shah.
; set hive.tez.container.size=?
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
s
[image: Inline image 1]
When I switched the execution engine to mr, the query finished in 30 mins.
Are there any knobs we have to tweak?
--
Regards,
Premal Shah.
try what Sergey
> suggested, the '$f0' is probably the auto-generated name for the count(0).
>
> Naming that column explicitly on both branches of the UNION ALL, might get
> CBO back up.
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
to create a table from
> the results. It does not happen to all CTAS queries.
>
> Not sure if that's related to Tez at all.
>
> Can try running it with
>
> set hive.cbo.enable=false;
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
--
Regards,
Premal Shah.
Sorry,
here's the link -
https://gist.github.com/premal/d054d4cc0ed00efdf60351ca2517db3d
On Wed, Nov 2, 2016 at 8:11 PM, Premal Shah wrote:
> Hi Prasanth,
> Here's a link to the hive log4js properties file.
>
> We are on Hive 2.0.1.
>
> We cannot use 2.1.0 because
lated to https://issues.apache.org/jira/browse/HIVE-11751
>
> The debug strings gets generated but gets filtered. Can you share your
> log4j2 properties file?
>
> What version of hive are you using?
>
> Thanks
> Prasanth
>
> On Nov 2, 2016, at 5:06 PM, Premal Shah wrote:
>
>
e /tmp/$USER/hive.log and see what's happening in detail.
>
>
>
> Cheers,
>
> Gopal
>
--
Regards,
Premal Shah.
0
>
> If you are more adventurous and want to run LLAP on an unsupported
> platform, I maintain scripts which will configure and install it
>
> https://github.com/t3rmin4t0r/tez-autobuild/blob/llap/README.md
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
l? My guess is this happens only in
> DEBUG log level.
>
> Thanks
> Prasanth
>
>
>
>
> On Fri, Oct 28, 2016 at 9:40 PM -0700, "Premal Shah" <
> premal.j.s...@gmail.com> wrote:
>
> Hive 2.0.1
> Hadoop 2.7.2
> Tex 0.8.4
>
> We have a UDF in
y. We can't get tez to run the query on the cluster.
The hive shell starts with an Xmx of 4G.
If I set hive.execution.engine = mr, then the query works, because it runs
on the hadoop cluster.
What should we change to avoid this problem?
Thanx
--
Regards,
Premal Shah.
partition the tables so that the joins are faster?
--
Regards,
Premal Shah.
esn't work.
> Good luck.
>
> STORED AS orc tblproperties ("orc.compress.size"="8192");
>
>
> On Thu, May 15, 2014 at 8:11 PM, Premal Shah wrote:
>
>> I have a table in hive stored as text file with 3283 columns. All columns
>> are of string dat
Sorry for the double post. I did not show up for a while and then I could
not get to the archives page, so I thought I'd needed to resend.
On Fri, May 16, 2014 at 12:54 AM, Premal Shah wrote:
> I have a table in hive stored as text file with 3283 columns. All columns
> are of strin
505K->2249K(83008K), 0.0097240 secs]
34.779: [GC 28384K(4177280K), 0.0014050 secs]
Anything I can tweak to make it work?
--
Regards,
Premal Shah.
505K->2249K(83008K), 0.0097240 secs]
34.779: [GC 28384K(4177280K), 0.0014050 secs]
Anything I can tweak to make it work?
--
Regards,
Premal Shah.
25 matches
Mail list logo
