Re: Re: Migrating Hive 3 to Hive 4

Hi Zhao, 1) Just copying acid files from source to target won't work. it needs associated metadata as well. You can use REPL DUMP (at source) & REPL LOAD (at target) to copy ACID tables from one cluster to another. You can find more information in this doc. https://hive.apache.org/docs/latest/hi

Re: Re: Migrating Hive 3 to Hive 4

Hive 3 cannot read ACID tables created by Hive 4, as Hive 4 is forward compatible only, not backward compatible.

Re: Re: Performance evaluation of Trino 468, Spark 4.0.0-RC2, and Hive 4 on Tez/MR3

Evaluating Hive4-LLAP can be of interest to many users in this mailing list, but it's a lot of work and we are not sure if we can finish parameter tuning to achieve the best performance. For Hive-Tez, DAGAppMaster is reused across queries. Only worker containers are not reused across queries. ---

Re: Re: Blog article 'Performance Tuning for Single-table Queries'

Hello, The blog post is informative but examines the performance of one single query thus it may not be enough to justify a change in the default configuration. If there are more comprehensive benchmarks that show that changing the default value of hive.optimize.reducededuplication is beneficial f

Re: Re: Re: Merge Operation Failing Results in this SQL Error [40000] [42000]

Hi Clinton, I'd like you to start a new thread about the problem with MV. It is far from the original syntax problem, and MV is a fairly advanced feature. A new thread with a proper title would attract more appropriate people. You can also present the minimal SQLs that reproduce the problem on th

Re: Re: Merge Operation Failing Results in this SQL Error [40000] [42000]

Thanks Lisoda for those insights. @Okumin , this is what I observed when checking the log files. Attached is a log file and the hive-site.xml file configuration. I have observed this error comes when the execution engine is set to Tez , the moment i switch to MR the issue does not come up. This

Re: Re: Merge Operation Failing Results in this SQL Error [40000] [42000]

Hi Clinton, Thanks for sharing your problem. If you provide more information, such as a dataset or queries, we can reproduce it and file the problem. Hi Lisoda, Thanks for giving us real examples. Interesting. Can I understand the first problem that happens when there is a big data file in an Ic

Re: Re: Merge Operation Failing Results in this SQL Error [40000] [42000]

Hello Lisoda, Thanks for this information. On Sun, Sep 1, 2024 at 4:04 PM lisoda wrote: > Hello Clinton: > > We have actually encountered the same issue where, in many cases, querying > Iceberg does not meet expected efficiency, falling short of regular > ORC/Parquet tables in speed. Since the

Re: Re: Support java/11/17/21

I know about that, I only flagged that [1] :-) That got sorted by upgrading the protobuf to 3.23, [2], & merging [3] Hadoop can't even compile JDK-8+, there are bunch of issue, Jersey being the biggest one, All the JDK upgrade tickets are open https://issues.apache.org/jira/browse/HADOOP-16795 ht

Re: Re: Next Hive 4.0.1 minor release

Hi, I'm posting some more suggestions. # We may label HIVE-28352 as "hive-4.0.1-must" I found that users were unable to upgrade some system tables from Hive 3 through schematool. It would be better to fix. https://issues.apache.org/jira/browse/HIVE-28352 # We may drop HIVE-24167 I am the ticket

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Thanks again everyone :) On Fri, Apr 19, 2024, 2:15 AM Rajesh Balamohan wrote: > Congratulations Simhadri. :) > > ~Rajesh.B > > On Fri, Apr 19, 2024 at 2:02 AM Aman Sinha wrote: > >> Congrats Simhadri ! >> >> On Thu, Apr 18, 2024 at 12:25 PM Naveen Gangam >> wrote: >> >>> Congrats Simhadri. Lo

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Congratulations Simhadri. :) ~Rajesh.B On Fri, Apr 19, 2024 at 2:02 AM Aman Sinha wrote: > Congrats Simhadri ! > > On Thu, Apr 18, 2024 at 12:25 PM Naveen Gangam > wrote: > >> Congrats Simhadri. Looking forward to many more contributions in the >> future. >> >> On Thu, Apr 18, 2024 at 12:25 PM

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Congrats Simhadri ! On Thu, Apr 18, 2024 at 12:25 PM Naveen Gangam wrote: > Congrats Simhadri. Looking forward to many more contributions in the > future. > > On Thu, Apr 18, 2024 at 12:25 PM Sai Hemanth Gantasala > wrote: > >> Congratulations Simhadri 🎉 well deserved >> >> On Thu, Apr 18, 2024

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Congrats Simhadri. Looking forward to many more contributions in the future. On Thu, Apr 18, 2024 at 12:25 PM Sai Hemanth Gantasala wrote: > Congratulations Simhadri 🎉 well deserved > > On Thu, Apr 18, 2024 at 8:41 AM Pau Tallada wrote: > >> Congratulations >> >> Missatge de Alessandro Soli

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Congratulations Simhadri 🎉 well deserved On Thu, Apr 18, 2024 at 8:41 AM Pau Tallada wrote: > Congratulations > > Missatge de Alessandro Solimando del dia > dj., 18 d’abr. 2024 a les 17:40: > >> Great news, Simhadri, very well deserved! >> >> On Thu, 18 Apr 2024 at 15:07, Simhadri G wrote:

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Congratulations Missatge de Alessandro Solimando del dia dj., 18 d’abr. 2024 a les 17:40: > Great news, Simhadri, very well deserved! > > On Thu, 18 Apr 2024 at 15:07, Simhadri G wrote: > >> Thanks everyone! >> I really appreciate it, it means a lot to me :) >> The Apache Hive project and i

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Great news, Simhadri, very well deserved! On Thu, 18 Apr 2024 at 15:07, Simhadri G wrote: > Thanks everyone! > I really appreciate it, it means a lot to me :) > The Apache Hive project and its community have truly inspired me . I'm > grateful for the chance to contribute to such a remarkable pro

Re: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Thanks everyone! I really appreciate it, it means a lot to me :) The Apache Hive project and its community have truly inspired me . I'm grateful for the chance to contribute to such a remarkable project. Thanks! Simhadri Govindappa On Thu, Apr 18, 2024 at 6:18 PM Sankar Hariappan wrote: > Congr

RE: Re: [ANNOUNCE] New Committer: Simhadri Govindappa

Congrats Simhadri! -Sankar From: Butao Zhang Sent: Thursday, April 18, 2024 5:39 PM To: user@hive.apache.org; dev Subject: [EXTERNAL] Re: [ANNOUNCE] New Committer: Simhadri Govindappa You don't often get email from butaozha...@163.com. Learn why this is important

Re: Re: Hive's performance for querying the Iceberg table is very poor.

Please ensure hive.stats.autogather is enabled as well. On Fri, Nov 10, 2023, 2:57 PM Denys Kuzmenko wrote: > `hive.iceberg.stats.source` controls where the stats should be sourced > from. When it's set to iceberg (default), we should go directly to iceberg > and bypass HMS. >

Re: Re: Hive's performance for querying the Iceberg table is very poor.

`hive.iceberg.stats.source` controls where the stats should be sourced from. When it's set to iceberg (default), we should go directly to iceberg and bypass HMS.

Re: Re: Hive's performance for querying the Iceberg table is very poor.

Can you please check this property? We need ensure it is true. set hive.compute.query.using.stats=true; In addition, it looks like the table created by spark has lots of data. Can you create a new table and insert into several values by spark, and then create & count(*) this location_based_tab

Re: Re: Hive's performance for querying the Iceberg table is very poor.

Could you please provide detailed steps to reproduce this issue? e.g. how do you create the table? Thanks, Butao Zhang Replied Message | From | lisoda | | Date | 11/9/2023 14:25 | | To | | | Subject | Re:Re: Re: Hive's performance for querying the Iceberg table is very poor. | In

Re: Re: Hive's performance for querying the Iceberg table is very poor.

Hi lisoda. You can check this ticket https://issues.apache.org/jira/browse/HIVE-27347 which can use iceberg basic stats to optimize count(*) query. Note: it didn't take effect if having delete files. Thanks, Butao Zhang Replied Message | From | lisoda | | Date | 11/9/2023 10:43 |

Re: Re: Hive's performance for querying the Iceberg table is very poor.

HIVE-27734 is in progress, as I see we have a POC attached to the ticket, we should have it in 2-3 week I believe. > Also, after the release of 4.0.0, will we be able to do all TPCDS queries on ICEBERG except for normal HIVE tables? Yep, I believe most of the TPCDS queries would be supported even

Re: Re: Tez & fetch task conversion

Hi Wojtek, Thanks for explaining the detail. I understand you have a larger amount of data than `hive.fetch.task.conversion.threshold`. Taking a glance, SimpleFetchOptimizer is likely to respect LIMIT if `hive.fetch.task.caching` is disabled and all predicates are for partition pruning. The case

RE: RE: Issue with the "hive.io.file.readcolumn.names" property

Hi, /cc Peter, as you might have some thoughts based on your experience with Iceberg :) I'm noticed another odd behavior with the "hive.io.file.readcolumn.names" property. Consider this query that reads from two separate tables at once: SELECT * FROM ( SELECT num as number,

Re: RE: Re: Custom OutputCommitter not called by Tez

unsubscribe On Sat, Apr 30, 2022 at 9:10 PM Julien Phalip wrote: > After all, I was able to have my MetaHook class' commitInsertTable() > method be properly called by Tez. However, it looks like it's in fact a > different instance of that class, and therefore it doesn't share the same > Configur

RE: RE: Re: Custom OutputCommitter not called by Tez

After all, I was able to have my MetaHook class' commitInsertTable() method be properly called by Tez. However, it looks like it's in fact a different instance of that class, and therefore it doesn't share the same Configuration object as the one that was initialized at the beginning of the job. So

RE: RE: Detecting write mode (append, overwrite) in custom storage handler

I realize this is in fact quite related to another thread that I recently started: https://lists.apache.org/thread/s0pzmgmq6trdjtxc50qwpww2dlzxql9b So this discussion could continue there. On 2022/04/30 04:08:27 Julien Phalip wrote: > I've noticed that the DefaultHiveMetaHook > < https://github.co

RE: RE: Re: Custom OutputCommitter not called by Tez

Hi Peter, So I've looked into the approach that you've pointed at in this pull request (https://github.com/apache/hive/pull/2161), which is to rely on HiveMetaHook.commitInsertTable() instead of the "traditional" OutputCommitter.commitJob(). I've tried to implement a similar approach, however som

RE: Re: Custom OutputCommitter not called by Tez

Hi Peter, Looking at https://issues.apache.org/jira/browse/TEZ-4279, it seems that the fix might have been applied to 0.9.3. Is that correct? If so, do you think that just upgrading Tez to that version might be enough to allow the "setUpJob()", "commitJob()" and "abortJob()" to be called appropria

RE: Re: Custom OutputCommitter not called by Tez

Thanks Peter. By chance could I get things to work by keeping my current version of Hive (3.1.2) and only upgrading Tez? Which version(s) should I use? Thank you, Julien On 2022/04/27 08:59:08 Peter Vary wrote: > We had the same issue with the IcebergOutputCommitter. > > The first solution was

Re: Re: How can I know use execute or executeQuery

MSCK REPAIR TABLE hivetest.mytable1 I know this reason but if I ResultSet rs = st.executeQuery("msck repair table hivetest.chinese_part1"); I will get a error The query did not generate a result set! igyu From: igyu Date: 2021-09-16 08:59 To: user Subject: Re: Re: How can

Re: Re: How can I know use execute or executeQuery

thanks I have a other perblem when I run "MSCK REPAIR TABLE hivetest.mytable1" I get a error Encountered: IDENTIFIER Expected: ALTER, COMMENT, COMPUTE, COPY, CREATE, DELETE, DESCRIBE, DROP, EXPLAIN, GRANT, INSERT, INVALIDATE, LOAD, REFRESH, REVOKE, SELECT, SET, SHOW, TRUNCATE, UPDATE, UPSERT

Re: Re: Rlike '\s' couldn't get the space

It did help, thanks! Regards, 孙志禹 From: Tanvi Thacker Date: 2018-11-15 06:06 To: user Subject: Re: Rlike '\s' couldn't get the space Hi, You need to use "rlike '\\s' to escape the single backslash. Regards Tanvi Thacker On Mon, Nov 12, 2018 at 3:02 AM anci_...@yahoo.com wrote: Dears,

Re: Re: Question about INSERT OVERWRITE TABLE with dynamic partition

Thanks, I think it's the proper explanation. For the query result in the second query is null, there won't be a partition name generated in dynamic partition step, so the system doesn't know which partition to overwrite. Thanks very much! Regards, 孙志禹 From: Tanvi Thacker Date: 2018-10-25

Re: Re: Question about OVER clause

Thanks，but the article talked too little to help. Actually, it only told me that by using this code we would accumulate all records of which v_date was less than or equal to current row. But the question is, what will happen by the code below? (the field v_date is a string value with the format

Re: Re: LineageLogger can't getEdges

is this the reason ? code in Optimizer.java if (postExecHooks.contains("org.apache.hadoop.hive.ql.hooks.PostExecutePrinter") || postExecHooks.contains("org.apache.hadoop.hive.ql.hooks.LineageLogger")) { transformations.add(new Generator()); } 2018-09-13 lk_hadoop 发件人："

Re: Re: Edit wiki of hive permissions

You now have edit permissions on the wiki. Welcome to the Hive wiki team, Chuikai You! -- Lefty On Tue, Jun 5, 2018 at 2:24 AM yo...@jpush.cn wrote: > I'm sorry,The correct account is *youchuikai * > Thank you. > -- > yo...@jpush.cn > > > *From:* Lefty Leverenz >

Re: Re: Edit wiki of hive permissions

I'm sorry,The correct account is youchuikai Thank you. yo...@jpush.cn From: Lefty Leverenz Date: 2018-06-05 10:46 To: youck CC: user Subject: Re: Edit wiki of hive permissions The permissions page doesn't recognize youchui...@163.com as a Confluence user. Do you have a Confluence account? S

Re: RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

Hey Eric , Thank you for your response . For CMS, whats the max heap size thats worked well for you. I was going through the article : https://www.cloudera.com/documentation/enterprise/5-9-x/topics/admin_hive_tuning.html . Wondering if you followed best practices similar to the suggestions in the

Re: RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

Hey Richard, Glad to know what G1 works well for you. Would be kind enough to share more details ? What are the jvm settings that work well for you? Whats the max heap size ? With these settings you have what the max HS2 session concurrency that you are able to achieve ? Would really appreciate

Re: RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

Use JDK8 and try G1 first, we saw lower GC pause under G1 especially in our large memory environment  On Wednesday, November 29, 2017, 8:14:03 AM PST, eric wong wrote: -- 已转发邮件 -- From: eric wong To: user@hive.apache.org Cc:  Bcc:  Date: Thu, 30 Nov 2017 00:07:47 +

RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

> > > -- 已转发邮件 -- > From: eric wong > To: user@hive.apache.org > Cc: > Bcc: > Date: Thu, 30 Nov 2017 00:07:47 +0800 > Subject: Re: For Apache Hive HS2 , what is the largest heap size setting > that works well? > > 1）Long GC pause should has detailed reson, like InitialMark/Remark、F

Re: Re: How to use HPL/SQL connect with impala and Hive

Looks like you are trying to connect not through HiveServer2. You should use: hplsql.conn.default hive2conn hplsql.conn.hive2conn org.apache.hive.jdbc.HiveDriver;jdbc:hive2://:1;; Thanks, Dmitry On Thu, Nov 23, 2017 at 10:32 AM, 梁凯 wrote: > Dear， > > Thanks for your help. I tr

Re: Re: [Questio]Which record does Hive give the bigger number when I use row_number

#yiv2801412653 body {line-height:1.5;}#yiv2801412653 blockquote {margin-top:0px;margin-bottom:0px;margin-left:0.5em;}#yiv2801412653 body {font-size:11pt;color:rgb(0, 0, 0);line-height:1.5;}#yiv2801412653 body {font-size:11pt;color:rgb(0, 0, 0);line-height:1.5;}Thanks a lot! It helps me so much.

Re: Re: where kylin global dict store?

I find the Globel Dictionary infomation in HBase table ,sorry for disturb 446463...@qq.com From: 446463...@qq.com Date: 2017-01-06 10:48 To: user; dev CC: user Subject: Re: Re: where kylin global dict store? I am sorry for not making a point for my question I used a Global Dictionary cloumn

Re: Re: where kylin global dict store?

I am sorry for not making a point for my question I used a Global Dictionary cloumn when I build a cube .but I can't find anything with this Global Dictionary infomation related with my cube in HBase table 'kylin_metadata' I searched all rowkeys in HBase table 'kylin_metadata' and filter out

Re: Re: I delete my table in hive,but the file in HDFS not be deleted

not external this is create table statement: - create table if not exists user_info_20161207 (rowKey string,timestamp string,sex string ,birthday string ,native_place string ) row format delimited fields terminated by ',' -- I kown the ow

Re: Re: load data Failed with exception java.lang.IndexOutOfBoundsException

file has more than two characters,that is a little interesting. I can not understand the result of function checkInputFormat is OrcInputFormat,maybe that is just right. Thanks. From: Stephen Sprague<mailto:sprag...@gmail.com> Date: 2016-09-09 12:47 To: user@hi

Re: Re: load data Failed with exception java.lang.IndexOutOfBoundsException

>at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat. validateInput(OrcInputFormat.java:508) would it be safe to assume that you are trying to load a text file into an table stored as ORC? your create table doesn't specify that explicitly so that means you have a setting in your configs that says

Re: Re: load data Failed with exception java.lang.IndexOutOfBoundsException

Yes, based on my testing,it is wrong from 0 to 99 with the content of file 1.dat, whether the column type is string or int. hive.log: 2016-09-09T09:10:40,978 INFO [d1e08abd-5f8b-4149-a679-00ba6b4f4ab9 main]: CliDriver (SessionState.java:printInfo(1029)) - Hive-on-MR is deprecated in Hive 2 a

Re: Re: hive will die or not?

I am afraid your points (the original thread owner) are manifestly misleading or at best half baked. Given a set of parameters one can argue from any angle. Why use Spark but not Flink. Why use this and not that. These are cyclic arguments. - Hive can use Spark as its execution engine with exc

Re: Re: Re: hive will die or not?

regards wenli On 08/08/2016 10:16, Edward Capriolo wrote: A few entities going to "kill/take out/better than hive" I seem to remember HadoopDb, Impala, RedShift , voltdb... But apparent hive is still around and probably faster http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fa

Re: Re: hive will die or not?

A few entities going to "kill/take out/better than hive" I seem to remember HadoopDb, Impala, RedShift , voltdb... But apparent hive is still around and probably faster http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fast-or-is-it-final On Sun, Aug 7, 2016 at 9:49 PM, 理 wrote:

Re: Re: hive will die or not?

in my opinion, multiple engine is not advantage, but reverse. it disperse the dev energy. consider the activity ,sparksql support all tpc ds without modify syntax! but hive cannot. consider the tech, dag, vectorization, etc sparksql also has, seems the code is more ef

Re: Re-arrange columns

Hi, I think I had similar issues to yours. Did you look in the Hive documentation at what the CASCADE keyword does on ADD or CHANGE COLUMNS statements? https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterColumn >From what I understand, the behavior of Hive

Re: Re: [ANNOUNCE] Apache Hive 2.1.0 Released

Thank you On 2016-06-22 01:37 , Thejas Nair Wrote: Thanks for your hard work and patience in driving the release Jesus! :) On Tue, Jun 21, 2016 at 10:18 AM, Jesus Camachorodriguez wrote: > The Apache Hive team is proud to announce the release of Apache Hive > version 2.1.0. > > The Apache Hive

Re: Re: The build-in indexes in ORC file does not work.

terminal_type =0, 260,000,000 rows, almost cover half of the whole data.terminal_type =25066, just 3800 rows. orc tblproperties("orc.compress"="SNAPPY","orc.compress.size"="262141","orc.stripe.size"="268435456","orc.row.index.stride"="10","orc.create.index"="true","orc.bloom.filter.columns"

Re: Re: The build-in indexes in ORC file does not work.

Hi professor Gopal, > Most of your ~300s looks to be the fixed overheads of setting up each task. Maybe you are right. Perhaps the orc indexes work normally in hive, Just because the fixed time overhead is too long, so I think the performance improement is not obvious, I will check this later.

Re: Re: [VOTE] Hive 2.0 release plan

Agreed, any plans for Hive 1.3? Will Hive 2.0 be a breaking release for those running 1.x? On Sun, Nov 15, 2015 at 7:07 PM, Wangwenli wrote: > Good News, *Any release plan for hive 1.3* ??? > > -- > Wangwenli > > > *From:* Gopal Vijayaraghavan > *Date:* 2015-1

Re: Re: [VOTE] Hive 2.0 release plan

Good News, Any release plan for hive 1.3 ??? Wangwenli From: Gopal Vijayaraghavan Date: 2015-11-14 14:21 To: d...@hive.apache.org CC: user@hive.apache.org Subject: Re: [VOTE

Re: Re: why UserGroupInformation equals method depend on the subject object

thanks benoy and chris's reply, i raised here: HADOOP-12529 wenli Regards From: Chris Nauroth Date: 2015-10-30 05:31 To: user; user@hive.apache.or

Re: Re: HiveServer2 load data inpath fails

Hi Vineet, Were you able to find anything in HS2 log? I was just able to run 'load data inpath' with hive ver. 0.14 without any issue. My env is hive.execution.engine=tez though. Also Based on your error message, it may be worth checking what value is set for datanucleus.connectionPoolingType. I

Re: Re: HiveServer2 load data inpath fails

Hi I tried truncating the table(ensuring no data files are present in the warehouse directory) and then used load data inpath with/without overwrite option but nothing worked. Hive directory is owned by hive user but my existing user which is a application user owns the complete access on hive wa

Re: Re: HiveServer2 load data inpath fails

Hi, I already stated my requirement since the files are generating on hdfs and i want to load the file from hdfs to hive instead of local file system. Moreover in production like environment one just have port access to HS2 node and no direct access to machine, in that particular case how do we e

Re: Re: HiveServer2 load data inpath fails

Place the file on the hiveserver2 host local filesystem . Then try load data local inpath r7raul1...@163.com From: Vineet Mishra Date: 2015-10-21 03:05 To: user CC: cdh-user Subject: Re: HiveServer2 load data inpath fails Any update around this? 0: jdbc:hive2://hadoop-hs2:1> load dat

Re: Re: Hiveserver2 client stdout

Amazing! Thank you Nicholas, exactly what I needed! On Mon, Oct 19, 2015 at 11:03 PM, Nicholas Hakobian < nicholas.hakob...@rallyhealth.com> wrote: > If you want to retrieve the STDOUT logs from the HiveServer2 Thrift > server, you can see how beeline does it here: > > > https://github.com/apache

Re: Re: Hiveserver2 client stdout

If you want to retrieve the STDOUT logs from the HiveServer2 Thrift server, you can see how beeline does it here: https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L958-L963 I used this as an example of how to pull the query logs for a recent projec

Re: Re: Hiveserver2 client stdout

Right, with beeline CLI, I can see logs just fine. But with the java JDBC client code ( https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients) I can't see stdout logs On Mon, Oct 19, 2015 at 10:26 PM, r7raul1...@163.com wrote: > You can check more option from > https://cwiki.apach

Re: Re: Hiveserver2 client stdout

You can check more option from https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients . --showWarnings=[true/false] Display warnings that are reported on the connection after issuing any HiveQL commands. Default is false. Usage: beeline --showWarnings=true --showNestedErrs=[true/f

Re: Re: Hiveserver2 client stdout

Or you can use --verbose=[true/false] when use jdbc client. Show verbose error messages and debug information (true) or do not show (false). Default is false. Usage: beeline --verbose=true r7raul1...@163.com From: r7raul1...@163.com Date: 2015-10-20 13:07 To: user Subject: Re: Re

Re: Re: Hiveserver2 client stdout

That would just increase the log level on the server side right? YES You can check the log in detail. r7raul1...@163.com From: Jerrick Hoang Date: 2015-10-20 11:55 To: user Subject: Re: Hiveserver2 client stdout That would just increase the log level on the server side right? I want th

Re: Re: regarding hiveserver2 DeRegisterWatcher

Hi Xuefu, Thanks for you response. Let me express more: /** * The watcher class which sets the de-register flag when the znode corresponding to this server // 3. from the declare, it seems, the purpose is to stop this hs2 instance forever, but this seems not correct in case the znode

Re: Re: binary column data consistency in hive table copy

if use CTAS then a MR job occures. Maybe the problem is in the MR job. 2015-09-15 xihuyu2000 发件人：Jason Dere 发送时间：2015-09-15 06:00 主题：Re: binary column data consistency in hive table copy 收件人："user@hive.apache.org" 抄送： Looks like your table is using text storage format. Binary data needs to

Re: Re: sql mapjoin very slow

Yes , I am accidentally joining on a Double. keys: 0 UDFToDouble(nav_tcdt) (type: double) 1 UDFToDouble(site_categ_id) (type: double) 2 UDFToDouble(site_categ_id) (type: double) 3 UDFToDouble(mg_brand_id) (type: double)

Re: Re: sql mapjoin very slow

lass default value is fs. Mean store statistics in local filesystem. Any one can tell what is the file path to store statistics ? r7raul1...@163.com From: r7raul1...@163.com Date: 2015-08-28 13:03 To: user Subject: Re: Re: sql mapjoin very slow I increase hive.hashtable.initialC

Re: Re: sql mapjoin very slow

the issue is not obvious (I.e the above). If there’s no obvious problem you might consider not using map join. From: "r7raul1...@163.com" Reply-To: user Date: Thursday, August 27, 2015 at 18:51 To: user Subject: Re: Re: sql mapjoin very slow I use MR. My mapjoin config as showed

Re: Re: sql mapjoin very slow

I use MR. My mapjoin config as showed in follow picture: r7raul1...@163.com From: Sergey Shelukhin Date: 2015-08-28 09:21 To: user Subject: Re: sql mapjoin very slow Are you using MR and Tez? You could try optimized hash table in case of Tez, although it’s supposed to improve memory, not nec

Re: Re: hive -e run tez query error

BTW , my hadoop version is 2.3.0-cdh5.0.2 r7raul1...@163.com From: Jeff Zhang Date: 2015-06-29 13:23 To: user Subject: Re: hive -e run tez query error > But keeping that client cache disabled when running against trunk generally > kills queries all the time with occasional errors like these.

Re: Re: delta file compact take no effect

My hive version is 1.1.0 r7raul1...@163.com From: Alan Gates Date: 2015-06-18 23:25 To: user Subject: Re: delta file compact take no effect Which version of Hive are you running? A number of deadlock issues were resolved in HIVE-10500 which was released in Hive 1.2. Based on your log it ap

Re: Re: delta file compact take no effect

s: 30 r7raul1...@163.com From: r7raul1...@163.com Date: 2015-06-18 08:37 To: user Subject: Re: Re: delta file compact take no effect Thank you! I will try r7raul1...@163.com From: Alan Gates Date: 2015-06-18 08:33 To: user Subject: Re: delta file compact take no effect See https://cwiki.apach

Re: Re: delta file compact take no effect

Thank you! I will try r7raul1...@163.com From: Alan Gates Date: 2015-06-18 08:33 To: user Subject: Re: delta file compact take no effect See https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration Compaction is initiated by the thrift metastore serve

Re: Re: delta file compact take no effect

My config is on my client. What is metastore config? r7raul1...@163.com From: Alan Gates Date: 2015-06-17 13:42 To: user Subject: Re: delta file compact take no effect Is the config you give on your metastore or your client? The worker thread and initiator must be started on the metastore.

Re: Re: delta file compact take no effect

Any help? r7raul1...@163.com From: r7raul1...@163.com Date: 2015-06-11 15:53 To: user Subject: Re: Re: delta file compact take no effect SHOW COMPACTIONS; I can see some info Database Table Partition Type State Worker Start Time default u_data_txn NULL MAJOR initiated NULL 0 Time taken

Re: Re: delta file compact take no effect

SHOW COMPACTIONS; I can see some info Database Table Partition Type State Worker Start Time default u_data_txn NULL MAJOR initiated NULL 0 Time taken: 0.024 seconds, Fetched: 2 row(s) But after that I still see many delta file. r7raul1...@163.com From: Elliot West Date: 2015-06-11 15:25 To

Re: Re: Beeline connect to hiveserver2 didn't print log.

I chang beeline-log4j.properties to log4j.rootLogger=INFO, console. When I use (beeline -u jdbc:hive2:// ) and (set hive.execution.engine=mr) , the query log display scan complete in 5ms Connecting to jdbc:hive2:// 15/05/27 13:06:40 [main]: WARN conf.HiveConf: HiveConf of name hive.metasto

Re: Re: PL/HQL - Procedural SQL-on-Hadoop

You can refer to the cursor columns using cur_name.col_name syntax Great job! r7raul1...@163.com From: Dmitry Tolpeko Date: 2015-05-01 03:43 To: user Subject: Re: PL/HQL - Procedural SQL-on-Hadoop Hi everyone, PL/HQL 0.3.1 released today. There are some features I hope you find interesting

Re: Re: upgrade of Hadoop cluster from 5.1.2 to 5.2.1.

are you member of group "*g_**hdp_storeops*" ? On Mon, May 25, 2015 at 6:21 AM, r7raul1...@163.com wrote: > config hdfs acl > > http://zh.hortonworks.com/blog/hdfs-acls-fine-grained-permissions-hdfs-files-hadoop/ > > -- > r7raul1...@163.com > > > *From:* Anupam sinha

Re: Re: upgrade of Hadoop cluster from 5.1.2 to 5.2.1.

config hdfs acl http://zh.hortonworks.com/blog/hdfs-acls-fine-grained-permissions-hdfs-files-hadoop/ r7raul1...@163.com From: Anupam sinha Date: 2015-05-21 12:44 To: user Subject: Re: upgrade of Hadoop cluster from 5.1.2 to 5.2.1. Hello everyone, i am a member of nested group which has select

Re: Re: hive sql on tez run forever

I see only 1 reduce run forerver. Skew join? r7raul1...@163.com From: Eugene Koifman Date: 2015-05-12 01:43 To: user CC: r7raul1...@163.com Subject: Re: hive sql on tez run forever This isn’t a valid rewrite. if a(x,y) has 1 row (1,2) and b(x,z) has 1 row (1,1) then the 1st query will produce

Re: Re: skewjoin problem

my sql no group. The sql cause the problem : from dw.fct_traffic_navpage_path_detl t left outer join dw.univ_parnt_tranx_comb_detl o on t.ordr_code = o.parnt_ordr_code and t.cart_prod_id = o.comb_prod_id and o.ds = '{$label}' select ordr_code,count(*) as a from dw.fct_traffic_navpage_path_d

Re: Re: join on different data type

Also. Refer section "*Allowed Implicit Conversions*" on https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types It had given a try on joining tables on STRING and INT column , it should work through implicit conversions. Regards Sanjiv Singh Mob : +091 9990-447-339 On Mon,

Re: Re: How to config hive when using namenode HA

Thank you! r7raul1...@163.com From: Gerald-G Date: 2015-05-06 10:35 To: user Subject: Re: How to config hive when using namenode HA Upgrading the Hive Metastore to Use HDFS HA Using the Command Line To configure the Hive metastore to use HDFS HA, change the records to reflect the location spe

Re: Re: join on different data type

IF join on A.col1 == B.col2 col1 is string col2 is bigint The automatic conversion is support? All cast to string? r7raul1...@163.com From: Gopal Vijayaraghavan Date: 2015-05-04 16:10 To: user CC: r7raul1...@163.com Subject: Re: join on different data type > If A.col1 is of DOUBLE type, >

Re: Re: join on different data type

Thank you! r7raul1...@163.com From: Gopal Vijayaraghavan Date: 2015-05-04 16:10 To: user CC: r7raul1...@163.com Subject: Re: join on different data type > If A.col1 is of DOUBLE type, > but B.col2 is of BIGINT, The automatic conversion is not acceptable according to the java language spe

Re: Re: how to set column level privileges

Column level security in hive was added at HIVE-5837 It has the PDF link for your readings. https://cwiki.apache.org/confluence/display/Hive/AuthDev talks about setting column level permissions On Thu, Mar 26, 2015 at 4:39 PM, Allen wrote: > Th

Re: Re: SHOW COMPACTIONS fail with remote metastore

My hive version is hive 0.14.0 , type SHOW COMPACTIONS in cli return exception. But I type sql in the cli ,the sql run successful on mr. r7raul1...@163.com From: Alan Gates Date: 2015-03-19 00:17 To: user Subject: Re: SHOW COMPACTIONS fail with remote metastore When you say "but run sql the

Re: Re: ERROR OutOfMemoryError: Java heap space

Use yarn logs -applicationId to see detail log. r7raul1...@163.com From: Jadhav Shweta Date: 2015-02-26 15:00 To: user Subject: Re: ERROR OutOfMemoryError: Java heap space Hi, I tried running same query from Simple JDBC Client instead of Spring Batch framework its was working fine. But now

  1   2   >