[jira] [Comment Edited] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2018-02-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363616#comment-16363616 ] liyunzhang edited comment on HIVE-17879 at 2/14/18 8:08 AM: Mo

[jira] [Commented] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2018-02-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363616#comment-16363616 ] liyunzhang commented on HIVE-17879: --- Most work of HIVE-17879.patch is belongs to https

[jira] [Updated] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2018-02-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17879: -- Attachment: HIVE-17879.patch > Can not find java.sql.date in JDK9 when building hive > -

[jira] [Commented] (HIVE-14285) Explain outputs: map-entry ordering of non-primitive objects.

2018-02-12 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361692#comment-16361692 ] liyunzhang commented on HIVE-14285: --- [~kgyrtkirk]: {code} I'm starting to remember...I

[jira] [Commented] (HIVE-14285) Explain outputs: map-entry ordering of non-primitive objects.

2018-02-11 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360203#comment-16360203 ] liyunzhang commented on HIVE-14285: --- [~kgyrtkirk]: I want to ask a question that the wo

[jira] [Commented] (HIVE-14285) Explain outputs: map-entry ordering of non-primitive objects.

2018-02-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359784#comment-16359784 ] liyunzhang commented on HIVE-14285: --- [~kgyrtkirk]: I found some problem with {{ExplainTa

[jira] [Commented] (HIVE-18340) Dynamic Min-Max/BloomFilter runtime-filtering in HoS

2018-02-07 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356365#comment-16356365 ] liyunzhang commented on HIVE-18340: --- [~stakiar]: {quote} Hive-on-Tez's has an implementa

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350077#comment-16350077 ] liyunzhang edited comment on HIVE-18301 at 2/2/18 9:59 AM: --- [~xu

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.3.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350077#comment-16350077 ] liyunzhang edited comment on HIVE-18301 at 2/2/18 9:54 AM: --- [~xu

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350077#comment-16350077 ] liyunzhang commented on HIVE-18301: --- [~xuefuz]: {quote} Instead of putting the input pa

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-31 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.2.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-31 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346418#comment-16346418 ] liyunzhang commented on HIVE-18301: --- [~lirui]: thanks for question. {quote} The IOCon

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340717#comment-16340717 ] liyunzhang commented on HIVE-18301: --- for the problem it may make the RDD bigger: origin

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.1.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: (was: HIVE-18301.1.patch) > Investigate to enable MapInput cache in Hive on Spark >

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Status: Patch Available (was: Open) HIVE-18301.1.patch to trigger Hive QA > Investigate to enable MapI

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.1.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Comment Edited] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached

2018-01-25 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340647#comment-16340647 ] liyunzhang edited comment on HIVE-18378 at 1/26/18 7:21 AM: [~

[jira] [Assigned] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached

2018-01-25 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-18378: - Assignee: liyunzhang > Explain plan should show if a Map/Reduce Work is being cached > --

[jira] [Commented] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached

2018-01-25 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340647#comment-16340647 ] liyunzhang commented on HIVE-18378: --- [~stakiar]: can you assign this jira to me ? I gues

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338586#comment-16338586 ] liyunzhang edited comment on HIVE-18301 at 1/25/18 4:05 AM: [~

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338586#comment-16338586 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote} does the proposed solution mean

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336946#comment-16336946 ] liyunzhang edited comment on HIVE-18301 at 1/24/18 5:47 AM: In

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336946#comment-16336946 ] liyunzhang edited comment on HIVE-18301 at 1/24/18 5:39 AM: In

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336946#comment-16336946 ] liyunzhang commented on HIVE-18301: --- In HIVE-18301.patch, it provides one solution to tr

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.patch > Investigate to enable MapInput cache in Hive on Spark > -

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336941#comment-16336941 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote} And we won't be able to tell the

[jira] [Commented] (HIVE-8436) Modify SparkWork to split works with multiple child works [Spark Branch]

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336431#comment-16336431 ] liyunzhang commented on HIVE-8436: -- [~csun]:  thanks for reply, {quote} without the copy

[jira] [Updated] (HIVE-18289) Support Parquet/Orc format when enable rdd cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18289: -- Summary: Support Parquet/Orc format when enable rdd cache in Hive on Spark (was: Fix jar dependency whe

[jira] [Commented] (HIVE-8436) Modify SparkWork to split works with multiple child works [Spark Branch]

2018-01-21 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333870#comment-16333870 ] liyunzhang commented on HIVE-8436: -- [~csun]:  can you spend some time to explain why need

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-17 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330190#comment-16330190 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote}My understanding is if the HadoopR

[jira] [Commented] (HIVE-17573) LLAP: JDK9 support fixes

2018-01-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312596#comment-16312596 ] liyunzhang commented on HIVE-17573: --- [~gopalv]: thanks for your reply and tool. bq.JDK

[jira] [Commented] (HIVE-17573) LLAP: JDK9 support fixes

2018-01-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16310797#comment-16310797 ] liyunzhang commented on HIVE-17573: --- [~gopalv]: It seems that on tpch_text_10(10g), ther

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16309361#comment-16309361 ] liyunzhang commented on HIVE-18301: --- Here some update about NPE: the normal case when e

[jira] [Issue Comment Deleted] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Comment: was deleted (was: [~stakiar]: the original purpose to change M->R to M->M->R is to let Combine

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.5.patch > Enable SharedWorkOptimizer in tez on HOS >

[jira] [Comment Edited] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308995#comment-16308995 ] liyunzhang edited comment on HIVE-17486 at 1/3/18 1:40 AM: --- [~st

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308994#comment-16308994 ] liyunzhang commented on HIVE-17486: --- [~stakiar]: the original purpose to change M->R to

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308995#comment-16308995 ] liyunzhang commented on HIVE-17486: --- [~stakiar]: the original purpose to change M->R to

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307619#comment-16307619 ] liyunzhang commented on HIVE-18301: --- thanks for Rui's comment, will spend some time to i

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-27 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304359#comment-16304359 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote} My understanding is these informa

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-22 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16301713#comment-16301713 ] liyunzhang commented on HIVE-18301: --- [~xuefuz],[~csun]: I read jiras about MapInput IOCo

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-21 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16301027#comment-16301027 ] liyunzhang commented on HIVE-18301: --- [~xuefuz],[~lirui]: Is there any way to store IOcon

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-21 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300999#comment-16300999 ] liyunzhang commented on HIVE-18301: --- The reason why have this NPE is because [org.apach

[jira] [Resolved] (HIVE-17755) NPE exception when running TestAcidOnTez#testGetSplitsLocks with "

2017-12-20 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang resolved HIVE-17755. --- Resolution: Not A Problem > NPE exception when running TestAcidOnTez#testGetSplitsLocks with " > -

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-20 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299384#comment-16299384 ] liyunzhang commented on HIVE-18148: --- +1 > NPE in SparkDynamicPartitionPruningResolver >

[jira] [Commented] (HIVE-18289) Fix jar dependency when enable rdd cache in Hive on Spark

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297977#comment-16297977 ] liyunzhang commented on HIVE-18289: --- if running in parquet, the exception is {code} Job

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297921#comment-16297921 ] liyunzhang commented on HIVE-18148: --- it's better to remove the DPP which the statistics

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297899#comment-16297899 ] liyunzhang commented on HIVE-18148: --- I can understand the reason why you need to remove

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297897#comment-16297897 ] liyunzhang edited comment on HIVE-18301 at 12/20/17 5:23 AM: -

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297897#comment-16297897 ] liyunzhang commented on HIVE-18301: --- yes. I can achieve the merge the same tables into

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297875#comment-16297875 ] liyunzhang commented on HIVE-18148: --- sorry for reply late. still have 1 question about t

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-18 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Description: Before IOContext problem is found in MapTran when spark rdd cache is enabled in HIVE-8920.

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-18 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Description: Before IOContext problem is found in MapTran when spark rdd cache is enabled in HIVE-8920.

[jira] [Assigned] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-18 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-18301: - > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Assigned] (HIVE-18289) Fix jar dependency when enable rdd cache in Hive on Spark

2017-12-17 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-18289: - > Fix jar dependency when enable rdd cache in Hive on Spark > ---

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-16 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.4.patch update HIVE-17486.4.patch to fix the compilation error. > Enable SharedW

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.3.patch add spark_optimize_shared_work.q.out and update HIVE-17486.3.patch. Trigg

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Status: Patch Available (was: Open) > Enable SharedWorkOptimizer in tez on HOS > --

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292286#comment-16292286 ] liyunzhang commented on HIVE-17486: --- [~xuefuz]: before you mentioned that the reason to

[jira] [Comment Edited] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292203#comment-16292203 ] liyunzhang edited comment on HIVE-17486 at 12/15/17 9:15 AM: -

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.2.patch [~lirui] and [~xuefuz]: I have updated HIVE-17486.2.patch and designed

[jira] [Commented] (HIVE-18282) Spark tar is downloaded every time for itest

2017-12-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292052#comment-16292052 ] liyunzhang commented on HIVE-18282: --- [~stakiar]: http://d3jw87u4immizc.cloudfront.net/s

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290256#comment-16290256 ] liyunzhang commented on HIVE-18148: --- {code} grep -C2 "hive.auto.convert.join" $HIVE_SO

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290243#comment-16290243 ] liyunzhang commented on HIVE-18148: --- [~lirui]: If this is found in common join please a

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288899#comment-16288899 ] liyunzhang commented on HIVE-18148: --- sorry i have not run the qtest. just copy the query

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288864#comment-16288864 ] liyunzhang commented on HIVE-18148: --- {quote} you can just run the new test with master b

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-12 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288814#comment-16288814 ] liyunzhang commented on HIVE-18148: --- [~lirui]: tried to use the example you provided. {

[jira] [Comment Edited] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285558#comment-16285558 ] liyunzhang edited comment on HIVE-17486 at 12/11/17 6:41 AM: -

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285558#comment-16285558 ] liyunzhang commented on HIVE-17486: --- [~xuefuz] {quote} It seems that we can do so whene

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285551#comment-16285551 ] liyunzhang commented on HIVE-17486: --- [~lirui]: {quote} could you provide more design and

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285549#comment-16285549 ] liyunzhang commented on HIVE-18150: --- [~Ferd]: [~stakiar] has uploaded the spark-2.2.0 p

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285473#comment-16285473 ] liyunzhang commented on HIVE-18150: --- [~ferd] : as we have finished the review of the pat

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285470#comment-16285470 ] liyunzhang commented on HIVE-17486: --- [~lirui] and [~xuefuz]: Please help view the [First

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285468#comment-16285468 ] liyunzhang commented on HIVE-18148: --- [~lirui]: I can not reproduce because you did not p

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-08 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283241#comment-16283241 ] liyunzhang commented on HIVE-17486: --- [~xuefuz] and [~lirui]: the second problem I found

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-07 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283012#comment-16283012 ] liyunzhang commented on HIVE-17486: --- [~xuefuz]: have uploaded the [design doc|https://d

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-07 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282966#comment-16282966 ] liyunzhang commented on HIVE-17486: --- [~xuefuz]: ok, will upload doc soon. > Enable Shar

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-06 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16281238#comment-16281238 ] liyunzhang commented on HIVE-17486: --- [~lirui], [~xuefuz]: can you spend some time to vie

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-06 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16281237#comment-16281237 ] liyunzhang commented on HIVE-18150: --- [~ferd], [~lirui],[~xuefuz] and [~stakiar]: 4 unit

[jira] [Updated] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-05 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18150: -- Attachment: (was: HIVE-18150.1.patch) > Upgrade Spark Version to 2.2.0 > ---

[jira] [Updated] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-05 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18150: -- Attachment: HIVE-18150.1.patch reattach patch to trigger Hive QA. > Upgrade Spark Version to 2.2.0 > --

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16278167#comment-16278167 ] liyunzhang commented on HIVE-17486: --- Here record the problems currently I met 1. I want

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.1.patch > Enable SharedWorkOptimizer in tez on HOS >

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16278065#comment-16278065 ] liyunzhang commented on HIVE-18150: --- test two qfiles by TestMiniSparkOnYarnCliDriver, se

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276547#comment-16276547 ] liyunzhang commented on HIVE-17486: --- will upload patch later and describe the problems i

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: explain.28.share.false explain.28.share.true > Enable SharedWorkOptimizer in

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276526#comment-16276526 ] liyunzhang commented on HIVE-17486: --- explain.28.scan.share.true we can see that there i

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: (was: explain.28.scan.share.true) > Enable SharedWorkOptimizer in tez on HOS > -

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: (was: explain.28.scan.share.false) > Enable SharedWorkOptimizer in tez on HOS >

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276515#comment-16276515 ] liyunzhang commented on HIVE-18150: --- trigger HIVE-QA to find the failed test cases > Up

[jira] [Updated] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18150: -- Attachment: HIVE-18150.1.patch > Upgrade Spark Version to 2.2.0 > -- > >

[jira] [Assigned] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-18150: - Assignee: liyunzhang (was: Sahil Takiar) > Upgrade Spark Version to 2.2.0 >

[jira] [Updated] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18150: -- Status: Patch Available (was: Open) > Upgrade Spark Version to 2.2.0 > -- >

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: explain.28.scan.share.false explain.28.scan.share.true I set the flag {{hive

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276289#comment-16276289 ] liyunzhang commented on HIVE-18150: --- [~stakiar]: any progress on this jira? If you have

[jira] [Issue Comment Deleted] (HIVE-18080) Performance degradation on VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled

2017-11-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18080: -- Comment: was deleted (was: It seems after If expression is vectorized by AVX2 instructions. It only co

[jira] [Commented] (HIVE-18080) Performance degradation on VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled

2017-11-22 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16263873#comment-16263873 ] liyunzhang commented on HIVE-18080: --- It seems after If expression is vectorized by AVX2

[jira] [Updated] (HIVE-18080) Performance degradation on VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled

2017-11-22 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18080: -- Attachment: IFExpression_AVX2_Instruction.png > Performance degradation on > VectorizedLogicBench#IfExp

  1   2   >