[
https://issues.apache.org/jira/browse/HIVE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855381#comment-15855381
]
Hive QA commented on HIVE-15682:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12851281/HIVE-15682.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10175 tests
executed
*Failed tests:*
{noformat}
TestCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=13)
[avro_joins.q,serde_reported_schema.q,annotate_stats_join_pkfk.q,udf_unix_timestamp.q,union22.q,describe_comment_nonascii.q,orc_analyze.q,schema_evol_orc_acidvec_part_update.q,partition_date.q,stats15.q,tez_join_result_complex.q,input36.q,alter_numbuckets_partitioned_table2_h23.q,transform_ppr1.q,spark_vectorized_dynamic_partition_pruning.q,unionDistinct_2.q,udaf_histogram_numeric.q,authorization_index.q,auto_join26.q,list_bucket_dml_3.q,alter_table_partition_drop.q,cbo_stats.q,vector_count.q,decimal_trailing.q,parquet_types_vectorization.q,smb_mapjoin_22.q,vector_decimal_6.q,autoColumnStats_8.q,input5.q,sample1.q]
TestCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=25)
[insert_overwrite_directory.q,notable_alias3.q,avro_decimal_native.q,orc_merge9.q,groupby_ppd.q,input41.q,udf_exp.q,skewjoin_mapjoin2.q,udf_concat_insert1.q,groupby2_map.q,groupby_grouping_id3.q,stats_only_null.q,insert_into3.q,vectorization_6.q,explode_null.q,input_lazyserde.q,timestamp_1.q,sort_merge_join_desc_7.q,timestamp_literal.q,skewjoinopt8.q,parquet_ppd.q,stats6.q,alias_casted_column.q,alter_merge.q,create_union_table.q,partition_timestamp_win.q,input_testxpath4.q,correlationoptimizer15.q,stats9.q,orc_merge3.q]
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3409/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3409/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3409/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12851281 - PreCommit-HIVE-Build
> Eliminate the dummy iterator and optimize the per row based reducer-side
> processing
> -----------------------------------------------------------------------------------
>
> Key: HIVE-15682
> URL: https://issues.apache.org/jira/browse/HIVE-15682
> Project: Hive
> Issue Type: Improvement
> Components: Spark
> Affects Versions: 2.2.0
> Reporter: Xuefu Zhang
> Assignee: Xuefu Zhang
> Attachments: HIVE-15682.patch
>
>
> HIVE-15580 introduced a dummy iterator per input row which can be eliminated.
> This is because {{SparkReduceRecordHandler}} is able to handle single key
> value pairs. We can refactor this part of code 1. to remove the need for a
> iterator and 2. to optimize the code path for per (key, value) based (instead
> of (key, value iterator)) processing. It would be also great if we can
> measure the performance after the optimizations and compare to performance
> prior to HIVE-15580.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
