This is an automated email from the ASF dual-hosted git repository. stigahuang pushed a commit to branch branch-4.1.2 in repository https://gitbox.apache.org/repos/asf/impala.git
commit c6223b2aeb8ae23a094551aa2abc8fab75e13165 Author: stiga-huang <[email protected]> AuthorDate: Tue Feb 28 08:48:50 2023 +0800 IMPALA-11953: Declare num_trues and num_falses in TIntermediateColumnStats as optional TIntermediateColumnStats is the representation of incremental stats which are stored in HMS partition properties using keys like "impala_intermediate_stats_chunk0", "impala_intermediate_stats_chunk1", "impala_intermediate_stats_chunk2", etc. Fields in TIntermediateColumnStats should be optional to ensure backward compatibility. IMPALA-8205 adds two required fields, num_trues and num_falses, in TIntermediateColumnStats. This breaks the incremental stats loading in higher versions of Impala if the stats are generated by older Impala versions (< 4.0). This patch changes the fields to be optional. Tests: - Verified the incremental stats generated by CDH Impala cluster can be loaded by CDP Impala cluster with this fix. Change-Id: I4f74d5d0676e7ce9eb4ea8061a15610846db3ca5 Reviewed-on: http://gerrit.cloudera.org:8080/19555 Reviewed-by: Riza Suminto <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- common/thrift/CatalogObjects.thrift | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/common/thrift/CatalogObjects.thrift b/common/thrift/CatalogObjects.thrift index 8aa1116c6..7f095fbc1 100644 --- a/common/thrift/CatalogObjects.thrift +++ b/common/thrift/CatalogObjects.thrift @@ -201,6 +201,8 @@ struct TColumnStats { // Intermediate state for the computation of per-column stats. Impala can aggregate these // structures together to produce final stats for a column. +// Fields should be optional for backward compatibility since this is stored in HMS +// partition properties. struct TIntermediateColumnStats { // One byte for each bucket of the NDV HLL computation 1: optional binary intermediate_ndv @@ -221,8 +223,8 @@ struct TIntermediateColumnStats { 6: optional i64 num_rows // The number of true and false value, of the column - 7: required i64 num_trues - 8: required i64 num_falses + 7: optional i64 num_trues + 8: optional i64 num_falses // The low and the high value 9: optional Data.TColumnValue low_value
