[
https://issues.apache.org/jira/browse/IMPALA-4568?focusedWorklogId=995099&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-995099
]
ASF GitHub Bot logged work on IMPALA-4568:
------------------------------------------
Author: ASF GitHub Bot
Created on: 08/Dec/25 03:53
Start Date: 08/Dec/25 03:53
Worklog Time Spent: 10m
Work Description: zhangyifan27 opened a new pull request, #83:
URL: https://github.com/apache/impala/pull/83
(no comment)
Issue Time Tracking
-------------------
Worklog Id: (was: 995099)
Remaining Estimate: 0h
Time Spent: 10m
> Cache Parquet footer cache to speedup scans & predicate evaluation against
> Min/Max indexes
> ------------------------------------------------------------------------------------------
>
> Key: IMPALA-4568
> URL: https://issues.apache.org/jira/browse/IMPALA-4568
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.7.0
> Reporter: Mostafa Mokhtar
> Priority: Major
> Labels: performance
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Implement an LRU based footer cache for Parquet to speedup scans & predicate
> evaluation against Min/Max indexes.
> Footer processing time can spike to 100s of ms, the cache should greatly help
> here.
> {code}
> FooterProcessingTime: (Avg: 12.142ms ; Min: 1.045ms ; Max: 279.706ms ;
> Number of samples: 323)
> FooterProcessingTime: (Avg: 6.935ms ; Min: 1.103ms ; Max: 226.230ms ;
> Number of samples: 327)
> FooterProcessingTime: (Avg: 18.576ms ; Min: 1.162ms ; Max: 560.226ms ;
> Number of samples: 306)
> FooterProcessingTime: (Avg: 12.607ms ; Min: 1.152ms ; Max: 222.173ms ;
> Number of samples: 304)
> FooterProcessingTime: (Avg: 11.916ms ; Min: 1.138ms ; Max: 318.401ms ;
> Number of samples: 317)
> FooterProcessingTime: (Avg: 17.501ms ; Min: 1.156ms ; Max: 659.679ms ;
> Number of samples: 309)
> FooterProcessingTime: (Avg: 7.654ms ; Min: 907.422us ; Max: 380.254ms ;
> Number of samples: 322)
> FooterProcessingTime: (Avg: 13.032ms ; Min: 1.119ms ; Max: 412.419ms ;
> Number of samples: 314)
> FooterProcessingTime: (Avg: 14.271ms ; Min: 1.239ms ; Max: 209.740ms ;
> Number of samples: 295)
> FooterProcessingTime: (Avg: 10.850ms ; Min: 1.217ms ; Max: 311.340ms ;
> Number of samples: 311)
> FooterProcessingTime: (Avg: 15.343ms ; Min: 803.346us ; Max: 1s108ms ;
> Number of samples: 304)
> FooterProcessingTime: (Avg: 13.697ms ; Min: 1.088ms ; Max: 243.107ms ;
> Number of samples: 296)
> FooterProcessingTime: (Avg: 11.365ms ; Min: 1.268ms ; Max: 397.234ms ;
> Number of samples: 310)
> FooterProcessingTime: (Avg: 9.690ms ; Min: 1.213ms ; Max: 410.280ms ;
> Number of samples: 322)
> FooterProcessingTime: (Avg: 11.259ms ; Min: 1.192ms ; Max: 210.212ms ;
> Number of samples: 299)
> FooterProcessingTime: (Avg: 14.689ms ; Min: 1.277ms ; Max: 414.577ms ;
> Number of samples: 311)
> FooterProcessingTime: (Avg: 11.807ms ; Min: 1.088ms ; Max: 535.455ms ;
> Number of samples: 321)
> FooterProcessingTime: (Avg: 13.488ms ; Min: 1.155ms ; Max: 449.439ms ;
> Number of samples: 311)
> FooterProcessingTime: (Avg: 12.501ms ; Min: 1.035ms ; Max: 298.715ms ;
> Number of samples: 302)
> FooterProcessingTime: (Avg: 7.310ms ; Min: 1.032ms ; Max: 182.076ms ;
> Number of samples: 308)
> {code}
> Note that the above timer includes both the I/O time and the time taken to
> decode the thrift and resolve columns, etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
