GaneshPatil7517 commented on PR #19848: URL: https://github.com/apache/datafusion/pull/19848#issuecomment-3760766374
> I don't think we can implement this until now, the arrow-rs and parquet both not support it. Thank you for the feedback @zhuqi-lucas! You're absolutely right - that's exactly why this is **Phase 1 infrastructure only**. This PR does not attempt to implement actual page-level reversal. Instead, it: 1. **Establishes the infrastructure foundation** - Adds the `reverse_pages` flag and wires it through the ParquetSource → ParquetOpener chain, following the existing `reverse_row_groups` pattern. 2. **Defers implementation to Phase 2** - The PR includes code comments noting that actual page-level reversal will be implemented in Phase 2 once arrow-rs and parquet add the necessary public APIs. 3. **Aligns with DataFusion patterns** - Uses the same builder pattern and infrastructure approach as the existing reverse_row_groups feature. The goal of Phase 1 is to establish the groundwork so that when arrow-rs and parquet add page-level reversal support, the DataFusion integration will be straightforward. Would you like me to add more explicit documentation in the code comments clarifying this Phase 1 vs Phase 2 split? I'm happy to update the PR if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
