GitHub user yjhjstz added a comment to the discussion: [Proposal] Cloudberry Multi-Catalog System Design Proposal
**Question 1: What is the expected time to 'go live' with this feature?** We expect to deliver an initial working version of the multi-catalog system within **3-6 months**, depending on community participation and contribution levels. Our implementation approach will be: - **Phase 1 (Months 1-2)**: Core MVS infrastructure and catalog registry - **Phase 2 (Months 2-3)**: **Iceberg catalog as the first implementation** - this gives us immediate value for modern data lakehouse use cases - **Phase 3 (Months 3-4)**: FDW integration and basic query execution - **Phase 4 (Months 4-6)**: Performance optimization and production hardening **Why Iceberg First?** - **High Demand**: Iceberg is the most requested lakehouse format in the community - **Mature Ecosystem**: Official Rust client (iceberg-rust) and well-defined REST catalog spec - **Proven Architecture**: Apache Doris has demonstrated successful Iceberg integration patterns we can adapt - **Immediate Value**: Users can query S3/cloud data lakes directly without data movement We're committed to delivering this feature incrementally with regular community updates and feedback integration. The Iceberg catalog will serve as our foundation for expanding to other catalog types (Hive, Delta Lake, etc.) in subsequent releases. **Question 2: Does this mean PXF will be gradually deprecated, or can it be integrated as one of the catalogs?** PXF will **NOT be deprecated**. Instead, it will be integrated as one of the catalog plugins within the multi-catalog framework. This approach: - **Preserves existing PXF investments** and user workflows - **Provides unified access** through `catalog.database.table` syntax - **Enables gradual migration** from file-based to metadata-rich catalog systems - **Supports hybrid architectures** where both legacy and modern data coexist Users will benefit from both PXF's mature connector ecosystem and the new metadata-aware lakehouse catalogs under a single, consistent interface. Links are fixed. GitHub link: https://github.com/apache/cloudberry/discussions/1297#discussioncomment-14046719 ---- This is an automatically sent email for dev@cloudberry.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@cloudberry.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
