GitHub user yjhjstz added a comment to the discussion: [Proposal] Apache Cloudberry as PostgreSQL 19 Extensions
Hi @igor-suhorukov, Thanks for this impressive experiment. It shows something many of us assumed was impossible: distributed snapshots, 2PC and shared reader/writer transactions running on a small set of dormant hooks, with ORCA compiled unmodified behind `planner_hook`. I agree that an extension-based Cloudberry is the right long-term direction. Following PostgreSQL a few majors behind is a structural problem, and this is the most credible answer to it I have seen. That said, I don't think the community should adopt the prototype as it is, for five reasons: 1. **It is a different product, not the next version.** Upgrading requires dump/restore, GUCs are renamed to `gp.*`, the shared catalogs and TDE are gone, and DDL command tags change. For existing 2.x users that is a migration to a new system, which makes it hard to position as Cloudberry's successor. 2. **The riskiest parts are not yet verified.** Transaction semantics rest on new mechanisms: `XactAdoptTransactionState()`, segments waiting on prepared transactions, and a replication slot holding back VACUUM. `isolation2`, the suite that exercises exactly these paths, has not passed yet, and overall coverage is 691 of 1,180 test files. Until those pass under concurrency, correctness is an open question. 3. **The performance cost is not measured.** Dispatching each slice as `SELECT gp_internal.exec_fragment(...)` over libpq, and passing the snapshot as `SET LOCAL`, adds overhead on every query. The TPC results (SF1, one container, parallel off) show correct answers, not performance. We would need short-query latency, high-concurrency throughput and larger-scale numbers before comparing it with the native dispatcher. 4. **It is still a fork, only a smaller one.** Of the 22 hooks, four look upstreamable. The other 18 mean maintaining a patched PostgreSQL indefinitely, so users still can't run on stock or managed PostgreSQL. The maintenance burden shrinks but does not go away. 5. **Code ownership.** About 600 commits written in 12 days is a remarkable result, but the community would inherit a ~33k-line translator and a new transaction layer that nobody here knows deeply yet. As @leborchuk said, rebasing and long-term evolution depend on people who own and understand the code. My suggestions: 1. **Evolve from `main` incrementally instead of replacing it.** Start with modules that have clear boundaries and need no deep kernel changes, such as AO/AOCS as table AMs, external tables as FDWs, and the `gp_sql` syntax layer. Use `pg19/` as the reference implementation and requirements list, not as a drop-in replacement. 2. **Upstream the general-purpose hooks first.** The table AM registry, smgr file events, parser hook and OID hook would benefit PostgreSQL broadly, and Cloudberry can back them as a real-world user. Every hook accepted upstream lowers the cost of the extension approach for good. 3. **Validate the transaction layer before going further.** Get `isolation2` passing and publish short-query latency and concurrency benchmarks against the native dispatcher. That data should decide whether the MPP core can move to extensions, not the other way round. 4. **On Route B: don't port it; keep reducing ORCA fallbacks.** Porting 30–45k lines of the MPP planner would recreate the very merge burden this effort is trying to remove. Fallback reductions (multi-level partitions, aggregate `FILTER`, `LATERAL`, non-default collations) benefit both `main` and the extension port. ORCA is also the natural first shared component, since its core does not depend on PostgreSQL headers and only the translator tracks the PostgreSQL version. 5. **Host it as a separate experimental repository.** Something like `cloudberry-pg-ext` keeps the work visible and open to contributors without implying it is the official roadmap. That separation can be revisited once items 2 and 3 have results. GitHub link: https://github.com/apache/cloudberry/discussions/2065#discussioncomment-18701062 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
