GitHub user MisterRaindrop added a comment to the discussion: [DISCUSSION] Making Apache Cloudberry an Agent-Native Analytical Database
**A concrete version of option B, for reference** Following up on the question above about where `lance_fdw` should live: to make it easier to discuss, I've prepared what option B would look like on my fork. This isn't meant to pre-empt the decision. If the community prefers to keep `lance_fdw` as a separate project, I'm happy to continue it there. **What it looks like** - `contrib/lance_fdw` is built only with `configure --with-lance`, which is off by default. configure looks for `<lance/lance.h>` and `liblance_c` through `--with-includes` and `--with-libraries`, much as it does for libxml2. Only `contrib/lance_fdw` links the library. - No Rust code, crates or cargo invocation are added to the tree. The only bundled third-party code is Apache Arrow nanoarrow 0.7.0 (Apache-2.0). - On the CI question I asked earlier: rather than depending on a prebuilt binary, a new `ic-lance-fdw` entry builds the pinned lance-c revision from source and caches it. The cold build takes about 15 minutes. Like `ic-datalake-fdw`, it then builds the extension with PGXS, generates the test datasets, serves them from a local SeaweedFS S3 gateway, and runs the regression suites. It currently passes on Rocky 8, 9 and 10. **A possible split**, following the outline above. Each step builds and passes its own suites: 1. Build integration, CI, options, `IMPORT FOREIGN SCHEMA`, and the parallel read-only scan with type mapping and memory accounting 2. Filter and projection pushdown 3. Vector operators and Top-K pushdown, including cancel during a search 4. Distributed Top-K The extension would start at 1.0. If this direction seems reasonable, I could open a tracking issue that lists these four PRs and the two `mcp-server` PRs (Lance reads, and Phase 4 result write-back), and then open the PRs for review. I'd be glad to hear any concerns or preferences first, about the approach, the split, or the CI setup. Thanks also to @igor-suhorukov for the `pg_duckdb` suggestion. As noted in my reply, the two look complementary rather than interchangeable. @my-ship-it @tuhaihe GitHub link: https://github.com/apache/cloudberry/discussions/1967#discussioncomment-18949885 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
