Dear Pulsar community members, PIP-62[1], "PIP 62: Move connectors, adapters and Pulsar Presto to separate repositories" was created in April 2020. The repositories for pulsar-connectors, pulsar-adapters and pulsar-sql were created about a year ago [2].
What is the current roadmap for completing PIP-62 and moving pulsar-connectors and pulsar-sql out of apache/pulsar repository? BR, Lari [1] https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar-Presto-to-separate-repositories [2] https://lists.apache.org/thread.html/r9e6ec742e2896da1f0ce7d4adc7cb84fc6db6dbf797732ccdd50fb86%40%3Cdev.pulsar.apache.org%3E Other email threads: * [Discuss] Don't include presto/trino in the normal Pulsar distribution - https://lists.apache.org/thread/jn96tct54mn0tvdot62vdslrvs38fm6d * Updates on Presto connector for PIP-62 - https://lists.apache.org/thread/f9n6sc2mrboq5sxhjbr7gvdl8vqp9fpk On Tue, Nov 2, 2021 at 3:59 PM Nicolò Boschi <boschi1...@gmail.com> wrote: > Resurrecting this thread. > > 2.9 is almost released and it hasn't been merged yet > > Extending the discussion to other connectors, it looks like there has been > no progress on PIP-62. > My concern is that a lot of Pulsar IO connectors dependencies we are > running are obsolete with several security reports > > I see there are interesting comments in the issue ( > https://github.com/apache/pulsar/issues/10219) and Sijie exported the > pulsar-io dir to https://github.com/apache/pulsar-connectors but it's > outdated > > From my point of view, we have to: > - reimport all the connectors source codes with newest ones (including > integration tests) > - add periodic CI jobs for connectors to run against master, 2.9-latest, > 2.8-latest, 2.7-latest to verify breaking changes > - define a release cycle/management for connectors (we should improve the > PIP doc). IMO it's not clear if each connector will have its own release > versions and how we'll handle it (git tags, artifacts deployment..) > - update pulsar release script in order to get the connectors artifacts > (retrieving the .nar or building it from source?) > - update docs > - remove pulsar-io dir from Pulsar repo > > It's the perfect timing to schedule this work for 2.10 > > What is missing? How's the situation? Is there a roadblock I haven't seen? > I think it's better to take another discussion for Presto since it will > come to another end > > > Il giorno sab 14 ago 2021 alle ore 15:21 Enrico Olivelli < > eolive...@gmail.com> ha scritto: > > > Sijie > > > > Il Ven 13 Ago 2021, 22:00 Sijie Guo <guosi...@gmail.com> ha scritto: > > > > > You can follow the progress at > > https://github.com/trinodb/trino/pull/8020. > > > > > > > Thanks for the pointer > > > > > > > > The original code doesn't conform to TrinoDB's standard. Marvin is > > > actively following up on that. > > > > > > Our goal is still to get this completed as part of the 2.9 release. > > > > > > > Wonderful > > > > Thanks > > Enrico > > > > > > > > - Sijie > > > > > > On Fri, Aug 13, 2021 at 2:04 AM Enrico Olivelli <eolive...@gmail.com> > > > wrote: > > > > > > > > Hello, > > > > How is the Presto work going ? > > > > IIRC the plan was to remove it from the Pulsar code base and let it > be > > > > hosted at Trino. > > > > > > > > If this is not going to happen within the 2.9.0 release timeline > > > > (September?) I would prefer to upgrade to "Trino". > > > > Probably we will have a downside problem that recent versions of > > > > Presto/Trino do not work on JDK8 but only on JDK11. > > > > > > > > I believe that in that case we could open a separate thread to say > that > > > > Pulsar SQL in 2.9.0 will work only on JDK11. > > > > In Pulsar 2.8.0 we added official compatibility with JDK11 (and it is > > the > > > > preferred version, as it is the version used in the Docker images), > so > > > > requiring JDK11 for Pulsar SQL 2.9.0 does not sound bad to me. > > > > > > > > My primary concern is that the version of Presto that we are running > is > > > > obsolete and there are several security reports against it or its > third > > > > party dependencies. > > > > > > > > Thoughts ? > > > > > > > > Enrico > > > > > > > > -- > Nicolò Boschi >
