Hi Marvin, Great work on the Trino PR! It's been a lot of work to get it to match the Trino code conventions.
I hope we could drop Presto & Pulsar SQL from the apache/pulsar code repository as planned in PIP-62[1], "PIP 62: Move connectors, adapters and Pulsar Presto to separate repositories", which was created in April 2020. Let's work together to complete this effort. Is there anything that others could help with to complete the Trino PR https://github.com/trinodb/trino/pull/8020 ? BR, Lari [1] https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar- Presto-to-separate-repositories On Wed, Nov 17, 2021 at 3:40 PM Zhengxin Cai <cai19930...@gmail.com> wrote: > Hi there, > I think the pr is still open, https://github.com/trinodb/trino/pull/8020, > will try to push it. > But even after the pr is merged, I actually still think we might still want > to keep a copy of the connector in Pulsar repo and push changes to Trino > repo periodically, as this will allow much faster bug fix and feature > iteration. > Best, > Marvin, > > Lari Hotari <lhot...@apache.org> 于2021年11月17日周三 下午2:19写道： > > > Dear Pulsar community members, > > > > PIP-62[1], "PIP 62: Move connectors, adapters and Pulsar Presto to > separate > > repositories" was created in April 2020. The repositories for > > pulsar-connectors, pulsar-adapters and pulsar-sql were created about a > year > > ago [2]. > > > > What is the current roadmap for completing PIP-62 and moving > > pulsar-connectors and pulsar-sql out of apache/pulsar repository? > > > > BR, > > > > Lari > > > > [1] > > > > > https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar-Presto-to-separate-repositories > > [2] > > > > > https://lists.apache.org/thread.html/r9e6ec742e2896da1f0ce7d4adc7cb84fc6db6dbf797732ccdd50fb86%40%3Cdev.pulsar.apache.org%3E > > > > Other email threads: > > * [Discuss] Don't include presto/trino in the normal Pulsar distribution > - > > https://lists.apache.org/thread/jn96tct54mn0tvdot62vdslrvs38fm6d > > * Updates on Presto connector for PIP-62 - > > https://lists.apache.org/thread/f9n6sc2mrboq5sxhjbr7gvdl8vqp9fpk > > > > On Tue, Nov 2, 2021 at 3:59 PM Nicolò Boschi <boschi1...@gmail.com> > wrote: > > > > > Resurrecting this thread. > > > > > > 2.9 is almost released and it hasn't been merged yet > > > > > > Extending the discussion to other connectors, it looks like there has > > been > > > no progress on PIP-62. > > > My concern is that a lot of Pulsar IO connectors dependencies we are > > > running are obsolete with several security reports > > > > > > I see there are interesting comments in the issue ( > > > https://github.com/apache/pulsar/issues/10219) and Sijie exported the > > > pulsar-io dir to https://github.com/apache/pulsar-connectors but it's > > > outdated > > > > > > From my point of view, we have to: > > > - reimport all the connectors source codes with newest ones (including > > > integration tests) > > > - add periodic CI jobs for connectors to run against master, > 2.9-latest, > > > 2.8-latest, 2.7-latest to verify breaking changes > > > - define a release cycle/management for connectors (we should improve > the > > > PIP doc). IMO it's not clear if each connector will have its own > release > > > versions and how we'll handle it (git tags, artifacts deployment..) > > > - update pulsar release script in order to get the connectors artifacts > > > (retrieving the .nar or building it from source?) > > > - update docs > > > - remove pulsar-io dir from Pulsar repo > > > > > > It's the perfect timing to schedule this work for 2.10 > > > > > > What is missing? How's the situation? Is there a roadblock I haven't > > seen? > > > I think it's better to take another discussion for Presto since it will > > > come to another end > > > > > > > > > Il giorno sab 14 ago 2021 alle ore 15:21 Enrico Olivelli < > > > eolive...@gmail.com> ha scritto: > > > > > > > Sijie > > > > > > > > Il Ven 13 Ago 2021, 22:00 Sijie Guo <guosi...@gmail.com> ha scritto: > > > > > > > > > You can follow the progress at > > > > https://github.com/trinodb/trino/pull/8020. > > > > > > > > > > > > > Thanks for the pointer > > > > > > > > > > > > > > The original code doesn't conform to TrinoDB's standard. Marvin is > > > > > actively following up on that. > > > > > > > > > > Our goal is still to get this completed as part of the 2.9 release. > > > > > > > > > > > > > Wonderful > > > > > > > > Thanks > > > > Enrico > > > > > > > > > > > > > > - Sijie > > > > > > > > > > On Fri, Aug 13, 2021 at 2:04 AM Enrico Olivelli < > eolive...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > Hello, > > > > > > How is the Presto work going ? > > > > > > IIRC the plan was to remove it from the Pulsar code base and let > it > > > be > > > > > > hosted at Trino. > > > > > > > > > > > > If this is not going to happen within the 2.9.0 release timeline > > > > > > (September?) I would prefer to upgrade to "Trino". > > > > > > Probably we will have a downside problem that recent versions of > > > > > > Presto/Trino do not work on JDK8 but only on JDK11. > > > > > > > > > > > > I believe that in that case we could open a separate thread to > say > > > that > > > > > > Pulsar SQL in 2.9.0 will work only on JDK11. > > > > > > In Pulsar 2.8.0 we added official compatibility with JDK11 (and > it > > is > > > > the > > > > > > preferred version, as it is the version used in the Docker > images), > > > so > > > > > > requiring JDK11 for Pulsar SQL 2.9.0 does not sound bad to me. > > > > > > > > > > > > My primary concern is that the version of Presto that we are > > running > > > is > > > > > > obsolete and there are several security reports against it or its > > > third > > > > > > party dependencies. > > > > > > > > > > > > Thoughts ? > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > -- > > > Nicolò Boschi > > > > > >
