+1 I think this is a good idea. The tradeoff here is that testing against a standalone "cluster" is even more different than testing against a distributed Pulsar cluster. I think this is fine, but if we accept this PIP, I think we should update our guidance on the Wiki, which currently recommends running release candidate validation tests against a standalone cluster [0].
- Michael [0] https://github.com/apache/pulsar/wiki/Release-Candidate-Validation#validate-pubsub-and-java-functions On Wed, Dec 15, 2021 at 5:09 AM Enrico Olivelli <eolive...@gmail.com> wrote: > > +1 > > @Haiting > Pulsar Standalone is usually used for local development, I am not sure you > need a migration tool. > You can still keep the configuration to use previously (standalone.conf) > > Enrico > > Il giorno mer 15 dic 2021 alle ore 10:56 Haiting Jiang < > jianghait...@apache.org> ha scritto: > > > +1 (non-binding) > > > > Out of the scope of this PIP, a follow up data migration tool could be > > added to help migration from old settings without data loss. > > > > --- > > Haiting > > > > On 2021/12/15 01:04:21 Hang Chen wrote: > > > +1 > > > > > > Thanks, > > > Hang > > > > > > PengHui Li <peng...@apache.org> 于2021年12月15日周三 07:52写道： > > > > > > > > +1 > > > > > > > > Penghui > > > > > > > > On Wed, Dec 15, 2021 at 1:18 AM Matteo Merli <mme...@apache.org> > > wrote: > > > > > > > > > https://github.com/apache/pulsar/issues/13302 > > > > > > > > > > Copying here for quoting convenience > > > > > ---- > > > > > > > > > > > > > > > > > > > > > > > > > ## Motivation > > > > > > > > > > Pulsar standalone is the "Pulsar in a box" version of a Pulsar > > cluster, > > > > > where > > > > > all the components are started within the context of a single JVM > > process. > > > > > > > > > > Users are using the standalone as a way to get quickly started with > > Pulsar > > > > > or > > > > > in all the cases where it makes sense to have a single node > > deployment. > > > > > > > > > > Right now, the standalone is starting by default with many > > components, > > > > > several of > > > > > which are quite complex, since they are designed to be deployed in a > > > > > distributed > > > > > fashion. > > > > > > > > > > ## Goal > > > > > > > > > > Simplify the components of Pulsar standalone to achieve: > > > > > > > > > > 1. Reduce complexity > > > > > 2. Reduce startup time > > > > > 3. Reduce memory and CPU footprint of running standalone > > > > > > > > > > ## Proposed changes > > > > > > > > > > The proposal here is to change some of the default implementations > > that are > > > > > used for the Pulsar standalone. > > > > > > > > > > 1. **Metadata Store implementation** --> > > > > > Change from ZooKeeper to RocksDB > > > > > > > > > > 2. **Pulsar functions package backend** --> > > > > > Change from using DistributedLog to using local filesystem, > > storing > > > > > the > > > > > jars directly in the data folder instead of uploading them > > into BK. > > > > > > > > > > 3. **Pulsar functions state store implementation** --> > > > > > Change the state store to be backed by a MetadataStore based > > backed, > > > > > with the RocksDB implementation. > > > > > > > > > > 4. **Table Service** --> > > > > > Do not start BK table service by default > > > > > > > > > > ## Compatibility considerations > > > > > > > > > > In order to avoid compatibility issues where users have existing > > Pulsar > > > > > standalone services that they want to upgrade without conflicts, we > > will > > > > > follow the principle of keeping the old defaults where there is > > existing > > > > > data on the disk. > > > > > > > > > > We will add a file, serving the purpose as a flag, in the > > `data/standalone` > > > > > directory, for example `new-2.10-defaults`. > > > > > > > > > > If the file is present, or if the data directory is completely > > missing, we > > > > > will adopt the new set of default configuration settings. > > > > > > > > > > If the file is not there, we will continue to use existing defaults > > and we > > > > > will > > > > > not break the upgrade operation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Matteo Merli > > > > > <mme...@apache.org> > > > > > > > > > >
