BTW - Once the VOTE is complete to accept the codebase there is some paperwork.
http://incubator.apache.org/ip-clearance/ While this is the Incubator the Board has requested that the IPMC track IP Clearance. Everything is in SVN and the pages are updated every evening PDT. If help is needed then let me know. Regards, Dave > On Sep 4, 2019, at 9:01 AM, Enrico Olivelli <eolive...@gmail.com> wrote: > > For whom who is interested I am working on adding support for HerdDB in > Pulsar Manager. > > The pull request is still very raw, but Pulsar Manager seems to work > https://github.com/streamnative/pulsar-manager/pull/183 > > > > Il giorno mar 3 set 2019 alle ore 08:37 Yuva raj <uvar...@gmail.com> ha > scritto: > >> On Fri, 30 Aug 2019 at 07:09, Sijie Guo <guosi...@gmail.com> wrote: >> >>>> I see there are dependencies on websockets, I would a prefer a simple >>> polling model of the http, also it's enable by default in pulsar. >>> >>> I don't think we depend don pulsar websocket. All are http restful >>> requests. >>> >>>> For the ui the default persistence should be sqlite. Potentially >> packaged >>> by default. >>> >>> Initially the default is sqlite. But sqlite doesn't work if there are a >> lot >>> of topic metrics. >>> We switched to MySQL as default for supporting production traffic. We can >>> switch default to PostgresSQL (as pulsar dashboard). >>> >> +1 >> Yes, Switching to Postgres Would be great. Because Postgres license is more >> liberal and works well with Apache license ecosystem. >> >>> >>> - Sijie >>> >>> On Thu, Aug 29, 2019 at 6:23 PM Ali Ahmed <alahmed...@gmail.com> wrote: >>> >>>> I see there are dependencies on websockets, I would a prefer a simple >>>> polling model of the http, also it's enable by default in pulsar. >>>> >>>> For the ui the default persistence should be sqlite. Potentially >> packaged >>>> by default. >>>> >>>> -Ali >>>> >>>> On Thu, Aug 29, 2019 at 2:47 PM Enrico Olivelli <eolive...@gmail.com> >>>> wrote: >>>> >>>>> Il gio 29 ago 2019, 23:28 Sijie Guo <guosi...@gmail.com> ha scritto: >>>>> >>>>>> That sounds an interesting idea! >>>>> >>>>> >>>>> Awesome >>>>> >>>>> Does HerdDB support JDBC? If so, it should >>>>>> be pretty straightforward to enable HerdDB. >>>>>> >>>>> >>>>> Sure, as far as I know, the JDBC client is the only client really >> used >>> in >>>>> production. >>>>> >>>>> In a replicated environment it uses Zookeeper for metadata and >> service >>>>> discovery and Bookkeeper as WAL and if you have a pulsar cluster you >>>>> already have both of them (zk cluster an bookies) >>>>> >>>>> I will take a look to how Pulsar Manager uses JDBC, maybe it will >> be >>>> very >>>>> easy. >>>>> >>>>> I will be back with news >>>>> >>>>> >>>>> Enrico >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Thanks, >>>>>> Sijie >>>>>> >>>>>> On Thu, Aug 29, 2019 at 12:08 PM Enrico Olivelli < >>> eolive...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Maybe you can try to use HerdDB (1), it is a replicated SQL >>> database >>>>> that >>>>>>> can be run embedded in the JVM. >>>>>>> It is an SQL database that we developed in my company, in order >> to >>>>>>> explicitly replace MySQL. >>>>>>> It uses Bookkeeper to store the WAL and implement replication, >> but >>> it >>>>> can >>>>>>> run in standalone mode or in memory (for tests/dev). >>>>>>> >>>>>>> I can help in setting up a demo. >>>>>>> >>>>>>> Enrico >>>>>>> >>>>>>> (1) https://github.com/diennea/herddb >>>>>>> >>>>>>> Il gio 29 ago 2019, 16:59 Sijie Guo <guosi...@gmail.com> ha >>> scritto: >>>>>>> >>>>>>>> Hi Dave, >>>>>>>> >>>>>>>> It doesn't require MySQL. You can use any JDBC supported >>> databases. >>>>>>>> Initially we would use SQLite for development and change to >> MySQL >>>> for >>>>>>> more >>>>>>>> production-ready workloads. >>>>>>>> >>>>>>>> Hope this clarify your question. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Sijie >>>>>>>> >>>>>>>> On Thu, Aug 29, 2019 at 5:56 AM Dave Fisher <w...@apache.org> >>>> wrote: >>>>>>>> >>>>>>>>> Hi - >>>>>>>>> >>>>>>>>> I see that MySQL is used. This would be a GPL dependency. >> Does >>>> the >>>>>>> Pulsar >>>>>>>>> Manager require MySQL, or can other Databases that are >>> compliant >>>>> with >>>>>>>>> Apache Release Policy be used instead? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Dave >>>>>>>>> >>>>>>>>>> On Aug 29, 2019, at 7:49 AM, Guangning E < >>> eguangn...@gmail.com >>>>> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> We have developed a new Pulsar web UI - pulsar-manager, >>> aiming >>>> at >>>>>>>>>> supporting managing Pulsar clusters running in different >>>>>> environments >>>>>>>>>> (on-premise data centers, cloud, and so on). We’d like to >>>>>> contribute >>>>>>>> the >>>>>>>>>> project back to the Pulsar community. >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> >>>>>>>>>> ## Motivation >>>>>>>>>> >>>>>>>>>> Currently, Pulsar has a monitoring tool - [Pulsar >>> Dashboard]( [ >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> http://pulsar.apache.org/docs/en/administration-dashboard/](http://pulsar.apache.org/docs/en/administration-dashboard/) >>>>>>>>>> ). >>>>>>>>>> However, it only focuses on the simple monitoring of >> Pulsar - >>>>>>>> collecting >>>>>>>>>> and displaying information such as statistics of tenants, >>>>>> namespaces, >>>>>>>>>> topics, subscriptions, and so on. It doesn’t provide any >>>>> management >>>>>>>>>> operations such as add, delete and update tenants, >>> namespaces, >>>>>>> topics, >>>>>>>>> and >>>>>>>>>> so on. When the scale of Pulsar cluster increases or the >>> number >>>>> of >>>>>>>>> clusters >>>>>>>>>> grows, using `pulsar-admin` to manage Pulsar can not >> satisfy >>>>>> demands. >>>>>>>>>> Therefore, Pulsar needs a simple and easy-to-use management >>>>> console >>>>>>> for >>>>>>>>>> administrators. >>>>>>>>>> >>>>>>>>>> Pulsar Manager is a web-based GUI management and monitoring >>>> tool >>>>>> that >>>>>>>>> helps >>>>>>>>>> administrators and users manage and monitor tenants, >>>> namespaces, >>>>>>>> topics, >>>>>>>>>> subscriptions, brokers, clusters, and so on, and supports >>>> dynamic >>>>>>>>>> configuration of multiple environments. >>>>>>>>>> >>>>>>>>>> ## Features >>>>>>>>>> >>>>>>>>>> Pulsar manager provides two main features: management and >>>>>> monitoring. >>>>>>>>>> >>>>>>>>>> ### Management >>>>>>>>>> >>>>>>>>>> 1. Environment : (operations on the environment) >>>>>>>>>> Create / Delete / Update / List / Get >>>>>>>>>> 2. Cluster : (operations on clusters) >>>>>>>>>> Create / Delete / Update / List / Get >>>>>>>>>> 3. Brokers : (operations on brokers) >>>>>>>>>> Heartbeat / Unload >>>>>>>>>> 4. Tenants: >>>>>>>>>> Create / Delete / Update / List >>>>>>>>>> 5. Namespaces: >>>>>>>>>> Create / Delete / List >>>>>>>>>> Manage Namespace Policy >>>>>>>>>> Unload >>>>>>>>>> Operations on Namespace Bundles >>>>>>>>>> 6. Topics >>>>>>>>>> Create / Delete / List >>>>>>>>>> Unload / Terminate / Offload / Compact >>>>>>>>>> Topic Details >>>>>>>>>> 7. Subscriptions >>>>>>>>>> Create / Delete / List >>>>>>>>>> Reset Cursor / Skip / Clear backlog / Unsubscribe >>>>>>>>>> 8. Namespace Isolation Policies >>>>>>>>>> Create / Delete / Update / Get / List >>>>>>>>>> 9. Failure Domains >>>>>>>>>> Create / Delete / Update / Get / List >>>>>>>>>> >>>>>>>>>> ### Monitoring: (display *aggregated* stats at different >>>> levels) >>>>>>>>>> >>>>>>>>>> 1. Tenants >>>>>>>>>> List the total number of namespaces per tenant (both >>> aggregated >>>>> and >>>>>>>>>> per-cluster basis) >>>>>>>>>> >>>>>>>>>> 2. Namespaces >>>>>>>>>> List the total number of topics per namespace >>>>>>>>>> List the aggregated stats per namespace (such as rate-in, >>>>> rate-out, >>>>>>>>>> throughput-in, and throughput-out) >>>>>>>>>> List the distributions of namespace bundles >>>>>>>>>> >>>>>>>>>> 3. Topics >>>>>>>>>> List the total number of partitions per topic >>>>>>>>>> List the aggregated stats per topic (such as rate-in, >>> rate-out, >>>>>>>>>> throughput-in, and throughput-out) >>>>>>>>>> Detailed stats per topic partition >>>>>>>>>> Detailed stats of storage per topic partition >>>>>>>>>> Detailed stats of subscription per topic and per partition >>>>>>>>>> Detailed stats of producers >>>>>>>>>> Detailed stats of consumers >>>>>>>>>> >>>>>>>>>> The detailed design proposal is in >>>>>>>>>> [ >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> https://docs.google.com/document/d/1C3meaHJzxX9wGDWQx-dC1b20yGeKsQtbwNiYuxZgOCE/edit#](https://docs.google.com/document/d/1C3meaHJzxX9wGDWQx-dC1b20yGeKsQtbwNiYuxZgOCE/edit#) >>>>>>>>>> >>>>>>>>>> Looking forward to any feedback. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Guangning >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> *Thanks* >> >> *Yuvaraj L* >>
