For whom who is interested I am working on adding support for HerdDB in
Pulsar Manager.

The pull request is still very raw, but Pulsar Manager seems to work
https://github.com/streamnative/pulsar-manager/pull/183



Il giorno mar 3 set 2019 alle ore 08:37 Yuva raj <uvar...@gmail.com> ha
scritto:

> On Fri, 30 Aug 2019 at 07:09, Sijie Guo <guosi...@gmail.com> wrote:
>
> > > I see there are dependencies on websockets, I would a prefer a simple
> > polling model of the http, also it's enable by default in pulsar.
> >
> > I don't think we depend don pulsar websocket. All are http restful
> > requests.
> >
> > > For the ui the default persistence should be sqlite. Potentially
> packaged
> > by default.
> >
> > Initially the default is sqlite. But sqlite doesn't work if there are a
> lot
> > of topic metrics.
> > We switched to MySQL as default for supporting production traffic. We can
> > switch default to PostgresSQL (as pulsar dashboard).
> >
> +1
> Yes, Switching to Postgres Would be great. Because Postgres license is more
> liberal and works well with Apache license ecosystem.
>
> >
> > - Sijie
> >
> > On Thu, Aug 29, 2019 at 6:23 PM Ali Ahmed <alahmed...@gmail.com> wrote:
> >
> > > I see there are dependencies on websockets, I would a prefer a simple
> > > polling model of the http, also it's enable by default in pulsar.
> > >
> > > For the ui the default persistence should be sqlite. Potentially
> packaged
> > > by default.
> > >
> > > -Ali
> > >
> > > On Thu, Aug 29, 2019 at 2:47 PM Enrico Olivelli <eolive...@gmail.com>
> > > wrote:
> > >
> > > > Il gio 29 ago 2019, 23:28 Sijie Guo <guosi...@gmail.com> ha scritto:
> > > >
> > > > > That sounds an interesting idea!
> > > >
> > > >
> > > > Awesome
> > > >
> > > > Does HerdDB support JDBC? If so, it should
> > > > > be pretty straightforward to enable HerdDB.
> > > > >
> > > >
> > > > Sure, as far as I know, the JDBC client is the only client really
> used
> > in
> > > > production.
> > > >
> > > > In a replicated environment it uses Zookeeper for metadata and
> service
> > > > discovery and Bookkeeper as WAL and if you have a pulsar cluster you
> > > > already have both of them (zk cluster an bookies)
> > > >
> > > > I will take a look to how Pulsar Manager   uses JDBC, maybe it will
> be
> > > very
> > > > easy.
> > > >
> > > > I will be back with news
> > > >
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Thanks,
> > > > > Sijie
> > > > >
> > > > > On Thu, Aug 29, 2019 at 12:08 PM Enrico Olivelli <
> > eolive...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Maybe you can try to use HerdDB (1), it is a replicated SQL
> > database
> > > > that
> > > > > > can be run embedded in the JVM.
> > > > > > It is an SQL database that we developed in my company, in order
> to
> > > > > > explicitly replace MySQL.
> > > > > > It uses Bookkeeper to store the WAL and implement replication,
> but
> > it
> > > > can
> > > > > > run in standalone mode or in memory (for tests/dev).
> > > > > >
> > > > > > I can help in setting up a demo.
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > > (1) https://github.com/diennea/herddb
> > > > > >
> > > > > > Il gio 29 ago 2019, 16:59 Sijie Guo <guosi...@gmail.com> ha
> > scritto:
> > > > > >
> > > > > > > Hi Dave,
> > > > > > >
> > > > > > > It doesn't require MySQL. You can use any JDBC supported
> > databases.
> > > > > > > Initially we would use SQLite for development and change to
> MySQL
> > > for
> > > > > > more
> > > > > > > production-ready workloads.
> > > > > > >
> > > > > > > Hope this clarify your question.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Sijie
> > > > > > >
> > > > > > > On Thu, Aug 29, 2019 at 5:56 AM Dave Fisher <w...@apache.org>
> > > wrote:
> > > > > > >
> > > > > > > > Hi -
> > > > > > > >
> > > > > > > > I see that MySQL is used. This would be a GPL dependency.
> Does
> > > the
> > > > > > Pulsar
> > > > > > > > Manager require MySQL, or can other Databases that are
> > compliant
> > > > with
> > > > > > > > Apache Release Policy be used instead?
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Dave
> > > > > > > >
> > > > > > > > > On Aug 29, 2019, at 7:49 AM, Guangning E <
> > eguangn...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > We have developed a new Pulsar web UI - pulsar-manager,
> > aiming
> > > at
> > > > > > > > > supporting managing Pulsar clusters running in different
> > > > > environments
> > > > > > > > > (on-premise data centers, cloud, and so on). We’d like to
> > > > > contribute
> > > > > > > the
> > > > > > > > > project back to the Pulsar community.
> > > > > > > > >
> > > > > > > > > ---
> > > > > > > > >
> > > > > > > > > ## Motivation
> > > > > > > > >
> > > > > > > > > Currently, Pulsar has a monitoring tool - [Pulsar
> > Dashboard]( [
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://pulsar.apache.org/docs/en/administration-dashboard/](http://pulsar.apache.org/docs/en/administration-dashboard/)
> > > > > > > > > ).
> > > > > > > > > However, it only focuses on the simple monitoring of
> Pulsar -
> > > > > > > collecting
> > > > > > > > > and displaying information such as statistics of tenants,
> > > > > namespaces,
> > > > > > > > > topics, subscriptions, and so on. It doesn’t provide any
> > > > management
> > > > > > > > > operations such as add, delete and update tenants,
> > namespaces,
> > > > > > topics,
> > > > > > > > and
> > > > > > > > > so on. When the scale of Pulsar cluster increases or the
> > number
> > > > of
> > > > > > > > clusters
> > > > > > > > > grows, using `pulsar-admin` to manage Pulsar can not
> satisfy
> > > > > demands.
> > > > > > > > > Therefore, Pulsar needs a simple and easy-to-use management
> > > > console
> > > > > > for
> > > > > > > > > administrators.
> > > > > > > > >
> > > > > > > > > Pulsar Manager is a web-based GUI management and monitoring
> > > tool
> > > > > that
> > > > > > > > helps
> > > > > > > > > administrators and users manage and monitor tenants,
> > > namespaces,
> > > > > > > topics,
> > > > > > > > > subscriptions, brokers, clusters, and so on, and supports
> > > dynamic
> > > > > > > > > configuration of multiple environments.
> > > > > > > > >
> > > > > > > > > ## Features
> > > > > > > > >
> > > > > > > > > Pulsar manager provides two main features: management and
> > > > > monitoring.
> > > > > > > > >
> > > > > > > > > ### Management
> > > > > > > > >
> > > > > > > > > 1. Environment : (operations on the environment)
> > > > > > > > > Create / Delete / Update / List / Get
> > > > > > > > > 2. Cluster : (operations on clusters)
> > > > > > > > > Create / Delete / Update / List / Get
> > > > > > > > > 3. Brokers : (operations on brokers)
> > > > > > > > > Heartbeat / Unload
> > > > > > > > > 4. Tenants:
> > > > > > > > > Create / Delete / Update / List
> > > > > > > > > 5. Namespaces:
> > > > > > > > > Create / Delete / List
> > > > > > > > > Manage Namespace Policy
> > > > > > > > > Unload
> > > > > > > > > Operations on Namespace Bundles
> > > > > > > > > 6. Topics
> > > > > > > > > Create / Delete / List
> > > > > > > > > Unload / Terminate / Offload / Compact
> > > > > > > > > Topic Details
> > > > > > > > > 7. Subscriptions
> > > > > > > > > Create / Delete / List
> > > > > > > > > Reset Cursor / Skip / Clear backlog / Unsubscribe
> > > > > > > > > 8. Namespace Isolation Policies
> > > > > > > > > Create / Delete / Update / Get / List
> > > > > > > > > 9. Failure Domains
> > > > > > > > > Create / Delete / Update / Get / List
> > > > > > > > >
> > > > > > > > > ### Monitoring: (display *aggregated* stats at different
> > > levels)
> > > > > > > > >
> > > > > > > > > 1. Tenants
> > > > > > > > > List the total number of namespaces per tenant (both
> > aggregated
> > > > and
> > > > > > > > > per-cluster basis)
> > > > > > > > >
> > > > > > > > > 2. Namespaces
> > > > > > > > > List the total number of topics per namespace
> > > > > > > > > List the aggregated stats per namespace (such as rate-in,
> > > > rate-out,
> > > > > > > > > throughput-in, and throughput-out)
> > > > > > > > > List the distributions of namespace bundles
> > > > > > > > >
> > > > > > > > > 3. Topics
> > > > > > > > > List the total number of partitions per topic
> > > > > > > > > List the aggregated stats per topic (such as rate-in,
> > rate-out,
> > > > > > > > > throughput-in, and throughput-out)
> > > > > > > > > Detailed stats per topic partition
> > > > > > > > > Detailed stats of storage per topic partition
> > > > > > > > > Detailed stats of subscription per topic and per partition
> > > > > > > > > Detailed stats of producers
> > > > > > > > > Detailed stats of consumers
> > > > > > > > >
> > > > > > > > > The detailed design proposal is in
> > > > > > > > > [
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1C3meaHJzxX9wGDWQx-dC1b20yGeKsQtbwNiYuxZgOCE/edit#](https://docs.google.com/document/d/1C3meaHJzxX9wGDWQx-dC1b20yGeKsQtbwNiYuxZgOCE/edit#)
> > > > > > > > >
> > > > > > > > > Looking forward to any feedback.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Guangning
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
> *Thanks*
>
> *Yuvaraj L*
>

Reply via email to