Thanks very much Gord for the pointers.
I have looked at this file reload feature and it addresses some part of my
problem description (avoiding the restart of the ceilometer services to reflect
the pipeline changes). However, this still requires users to directly access
the internal ceilometer file (pipeline.yaml) to make any updates to pipeline.
There are no APIs introduced as part of this feature to allow user to
configure/unconfigure these pipeline. In one of the deployment that we are
currently looking for, we wanted to expose only ceilometer API service to the
users and block access to any internal modules of ceilometer. In the same
deployment, we are also thinking of allowing users to configure/unconfigure a
pipeline for a set of meters for only a subset of resources instead of getting
notified about events/meters of all the resources as shown below.
sources:
- name: user1_meter_source
meters:
- "cpu_util"
query/filter:
- {{field:"project_id",op:"eq",value:"111111"}}
sinks:
- user1_meter_sink
sinks:
- name: user1_meter_sink
transformers:
publishers:
- udp://<IP>:<port>
We have found a related blueprint [1], which is exactly addressing this dynamic
provisioning of pipelines via an API and making use of a new data store to
store this configuration. But seems like that blueprint was abandoned due its
limited usage. We agree that it may not be that frequent operation to change
the pipeline configuration in all deployments, but in certain deployments,
where applications desire to turn ON/OFF receiving these notifications based on
some other trigger (like anomaly detection, where application might have sensed
an anomaly based on some set of meters and wanted to turn ON few additional set
of event/meters to confirm that and turn OFF once the anomaly is confirmed).
Thanks
Srikanth
[1]
https://wiki.openstack.org/wiki/Ceilometer/blueprints/Configuration-via-data-store
From: gord chung [mailto:[email protected]]
Sent: Thursday, November 19, 2015 9:08 PM
To: [email protected]
Subject: Re: [openstack-dev] [Ceilometer]:Subscribe and Publish Notification
frame work in Ceilometer !
On 19/11/15 10:26 PM, Srikanth Vavilapalli wrote:
Hi Gord
On your second point, Yes, Ceilometer does provide a framework to capture a
notification and republish to multiple "publish targets" in addition to the
collector service using udp/kafka/notification as the transport mechanisms...
We believe this is how "Event Alarm Evaluator" module in Aodh project get
notified directly from Notification Agents.
However seems like the configuration of these additional "publish targets" is
supported only through updating the pipeline_cfg_file and restarting the
corresponding ceilometer services. i.e. the users need to manually update the
pipeline config files to insert their "publish targets" in the sink-publisher
configuration for a set of event filters of their interest. This type of
provisioning is very static.
As per our understanding, ceilometer currently does not provide means for users
to dynamically register/unregister their "publish targets" with ceilometer
framework for a subset events of their interest? i.e User invokes a ceilometer
API with a set of event filters and associated publish targets, that can be
stored in a data store, which will eventually be used by the ceilometer
Publisher to dispatch the notification to those configured destinations in
addition to the statically configured "publish targets". Plz let us know if our
understanding is wrong or if there are any other means to achieve the above
functionality. We believe this as a very key functionality needed to build
latency sensitive (sub-second) analytics application on-top of ceilometer
framework. We are seeking the feedback from community on having this kind of
functionality inside ceilometer before proceeding with blueprint submission.
actually, in Liberty you can configure a reload[1][2] to refresh pipeline when
a change happens. based on survey results, most operators don't need this
functionality so it's off by default but it seems you do. :)
[1]
https://specs.openstack.org/openstack/ceilometer-specs/specs/liberty/reload-file-based-pipeline-configuration.html
[2]
https://github.com/openstack/ceilometer/blob/master/ceilometer/pipeline.py#L50-L58
cheers,
--
gord
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
