This discussion needs to be happening on openstack-dev too, so
cc'ing that list in as well. The top of the thread is at
http://lists.openstack.org/pipermail/openstack/2016-April/015864.html
On Tue, 12 Apr 2016, Chris Dent wrote:
On Tue, 12 Apr 2016, Nadya Shakhat wrote:
I'd like to discuss one question with you. Perhaps, you remember that
in Liberty we decided to get rid of transformers on polling agents [1]. I'd
like to describe several issues we are facing now because of this decision.
1. pipeline.yaml inconsistency.
The original idea with centralizing the transformers to just the
notification agents was to allow a few different things, only one of
which has happened:
* Make the pollster code less complex with few dependencies,
easing deployment options (this has happened), maintenance
and custom pollsters.
With the transformers in the pollsters they must maintain a
considerable amount of state that makes effective use of eventlet
(or whatever the chosen concurrent solution is) more difficult.
The ideal pollster is just something that spits out a dumb piece
of identified data every interval. And nothing else.
* Make it far easier to use and conceptualize the use of pollsters
outside of the ceilometer environment as simple data collectors.
In that context transformation would occur only close to the data
consumption not at the data production.
This, following the good practice of services doing one thing
well.
* Migrate away from the pipeline.yaml that conflated sources and
sinks to a model that is good both for computers and humans:
* sources over here
* sinks over here
That these other things haven't happened means we're in an awkward
situation.
Are the options the following?
* Do what you suggest and pull transformers back into the pollsters.
Basically revert the change. I think this is the wrong long term
solution but might be the best option if there's nobody to do the
other options.
* Implement a pollster.yaml for use by the pollsters and consider
pipeline.yaml as the canonical file for the notification agents as
there's where the actual _pipelines_ are. Somewhere in there kill
interval as a concept on pipeline side.
This of course doesn't address the messaging complexity. I admit
that I don't understand all the issues there but it often feels
like we are doing that aspect of things completely wrong, so I
would hope that before we change things there we consider all the
options.
What else?
One probably crazy idea: What about figuring out the desired end-meters
of common transformations and making them into dedicated pollsters?
Encapsulating that transformation not at the level of the polling
manager but at the individual pollster.
--
Chris Dent (╯°□°)╯︵┻━┻ http://anticdent.org/
freenode: cdent tw: @anticdent
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
