On Thu, Apr 9, 2015 at 2:15 AM, Clint Byrum <cl...@fewbar.com> wrote:
> Excerpts from Angus Salkeld's message of 2015-04-06 19:55:37 -0700: > > Hi all > > > > For quite some time we (Heat team) have wanted to be able to send > messages > > to our > > users (by user I do not mean the Operator, but the User that is > interacting > > with the client). > > > > What do I mean by "user messages", and how do they differ from our > current > > log messages > > and notifications? > > - Our current logs are for the operator and have information that the > user > > should not have > > (ip addresses, hostnames, configuration options, other tenant info > etc..) > > - Our notifications (that Ceilometer uses) *could* be used, but I am not > > sure if it quite fits. > > (they seem a bit heavy weight for a log message and aimed at higher > level > > events) > > > > These messages could be (based on Heat's use case): > > > > - Specific user oriented log messages (distinct from our normal operator > > logs) > > These currently go in the Heat events API, yes? > Well I wanted a replacement for our events table that could also take smaller messages and fulfil some other use cases (be a stream, that the user doesn't have to poll). We are sending RPC notifications at the moment: stack.<action>.begin/end/error autoscaling.begin/end/error We need to add: per resource notifications (to replace the event table) Add some useful log-like messages: "x property is deprecated, please update your template to use Y" There are also a bunch of other log messages that should really be going to the user, but just end up in the Operator's log file. > > - Deprecation messages (if they are using old resource > properties/template > > features) > > I think this could fit with the bits above. > > > - Progress and resource state changes (an application doesn't want to > poll > > an api for a state change) > > These also go in the current Heat events. > > > - Automated actions (autoscaling events, time based actions) > > As do these? > > > - Potentially integrated server logs (from in guest agents) > > > > I wanted to raise this to "[all]" as it would be great to have a general > > solution that > > all projects can make use of. > > > > What do we have now: > > - The user can not get any kind of log message from services. The closest > > thing > > ATM is the notifications in Ceilometer, but I have the feeling that > these > > have a different aim. > > - nova console log > > - Heat has a DB "event" table for users (we have long wanted to get rid > of > > this) > > So if we forget the DB part of it, the API is also lacking things like > pagination and search that one would want in an event/logging API. > I'd almost rather spend the time getting an OpenStack wide solution then switch our client (and API) to use that. > > > > > What do other clouds provide: > > - https://devcenter.heroku.com/articles/logging > > - https://cloud.google.com/logging/docs/ > > - https://aws.amazon.com/blogs/aws/cloudwatch-log-service/ > > - http://aws.amazon.com/cloudtrail/ > > (other examples...) > > > > What are some options we could investigate: > > 1. remote syslog > > The user provides a rsyslog server IP/port and we send their messages > > to that. > > [pros] simple, and the user could also send their server's log > messages > > to the same > > rsyslog - great visibility into what is going on. > > > > There are great tools like loggly/logstash/papertrailapp > that > > source logs from remote syslog > > It leaves the user in control of what tools they get to > use. > > > > [cons] Would we become a spam agent (just sending traffic to an > > IP/Port) - I guess that's how remote syslog > > works. I am not sure if this is an issue or not? > > > > This might be a lesser solution for the use case of "an > > application doesn't want to poll an api for a state change" > > > > I am not sure how we would integrate this with horizon. > > > > I think this one puts too much burden on the user to setup a good > receiver. > > > 2. Zaqar > > We send the messages to a queue in Zaqar. > > [pros] multi tenant OpenStack project for messaging! > > > > [cons] I don't think Zaqar is installed in most installations (tho' > > please correct me here if this > > is wrong). I know Mirantis does not currently support > Zaqar, > > so that would be a problem for me. > > > > There is not the level of external tooling like in option > "1" > > (logstash and friends) > > > > I agree with your con, and would also add that after the long > discussions we had in the past we had some concerns about scaling. > > > 3. Other options: > > Please chip in with suggestions/links! > > > > There's this: > > https://wiki.openstack.org/wiki/Cue > > I think that could be a bit like 1, but provide the user with an easy > target for the messages. > > I also want to point out that what I'd actually rather see is that all > of the services provide functionality like this. Users would be served > by having an event stream from Nova telling them when their instances > are active, deleted, stopped, started, error, etc. > > Also, I really liked Sandy's suggestion to use the notifications on the > backend, and then funnel them into something that the user can consume. > The project they have, yagi, for putting them into atom feeds is pretty > interesting. If we could give people a simple API that says "subscribe > to Nova/Cinder/Heat/etc. notifications for instance X, and put them > in an atom feed", that seems like something that would make sense as > an under-the-cloud service that would be relatively low cost and would > ultimately reduce load on API servers. > "an under-the-clould service" ? - That is not what I am after here. What I am really after is a general OpenStack solution for how end users can consume service notifications (and replace "heat event-list"). Right now there is "ceilometer event-list", but as some Ceilometer devs have said, they don't want to store every notification that comes. So is the yagi + atom hopper solution something we can point end-users to? Is it per-tenant etc... Sandy, do you have a write up somewhere on how to set this up so I can experiment a bit? Maybe this needs to be a part of Cue? -Angus > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
