On Dec 5, 2013, at 1:36 AM, Maru Newby <ma...@redhat.com> wrote: > > On Dec 3, 2013, at 12:18 AM, Joe Gordon <joe.gord...@gmail.com> wrote: > >> >> >> >> On Sun, Dec 1, 2013 at 7:04 PM, John Dickinson <m...@not.mn> wrote: >> Just to add to the story, Swift uses "X-Trans-Id" and generates it in the >> outer-most "catch_errors" middleware. >> >> Swift's catch errors middleware is responsible for ensuring that the >> transaction id exists on each request, and that all errors previously >> uncaught, anywhere in the pipeline, are caught and logged. If there is not a >> common way to do this, yet, I submit it as a great template for solving this >> problem. It's simple, scalable, and well-tested (ie tests and running in >> prod for years). >> >> https://github.com/openstack/swift/blob/master/swift/common/middleware/catch_errors.py >> >> Leaving aside error handling and only focusing on the transaction id (or >> request id) generation, since OpenStack services are exposed to untrusted >> clients, how would you propose communicating the appropriate transaction id >> to a different service? I can see great benefit to having a glance >> transaction ID carry through to Swift requests (and so on), but how should >> the transaction id be communicated? It's not sensitive info, but I can >> imagine a pretty big problem when trying to track down errors if a client >> application decides to set eg the X-Set-Transaction-Id header on every >> request to the same thing. >> >> -1 to cross service request IDs, for the reasons John mentions above. >> >> >> Thanks for bringing this up, and I'd welcome a patch in Swift that would use >> a common library to generate the transaction id, if it were installed. I can >> see that there would be huge advantage to operators to trace requests >> through multiple systems. >> >> Another option would be for each system that calls an another OpenStack >> system to expect and log the transaction ID for the request that was given. >> This would be looser coupling and be more forgiving for a heterogeneous >> cluster. Eg when Glance makes a call to Swift, Glance cloud log the >> transaction id that Swift used (from the Swift response). Likewise, when >> Swift makes a call to Keystone, Swift could log the Keystone transaction id. >> This wouldn't result in a single transaction id across all systems, but it >> would provide markers so an admin could trace the request. >> >> There was a session on this at the summit, and although the notes are a >> little scarce this was the conclusion we came up with. Every time a cross >> service call is made, we will log and send a notification for ceilometer to >> consume, with the request-ids of both request ids. One of the benefits of >> this approach is that we can easily generate a tree of all the API calls >> that are made (and clearly show when multiple calls are made to the same >> service), something that just a cross service request id would have trouble >> with. > > Is wise to trust anything a client provides to ensure traceability? If a > user receives a request id back from Nova, then submits that request id in an > unrelated request to Neutron, the traceability would be effectively > corrupted. If the consensus is that we don't want to securely deliver > request ids for inter-service calls, how about requiring a service to log its > request id along with the request id returned from a call to another service > to achieve the a similar result?
Yes, this is what I was proposing. I think this is the best path forward. > The catch is that every call point (or client instantiation?) would have to > be modified to pass the request id instead of just logging at one place in > each service. Is that a cost worth paying? Perhaps this is my ignorance of how other projects work today, but does this not already happen? Is it possible to get a response from an API call to an OpenStack project that doesn't include a request id? > > > m. > > >> >> https://etherpad.openstack.org/p/icehouse-summit-qa-gate-debugability >> >> >> With that in mind I think having a standard x-openstack-request-id makes >> things a little more uniform, and means that adding new services doesn't >> require new logic to handle new request ids. > > > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
