Karim Yaghmour writes: > > Aaron Cohen wrote: > > I've got a quick question and I just want to be clear that it > > doesn't have a political agenda behind it. > > :) > > > Here goes, why can't LTT and/or relayfs, work similar to the way > > syslog does and just fill a buffer (aka ring-buffer or whatever is > > appropriate), while a userspace daemon of some kind periodically reads > > that buffer and massages it. I'm probably being naive but if the > > difficulty is with huge several hundred-gig files, the daemon if it > > monitors the buffer often enough could stuff it into a database or > > whatever high-performance format you need. > > Because of the bandwidth it is not possible to do any sort of live > processing of any kind. The only thing the daemon can possibly do > is write large blocks of tracing info to disk as rapidly as possible.
I have to disagree. Awhile back, if you remember, I posted a patch to the LTT daemon that would monitor the trace stream in real time, and process it using an embedded Perl interpreter, no less: http://marc.theaimsgroup.com/?l=linux-kernel&m=109405724500237&w=2 It didn't seem to have any problems keeping up with the trace stream even though it was monitoring all LTT event types (and a couple of others - custom events injected using kprobes) and not doing any filtering in the kernel, through kernel compiles, normal X traffic, etc. I don't know what volume of event traffic would cause this model to break down, but I think it shows that at least some level of non-trivial live processing is possible... Tom > > > It also seems to me that Linus' nascent "splice and tee" work would > > be really useful for something like this to avoid a lot of unnecessary > > copying by the userspace daemon. > > There is no copying by the userspace daemon. All it does is open(), > then mmap(), and then it sleeps until it is woken up by the ltt > kernel subsystem. When that happens, it only does a write() on the > mmaped area, tells the ltt subsystem that it commited X number of > sub-buffers and goes back asleep. This is all zero-copy. > > Karim > -- > Author, Speaker, Developer, Consultant > Pushing Embedded and Real-Time Linux Systems Beyond the Limits > http://www.opersys.com || [EMAIL PROTECTED] || 1-866-677-4546 -- Regards, Tom Zanussi <[EMAIL PROTECTED]> IBM Linux Technology Center/RAS - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
