System and processor info, marked as 'logged once' in gist shared by Piotr, should still be logged instead of registered as metrics, right?
On Thu, Oct 5, 2017 at 2:38 AM, Till Rohrmann <trohrm...@apache.org> wrote: > Thanks for the proposal Piotr. I like it a lot since it will help people to > better understand their system. I would also be in favour of adding them to > the system metrics. I think o.a.f.runtime.metrics.util.MetricUtils is the > right place to start. Given the small dependency footprint and the > compatible license, I would be in favour of option 1. > > Cheers, > Till > > > On Thu, Oct 5, 2017 at 11:19 AM, Piotr Nowojski <pi...@data-artisans.com> > wrote: > > > +1 thanks for pointing this out. It makes sense to just expand those > > system metrics (I was not aware of them). > > > > > On Oct 4, 2017, at 6:07 PM, Greg Hogan <c...@greghogan.com> wrote: > > > > > > What if we added these as system metrics and added a way to write > > metrics to a (separate?) log file? > > > > > > > > >> On Oct 4, 2017, at 10:13 AM, Piotr Nowojski <pi...@data-artisans.com> > > wrote: > > >> > > >> Hi, > > >> > > >> Lately I was debugging some weird test failures on Travis and I needed > > to look into metrics like: > > >> - User, System, IOWait, IRQ CPU usages (based on CPU ticks since > > previous check) > > >> - System wide memory consumption (including making sure that swap was > > disabled) > > >> - network usage > > >> - etc… > > >> > > >> Without an access to the machines itself. For this purpose I > > implemented some periodic daemon thread logger. Log output looked like > this: > > >> > > >> https://gist.github.com/pnowojski/8b863abb0fb08ac75b62627feadbd2f7 < > > https://gist.github.com/pnowojski/8b863abb0fb08ac75b62627feadbd2f7> > > >> > > >> I think it would be nice to add this feature to Flink itself, by > > extending existing MemoryLogger. Same lack of information that I had with > > travis could easily happen on productional environments. The problem is > > that there is no easy way to obtain such kind of information without > using > > some external libraries (think about cross platform support). I have used > > for that: > > >> > > >> https://github.com/oshi/oshi <https://github.com/oshi/oshi> > > >> > > >> It has some minimal additional dependencies, one thing worth noting is > > a JNA - it’s JAR weights ~1MB. We would have two options to add this > > feature: > > >> > > >> 1. Include this oshi dependency in flink-runtime > > >> 2. Wrap oshi into flink-contrib/flink-resource-logger module and make > > this new module an optional/dynamically loaded dependency by > flink-runtime > > (used only if user manually copies flink-resource-logger.jar to a class > > path). > > >> > > >> I would lean toward 1., since that’s a powerful tool and it’s > > dependencies are pretty minimal (except this JNA’s jar size). What do you > > think? > > >> > > >> Piotrek > > > > >
