HDFS-8880 does not require HDFS-9114. It is configured like any other logger by adding '-Dnamenode.metrics.logger=DEBUG,NNMETRICSRFA' or equivalent to HADOOP_NAMENODE_OPTS. Your concerns sound specific to 9114 and not inherent to 8880.
On 10/6/15, 1:28 PM, "Allen Wittenauer" <a...@altiscale.com> wrote: > >As I mentioned in the original message, take a look at HDFS-9114, which is >basically making HDFS-8880 operable. > >Generating a sink that is not a sink (which is what HDFS-8880 basically does), >isn’t particularly practical. 8880 would be much more appealing if it was a >sink. If FileSink isn’t fixable, then let's make a Log4jSink which would >basically be FileSink with those issues fixed. > >If not all NN metrics are getting logged, then that’s probably a bug in *all* >of the metrics systems, no? Having them show up only in this one off sounds >absolutely terrible. > >On Oct 6, 2015, at 1:11 PM, Arpit Agarwal <aagar...@hortonworks.com> wrote: > >> There is no property called hadoop.metrics.log.file. Where do you see it? >> And there is no property that would show up in the command-line of every >> process. namenode.metrics.logger would show up in the NN command-line and >> even that is off by default. >> >> FileSink looks limited for practical use. >> - No support for rolling output files. >> - The format is hard-coded and not practical to grep. Changing it is >> incompatible. >> - Won't write to new output files unless I create the files first with >> 'touch'. >> >> - Not all NN metrics are logged. These last two are likely bugs. >> >> >> >> >> >> >> On 10/6/15, 11:46 AM, "Allen Wittenauer" <a...@altiscale.com> wrote: >> >>> Current state is the custom stuff in HDFS-8880 is mostly undocumented >>> except for some tuning bits in hdfs-default.xml vs. the javadocs for >>> metrics2. Neither of which is ideal. >>> >>> There’s no doubt the documentation for all of the metrics2 sinks are… >>> sparse. Which is likely what led to the duplicate functionality. :( >>> >>> On Oct 6, 2015, at 11:29 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: >>> >>>> If it's duplicate we should probably back it out, but taking a step back, >>>> is the issue that there isn't good documentation about configuring Metrics2 >>>> / FileSync? I see the API docs, but a user-focused guide on how to >>>> configure Metrics2 would probably be a welcome addition. >>>> >>>> HBase has a blog at https://blogs.apache.org/hbase/ this could also be good >>>> content for a blog post. >>>> >>>> Best, >>>> Andrew >>>> >>>> On Tue, Oct 6, 2015 at 11:12 AM, Allen Wittenauer <a...@altiscale.com> >>>> wrote: >>>> >>>>> >>>>> Folks, >>>>> >>>>> I’ve been looking over HDFS-8880 and it’s various follow-on >>>>> JIRAs. The intentions are good, but the implementation is >>>>> mostly/effectively a duplicate of the FileSink that’s already part of the >>>>> Hadoop metrics subsystem. (which therefore means it works with all >>>>> daemons, >>>>> out of the box already). Reading through HDFS-9114, it’s pretty obvious >>>>> now that users are going to get *very* confused as just what happens when >>>>> they set the “hadoop.metrics.log.file” property. It’s opening a pandora’s >>>>> box of work, since that property only partially works with one >>>>> sub-project, >>>>> will show up on the command line of every daemon, and isn’t documented... >>>>> >>>>> I’d like to see this series of patches reverted (they haven’t >>>>> shipped yet, so now is the time!) and effort placed into updating the >>>>> metrics2 FileSink to have whatever functionality is missing. >>>>> >>>>> Thoughts? >>> >>> > >
