Re: HDFS audit log

2015-05-05 Thread Colin McCabe
I think HDFS INotify is a better choice if you need: * guaranteed backwards compatibility * rapid and unambiguous parsing (via protobuf) * clear Java API for retrieving the data (I.e. not rsync on a text file) * ability to resume reading at a given point if the consumer process fails We are using

Re: HDFS audit log

2015-04-27 Thread Andrew Wang
Hi Allen, thanks for bringing this up. Two comments: - Is there a specification for the audit log format? i.e. is it something structured like JSON? I think I asked you this in-person, and you said it's something custom. I doubt that we can just "freeze" the format. In recent times we've added thi