However, adding port is to modify the internal content of the IP field, which has little impact on the overall layout.
In our cluster, we parse the audit log through Vector and send the data to Kafka, which is unaffected. tom lee <tomlees...@gmail.com> 于2021年10月11日周一 下午2:44写道： > Thank Ayush for reminding me. I also have similar concerns, so I published > this discussion, hoping to let the members of the community know about this > matter and then give suggestions. > > Ayush Saxena <ayush...@gmail.com> 于2021年10月11日周一 下午2:38写道： > >> Hey >> I am not sure whether we can directly go and change this. Any changes to >> Audit Log format are considered incompatible. >> >> >> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/Compatibility.html#Audit_Log_Output >> >> -Ayush >> >> On 10-Oct-2021, at 7:57 PM, tom lee <tomlees...@gmail.com> wrote: >> >> Hi all, >> >> In our production environment, we occasionally encounter a problem where a >> user submits an abnormal computation task, causing a sudden flood of >> requests, which causes the queueTime and processingTime of the Namenode to >> rise very high, causing a large backlog of tasks. >> >> We usually locate and kill specific Spark, Flink, or MapReduce tasks based >> on metrics and audit logs. Currently, IP and UGI are recorded in audit >> logs, but there is no port information, so it is difficult to locate >> specific processes sometimes. Therefore, I propose that we add the port >> information to the audit log, so that we can easily track the upstream >> process. >> >> Currently, some projects contain port information in audit logs, such as >> Hbase and Alluxio. I think it is also necessary to add port information >> for >> HDFS audit logs. >> >> I submitted a PR(https://github.com/apache/hadoop/pull/3538), which has >> been tested in our test environment, and both RPC and HTTP are in effect. >> I >> look forward to your discussion on possible problems and suggestions for >> modification. I will actively update the PR. >> >> Best Regards, >> Tom >> >>
