[
https://issues.apache.org/jira/browse/NIFI-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381050#comment-16381050
]
ASF GitHub Bot commented on NIFI-4809:
--------------------------------------
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/2430
Ugh that's true, I'm not a fan of dynamic keys. Since the schema would be
generated by the reporting task, then we could create a schema for the example
above, but then each flow file would have its own schema, and even worse, each
flow file would have only one metric, so at that point it's not really
conducive to record processing. As an alternative we could convert the output
(before or after JSON conversion) to an altered spec with a consistent schema
definition.
For the Ambari reporting task, since Ambari is expecting this format, then
fine; if we keep the same spec in this new reporting task for consistency, then
I'd hope to see a template on the Wiki using the new reporting task with a
JoltTransformJSON processor to do the aforementioned transformation, along with
including an AvroSchemaRegistry that contains the schema definition for the
files coming out of the JoltTransformJSON processor. This would allow us to
keep a consistent (standard) defined format (albeit non-schema-friendly), but
offer a well-known solution to prepare the data for record-aware processors.
Thoughts?
> Implement a SiteToSiteMetricsReportingTask
> ------------------------------------------
>
> Key: NIFI-4809
> URL: https://issues.apache.org/jira/browse/NIFI-4809
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Pierre Villard
> Assignee: Pierre Villard
> Priority: Major
>
> At the moment there is an AmbariReportingTask to send the NiFi-related
> metrics of the host to the Ambari Metrics Service. In a multi-cluster
> configuration, or when working with MiNiFi (Java) agents, it might not be
> possible for all the NiFi instances (NiFi and/or MiNiFi) to access the AMS
> REST API.
> To solve this problem, a solution would be to implement a
> SiteToSiteMetricsReportingTask to send the data via S2S to the "main" NiFi
> instance/cluster that will be able to publish the metrics into AMS (using
> InvokeHTTP). This way, it is possible to have the metrics of all the
> instances exposed in one AMS instance.
> I propose to send the data formatted as we are doing right now in the Ambari
> reporting task. If needed, it can be easily converted into another schema
> using the record processors once received via S2S.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
