[
https://issues.apache.org/jira/browse/NIP-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910393#comment-17910393
]
Joe Witt commented on NIP-2:
----------------------------
The spirit of driving better validation of a given component makes sense.
Does this mean a user would supply a preview input flowfile by uploading
content and entering attributes, then let it execute, we'd then capture the
output such that they can review which relationship it would have gone to and
what the output looks like?
The proposal seems like a considerable amount of work across all the layers.
What about repeatability of the validation?
And as an alternative what about considering 'unit tests' at the Process Group
level whereby users can register input flowfiles and provide assertions on
their results? I suspect validating across subnets of flows and in a
repeatable fashion is the more common requirement and it likely involves a lot
of the same work.
> Preview capability for processors
> ---------------------------------
>
> Key: NIP-2
> URL: https://issues.apache.org/jira/browse/NIP-2
> Project: NiFi Improvement Proposal
> Issue Type: New Feature
> Reporter: Matt Burgess
> Priority: Normal
>
> *Motivation*
> The ability to "send" a sample FlowFile through a processor in an existing
> flow, without being associated with the flow itself (provenance, I/O metrics,
> e.g.), rather by executing the configured processor using the provided input
> and returning the output could be highly beneficial for Flow Designers. It's
> somewhat analogous to the "Run Once" feature but aims to operate outside the
> actual flow framework so as not to potentially contaminate the various NiFi
> repositories keeping track of the flow of "real" data.
> *Scope*
> This would involve several changes and additions to the API and codebase, see
> the Description section for details.
> *Compatibility*
> This feature would be backwards-compatible as is it is new and can only be
> executed on a flow in a NiFi instance that supports this capability.
> *Description*
> This capability would provide a user to test a processor already in the flow
> during design before starting the flow with "real" data which can help debug
> flows while preventing data loss during development.
> Tasks:
> * REST API call(s) to POST data to endpoint(s) which respond with machine
> readable output (JSON, XML, e.g.)
> * An annotation on processors such as Previewable, perhaps the
> SideEffectFree annotation suffices but you cannot extend Java annotations so
> SideEffectFree processors may just be a good starting point
> * An established format for both the POST data and the output data (content
> and attributes, multiple FlowFiles, relationships, etc.)
> * UI changes to include:
> -- Context-sensitive (right-click) menu option to Preview
> -- Dialog to paste/upload/drag the input content
> -- Inject parameters (perhaps reuse the Verification capability here as
> prudent)
> -- Leverage the existing Content Viewer as prudent to show the output (both
> content and attributes in one place would be preferable, but code reuse in
> the UI helps with consistency and maintainability)
> * Documentation
> -- Add a section to the processors' documentation if they are Previewable, a
> common description could be provided to avoid the need for custom UIs
> -- Add a section to the User Guide to describe how to preview a Previewable
> processor and how to interpret the results
> -- Add a section as necessary to the Admin guide for allowing/disallowing
> Preview (see the Authorization section below)
> * Authorization
> -- Add a policy to allow/prevent previewing a processor (optional for this
> NIP as this is intended to be a "sandbox" operation)
> *Verification*
> As processors are augmented with the Previewable annotation, the pull
> requests should include unit tests and the overall documentation should
> describe how to use the Preview capability on a Previewable processor.
> *Alternatives*
> On development systems (or in development process groups), the GenerateXYZ
> processors can be used to inject data into the processor and the UI or the
> REST API can be used to inspect the output data from the connection. This can
> be quite cumbersome and requires a separate NiFi system or process group for
> testing flow development, then moving the tested flow into the target NiFi
> system.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
