Hello, Apologies in advance if this is the wrong list to send this type of query to. After short discussion with Chris Sampson on https://issues.apache.org/jira/browse/NIFI-8214 he proposed to send out email to this list; I hope it finds you well.
We (several of my colleagues) sometimes use a pattern where we build a nifi flow that gets initiated by a short json configuration file; the initial input file (or generated flowfile) contains simple configuration data for the rest of the flow and sets up things like Remote paths, users, testcase IDs, endpoints to hit, etc... as a file is easier to manage, maintain and archive than a graphical set of linked components with logic. This setup allows relative easy building of a generalized graphical logic flow with (less issue below) minimal branching as 'controlling where stuff goes or talks to' is effectively on the input file. With some processing and splitting upfront each flowfile will effectively go down it's intended path and carry the needed values as attributes for downstream usage where they act as dynamic configuration/parameters (or whatever terminology you wish to apply) and we are also able to use calculations/logic to pick output locations, etc. To make it more concrete just imagine a use case where a simple attribute value can decide using a production system or a test system endpoint where all logic is identical but the only difference will be an input and/or output location. Granted the example is simple but it does demonstrate how this can get out of hand with multiple branching locations and duplicated components. We do face the issue that several NiFi components do not support Attributes (or expression language) on several key configuration properties (some typical example components with this limitation: FTP components, ElasticPut, and there surely will be a few others based off the similar base-classes which I haven't used/found yet); this forces us into building dedicated routes+ duplicated pattern of components because a simple destination URI or an Remote Input/Output path cannot be dynamically adjusted but only be set through more pre-defined constants (parameters or through VarRegistry). This reduces flexibility quite a bit and introduces a lot of complexity to the graphical view as this obviously means more graphical clutter on the worksheet, a lot more connection lines, additional branching, etc... A minor change in logic or a property needs to be touched in several places making this relatively error-prone to maintain as well. I'd like to propose to extend - by default - fields like: Server endpoints, ports , Remote Paths, Input Paths (all these similar type of fields which are limited in several components) - with a default capability of using attributes/expression language (and by extension parameters and the variable registry); this will increase flexibility for many components and could de-dupe a lot of graphical flow logic and components. Best Regards, [cid:[email protected]]<http://www.voo.be/> Rogier Timmermans Lead Engineer VOD & OTT 46 Rue Jean Jaures, B-4030 Ans Tel: +31 (0) 6 5428 4192 [email protected]<mailto:[email protected]> Ce message transmis par voie ?lectronique ainsi que toutes ses annexes contiennent des informations qui peuvent ?tre confidentielles ou prot?g?es. Ces informations sont uniquement destin?es ? l'usage des personnes ou des entit?s pr?cis?es dans les champs 'A', 'Cc' et 'Cci'. Si vous n'?tes pas l'un de ces destinataires, soyez conscient que toute forme, partielle ou compl?te, de divulgation, copie, distribution ou utilisation de ces informations est strictement interdite. Si vous avez re?u ce message par erreur, veuillez nous en informer par t?l?phone ou par message ?lectronique et d?truire les informations imm?diatement. Ce message n'engage que son signataire et aucunement son employeur.
