Thank you for your replies. My goal was to have that shared ownership as Bryan suggested; however, the simplicity of bootstrapping these efforts within Apache NiFi MiNiFi C++, controlling dependencies, and overall ease of use by maintainers has led me to close the PR and place this assembly in MiNiFi C++. This enables the builder to package all necessary components within the produced archive and have access to Java processors within MiNiFI C++( they must purposefully enable this feature ). Just wanted to close the loop. Thank you for your consideration
On Wed, Feb 27, 2019 at 12:52 PM Bryan Bende <[email protected]> wrote: > I think this situation is sort of analogous to some other scenarios we > have run into, such as the integrations between NiFi and stream > processing. > > For example, the storm and spark integration lives in NiFi's code > base, and NiFi is responsible for determining if a new version of > those APIs is available, updating the dependency, and resolving any > necessary code changes. > > The Flink and Apex integration lives in each of those respective code > bases, so if Flink changes all of their APIs, they will be forced to > update the NiFi Flink source/sink in order to make their build pass. > > I can see value in either approach. If we consider the JNI stuff to be > a more generic capability, then I think I lean towards NiFi owning it. > > Even with that, MiNiFi will still be dependent on some version of > whatever JNI artifacts NiFi produces, so MiNiFi would still have > ownership over when to change that dependency and ensure that it still > works properly in a given release of MiNiFi. > > On Wed, Feb 27, 2019 at 12:02 PM Arpad Boda <[email protected]> wrote: > > > > Hi, > > > > In my opinion, there is another question here to be answered: what > version of NiFi NARs do we expect to work in MiNiFi? > > > > I see two different approaches here: > > -#1) MiNiFi is compatible with a given NiFi version, like 1.9.0, NARs in > that release work properly in MiNiFi. > > In this case the tests belong to MiNiFi and NiFi there becomes a > dependency -> the tests are required to pass when upgrading the dependency. > > -#2) MiNiFi is compatible with master. This means that whatever we > introduce in NiFi shouldn't break, so the tests more belong to NiFi. > > > > According to my experience preventing the merge of a breaking change is > easier than fixing later, which would make me vote for #2, however this > approach has a side effect, too: introducing a breaking change (that MiNiFi > needs to adapt) becomes a huge pain, it's much easier in #1. > > > > Regards, > > Arpad > > > > On 27/02/2019, 17:44, "Marc Parisi" <[email protected]> wrote: > > > > Hi , > > I've submitted a PR to Apache NiFi MiNiFi C++ that enables the > project > > to access and run NiFi processors. This requires that certain jars be > > included to bootstrap that process long with a set of NARs from > which we > > will access the desired components. Effectively this means that a > user can > > run the current set of C++ processors alongside Java processors. > > To facilitate this feature's use, I submitted a PR [1] to NiFi > with that > > assembles the appropriate JARs and a basic set of useful/necessary > NARs. > > I'd like opinions on where this should live. The options have been > to place > > it within Apache NiFi MiNIFi C++ ( specifically within the > aforementioned > > feature's build directory) as it only relates to that project and, > perhaps > > allowing us to more easily address problems...or as per the PR: > placing it > > in NiFi proper, hopefully building regression tests that notify us of > > issues before a release. > > > > I've been wavering on the options and want community opinion. The > end > > goal in either case is that if/when a user builds with this > functionality > > we can deliver a package that runs out of the box. This can be done > with > > either approach. > > > > The net positive of placing it in NiFi as I see it: sense that > it is an > > official feature and perhaps notification before NiFi release or in > > travis. Side effects include: questioning if this belongs here, > hair loss, > > and perhaps a sudden loss of appetite. > > > > The net positive of isolating this within MiNIFi C++: cleaner end > > solution, it's produced in the same place it's consumed, ability to > notify > > of problems in our CI and release process ( this could be viewed as a > > negative too ? ). > > > > TL;DR: Where feature Live? > > > > Any insight/opinions are appreciated. Thanks! > > > > [1] https://github.com/apache/nifi/pull/3330 > > > > >
