I created a Jira to document the new load balancing functionality [1] and just submitted a PR for it [2].
I think it would be beneficial for users if it could be reviewed and included in the release. [1] https://issues.apache.org/jira/browse/NIFI-5701 [2] https://github.com/apache/nifi/pull/3080 Thanks! -Drew > On Oct 16, 2018, at 11:13 AM, Jeff <jtsw...@gmail.com> wrote: > > Since NIFI-5562 is an improvement JIRA, and we're cutting it very close to > the release candidate, it may not make it into the release. However, since > Mark Payne is investigating some issues that may have been introduced in > 1.8.0, I can delay creating RC1. > > On Tue, Oct 16, 2018 at 10:24 AM Mark Payne <marka...@hotmail.com> wrote: > >> Jeff / all, >> >> I ran into an issue with NIFI-375 and re-opened the ticket. If a processor >> is stopped or started in a cluster, >> the stats that come back in the response are incorrect because the >> response is not being properly merged >> from all nodes in the cluster. >> >> I also have run into a couple of other issues that appear to have been >> introduced in 1.8. I have not yet created JIRA's >> for them because I want to understand the issues better before trying to >> write it up. I am seeing, for example, some errors >> in the logs indicating that provenance events don't have a FlowFile UUID >> assigned to them when being read from the >> repository. This seems to have been introduced in 1.8 so would like to get >> a resolution before releasing. >> >> Thanks >> -Mark >> >> >>> On Oct 15, 2018, at 6:33 PM, Jeff <jtsw...@gmail.com> wrote: >>> >>> NiFi Devs, >>> >>> The Release page [1] for 1.8.0 now reports that all issues are done! I'd >>> like to start the release candidate preparation tomorrow, around 1200 >> EST. >>> >>> Thanks to everyone for all the great work that's been done! 196 issues >>> resolved in this version with some great new features! >>> >>> [1] https://issues.apache.org/jira/projects/NIFI/versions/12343482 >>> >>> On Mon, Oct 15, 2018 at 7:30 AM Sivaprasanna <sivaprasanna...@gmail.com> >>> wrote: >>> >>>> Great. Thanks. :) >>>> >>>> - >>>> Sivaprasanna >>>> >>>> On Mon, Oct 15, 2018 at 7:09 AM Koji Kawamura <ijokaruma...@gmail.com> >>>> wrote: >>>> >>>>> Jeff, Sivasprasanna, >>>>> >>>>> NIFI-5698 (PR3073) Fixing DeleteAzureBlob bug is merged. >>>>> >>>>> Thanks, >>>>> Koji >>>>> On Mon, Oct 15, 2018 at 10:18 AM Koji Kawamura <ijokaruma...@gmail.com >>> >>>>> wrote: >>>>>> >>>>>> Thank you for the fix Sivaprasanna, >>>>>> I have Azure account. Reviewing it now. >>>>>> >>>>>> Koji >>>>>> On Sun, Oct 14, 2018 at 11:21 PM Jeff <jtsw...@gmail.com> wrote: >>>>>>> >>>>>>> Sivaprasanna, >>>>>>> >>>>>>> Thanks for submitting a pull request for that issue! Later today or >>>>>>> tomorrow I'll have to check to see if I've already used up my >>>> free-tier >>>>>>> access to Azure. If I still have access, I can review your PR and >>>>> we'll >>>>>>> get it into 1.8.0. >>>>>>> >>>>>>> On Sun, Oct 14, 2018 at 4:30 AM Sivaprasanna < >>>>> sivaprasanna...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> All - Just found one bug with DeleteAzureBlobStorage processor. It >>>>> was >>>>>>>> shared by one user on StackOverflow [1] and I later confirmed it. >>>> It >>>>> looks >>>>>>>> to be introduced by NIFI-4199. I have created a Jira [2] and made >>>> the >>>>>>>> necessary changes (not huge, just few lines) and raised a PR [3]. I >>>>> think, >>>>>>>> if we can spend a little time in getting it reviewed, we can mark >>>> it >>>>> for >>>>>>>> 1.8.0. Thoughts? >>>>>>>> >>>>>>>> [1] - >>>>>>>> >>>>>>>> >>>>> >>>> >> https://stackoverflow.com/questions/52766991/apache-nifi-deleteazureblobstorage-processor-is-throwing-an-error >>>>>>>> [2] - https://issues.apache.org/jira/browse/NIFI-5698 >>>>>>>> [3] - https://github.com/apache/nifi/pull/3073 >>>>>>>> >>>>>>>> - >>>>>>>> Sivaprasanna >>>>>>>> >>>>>>>> On Fri, Oct 12, 2018 at 9:05 PM Mike Thomsen < >>>> mikerthom...@gmail.com >>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> 4811 should be ready for review now. Rebased and cleaned it up >>>>> with a >>>>>>>> full >>>>>>>>> listing of the Spring dependencies. >>>>>>>>> >>>>>>>>> On Fri, Oct 12, 2018 at 11:23 AM Joe Witt <joe.w...@gmail.com> >>>>> wrote: >>>>>>>>> >>>>>>>>>> Jeff, >>>>>>>>>> >>>>>>>>>> I think for anything not tagged to 1.8.0 we just keep rolling. >>>>> For >>>>>>>>>> anything tagged 1.8.0 that should not be we should remove it >>>>> until >>>>>>>>>> ready. For things tagged to 1.8.0 that cannot be moved we >>>> should >>>>>>>>>> resolve. For the tagged 1.8.0 section you had. >>>>>>>>>> >>>>>>>>>> - NIFI-4811 < >>>> https://issues.apache.org/jira/browse/NIFI-4811> >>>>> - >>>>>>>> Use a >>>>>>>>>> newer version of spring-data-redis >>>>>>>>>> - PR 2856 <https://github.com/apache/nifi/pull/2856> >>>>>>>>>> *This needs to be resolved by either reverting the commit or >>>>> ensuring >>>>>>>>>> L&N accurately reflects all. We have to do this always and for >>>>> every >>>>>>>>>> nar. The process isnt easy or fun but it is necessary to >>>> produce >>>>>>>>>> valid ASF releases. Landing commits which change dependencies >>>>>>>>>> requires this due diligence. Now, we've put a lot of energy >>>> into >>>>>>>>>> updating Spring dependencies because some older Spring libs had >>>>>>>>>> vulnerabilities which while we likely aren't exposed to them we >>>>> want >>>>>>>>>> to fix in due course. So reverting may require more analysis >>>>> than if >>>>>>>>>> we were just get L&N fixed with this new change. I commented >>>> on >>>>> the >>>>>>>>>> JIRA. But this needs to be resolved. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> - NIFI-5426 < >>>> https://issues.apache.org/jira/browse/NIFI-5426> >>>>> - Use >>>>>>>>>> NIO.2 API for ListFile to avoid multiple disk reads >>>>>>>>>> - PR 2889 <https://github.com/apache/nifi/pull/2889> >>>>>>>>>> *This just needed to be marked resolved. The commit went in >>>> the >>>>> day >>>>>>>>>> after we cut 1.7.1. So this one is sorted. >>>>>>>>>> >>>>>>>>>> - NIFI-5448 < >>>> https://issues.apache.org/jira/browse/NIFI-5448> >>>>> - >>>>>>>>> Failed >>>>>>>>>> EL date parsing live-locks processors without a failure >>>>> relationship >>>>>>>>>> * The commit needs to be reverted. I'm working on that now. >>>>> Once the >>>>>>>>>> discsusion/concerns are addressed this can get dealt with. >>>>>>>>>> >>>>>>>>>> - NIFI-5665 < >>>> https://issues.apache.org/jira/browse/NIFI-5665> >>>>> - >>>>>>>>> Upgrade >>>>>>>>>> io.netty dependencies >>>>>>>>>> * This looks important to get resolved if possible as old netty >>>>> libs >>>>>>>>>> are on the list of things with vulnerabilities. >>>>>>>>>> >>>>>>>>>> - NIFI-5686 < >>>> https://issues.apache.org/jira/browse/NIFI-5686> >>>>> - >>>>>>>> Test >>>>>>>>>> failure in TestStandardProcessScheduler >>>>>>>>>> - PR 3062 <https://github.com/apache/nifi/pull/3062> >>>>>>>>>> * This has a PR but a test, possibly two, failed in one of the >>>>> travis >>>>>>>>>> runs and it is clearly related. I ignored one of those tests >>>> in >>>>> a >>>>>>>>>> previous run. We must deal with brittle tests. But the >>>>> underlying >>>>>>>>>> problem is important to solve here so either the tests needs >>>>> improved >>>>>>>>>> or we still have an issue. Not clear but worth some focus. >>>>>>>>>> >>>>>>>>>> note: I intend to reference updates to libraries that have >>>> known >>>>>>>>>> vulnerabilities and do so in a far less subtle manner than we >>>>> had. We >>>>>>>>>> aren't acknowledging that NiFi is or exposes vulnerabilities >>>> but >>>>> we >>>>>>>>>> are and should be clear when we're updating dependencies that >>>> do >>>>> have >>>>>>>>>> them (even if we're not exposed to them) so that some of these >>>>> commits >>>>>>>>>> aren't so mysterious. It creates far more confusion than is >>>>> worth. >>>>>>>>>> We still will follow the ASF/NiFi security handling policy but >>>> I >>>>> no >>>>>>>>>> longer intend to treat due course dependency updates as if they >>>>> need >>>>>>>>>> to be a secret. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Joe >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Oct 12, 2018 at 3:32 AM Jeff <jtsw...@gmail.com> >>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hello everyone! Next week is probably a good timeframe to >>>> aim >>>>> for a >>>>>>>>>>> release candidate, with two major feature PRs recently merged >>>>> to >>>>>>>>> master: >>>>>>>>>>> >>>>>>>>>>> - NIFI-5516 < >>>>> https://issues.apache.org/jira/browse/NIFI-5516> - >>>>>>>>> Allow >>>>>>>>>>> data in a Connection to be Load-Balanced across cluster >>>>>>>>>>> - NIFI-5585 < >>>>> https://issues.apache.org/jira/browse/NIFI-5585> - >>>>>>>>>> Prepare >>>>>>>>>>> Nodes to be Offloaded >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> To recap, here's a list of other JIRAs mentioned in this >>>>> thread: >>>>>>>>>>> >>>>>>>>>>> - NIFI-5402 < >>>>> https://issues.apache.org/jira/browse/NIFI-5402> - >>>>>>>>>> Reduce >>>>>>>>>>> artifact size by only building .zip archive >>>>>>>>>>> - NIFI-5462 < >>>>> https://issues.apache.org/jira/browse/NIFI-5462> - >>>>>>>>>> Refactor >>>>>>>>>>> TLS Toolkit >>>>>>>>>>> - NIFI-5485 < >>>>> https://issues.apache.org/jira/browse/NIFI-5485> - >>>>>>>>>> Enable >>>>>>>>>>> TLS Toolkit (client/server) to sign certificates with >>>>> external CA >>>>>>>>>>> certificate >>>>>>>>>>> - NIFI-5537 < >>>>> https://issues.apache.org/jira/browse/NIFI-5537> - >>>>>>>>>> Create >>>>>>>>>>> Neo4J cypher execution processor >>>>>>>>>>> - PR 2956 <https://github.com/apache/nifi/pull/2956> >>>>>>>>>>> - Mike Thomsen, this was the specific JIRA to which you >>>>> were >>>>>>>>>>> referring, right? >>>>>>>>>>> - NIFI-5582 < >>>>> https://issues.apache.org/jira/browse/NIFI-5582> - >>>>>>>>>> Integrate >>>>>>>>>>> legacy behavior of HashAttribute into >>>>> CryptographicHashAttribute >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> These JIRAs are marked with a fix version of 1.8.0 that are >>>> not >>>>>>>>> currently >>>>>>>>>>> resolved: >>>>>>>>>>> >>>>>>>>>>> - NIFI-4811 < >>>>> https://issues.apache.org/jira/browse/NIFI-4811> - >>>>>>>>> Use a >>>>>>>>>>> newer version of spring-data-redis >>>>>>>>>>> - PR 2856 <https://github.com/apache/nifi/pull/2856> >>>>>>>>>>> - NIFI-5426 < >>>>> https://issues.apache.org/jira/browse/NIFI-5426> - >>>>>>>> Use >>>>>>>>>>> NIO.2 API for ListFile to avoid multiple disk reads >>>>>>>>>>> - PR 2889 <https://github.com/apache/nifi/pull/2889> >>>>>>>>>>> - NIFI-5448 < >>>>> https://issues.apache.org/jira/browse/NIFI-5448> - >>>>>>>>>> Failed >>>>>>>>>>> EL date parsing live-locks processors without a failure >>>>>>>> relationship >>>>>>>>>>> - NIFI-5665 < >>>>> https://issues.apache.org/jira/browse/NIFI-5665> - >>>>>>>>>> Upgrade >>>>>>>>>>> io.netty dependencies >>>>>>>>>>> - NIFI-5686 < >>>>> https://issues.apache.org/jira/browse/NIFI-5686> - >>>>>>>>> Test >>>>>>>>>>> failure in TestStandardProcessScheduler >>>>>>>>>>> - PR 3062 <https://github.com/apache/nifi/pull/3062> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Oct 4, 2018 at 6:39 AM Mike Thomsen < >>>>> mikerthom...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> That's a fair point. Only thing I could add there is that I >>>>> think >>>>>>>> we >>>>>>>>>> should >>>>>>>>>>>> consider a targeted burn down on the PR list as part of >>>> 1.9. >>>>> There >>>>>>>>> are >>>>>>>>>> a >>>>>>>>>>>> lot of PRs from the last several months that would be good >>>>>>>> candidates >>>>>>>>>> to >>>>>>>>>>>> see if we can close them out like MarkLogic and Pulsar. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Oct 3, 2018 at 10:34 PM Joe Witt < >>>> joe.w...@gmail.com >>>>>> >>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Mike, >>>>>>>>>>>>> >>>>>>>>>>>>> Processors in particularly are among the toughest at this >>>>> point. >>>>>>>>> We >>>>>>>>>>>>> have very very little headroom on dependency size for the >>>>> full >>>>>>>>> build >>>>>>>>>>>>> size that we upload to ASF infra and mirrors. That and >>>> the >>>>>>>> license >>>>>>>>>>>>> review work involved in each... >>>>>>>>>>>>> >>>>>>>>>>>>> We should really create a way to publish processors on >>>> more >>>>>>>>> frequent, >>>>>>>>>>>>> irregular intervals where the release work and size/etc.. >>>>> are far >>>>>>>>>> less >>>>>>>>>>>>> problematic. We have another discuss thread on that so >>>>> I'll >>>>>>>> leave >>>>>>>>> it >>>>>>>>>>>>> there for discussion. I do share your view that this >>>>> processor >>>>>>>>>> (among >>>>>>>>>>>>> several others outstanding) would be really useful but i >>>> am >>>>>>>>>> definitely >>>>>>>>>>>>> thinking we should keep release pace up. Release more >>>>>>>>>> often...release >>>>>>>>>>>>> processors separately, etc.. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> Joe >>>>>>>>>>>>> On Wed, Oct 3, 2018 at 9:30 PM Mike Thomsen < >>>>>>>>> mikerthom...@gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would like to see the Neo4J work that mans2singh is >>>>> doing get >>>>>>>>>>>> included. >>>>>>>>>>>>>> Being able to at least partially support a popular >>>> graph >>>>>>>> database >>>>>>>>>> would >>>>>>>>>>>>> be >>>>>>>>>>>>>> a nice feather in our cap. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Oct 3, 2018 at 5:12 PM Andy LoPresto < >>>>>>>>> alopre...@apache.org >>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am currently working on a TLS Toolkit refactor >>>>> (NIFI-5462 & >>>>>>>>>>>>> NIFI-5485) >>>>>>>>>>>>>>> and HashAttribute updates (NIFI-5582). I believe >>>> there >>>>> are a >>>>>>>>>> couple >>>>>>>>>>>>> upgrade >>>>>>>>>>>>>>> PRs open, and I would really like to see NIFI-5402 >>>> (no >>>>>>>> .tar.gz >>>>>>>>>> in the >>>>>>>>>>>>>>> build) tackled for this release as well. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Andy LoPresto >>>>>>>>>>>>>>> alopre...@apache.org >>>>>>>>>>>>>>> *alopresto.apa...@gmail.com < >>>>> alopresto.apa...@gmail.com>* >>>>>>>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E >>>>> F65B >>>>>>>> 2F7D >>>>>>>>>> EF69 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Oct 3, 2018, at 11:16 AM, Joe Witt < >>>>> joew...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Jeff - thanks again for volunteering. I just went >>>>> through >>>>>>>> the >>>>>>>>>> open >>>>>>>>>>>>>>> items tagged to 1.8.0 to try and shake some loose, >>>>> close down >>>>>>>>>> ones >>>>>>>>>>>>>>> that appear to be done but forgotten, and initiate >>>>> resolution >>>>>>>>> on >>>>>>>>>> one >>>>>>>>>>>>>>> that is in a dangling state. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Another very nice release shaping up here. All the >>>>> work >>>>>>>> around >>>>>>>>>> load >>>>>>>>>>>>>>> balancing and node offloading is awesome. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>> On Wed, Oct 3, 2018 at 2:06 PM Jeff < >>>>> jsto...@apache.org> >>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It looks like we're getting close to a point where we >>>>> could >>>>>>>>>> release >>>>>>>>>>>>> NiFi >>>>>>>>>>>>>>> 1.8.0The release tracking page for version 1.8.0 [1] >>>>> shows 3 >>>>>>>>> "in >>>>>>>>>>>>> progress" >>>>>>>>>>>>>>> and 9 "to do" issues. In addition to what has been >>>>> tagged >>>>>>>> with >>>>>>>>> a >>>>>>>>>> fix >>>>>>>>>>>>>>> version of 1.8.0, it looks like NIFI-5516 and >>>>> NIFI-5585 are >>>>>>>>>> close to >>>>>>>>>>>>>>> completion. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Are there other JIRAs that the community considers >>>>> necessary >>>>>>>>> for >>>>>>>>>> the >>>>>>>>>>>>>>> release that are close to being resolved, with the >>>>> goal of >>>>>>>>>> getting a >>>>>>>>>>>>>>> release candidate out in the next couple of weeks? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm happy to perform the release manager duties! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] >>>>>>>>>> https://issues.apache.org/jira/projects/NIFI/versions/12343482 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>> >>>> >> >>
