Unsubscribe
-Original Message-
From: Midhun Mohan
Sent: Wednesday, March 18, 2020 8:29 AM
To: dev@nifi.apache.org
Subject: Re: Make invokehttp to process faster to process more than 20k records
Hey Mike , I meant like when I try posting that many records it is taking bit
time. Just che
Looks like a great start! Very much like the direction.
Would be ideal to also see mock up of how we would envison NIFI-951 to be
factored in.
Thanks
Rick
-Original Message-
From: Rob Moran [mailto:rmo...@gmail.com]
Sent: Wednesday, January 06, 2016 12:09 PM
To: dev@nifi.apache.org
Su
ithdraw this proposal at this time?
>
> Thanks
> Joe
>
>> On Fri, Sep 25, 2015 at 4:25 AM, Rick Braddy wrote:
>> Yes. Replication of directory tree via Nifi similar to rsync.
>>
>> -Original Message-
>> From: Joe Skora [mailto:jsk...@gmail
Joe,
This reminds me... are there any entry or exit criteria (from a defects
perspective) established for NiFi releases? In other words, what is the
criteria for determining when the code is ready for release and production use?
Thanks
Rick
-Original Message-
From: Joe Witt [mailto:jo
How about auto-rejecting emails from all unregistered senders, since one must
register before sending email this group anyway?
Spammers won't take time to register.
Rick
> On Oct 30, 2015, at 8:19 AM, Joe Witt wrote:
>
> Team,
>
> As a moderator I'm getting tons of spam emails to dev@nifi.
e open in both directions through a firewall. Even with
those iptables rules, it seems something is missing. I will figure it out
eventually, and let everyone know what's required to use Nifi across firewall
boundaries.
Rick
-----Original Message-
From: Rick Braddy
Sent: Monday, Oct
+1
> On Oct 9, 2015, at 7:16 AM, Matt Gilman wrote:
>
> Joe,
>
> We've been receiving a lot of feedback lately regarding the Purge/Clear
> Queue capability. Because of this we'd like to introduce that feature into
> the 0.4.0 release with the Viewing and Removing of individual FlowFiles
> into
ility to add a custom authentication provider will be important for
> such usecases.
>
>> On Mon, Oct 5, 2015 at 10:10 PM, Tony Kurc wrote:
>>
>> I'd like to see Duo Web two-factor https://www.duosecurity.com/docs/duoweb
>>
>>> On Mon, Oct 5, 2015 at 10:00
anks Rick. If you were to say which of that you'd want 'first' and
> then which you can see coming later please advise.
>
> All: Please do just that - let us know which you need 'now' and which
> you can wait on.
>
> Thanks
> Joe
>
>> On Mon,
Matt,
Here you go:
- 2-factor Google Authenticator to supplement password auth (e.g. to
strengthen password with mobile phone onetime ID or other support strong auth
options)
- Recaptcha required after N failed password login attempts to block brute
force attacks (e.g. 5 failed logins, then
what's required to use Nifi across firewall
boundaries.
Rick
-Original Message-
From: Rick Braddy
Sent: Monday, October 05, 2015 10:18 AM
To: dev@nifi.apache.org
Subject: RE: Remote process group networking
Let me ask this in a simpler way... for Nifi Remote Process Group
communica
Let me ask this in a simpler way... for Nifi Remote Process Group
communications across firewall boundaries, which ports must be open through
firewalls between a source node running the local graph processes and the
Remote Process Group node?
Rick
-Original Message-
From: Rick Braddy
I have a question about network paths required for proper operation of remote
process groups.
By default, the initial connection from source node to remote process group
target node is on port 8080. Then, there's a second port (e.g., I set it to
8081 and a setting for whether it's SSL secured
oversimplification, but for the purposes of understanding, is the
intent to mirror directory tree with NiFi similar to rsync?
On Wed, Sep 23, 2015 at 11:26 PM, Rick Braddy wrote:
> Joe,
>
> Thanks for the quick response.
>
> Yes, I can add to the Wiki once access has been granted. Fu
work with you to bring it
in. It is a non-trivial contribution for sure. Folks often do not consider
all the nasty gotchas that can occur in something as seemingly simple as File
IO.
Thanks
Joe
[1] https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals
On Wed, Sep 23, 2015 at
This thread proposes community review/comments of modified versions of GetFile
and PutFile for potential future adoption by the Nifi community. For those who
want to jump straight to the code, here's the review repository location for
the current version: https://github.com/rickbraddy/nifishar
ip is
>> unused and that is ok vs our current approach which says each
>> relationship must be chosen as terminated or connected.
>>
>> Thanks
>> Joe
>>
>>> On Mon, Sep 21, 2015 at 11:26 PM, Rick Braddy wrote:
>>> Me, too.
>>>
>>
yet though. It simply provides a way for the processor to
>>> developer to signal the default behavior is that a relationship is
>>> unused and that is ok vs our current approach which says each
>>> relationship must be chosen as terminated or connected.
>>>
&
Me, too.
And I need the ability to determine which connection(s) are linked to each
relationship, along with a way to auto-terminate unused relationships from my
processor.
Thanks
Rick
> On Sep 21, 2015, at 10:22 PM, Huang, Jie wrote:
>
> Hi all,
>
> I need to get the connection information
PM, Rick Braddy wrote:
>
> Can templates be deleted after being created or are they permanent for some
> reason?
>
> -Original Message-
> From: Mark Payne [mailto:marka...@hotmail.com]
> Sent: Thursday, September 17, 2015 7:10 PM
> To: dev@nifi.apache.org
> Su
Can templates be deleted after being created or are they permanent for some
reason?
-Original Message-
From: Mark Payne [mailto:marka...@hotmail.com]
Sent: Thursday, September 17, 2015 7:10 PM
To: dev@nifi.apache.org
Subject: Re: Developer questions
cb,
That is correct - in NiFi you ca
to the new lib directory.
On Wed, Sep 16, 2015 at 8:47 AM, Rick Braddy wrote:
> Thanks Bryan. That's easy!
>
> Rick
>
>
>
> > On Sep 15, 2015, at 6:40 PM, Bryan Bende wrote:
> >
> > Rick,
> >
> > You should be able to move the conf/flow.xml
Thanks Bryan. That's easy!
Rick
> On Sep 15, 2015, at 6:40 PM, Bryan Bende wrote:
>
> Rick,
>
> You should be able to move the conf/flow.xml.gz from your snapshot
> install to your new 0.3.0 conf directory.
>
> -Bryan
>
>> On Tuesday, September 15
How can I migrate prior flow graph from 0.3.0-SNAPSHOT to the new 0.3.0 tree?
Rick
+1 Release this package as nifi-0.3.0
- KEYS file in source package only contains 4 signers, official release dist
has 6 signers: https://dist.apache.org/repos/dist/release/nifi/KEYS
(looks like Aldrin and Mark Payne were added to release keys but mi
thing was previously auto-terminated but now that relationship is connected
to another component, that "checkbox" is removed.
3: Would like to hear additional details about where files go from here, but
what you may be after could be an emerging pattern of use. Need to see how
this di
> -Mark
>
>
>> From: rbra...@softnas.com
>> To: dev@nifi.apache.org
>> Subject: Re: Auto-terminating default connections programmatically
>> Date: Mon, 14 Sep 2015 12:19:37 +
>>
>> Bump
>>
>>
Hi,
We have a situation whereby we want a processor that automatically discovers
some resources (e.g., filesystems), and then auto-publishes a number of
available relationships as output connections from the processor to be used to
configure other processors (one processor configuring another
Bump
> On Sep 12, 2015, at 4:39 PM, Rick Braddy wrote:
>
> Hi,
>
> Is there a way to auto-terminate certain relationships / connections
> automatically from within a processor?
>
> For example, a processor has three outputs, but by default we only want the
> &
Hi,
Is there a way to auto-terminate certain relationships / connections
automatically from within a processor?
For example, a processor has three outputs, but by default we only want the
"success" relationship to be required, to save the user the hassle of having to
manually auto-terminate re
the project matures, this kind of
thing, along with an API reference guide would be very helpful for newbie
developers.
Thanks again!
On my way again...
Rick
-Original Message-----
From: Rick Braddy [mailto:rbra...@softnas.com]
Sent: Saturday, September 12, 2015 3:31 PM
To: dev@nifi.ap
corded - the first incoming
> > flowfile, for which session.transfer() was never called.
> >
> > Questions:
> >
> > 1. Is it required to "dispose" of incoming flowfiles that have been
> > accessed via session.get() using session.transfer()?
>
ed to an unused relationship,
> such as "ORIGINAL" like the SplitText processor does?
>
> 3. I read about the "SupportsBatching" attribute, which is not set on my
> processor (or the original GetFile processor, which does not read from input
> queues). Gi
in batches, should this
attribute be set?
Thanks
Rick
-----Original Message-
From: Rick Braddy [mailto:rbra...@softnas.com]
Sent: Saturday, September 12, 2015 8:50 AM
To: dev@nifi.apache.org
Subject: RE: Transfer relationship not specified (FlowFileHandlingException)
More information..
d and processed
just fine, but it seems that commit() calling checkpoint() doesn't like what
it's seeing overall.
Rick
-Original Message-
From: Rick Braddy [mailto:rbra...@softnas.com]
Sent: Saturday, September 12, 2015 7:42 AM
To: dev@nifi.apache.org
Subj
ified (FlowFileHandlingException)
Yep. Looks legit to me. Will try a unit test with a mixture of flowFiles
associated with content and without.
On Sat, Sep 12, 2015 at 8:12 AM, Rick Braddy wrote:
> Joe,
>
> Replies below.
>
> Rick
>
>
> -Original Message-
> From: Joe
nfirm which codebase you're running against. Latest HEAD of
master?
I'm using a snap from GitHub that's several weeks old from August 25th (it's
working fine with the original GetFile processor, which this code was derived
from)
Thanks
Joe
On Sat, Sep 12, 2015 at 7:21 A
-Original Message-----
From: Rick Braddy
Sent: Friday, September 11, 2015 8:26 PM
To: dev@nifi.apache.org
Subject: RE: Transfer relationship not specified (FlowFileHandlingException)
Mark,
The interesting thing is that session.transfer() is being called, as I have
stepped through it in the
Tim,
Based on what you describe, and not being familiar with Kafka or your
application, it sounds like breaking each row into a flowfile could make sense,
depending upon what you're needing to do downstream. There is overhead
associated with each FlowFile, as well as a provenance consideration
s Thread-3]
> c.s.c.processors.files.GetFileData
> GetFileData[id=8f5e644d-591c-4df1-8c79-feea118bd8c0] added
> StandardFlowFileRecord[uuid=a324aaff-a340-499d-9904-2421b2bfc4a8,claim=,offset=0,name=in,size=0]
> to flow ...
>
> So it seems there's some issue with each of the F
f the FlowFiles...
-Original Message-
From: Rick Braddy [mailto:rbra...@softnas.com]
Sent: Friday, September 11, 2015 6:00 PM
To: dev@nifi.apache.org
Subject: Transfer relationship not specified (FlowFileHandlingException)
Hi,
I have a processor that appears to be creating FlowFiles corr
Hi,
I have a processor that appears to be creating FlowFiles correctly (modified a
standard processor), but when it goes to commit() the session, an exception is
raised:
2015-09-11 17:37:24,690 ERROR [Timer-Driven Process Thread-6]
c.s.c.processors.files.GetFileData
[GetFileData[id=8f5e644d-5
I dropped back to an earlier rev of my NiFi run-time tree and all is well now.
Thanks for the help.
Rick
-Original Message-
From: Rick Braddy [mailto:rbra...@softnas.com]
Sent: Wednesday, September 09, 2015 5:35 PM
To: dev@nifi.apache.org
Subject: RE: NiFi service fails to start after
ur old NAR, then remove GetFileAttributes from
your graph, then you will be able to install your new NAR that doesn't contain
GetFileAttributes.
-- Mike
On Wed, Sep 9, 2015 at 6:11 PM, Rick Braddy wrote:
> Hi,
>
> One of the processor files I deleted from my NAR project is
> GetF
Hi,
One of the processor files I deleted from my NAR project is GetFileAttributes.
Now the NiFi service fails to start with these complaints below. I have deleted
the library in the lib folder, deleted work/nar folder, I even tried adding the
file back into the project to see if that might help
att
[1] https://issues.apache.org/jira/browse/NIFI-943
[2] https://issues.apache.org/jira/browse/NIFI-78
On Sun, Sep 6, 2015 at 6:59 PM, Rick Braddy wrote:
> Hi,
>
> I have been doing some testing with ExecuteProcess processor to learn
> more about how Nifi processors work and perform I/O.
d see if it contains
both the old class and the new class?
Dan Bress
Software Engineer
ONYX Consulting Services
____
From: Rick Braddy
Sent: Wednesday, September 9, 2015 3:25 PM
To: dev@nifi.apache.org
Subject: RE: How to delete docs for renamed processor
Dan
Also did you do a 'mvn clean install' on your nar, or just 'mvn install'?
Dan Bress
Software Engineer
ONYX Consulting Services
________
From: Rick Braddy
Sent: Wednesday, September 9, 2015 3:00 PM
To: dev@nifi.apache.org
Subject: RE: How to d
Thanks Mark. Unfortunately, that didn't work - it's still there.
Rick
-Original Message-
From: Mark Payne [mailto:marka...@hotmail.com]
Sent: Wednesday, September 09, 2015 1:45 PM
To: dev@nifi.apache.org
Subject: RE: How to delete docs for renamed processor
Rick,
You can try deleting
Hi,
Hopefully a quick answer. I have renamed a custom processor and I still see
the processor by the old name showing up in the user docs. Is there a way to
delete the docs related to the old processor name?
Thanks
Rick
sion.
It's documented in the developer guide under "Session Rollback".
Joe- - - - - - Joseph Percivalllinkedin.com/in/Percivalle:
joeperciv...@yahoo.com
On Tuesday, September 8, 2015 2:54 PM, Rick Braddy
wrote:
Hi,
During development of some new processors, I hav
Hi,
During development of some new processors, I have been looking closely at
standard processors to understand best practices. The Developer Guide suggests
that one should call session "commit()" upon completion of onTrigger() session
processing, which makes sense. However, I notice in a numb
Hi,
I have been doing some testing with ExecuteProcess processor to learn more
about how Nifi processors work and perform I/O. This has led to a couple of
questions.
1) Nifi API docs - are there API docs planned for what appear to be
"internal" API's that are commonly used by processors
Ravi,
We process large numbers of files and noticed the I/O overhead as well. I put
the content, FlowFiles and provenance repositories on SSD and was then able to
achieve high enough throughput with our workloads to consistently saturate a
1Gb pipe with lots of parallel processing, which was f
ance,
>
>
>
> This would enable DEBUG level logging for every GetFile in your flow.
>
> Matt
>
>> On Fri, Sep 4, 2015 at 4:48 PM, Rick Braddy wrote:
>>
>> I know there's a way to enable DEBUG logging... thought I saw it somewhere
>>
I know there's a way to enable DEBUG logging... thought I saw it somewhere in
the docs, but just researched and the Developer's Guide says to use
ProcessorLog for DEBUG, but doesn't say how to enable debug logging (or what
the logging levels are, how to change logging levels system-wide or per
dle and restart Nifi service... so it appears that since
the Maven repository is associated with my local home directory, that may be
why it's failing.
I will create a link and see if that resolves the issue.
Thanks for the pointer.
Rick
-Original Message-
From: Rick Braddy [mailto:
ailing.
I will create a link and see if that resolves the issue.
Thanks for the pointer.
Rick
-Original Message-----
From: Rick Braddy [mailto:rbra...@softnas.com]
Sent: Friday, September 04, 2015 1:00 PM
To: dev@nifi.apache.org
Subject: RE: POM dependency failures
Bryan,
Yes. The 0.3.0-S
,
Can you check in your local Maven repository to see if the 0.3.0-SNAPSHOTs are
in there? It is typically in HOME/.m2
As one example ~/.m2/repository/org/apache/nifi/nifi-api/ should have a
sub-directory for 0.3.0-SNAPSHOT.
-Bryan
On Fri, Sep 4, 2015 at 2:50 PM, Rick Braddy wrote:
> Yes
associate poms.
On Fri, Sep 4, 2015 at 1:39 PM, Rick Braddy wrote:
> Hi,
>
> Does anyone know why these dependencies would suddenly start failing
> or how to go about resolving?
>
> [WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is
> missing, no dependency
Hi,
Does anyone know why these dependencies would suddenly start failing or how to
go about resolving?
[WARNING] The POM for org.apache.nifi:nifi-api:jar:0.3.0-SNAPSHOT is missing,
no dependency information available
[WARNING] The POM for org.apache.nifi:nifi-processor-utils:jar:0.3.0-SNAPSHOT
cific commands'. Tricky road there.
Thanks
Joe
On Wed, Sep 2, 2015 at 6:59 PM, Rick Braddy wrote:
> Further troubleshooting today... used same "find" command via ExecuteProcess
> standard processor. It works fine, so there's something wrong with my
> customized
Further troubleshooting today... used same "find" command via ExecuteProcess
standard processor. It works fine, so there's something wrong with my
customized processor... will debug it to resolve.
-Original Message-----
From: Rick Braddy [mailto:rbra...@softnas.com]
Hi,
I have a slightly modified version of ExecuteProcess that's been customized to
do various "find targetdir -print" style commands, which generates standard
output that results in a FlowFile. Large outputs (long directory listings of
100 lines or more) work perfectly; however, it appears tha
Hi,
I have a Nifi design question. In order to process extremely large files (any
size), we intend to create a processor that reads the file in "chunks" and
sends as a multi-part FlowFile series, which will avoid using up all available
content repository and/or JVM space.
One way would be to c
just skip
them:
mvn clean package -DskipTests
Regards,
Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com
On Tue, Sep 1, 2015 at 8:53 AM, Rick Braddy wrote:
> Hi,
>
> Is there a way to speed up rebuilding the entire Nifi source tree
ean package -DskipTests
>
> Regards,
>
>
>
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
> On Tue, Sep 1, 2015 at 8:53 AM, Rick Braddy wrote:
>
>> Hi,
>>
>> Is there a way to speed up rebuilding
Hi,
Is there a way to speed up rebuilding the entire Nifi source tree using Maven
after making a minor change to a standard processor java source file? I mean,
by-passing all the time-consuming unit tests that waste time during the build
process and just: 1) rebuild source files that changed,
x27;t work after that.
-Bryan
On Mon, Aug 31, 2015 at 7:07 PM, Rick Braddy wrote:
> Hi,
>
> I'm developing a new processor that I want to test alongside the
> standard processors (like GetFile). I have added the new .java file
> and it's compiling just fine; however, it&#
Hi,
I'm developing a new processor that I want to test alongside the standard
processors (like GetFile). I have added the new .java file and it's compiling
just fine; however, it's not showing up in the processor list of the GUI.
The new processor has been added to:
~/nifi/nifi-nar-bundles/ni
at
is in your processors initialization logic. It will suspend launching until the
remote debugger is attached.
Matt
On Sun, Aug 30, 2015 at 3:41 PM, Rick Braddy wrote:
> Joe,
>
> Thanks for that information. I will start with NetBeans, as I've used
> it before on some other
ay want to just script out commands to copy over the old
nar and restart nifi.
Thanks
Joe
On Sun, Aug 30, 2015 at 1:34 PM, Rick Braddy wrote:
> Hi,
>
> The Nifi Developer's Guide is a great read with lots of useful information.
> However, as a relative noob to Nifi deve
Hi,
The Nifi Developer's Guide is a great read with lots of useful information.
However, as a relative noob to Nifi development, I still have a few questions
that seemed unanswered (or perhaps I overlooked):
1. Recommended IDE for Nifi - given Nifi development involves both Java
and Ma
73 matches
Mail list logo
