Re: [ANNOUNCE] Apache NiFi 1.0.0-Beta release

2016-08-09 Thread Andrew Psaltis
Congrats on a really big step!! Excited to play with the bits. On Wed, Aug 10, 2016 at 12:07 AM, Joe Percivall wrote: > > Hello > The Apache NiFi team would like to announce the release of Apache NiFi > 1.0.0-BETA. > The upcoming 1.0.0 release will mark the culmination of a lot of work over > th

[ANNOUNCE] Apache NiFi 1.0.0-Beta release

2016-08-09 Thread Joe Percivall
Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.0.0-BETA. The upcoming 1.0.0 release will mark the culmination of a lot of work over the last few months with many new framework level features being added. This Beta release was cut to give our Apache NiFi users a chanc

Re: Question on template handling in 1.0

2016-08-09 Thread James Wing
Thanks for picking this up, Joe. The short recap: In 1.0, the import template option is provided only from the floating Actions toolbar rather than the Templates dialog as it was in 0.x. I asked on the ticket why it moved. If I understand Matt Gilman's explanation, the reason is that importing

Re: Releasing the Beta

2016-08-09 Thread Joe Witt
Seems all reasonable to me and nicely done on working the process Joe. On Tue, Aug 9, 2016 at 5:27 PM, Joe Percivall wrote: > Hello Team, > > > With the most recent vote thread, Apache NiFi 1.0.0-Beta was approved to be > released. This is the first "unstable" release[1] aimed at developers and

Releasing the Beta

2016-08-09 Thread Joe Percivall
Hello Team, With the most recent vote thread, Apache NiFi 1.0.0-Beta was approved to be released. This is the first "unstable" release[1] aimed at developers and testers Apache NiFi has cut and as such I have omitted or modified a couple steps from the usual release guide[2]. For the sake of t

Re: ExecuteSQL generated Avro schema

2016-08-09 Thread Bryan Bende
SplitAvro should have a property called Output Strategy that can be changed to "Bare Record" which should be the record without the schema. Would that work? On Tue, Aug 9, 2016 at 2:22 PM, Nabegh wrote: > Hi Bryan, > > SplitAvro will include the schema with the data. I'm trying to separate > th

Re: ExecuteSQL generated Avro schema

2016-08-09 Thread Nabegh
Hi Bryan, SplitAvro will include the schema with the data. I'm trying to separate them to reduce the message size that is being sent through the network. Thanks -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/ExecuteSQL-generated-Avro-schema-tp13020p1302

Re: ExecuteSQL generated Avro schema

2016-08-09 Thread Bryan Bende
Hello, I know this is not a direct answer to your question, but if you want to send Avro records at the end, is there a reason you couldn't do ExecuteSQL -> SplitAvro -> InvokeHttp ? I'm assuming there is more logic in ExecuteScript besides just converting JSON to Avro, but wanted to make sure yo

ExecuteSQL generated Avro schema

2016-08-09 Thread Nabegh
I'm trying to pull records from a database, split them, encode each record separately, send the encoded record over the network, and finally decode the record at the receiving end. The processors I'm using are ExecuteSQL => ConvertAvroToJSON => SplitText => ExecuteScript => InvokeHTTP ExecuteScr

Question on template handling in 1.0

2016-08-09 Thread Joe Witt
James Wing asked a good question on a closed JIRA that I didn't want to get lost (closed things get far less attention usually). https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413792#comment-15413792 Recomme

Re: Convention for handling trailing spaces in property values

2016-08-09 Thread Joe Witt
My preference is that we make it more clear in the UI when/where whitespace characters are present. Too hard to predict whether certain spaces are desirable or not to enforce a trim. On Tue, Aug 9, 2016 at 11:40 AM, Joey Frazee wrote: > Hey guys, I was recently testing a PR and was seeing incons

Convention for handling trailing spaces in property values

2016-08-09 Thread Joey Frazee
Hey guys, I was recently testing a PR and was seeing inconsistent FileNotFound exceptions using a path specified in a processor property. It ended up being because one of the flows had a trailing space in the value and the other didn't. I'm curious about whether there's any thoughts on what the

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

2016-08-09 Thread Andre
Bryan, Thanks for the message. I've seen that link previously. Challenge is Exchange admins have the ability to force the conversion of email into standard mime and suspect my email server does that. In any case, all sorted, I used an unused outlook license I had at home together with mailtrap.i

Re: [nifi] Git Push Summary

2016-08-09 Thread Joe Percivall
Hey Joe, Thanks for catching this! Somehow I did mess up and tagged the 0.7.0-RC2 commit as rel/nifi-1.0.0-BETA. I tried to fix it by overwriting the tag, deleting the previous tag or pushing out an empty one but I don't think I have access to modify am existing rel tag. I just pushed out a "r

Re: [nifi] Git Push Summary

2016-08-09 Thread Joe Skora
Based on a clone and the GitHub UI, this tag appears to be based on 0.7.0-RC2? On Mon, Aug 8, 2016 at 5:33 PM, wrote: > Repository: nifi > Updated Tags: refs/tags/rel/nifi-1.0.0-BETA [created] 0db515d57 >

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

2016-08-09 Thread Joe Witt
Definitely best course of action is to use our own originally created test data. This can at times be very difficult but perhaps what Bryan just pointed out helps. Alternatively, we can of course include test artifacts in our source repository but we must simply account for them in license and no

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

2016-08-09 Thread Bryan Rosander
Hi Andre, I found a superuser answer that seems like it might be helpful in forcing Outlook to use TNEF. http://superuser.com/questions/613014/how-do-i-force-outlook-to-send-an-email-message-to-have-a-winmail-dat-attachment#answer-638244 Hope that helps. Thanks, Bryan On Tue, Aug 9, 2016 at 9:

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

2016-08-09 Thread Andre
Hi Joe, I am aware of it, reason I called it out openly so if someone can try to assist. In the past, I have used created content within the junit or crafted it within my lab, but in the case of TNEF I could not find a way of creating the files (POI does not have this ability and outlook seems co

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

2016-08-09 Thread Joe Witt
Andre We cannot copy source material even for testing unless we fully and properly account for licensing and notice concerns. Thanks Joe On Aug 9, 2016 8:25 AM, "Andre" wrote: > All, > > PR817[1] introduces an winmail.dat extractor. > > Following people's feedback, I created a separate process

Re: NIFI-1170 (ie TailDir )

2016-08-09 Thread Mark Payne
Andre, The state provide offers an arbitrary set of key/value pairs via the Map interface. I'd recommend going with something like "file.0.name" => "/data/myfile.txt", "file.0.timestamp" => "147327302730", etc. Is there something that I'm missing, so that this won't work? Thanks -Mark > On

ExtractTNEFAttachments: (was Re: ListenSMTP processor)

2016-08-09 Thread Andre
All, PR817[1] introduces an winmail.dat extractor. Following people's feedback, I created a separate processor to handle the TNEF attachments. This means the typical deployment will look like: (ListenSMTP || GetPOP3) --> ExtractEmailAttachments --> RouteOnAttribute [filename=winmail.dat] --> Ex

Re: NIFI-1170 (ie TailDir )

2016-08-09 Thread Andre
Mark, I am preparing to start working on NIFI-1170 again and I was wondering, is a second level of state space something that can be done? "TailDir" (similar to Flume's tail processor) should be capable of holding the state of multiple files, however, I presently need to serialised, store, read a