Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/502
Hey @pvillard31 @aldrin - Just wanted to check if you have any
suggestions/recommendations for this pull request.
Thanks
---
If your project is set up for it, you can reply to this email and
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/502
Hi @pvillard31 @apiri - Just wanted to see if you have any
thoughts/recommendations for me.
Also, @pvillard31 were you able to run the integration tests and validate
that the entries were
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/502
Hey @pvillard31 - Thanks for your comments and I will try to address to
your questions below:
1. In order to run integration tests, we need to run apache ignite 1.6
locally (or create the
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/502
@pvillard31 - I've added provenance and corrected the merge and conflict
issue. Please let me know your thoughts.
---
If your project is set up for it, you can reply to this email and have
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/502
@pvillard31 - I messed up the merge may have to recreate the branch. Let
me fix that and then you can review it.
Thanks
---
If your project is set up for it, you can reply to this email and
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/502
Hi @pvillard31 - I've made the changes that you've recommended except for
the provenance since I could not find any method to support a collection of
flow files. Please let me know
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66547015
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/resources/ignite-client.xml
---
@@ -0,0 +1,26 @@
+
+
+http
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66546632
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/AbstractIgniteProcessor.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66546466
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/cache/PutIgniteCache.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66546338
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/cache/PutIgniteCache.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66546103
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/cache/AbstractIgniteCacheProcessor.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66545282
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/cache/PutIgniteCache.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66545243
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/cache/PutIgniteCache.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66545042
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/cache/PutIgniteCache.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66544841
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/cache/PutIgniteCache.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/502#discussion_r66531971
--- Diff:
nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/main/java/org/apache/nifi/processors/ignite/AbstractIgniteProcessor.java
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/502
Hey Folks:
Please let me know your thoughts/suggestions on this Nifi Ignite Put
Processor.
Thanks
---
If your project is set up for it, you can reply to this email and have your
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
@jvwing -
I've added displayName to PropertyDescriptor and updated name based on your
feedback.
I've not tried nifi on aws and mostly run it locally on my laptop that
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
@jvwing -
Thanks again for your thorough review.
Just for my own learning, if there any pointers in the errors that
indicated it was a memory issue, please let me know. Also, is there
GitHub user mans2singh opened a pull request:
https://github.com/apache/nifi/pull/502
Nifi-1972 Apache Ignite Put Cache Processor
Nifi Put Ignite cache processor which streams data into Ignite Cache.
You can merge this pull request into a Git repository by running:
$ git pull
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
Hey @jvwing
I've made the property name changes you recommended and corrected
maxBufferedTimeInterval - it was indeed millis and values were different from
what I had. I've
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
I also recommend some modifications to the property definitions in both
AbstractKinesisConsumer and AbstractKinesisProducer:
* Use `name` as a key and `displayName` for the UI text, rather
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
I have shared some [recommended edits to the property labels and doc
text](https://github.com/jvwing/nifi/commit/e431a9665ce5c2cbbabf2605cc088108ed74e7c5),
please take a look.
---
If your project
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
@jvwing -
I've updated the code to catch and log the exception and continue
processing kinesis records as you had recommended.
I can remove REL_FAILURE since it is not
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
Ok - I will add more logging, let me know if there is any other observation
you have.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
Hi @jvwing:
Regarding exception handling - Kinesis will automatically handle retrying
if there is kinesis client error from the last checkpoint created in the
GetKinesisStream
Github user mans2singh commented on the issue:
https://github.com/apache/nifi/pull/239
Hi @jvwing
Just wanted to check if you were able to test the processors and if you
have any additional review comments/recommendations for me.
Thanks
Mans
---
If
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239
Hey Folks: For this pull request - Travis-ci build is passing for me but
appveyor is failing in the nifi-framework-core module with the following error.
Please let me know if there is
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239
Hi @jvwing - I made the changes you had recommended (added unit tests for
get and put kinesis, renamed the get and put processors, added comments about
usage of DynamoDB and Cloud Watch
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239#issuecomment-222191326
Hi @jvwing
Thanks again for your comments:
I have am currently not using REL_FAILURE in GetKinesis as you've
mentioned, and wanted your advice o
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239#issuecomment-222020291
Hi @jvwing -
Thanks for your feedback.
Regarding changing the hierarchy to create a `AmazonWebServiceClient`
independent base class - I can create
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239#issuecomment-221567920
Hey @jvwing @olegz @apiri - Just wondering if you've had the time to review
this request. If there are any feedback comments/recommendations, please let
me
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239#issuecomment-220644414
Hey @olegz @apiri @jvwing @joewitt - I've tried to address your review
comments above. Please let me know if you have any feedback for me. Thanks
Mans
---
If
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/239#discussion_r63047926
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/consumer/GetKinesis.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/239#discussion_r63047805
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractBaseAWSProcessor.java
---
@@ -0,0
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/239#discussion_r63047632
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractBaseAWSProcessor.java
---
@@ -0,0
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239#issuecomment-218325113
Hi Nifi Folks, @apiri, @olegz, @joewitt :
Thanks for your review comments. I apologize I had to take care of work
related assignments and could not get back
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/239#issuecomment-214554805
@joewitt - I got busy with assignments at work and apologize for not
getting back to respond to the comments. Will it be possible to push this pull
request for a
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-208079556
Thanks @aldrin @jvwing @olegz @bbende for your advice/guidance in getting
this feature implemented.
---
If your project is set up for it, you can reply to this email
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-207081123
@apiri - Hi Aldrin - Do you have any recommendations for this pull request
? Thanks
---
If your project is set up for it, you can reply to this email and have your
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-203188914
@apiri @jvwing - Please let me know if you have any further
recommendations. Thanks
---
If your project is set up for it, you can reply to this email and have your
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r57593798
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/PutDynamoDB.java
---
@@ -0,0 +1,182
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r56741830
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/AbstractDynamoDBProcessor.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r56742665
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/PutDynamoDB.java
---
@@ -0,0 +1,182
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r56742670
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/PutDynamoDB.java
---
@@ -0,0 +1,182
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r56741885
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/PutDynamoDB.java
---
@@ -0,0 +1,182
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197183816
@aldrin @mattyb149 Thanks for your pointers and samples for using data
validator. I've made the changes and updated test cases. Please let me know
if the
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197122905
@aldrin - I've moved the check for buffer size and rebased as you had
recommended. I could not find the data size validator that you have mentioned
though. Pleas
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-196646278
@aldrin - I've added max buffer size limit check while getting flow files
from the session as you had recommended. I've also kept the batch size limit
in p
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-195833198
@apiri - I've rebased the branch to the current master and have resolve the
conflicts. However, on rebasing there were many updates from the master so I
coul
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-195415550
I've addressed the minor comments (log level + default value) and will work
on rebasing.
---
If your project is set up for it, you can reply to this email and
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195409880
@apiri - To make things simple we get rid of the batch size and we just
get one flow file at a time on each invocation of the onTrigger. We store the
flow files in
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195300721
@apiri - Just wanted to clarify - We will grab a batch of flow files and
check that the flow files combined size threshold is reached. Then we send the
batch to the
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195172976
@apiri Your recommendation for using buffer size is great. So, should we
add another property - total batch bytes, and then send them in batches of that
size ? If you
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/216#issuecomment-194116446
@markap14 - I've updated the code based on your comments. Please let me
know if you have any other recommendation. Thanks
---
If your project is set up for it
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-194109133
@jvwing - I've add check for empty json response and it will continue to
process without an error. I've also incorporated your comments about
description i
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-193951585
@jvwing - I've added more unit test cases for different types of
exceptions. Let me know what you think.
---
If your project is set up for it, you can reply to
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-193590384
@jvwing - I missed the case where the attribute would be null - thanks for
catching that. I've corrected the issue and included your test case among unit
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-193001802
I've done a clean build locally and it passes and also tested the case
which is failing in travis-ci and it passes. Also, travis is showing different
error for
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-192992232
@jvwing - Thanks or your feedback.
Based on your recommendations - I've added separate relations for
unprocessed and not found (latter only for get d
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/216#issuecomment-190237652
@aldrin @jvwing - Please let me know if there is anything else required for
this processor. Thanks
---
If your project is set up for it, you can reply to this email
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-190236625
@olegz @jvwing - Please let me know if you have any other recommendations
for me. Thanks
---
If your project is set up for it, you can reply to this email and have
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r54336504
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/ItemKeys.java
---
@@ -0,0 +1,52
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r54336495
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/DeleteDynamoDB.java
---
@@ -0,0 +1,152
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r54336499
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/GetDynamoDB.java
---
@@ -0,0 +1,180
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r54336478
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/DeleteDynamoDB.java
---
@@ -0,0 +1,152
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r54336475
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/AbstractDynamoDBProcessor.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/224#discussion_r54336468
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/dynamodb/AbstractDynamoDBProcessor.java
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/224#issuecomment-189404424
Hi James:
The unit tests are failing because the credentails file is not found in the
home folder. Â
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-188062568
Corrected title of pull request. Please let me know if there is anything
else required.
---
If your project is set up for it, you can reply to this email and have
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r53893312
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/kinesis/firehose/ITPutKinesisFirehose.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r53893291
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/kinesis/firehose/ITPutKinesisFirehose.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r53893255
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/PutKinesisFirehose.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r53893277
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/PutKinesisFirehose.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r53893236
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/PutKinesisFirehose.java
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r53893209
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r53893190
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-186941406
@apiri - Do you have any other comments/suggestions for me ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
GitHub user mans2singh opened a pull request:
https://github.com/apache/nifi/pull/239
Nifi 1540 - AWS Kinesis Get and Put Processors
Hi: I have implemented kinesis consumer and producer. I've created a new
base class for aws processors which do not depend on aws client dir
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-185913781
Just wanted to check if there is any other feedback/recommendation on
Kinesis Firehose put processor.
---
If your project is set up for it, you can reply to this
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-184885151
@apiri I've rebased the forked branch. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as wel
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/216#discussion_r53082519
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/lambda/ITPutLambdaTest.java
---
@@ -0,0 +1,146
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-184858822
BTW - I had a combined request for kinesis + proxy - if you would like to
review - https://github.com/apache/nifi/pull/213
---
If your project is set up for it, you
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/209#discussion_r53062954
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
---
@@ -91,6 +93,23
GitHub user mans2singh opened a pull request:
https://github.com/apache/nifi/pull/224
Nifi 1516 - AWS DynamoDB Get/Put/Delete Processors
The current implementation supports string and numbers for hash and range
keys. For put operations, a json document can be inserted to a
GitHub user mans2singh opened a pull request:
https://github.com/apache/nifi/pull/216
Nifi-1509 - Support aws lambda put processor
Support AWS Lambda processor
(https://issues.apache.org/jira/browse/NIFI-1509)
You can merge this pull request into a Git repository by running
GitHub user mans2singh opened a pull request:
https://github.com/apache/nifi/pull/213
Nifi 1489 (Support for http proxy) + Nifi 1495 (AWS Kinesis Firehose)
This pull request combines Http proxy enhancement for aws processors
(nifi-1489) and aws kinesis firehose processor (nifi-1495
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-182092294
Explicitly check that proxy host and port are both specified based on
comments (rather than using xor operator)
---
If your project is set up for it, you can reply to
GitHub user mans2singh opened a pull request:
https://github.com/apache/nifi/pull/209
Added support for using proxy for aws processors
Added http proxy support for aws processors
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
89 matches
Mail list logo
