[
https://issues.apache.org/jira/browse/NIFI-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430286#comment-16430286
]
ASF GitHub Bot commented on NIFI-1706:
--------------------------------------
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/2162
@patricker Thanks for the update, and sorry for my delayed response. I just
got the chance to review this today.
This has a conflict with #2504 . I can fix compilation level conflict by
removing AbstractDatabaseProcessor.onPropertyModified which has been deleted by
#2504.
However, It seems that we need to update other places, too. I encountered
few unit test failures at
TestGenerateTableFetch:
```
[ERROR] TestGenerateTableFetch.testAddedRows:202 expected:<4> but was:<0>
[ERROR] TestGenerateTableFetch.testAddedRowsRightBounded:402 expected:<4>
but was:<0>
[ERROR] TestGenerateTableFetch.testAddedRowsTwoTables:276 expected:<1>
but was:<0>
[ERROR] TestGenerateTableFetch.testAddedRowsWithCustomWhereClause:1112
expected:<5> but was:<0>
[ERROR] TestGenerateTableFetch.testMultipleColumnTypeMissing:1250
expected:<1> but was:<0>
```
I'll continue looking at these test failures. It'd be appreciated if you
can rebase the PR with the latest master and see how we can fix those, too.
Thanks!
> Extend QueryDatabaseTable to support arbitrary queries
> ------------------------------------------------------
>
> Key: NIFI-1706
> URL: https://issues.apache.org/jira/browse/NIFI-1706
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.4.0
> Reporter: Paul Bormans
> Assignee: Peter Wicks
> Priority: Major
> Labels: features
>
> The QueryDatabaseTable is able to observe a configured database table for new
> rows and yield these into the flowfile. The model of an rdbms however is
> often (if not always) normalized so you would need to join various tables in
> order to "flatten" the data into useful events for a processing pipeline as
> can be build with nifi or various tools within the hadoop ecosystem.
> The request is to extend the processor to specify an arbitrary sql query
> instead of specifying the table name + columns.
> In addition (this may be another issue?) it is desired to limit the number of
> rows returned per run. Not just because of bandwidth issue's from the nifi
> pipeline onwards but mainly because huge databases may not be able to return
> so many records within a reasonable time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
