[
https://issues.apache.org/jira/browse/NIFI-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcio Sugar updated NIFI-4924:
-------------------------------
Description:
I'm trying to copy data from one table on DB2 database "A" to the same table on
another DB2 database "B". Schemas are identical.
My flow is simply a QueryDatabaseTable reading from "A" and connected to a
PutDatabaseRecord writing to "B". The latter uses and AvroReader controller.
Each processor uses its own DBCPConnectionPool since "A" and "B" are on
different instances.
When I set the AvroReader to use the 'Schema Text' property, I get errors like
this:
{code:java}
PutDatabaseRecord[id=e3905091-0161-1000-028c-982c192bf16f]
org.apache.nifi.processors.standard.PutDatabaseRecord$$Lambda$438/1366813796@deb42
failed to process due to org.apache.nifi.processor.exception.ProcessException:
Failed to process
StandardFlowFileRecord[uuid=58ce220f-2b43-4875-bdb4-c704b093f9d7,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1519955790257-2, container=default,
section=2], offset=175623,
length=174960],offset=0,name=255478043043373,size=174960] due to
java.lang.ArrayIndexOutOfBoundsException: -40; rolling back session: Failed to
process
StandardFlowFileRecord[uuid=58ce220f-2b43-4875-bdb4-c704b093f9d7,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1519955790257-2, container=default,
section=2], offset=175623,
length=174960],offset=0,name=255478043043373,size=174960] due to
java.lang.ArrayIndexOutOfBoundsException: -40
{code}
Debugging the code I found what I believe to be a typo. In
PutDatabaseRecord.java, line 1031, the program is calling the getPrimaryKeys
method but passing a null as the 2nd. parameter:
{code:java}
try (final ResultSet pkrs = dmd.getPrimaryKeys(catalog, null, tableName))
{{code}
So in fact it's getting the primary keys for all the tables with the same name
found across all the schemas, which is wrong.
The proper call, I think, should set the schema name:
{code:java}
try (final ResultSet pkrs = dmd.getPrimaryKeys(catalog, schema, tableName))
{{code}
This is a subtle bug that can go unnoticed if the database doesn't have tables
with the same name in different schemas.
was:
I'm trying to copy data from one table on DB2 database "A" to the same table on
another DB2 database "B". Schemas are identical.
My flow is simply a QueryDatabaseTable reading from "A" and connected to a
PutDatabaseRecord writing to "B". The latter uses and AvroReader controller.
Each processor uses its own DBCPConnectionPool since "A" and "B" are on
different instances.
When I set the AvroReader to use the 'Schema Text' property, I get errors like
this:
{code:java}
PutDatabaseRecord[id=e3905091-0161-1000-028c-982c192bf16f]
org.apache.nifi.processors.standard.PutDatabaseRecord$$Lambda$438/1366813796@deb42
failed to process due to org.apache.nifi.processor.exception.ProcessException:
Failed to process
StandardFlowFileRecord[uuid=58ce220f-2b43-4875-bdb4-c704b093f9d7,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1519955790257-2, container=default,
section=2], offset=175623,
length=174960],offset=0,name=255478043043373,size=174960] due to
java.lang.ArrayIndexOutOfBoundsException: -40; rolling back session: Failed to
process
StandardFlowFileRecord[uuid=58ce220f-2b43-4875-bdb4-c704b093f9d7,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1519955790257-2, container=default,
section=2], offset=175623,
length=174960],offset=0,name=255478043043373,size=174960] due to
java.lang.ArrayIndexOutOfBoundsException: -40
{code}
Debugging the code I found what I believe to be a typo. In
PutDatabaseRecord.java, line 1031, the program is calling the getPrimaryKeys
method but passing a null as the 2nd. parameter:
{code:java}
try (final ResultSet pkrs = dmd.getPrimaryKeys(catalog, null, tableName))
{{code}
So in fact it's getting the primary keys for all the tables with the same name
found across all the schemas, which is wrong.
The proper call, I think, should set the schema name:
{code:java}
try (final ResultSet pkrs = dmd.getPrimaryKeys(catalog, schema, tableName))
{{code}
This is a subtle bug that can go unnoticed if the database doesn't have tables
with the same name in different schemas.
> PutDatabaseRecord throws ArrayIndexOutOfBoundsException where there is more
> than one table with the same name on the database (in different schemas)
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4924
> URL: https://issues.apache.org/jira/browse/NIFI-4924
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.5.0
> Environment: ubuntu 16.04
> nifi 1.5.0
> db2 v10.5.0.7
> Reporter: Marcio Sugar
> Priority: Major
>
> I'm trying to copy data from one table on DB2 database "A" to the same table
> on another DB2 database "B". Schemas are identical.
> My flow is simply a QueryDatabaseTable reading from "A" and connected to a
> PutDatabaseRecord writing to "B". The latter uses and AvroReader controller.
> Each processor uses its own DBCPConnectionPool since "A" and "B" are on
> different instances.
> When I set the AvroReader to use the 'Schema Text' property, I get errors
> like this:
> {code:java}
> PutDatabaseRecord[id=e3905091-0161-1000-028c-982c192bf16f]
> org.apache.nifi.processors.standard.PutDatabaseRecord$$Lambda$438/1366813796@deb42
> failed to process due to
> org.apache.nifi.processor.exception.ProcessException: Failed to process
> StandardFlowFileRecord[uuid=58ce220f-2b43-4875-bdb4-c704b093f9d7,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1519955790257-2, container=default,
> section=2], offset=175623,
> length=174960],offset=0,name=255478043043373,size=174960] due to
> java.lang.ArrayIndexOutOfBoundsException: -40; rolling back session: Failed
> to process
> StandardFlowFileRecord[uuid=58ce220f-2b43-4875-bdb4-c704b093f9d7,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1519955790257-2, container=default,
> section=2], offset=175623,
> length=174960],offset=0,name=255478043043373,size=174960] due to
> java.lang.ArrayIndexOutOfBoundsException: -40
> {code}
>
> Debugging the code I found what I believe to be a typo. In
> PutDatabaseRecord.java, line 1031, the program is calling the getPrimaryKeys
> method but passing a null as the 2nd. parameter:
> {code:java}
> try (final ResultSet pkrs = dmd.getPrimaryKeys(catalog, null, tableName))
> {{code}
> So in fact it's getting the primary keys for all the tables with the same
> name found across all the schemas, which is wrong.
>
> The proper call, I think, should set the schema name:
> {code:java}
> try (final ResultSet pkrs = dmd.getPrimaryKeys(catalog, schema, tableName))
> {{code}
>
> This is a subtle bug that can go unnoticed if the database doesn't have
> tables with the same name in different schemas.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
