[
https://issues.apache.org/jira/browse/NIFI-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083644#comment-16083644
]
Koji Kawamura commented on NIFI-4174:
-------------------------------------
There are two things, one is connection timeout setting, and the other is the
error message.
In your case, OracleDriver kept waiting to make a connection and NiFi framework
decided to time it out. In this case I don't think we can produce better error
message. By default, connection timeout is not set and a driver keeps waiting.
In order to set connection timeout, we need to set different JDBC properties
for different drivers:
For [Oracle Thin
driver|http://docs.oracle.com/cd/E18283_01/appdev.112/e13995/constant-values.html#oracle_jdbc_OracleConnection_CONNECTION_PROPERTY_THIN_READ_TIMEOUT],
use "oracle.net.CONNECT_TIMEOUT",
[MySQL|https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html]
uses "connectTimeout",
[PostgreSQL|https://jdbc.postgresql.org/documentation/91/connect.html] seems
using "loginTimeout".
By setting these JDBC property by 'User Defined property' (Dynamic processor
property), a driver can throw an Exception when it's timeout.
Even though, since NiFi framework uses async call and the root cause is deep in
a exception that NiFi catches, we need to change NiFi framework code to utilize
the meaningful exception message to show as a bulletin message.
By setting Oracle timeout, I got following stacktrace:
{code}
2017-07-12 17:06:09,483 ERROR [StandardProcessScheduler Thread-1]
o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due
to java.lang.RuntimeException: Failed while executing one
of processor's OnScheduled task.
java.lang.RuntimeException: Failed while executing one of processor's
OnScheduled task.
at
org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1482)
at
org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
at
org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException:
java.lang.reflect.InvocationTargetException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at
org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
... 9 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
at
org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1307)
at
org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1303)
... 6 common frames omitted
Caused by: org.apache.nifi.processor.exception.ProcessException:
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (IO Error: The Network Adapter could not establish
the connection)
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:275)
at sun.reflect.GeneratedMethodAccessor434.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89)
at com.sun.proxy.$Proxy145.getConnection(Unknown Source)
at
org.apache.nifi.processors.standard.AbstractDatabaseFetchProcessor.setup(AbstractDatabaseFetchProcessor.java:210)
at
org.apache.nifi.processors.standard.GenerateTableFetch.setup(GenerateTableFetch.java:155)
... 16 common frames omitted
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (IO Error: The Network Adapter could not establish
the connection)
at
org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
at
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:272)
... 23 common frames omitted
Caused by: java.sql.SQLRecoverableException: IO Error: The Network Adapter
could not establish the connection
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:673)
at
oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:715)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:385)
at
oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:30)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:564)
at org.apache.nifi.dbcp.DriverShim.connect(DriverShim.java:46)
at
org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at
org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at
org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
... 26 common frames omitted
{code}
> GenerateTableFetch does not work with oracle on Nifi 1.2
> --------------------------------------------------------
>
> Key: NIFI-4174
> URL: https://issues.apache.org/jira/browse/NIFI-4174
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.2.0
> Reporter: Jorge Machado
> Priority: Minor
>
> I'm trying to extract some data from a oracle DB.
> I'm getting :
> {code:java}
> 2017-07-11 16:19:29,612 WARN [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Timed out while waiting for
> OnScheduled of 'GenerateTableFetch' processor to finish. An attempt is made
> to cancel the task via Thread.interrupt(). However it does not guarantee that
> the task will be canceled since the code inside current OnScheduled operation
> may have been written to ignore interrupts which may result in a runaway
> thread. This could lead to more issues, eventually requiring NiFi to be
> restarted. This is usually a bug in the target Processor
> 'GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]' that needs to
> be documented, reported and eventually fixed.
> 2017-07-11 16:19:29,612 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.p.standard.GenerateTableFetch
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4] failed to invoke
> @OnScheduled method due to java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.; processor will not be
> scheduled to run for 30 seconds: java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
> ... 9 common frames omitted
> 2017-07-11 16:19:29,613 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method
> due to java.lang.RuntimeException: Timed out while executing one of
> processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
> ... 9 common frames omitted
> {code}
> Database Connection Pooling Service:
> jdbc:oracle:thin:@somehost:6779:someSID
> oracle.jdbc.OracleDriver
> /pathTo/ojdbc7.jar
> max wait time 500mil
> On the processor I have:
> Max Wait Time 10 seconds
> Partition Size 100
> i tryed schedule 0s and 10 000s result is the same
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
