[ 
https://issues.apache.org/jira/browse/KAFKA-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita updated KAFKA-15252:
---------------------------
    Description: 
We face a problem with restarting of the tasks, sometimes it leads to resource 
leak. 

We used the jdbc source connector and noticed an increasing of count of opened 
sessions on Vertica side. But this problem is applicable for all databases and 
possibly for all source connectors.

Our case is the next: 

1) Run jdbc source connector (io.confluent.connect.jdbc.JdbcSourceConnector) 
and set poll.interval.ms (86400000) > task.shutdown.graceful.timeout.ms (it's 
the property on Kafka-connect side, we set 10000)

2) Send POST /connectors/<connector_name>/tasks/<task_number>/restart

ER: count of session is the same as before restart

AR: count of session increases

The main problem is when 
org.apache.kafka.connect.runtime.Worker#stopAndAwaitTasks(java.util.Collection<org.apache.kafka.connect.util.ConnectorTaskId>)
  method is called it doesn't stop a source task itself. 
The source task stops only if polling process stops on source task side. 

  was:
We face a problem with the restarting of the tasks, sometimes it leads to 
resource leak. 

We used the jdbc source connector and noticed an increasing of count of opened 
sessions on Vertica side. But this problem is applicable for all databases and 
possibly for all source connectors.

Our case is the next: 

1) Run jdbc source connector (io.confluent.connect.jdbc.JdbcSourceConnector) 
and set poll.interval.ms (86400000) > task.shutdown.graceful.timeout.ms (it's 
the property on Kafka-connect side, we set 10000)

2) Send POST /connectors/<connector_name>/tasks/<task_number>/restart

ER: count of session is the same as before restart

AR: count of session increases

The main problem is when 
org.apache.kafka.connect.runtime.Worker#stopAndAwaitTasks(java.util.Collection<org.apache.kafka.connect.util.ConnectorTaskId>)
  method is called it doesn't stop a source task itself. 
The source task stops only if polling process stops on source task side. 


> Task is not stopped until the poll interval passes in case of task restarting.
> ------------------------------------------------------------------------------
>
>                 Key: KAFKA-15252
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15252
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Nikita
>            Priority: Major
>
> We face a problem with restarting of the tasks, sometimes it leads to 
> resource leak. 
> We used the jdbc source connector and noticed an increasing of count of 
> opened sessions on Vertica side. But this problem is applicable for all 
> databases and possibly for all source connectors.
> Our case is the next: 
> 1) Run jdbc source connector (io.confluent.connect.jdbc.JdbcSourceConnector) 
> and set poll.interval.ms (86400000) > task.shutdown.graceful.timeout.ms (it's 
> the property on Kafka-connect side, we set 10000)
> 2) Send POST /connectors/<connector_name>/tasks/<task_number>/restart
> ER: count of session is the same as before restart
> AR: count of session increases
> The main problem is when 
> org.apache.kafka.connect.runtime.Worker#stopAndAwaitTasks(java.util.Collection<org.apache.kafka.connect.util.ConnectorTaskId>)
>   method is called it doesn't stop a source task itself. 
> The source task stops only if polling process stops on source task side. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to