I've now added a diagram to illustrate the states of a SourceTask. The KIP is
essentially trying to give a clear signal to SourceTask when all work has
stopped. In particular, if a SourceTask has a session to the source system that
it uses in poll() and commit(), it now has a safe way to release this.
Andrew Schofield
IBM Event Streams
On 21/01/2019, 10:13, "Andrew Schofield" <andrew_schofi...@live.com> wrote:
Ryanne,
Thanks for your comments. I think my overarching point is that the various
states of a SourceTask and the transitions between them seem a bit loose and
that makes it difficult to figure out when the resources held by a SourceTask
can be safely released. Your "I can't tell from the documentation" comment is
key here __ Neither could I.
The problem is that stop() is a signal to stop polling. It's basically a
request from the framework to the task and it doesn't tell the task that it's
actually finished. One of the purposes of the KC framework is to make life easy
for a connector developer and a nice clean "all done now" method would help.
I think I'll add a diagram to illustrate to the KIP.
Andrew Schofield
IBM Event Streams
On 18/01/2019, 19:02, "Ryanne Dolan" <ryannedo...@gmail.com> wrote:
Andrew, do we know whether the SourceTask may be start()ed again? If
this
is the last call to a SourceTask I suggest we call it close(). I can't
tell
from the documentation.
Also, do we need this if a SourceTask can keep track of whether it was
start()ed since the last stop()?
Ryanne
On Fri, Jan 18, 2019, 12:02 PM Andrew Schofield
<andrew_schofi...@live.com
wrote:
> Hi,
> I’ve created a new KIP to enhance the SourceTask interface in Kafka
> Connect.
>
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-419%3A%2BSafely%2Bnotify%2BKafka%2BConnect%2BSourceTask%2Bis%2Bstopped&data=02%7C01%7C%7Cfa73e281fe0348a2740b08d67f8924b8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636836624328119778&sdata=v6BU3q3W4Q2RIkdWtHCCn5uCSTF%2BMAnbj%2F%2B2%2Flladco%3D&reserved=0
>
> Comments welcome.
>
> Andrew Schofield
> IBM Event Streams
>
>
