[
https://issues.apache.org/jira/browse/KAFKA-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Crain updated KAFKA-6373:
---------------------------------
Description:
I am receiving a confusing error from a Kafka Streams application. Most of the
time when I try to bring up just a single replica of the task for the first
time, I get this:
{noformat}
Detected a task that got migrated to another thread. This implies that this
thread missed a rebalance and dropped out of the consumer group. Trying to
rejoin the consumer group now.
org.apache.kafka.streams.errors.TaskMigratedException: Log end offset of [Name
of Topic]-36 should not change while restoring: old end offset 37559, current
offset 37561
{noformat}
The confusing thing is that [Name of Topic] is *not* a change log topic created
by the stream app. Rather it is a topic published from a completely different
service. And since that other service is publishing to that topic actively, of
course the end offset is constantly changing.
Here is a rough view of my stream topology. I'll call the topic that's showing
up in the above error "ExternalTableTopic".
{noformat}
externalTable = table(ExternalTableTopic)
stream(ExternalStreamTopic)
.leftJoin(externalTable, joiner)
.aggregate(aggregator, SomeQueryableStoreName)
{noformat}
...and that's it. If I take out the left join this appears not to happen. Is
it illegal to join a table to a stream if that table is being published from
somewhere else?
was:
I am receiving a confusing error from a Kafka Streams application. Most of the
time when I try to being up just a single replica of the task for the first
time, I get this:
{noformat}
Detected a task that got migrated to another thread. This implies that this
thread missed a rebalance and dropped out of the consumer group. Trying to
rejoin the consumer group now.
org.apache.kafka.streams.errors.TaskMigratedException: Log end offset of [Name
of Topic]-36 should not change while restoring: old end offset 37559, current
offset 37561
{noformat}
The confusing thing is that [Name of Topic] is *not* a change log topic created
by the stream app. Rather it is a topic published from a completely different
service. And since that other service is publishing to that topic actively, of
course the end offset is constantly changing.
Here is a rough view of my stream topology. I'll call the topic that's showing
up in the above error "ExternalTableTopic".
{noformat}
externalTable = table(ExternalTableTopic)
stream(ExternalStreamTopic)
.leftJoin(externalTable, joiner)
.aggregate(aggregator, SomeQueryableStoreName)
{noformat}
...and that's it. If I take out the left join this appears not to happen. Is
it illegal to join a table to a stream if that table is being published from
somewhere else?
> Log end offset of input table changing during restore
> -----------------------------------------------------
>
> Key: KAFKA-6373
> URL: https://issues.apache.org/jira/browse/KAFKA-6373
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 1.0.0
> Environment: Client 1.0.0, Brokers 1.0.0 with 1.0.0 message format
> and inter-broker protocol
> Reporter: Charles Crain
>
> I am receiving a confusing error from a Kafka Streams application. Most of
> the time when I try to bring up just a single replica of the task for the
> first time, I get this:
> {noformat}
> Detected a task that got migrated to another thread. This implies that this
> thread missed a rebalance and dropped out of the consumer group. Trying to
> rejoin the consumer group now.
> org.apache.kafka.streams.errors.TaskMigratedException: Log end offset of
> [Name of Topic]-36 should not change while restoring: old end offset 37559,
> current offset 37561
> {noformat}
> The confusing thing is that [Name of Topic] is *not* a change log topic
> created by the stream app. Rather it is a topic published from a completely
> different service. And since that other service is publishing to that topic
> actively, of course the end offset is constantly changing.
> Here is a rough view of my stream topology. I'll call the topic that's
> showing up in the above error "ExternalTableTopic".
> {noformat}
> externalTable = table(ExternalTableTopic)
> stream(ExternalStreamTopic)
> .leftJoin(externalTable, joiner)
> .aggregate(aggregator, SomeQueryableStoreName)
> {noformat}
> ...and that's it. If I take out the left join this appears not to happen.
> Is it illegal to join a table to a stream if that table is being published
> from somewhere else?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
