[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552065#comment-15552065
]
Mykola Polonskyi commented on KAFKA-3544:
-----------------------------------------
[~guozhang] greetings!
My comment a bit late but I catch error with similar stack-trace (like Greg),
with using of KGroupedTable#aggregation and the intermediate topic.
kafka streams 0.10.0.1
{code:kotlin}
private val streamId = "STREAM_USER_WITH_SKICARD"
connectionProperties.put(StreamsConfig.APPLICATION_ID_CONFIG, streamId)
val skicardByOwnerIdTopicName = "skicards_by_owner_id"
userTable.join(
skicardsTable.groupBy {
key, value -> KeyValue<String,
SkicardInfo>(value.skicardInfo.ownerId, value.skicardInfo)
}.aggregate(
{ mutableSetOf<SkicardInfo>() }, // accumulator definition
{ ownerIdString, skicardInfoObject, accumulator ->
accumulator.put(skicardInfo) },
{ ownerIdString, skicardInfoObject, accumulator ->
accumulator }, //XXX fix this one
skicardByOwnerIdSerde,
skicardByOwnerIdTopicName
),
{ userCreatedOrUpdated, skicardInfoSet ->
UserWithSkicardsCreatedOrUpdated(userCreatedOrUpdated.user, skicardInfoSet) }
).to(
Serdes.StringSerde(),
userWithSkicardsSerde,
USER_WITH_SKICARDS_TOPIC.name
)
{code}
And inside kafka got appeared topic
"STREAM_USER_WITH_SKICARD-skicards_by_owner_id-repartition"
And then incide the debug I see that code try to retrieve meta-info
(StreamPartitionAssignor.java:446) for topic skicards_by_owner_id-repartition
instead "STREAM_USER_WITH_SKICARD-skicards_by_owner_id-repartition".
Is that bug that you spoke about?
p.s. I`m not sure should I report it as new ticket that`s why provide the
additional comment here.
> Missing topics on startup
> -------------------------
>
> Key: KAFKA-3544
> URL: https://issues.apache.org/jira/browse/KAFKA-3544
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.0.0
> Reporter: Greg Fodor
> Assignee: Guozhang Wang
> Labels: semantics
>
> When running a relatively complex job with multiple tasks and state stores,
> on the first run I get errors due to some of the intermediate topics not
> existing. Subsequent runs work OK. My assumption is streams may be creating
> topics lazily, so if downstream tasks are initializing before their parents
> have had a chance to create their necessary topics then the children will
> attempt to start consuming from topics that do not exist yet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
