David Jacot created KAFKA-10795:
-----------------------------------
Summary: Automated protocol should expose Map keyed by
TopicPartition
Key: KAFKA-10795
URL: https://issues.apache.org/jira/browse/KAFKA-10795
Project: Kafka
Issue Type: Improvement
Reporter: David Jacot
Assignee: David Jacot
It is common in the AK protocol to encode topics and partitions. In these
cases, we usually have a list/map of topic objects which have a name attribute
and a partitions attribute. Then, the partitions attribute is a list/map of
partition objects which have at least a partition attribute.
{code:javascript}
{ "name": "Topics", "type": "[]OffsetForLeaderTopicResult", "versions": "0+",
"about": "Each topic we fetched offsets for.", "fields": [
{ "name": "Topic", "type": "string", "versions": "0+", "entityType":
"topicName",
"mapKey": true, "about": "The topic name." },
{ "name": "Partitions", "type": "[]EpochEndOffset", "versions": "0+",
"about": "Each partition in the topic we fetched offsets for.",
"fields": [
{ "name": "Partition", "type": "int32", "versions": "0+",
"about": "The partition index." },
...
]}
]}
{code}
The caveat is that most of the internals of AK works with Map<TopicPartition,
T> so we have to translate from the automated protocol data structures to the
the internal one, and vice-versa.
{code:java}
Map<TopicPartition, EpochEndOffset> endOffsets = new HashMap<>();
for (OffsetForLeaderTopicResult topic : response.data().topics()) {
for (EpochEndOffset partition : topic.partitions()) {
TopicPartition topicPartition = new TopicPartition(topic.topic(),
partition.partition());
endOffsets.put(topicPartition, partition);
}
}
{code}
Ideally, we would like to have the right data structures exposed by the
automated protocol.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
