fvaleri commented on pull request #11748:
URL: https://github.com/apache/kafka/pull/11748#issuecomment-1061881629
@mimaison looks like it works, but topic's partitions without data are not
replicated like before. Not sure it is intended behavior.
### Reproducer procedure
```shell
SOURCE="localhost:9090"
TARGET="localhost:9093"
export KAFKA_HOME="$HOME/.kafka/home"
$KAFKA_HOME/bin/kafka-topics.sh --bootstrap-server $SOURCE --create --topic
my-topic --config 'retention.ms=10000'
$KAFKA_HOME/bin/kafka-console-consumer.sh --bootstrap-server $SOURCE --group
my-group --topic my-topic
for i in $(seq 1 100); do echo $i; done |
$KAFKA_HOME/bin/kafka-console-producer.sh --bootstrap-server $SOURCE --topic
my-topic
# wait until the retention policy deletes the records and then start mm2
$KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server $TARGET
--describe --group my-group
```
### Test on 3.1.0 release
```shell
$ $KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server $SOURCE
--describe --group my-group
Consumer group 'my-group' has no active members.
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST CLIENT-ID
my-group my-topic 0 100 100 0
- - -
my-group my-topic 1 0 0 0
- - -
my-group my-topic 2 0 0 0
- - -
$ $KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server $TARGET
--describe --group my-group
Consumer group 'my-group' has no active members.
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST CLIENT-ID
my-group my-topic 0 100 0
-100 - - -
my-group my-topic 1 0 0 0
- - -
my-group my-topic 2 0 0 0
- - -
# mirrorMaker.out
[2022-03-08 15:14:49,985] TRACE latestDownstreamOffset 100 is larger than or
equal to convertedUpstreamOffset 100 for TopicPartition my-topic-0
(org.apache.kafka.connect.mirror.MirrorCheckpointTask)
[2022-03-08 15:14:49,985] TRACE latestDownstreamOffset 0 is larger than or
equal to convertedUpstreamOffset 0 for TopicPartition my-topic-1
(org.apache.kafka.connect.mirror.MirrorCheckpointTask)
[2022-03-08 15:14:49,985] TRACE latestDownstreamOffset 0 is larger than or
equal to convertedUpstreamOffset 0 for TopicPartition my-topic-2
(org.apache.kafka.connect.mirror.MirrorCheckpointTask)
```
### Test on branch mimaison:KAFKA-12635
```shell
$ $KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server $SOURCE
--describe --group my-group
Consumer group 'my-group' has no active members.
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST CLIENT-ID
my-group my-topic 0 0 0 0
- - -
my-group my-topic 1 100 100 0
- - -
my-group my-topic 2 0 0 0
- - -
$ $KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server $TARGET
--describe --group my-group
Consumer group 'my-group' has no active members.
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST CLIENT-ID
my-group my-topic 1 100 100 0
- - -
# mirrorMaker.out
[2022-03-08 15:56:50,978] TRACE latestDownstreamOffset 100 is larger than or
equal to convertedUpstreamOffset 100 for TopicPartition my-topic-1
(org.apache.kafka.connect.mirror.MirrorCheckpointTask)
```
If I now send additional 100 messages, this is what I have on the target
cluster:
```shell
$ $KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server $TARGET
--describe --group my-group
Consumer group 'my-group' has no active members.
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST CLIENT-ID
my-group my-topic 0 100 100 0
- - -
my-group my-topic 1 100 100 0
- - -
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
