2020-02-07 09:41:06 UTC - sky-big: @sky-big has joined the channel
----
2020-02-07 13:03:00 UTC - Shahar Fermon: @Shahar Fermon has joined the channel
----
2020-02-07 13:34:34 UTC - Guilherme Perinazzo: I'm also having issues
connecting to pulsar websocket on standalone
----
2020-02-07 14:22:37 UTC - markg: thank you all for the responses
----
2020-02-07 16:15:54 UTC - Sijie Guo: Yes there was an issue regarding websocket
running in stand-alone using 2.5. There was a fix. We will include that in
2.5.1 release
----
2020-02-07 17:40:20 UTC - Addison Higham: I rebased all my open PRs,
:crossed_fingers: I make the build gods happy :slightly_smiling_face:
man-bowing : Matteo Merli
----
2020-02-07 20:49:04 UTC - Devin G. Bost: Are all the bookie addresses in ZK
persistent instead of ephemeral?
----
2020-02-07 21:17:43 UTC - Devin G. Bost: I wonder if keeping persistent records
could be problematic in some cases because my team has seen some issues in
production where ZK ends up hanging onto old or dirty data (usually after a
problem that requires intervention), and we end up needing to manually remove
entries from ZK to try to get things back to a working state.
----
2020-02-07 21:29:41 UTC - Addison Higham: what are you seeing exactly? Brokers
are trying to talk to bookies that no longer exist?
----
2020-02-07 23:58:53 UTC - Eugen: Shouldn't
`producer.send();`
`producer.send();`
and
`producer.sendAsync();`
`producer.sendAsync();`
`producer.flush();`
be identical in terms of what and in what order the consumer will receive?
----
2020-02-07 23:59:23 UTC - Matteo Merli: Yes
----
2020-02-07 23:59:44 UTC - Matteo Merli: Assuming eclusive/failover subscription
----
2020-02-08 00:00:00 UTC - Eugen: Funny thing, it is not identical
----
2020-02-08 00:00:46 UTC - Eugen: I played around with deduplication unit tests,
and the last line here should fail:
```producer.newMessage().value("my-message-0".getBytes()).sequenceId(2).sendAsync();
producer.newMessage().value("my-message-1".getBytes()).sequenceId(3).sendAsync();
producer.newMessage().value("my-message-2".getBytes()).sequenceId(5).sendAsync();
producer.flush();
// Repeat the messages and verify they're not received by consumer
producer.newMessage().value("A".getBytes()).sequenceId(2).sendAsync();
producer.newMessage().value("B".getBytes()).sequenceId(6).sendAsync();
producer.flush(); // not that it SHOULD matter, as the .close() below would
(hopefully) imply flushing first
producer.close();
for (int i = 0; i < 3; i++) {
Message<byte[]> msg = consumer.receive();
assertEquals(new String(msg.getData()), "my-message-" + i);
consumer.acknowledge(msg);
}
Message<byte[]> msg = consumer.receive(1, TimeUnit.SECONDS);
assertNull(msg);```
----
2020-02-08 00:01:34 UTC - Eugen: because of my message "B", which, due to the
higher seqId is not a dupe, being added to the topic
----
2020-02-08 00:01:43 UTC - Eugen: however it does not fail!
----
2020-02-08 00:02:37 UTC - Eugen: now when I change the `sendAsync()` of both
(!) "A" and "B" to `send()`, that last line will fail, as it should
----
2020-02-08 00:03:04 UTC - Eugen: ok, if that is not what's supposed to happen,
I will look into it
----
2020-02-08 00:04:13 UTC - Eugen: interestingly, changing above code to only
send "B" synchronously does not make the test fail, which doesn't make much
sense to me, because asynchronously sending the duplicate "A" beforehand should
not have anything to do with synchronously sending "B"
----
2020-02-08 00:30:07 UTC - Sijie Guo: ephemeral
----
2020-02-08 00:30:34 UTC - Sijie Guo: would be helpful if you can describe more
about what you have done.
----
2020-02-08 02:28:37 UTC - Penghui Li: Looks like it’s related to the batch
message.
```if (sequenceId <= lastSequenceIdPushed) {
if (sequenceId <= lastSequenceIdPublished) {
log.warn("Message with sequence id {} is
definitely a duplicate", sequenceId);
} else {
<http://log.info|log.info>("Message with
sequence id {} might be a duplicate but cannot be determined at this time.",
sequenceId);
}
doBatchSendAndAdd(msg, callback, payload);
}```
You can check if A and B are in a batch.
----
2020-02-08 02:31:56 UTC - Penghui Li: If related to the batch message, we need
to optimize here
----
2020-02-08 07:37:54 UTC - Penghui Li: @Eugen I have added an issue on github to
track this problem <https://github.com/apache/pulsar/issues/6273>. Looks this
is a obvious bug.
----
