2020-02-10 17:03:34 UTC - Fred: @Fred has joined the channel ---- 2020-02-10 19:07:38 UTC - Ali: @Ali has joined the channel ---- 2020-02-10 20:49:08 UTC - Ali Ahmed: @tuteng Why are the pulsar images for 2.4.2 and 2.5.0 updated yesterday ? +1 : Eric Simon ---- 2020-02-10 22:26:42 UTC - Antti Kaikkonen: Could someone help me understand why creating a source connector with `--processing-guarantees EFFECTIVELY_ONCE` makes pulsar produce the messages one at a time? I think that this `future.join();` is causing it: <https://github.com/apache/pulsar/blob/e1f3409fd45b26bafb44c321f99f853ebbc50109/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/sink/PulsarSink.java#L237>
Pulsar waiting for the previous message to be acknowledged before producing the next message limits the throughput of this test source connector to just 90 msg/s compared to over 300 000 msg/s when using ATMOST_ONCE: <https://pastebin.com/rRkV0mTs>. I would imagine that this applies to normal pulsar functions as well, but I have not tested. ---- 2020-02-11 00:05:20 UTC - tuteng: I should only update the pulsar:latest image. In order to solve the problem of integration sql test failure, there should be some layers used in 2.4.2 and 2.5.0 +1 : Devin G. Bost ---- 2020-02-11 00:41:18 UTC - Devin G. Bost: If they’re all ephemeral, is there a reason that the tests use persistent instead? ----
