2019-06-18 16:17:58 UTC - David Kjerrumgaard: @David Kjerrumgaard has joined the channel ---- 2019-06-18 16:43:37 UTC - lan: @lan has joined the channel ---- 2019-06-19 07:38:10 UTC - Kenta Kusumoto: @Kenta Kusumoto has joined the channel ---- 2019-06-19 07:50:08 UTC - Kenta Kusumoto: Hi. I have a performance issue with Pulsar. I have a producer which publishes around 1000 messages per second. However the consumer doesn’t seem to be able to handle all messages. Can someone tell if Pulsar should be able to handle this rate of messages and where to start debugging the problem?
The code for the consumer is here: <https://github.com/HSLdevcom/transitdata-common/blob/master/src/main/java/fi/hsl/common/pulsar/PulsarApplication.java#L196> ---- 2019-06-19 07:50:25 UTC - Kenta Kusumoto: ``` { "msgRateIn" : 633.5893397602742, "msgThroughputIn" : 273775.8453950144, "msgRateOut" : 198.33661117639403, "msgThroughputOut" : 85757.96729333946, "averageMsgSize" : 432.10298566355397, "storageSize" : 70710127, "publishers" : [ { "msgRateIn" : 633.5893397602742, "msgThroughputIn" : 273775.8453950144, "averageMsgSize" : 432.0, "producerId" : 0, "metadata" : { }, "address" : "/172.17.0.1:42030", "producerName" : "standalone-0-34", "connectedSince" : "2019-06-19T07:18:01.492Z", "clientVersion" : "2.3.1" } ], "subscriptions" : { "hfp-dedup-subscription" : { "msgRateOut" : 198.33661117639403, "msgThroughputOut" : 85757.96729333946, "msgRateRedeliver" : 0.0, "msgBacklog" : 163603, "blockedSubscriptionOnUnackedMsgs" : false, "unackedMessages" : 0, "type" : "Exclusive", "activeConsumerName" : "4e4e6", "msgRateExpired" : 0.0, "consumers" : [ { "msgRateOut" : 198.33661117639403, "msgThroughputOut" : 85757.96729333946, "msgRateRedeliver" : 0.0, "consumerName" : "4e4e6", "availablePermits" : 700, "unackedMessages" : 0, "blockedConsumerOnUnackedMsgs" : false, "metadata" : { }, "address" : "/172.17.0.1:41998", "connectedSince" : "2019-06-19T07:14:19.837Z", "clientVersion" : "2.3.1" } ] } }, "replication" : { }, "deduplicationStatus" : "Disabled" } ``` ---- 2019-06-19 07:52:08 UTC - Kenta Kusumoto: the consumer’s `msgBacklog` grows over time. ----
