2020-06-12 12:03:36 UTC - Vil: @Vil has joined the channel ---- 2020-06-12 12:04:30 UTC - Vil: But doesn’t Pulsar Presto SQL work like this? Talking directly to Bookeeper? ---- 2020-06-12 14:17:40 UTC - Anup Ghatage: @Anup Ghatage has joined the channel ---- 2020-06-12 15:06:30 UTC - David Kjerrumgaard: If you want some contribution to this response please let us know. We all have some points to add. plus, we can address it as "a response from the Pulsar community" ---- 2020-06-12 15:46:55 UTC - Ganesh Raju: @Ganesh Raju has joined the channel ---- 2020-06-12 16:15:18 UTC - Sijie Guo: I would encourage people to blog their thoughts about this as well. There will be more responses from the community :slightly_smiling_face: ---- 2020-06-12 17:29:27 UTC - Derek Moore: Your comment is not approved yet! :wink:
I pointed Enrico to that article. I am trying to put together a very fair comparison of <http://Pravega.io|Pravega.io> and Kafka. I saw Kai's article and thought it was very biased. ---- 2020-06-12 19:48:02 UTC - Addison Higham: FYI: we just tried upgrading to 2.5.2 (well, with a few custom patches and backports, but basically same ones we had on 2.5.1). It went fine in all our regions except our most busy one, where the call to get `getManagedLedgerConfig` would consistently time out and prevent really any consumers/producers from being ablet to connect. It appears that some combination of scale and features (perhaps s3 offloading which we don't utilize that much in other regions?) brought our cluster to a halt ---- 2020-06-13 00:23:00 UTC - Devin G. Bost: Agreed. I noticed that the Confluent folks have paid advertising on the “Kafka vs Pulsar” search in Google. ---- 2020-06-13 00:25:33 UTC - Devin G. Bost: @Addison Higham I’m planning on upgrading our clusters soon, so thanks for sharing this. ---- 2020-06-13 00:26:57 UTC - Addison Higham: I need to confirm, but I think it is related to offloading changing a bit and some config I am missing or it just is not being passed properly ---- 2020-06-13 00:27:45 UTC - Devin G. Bost: Have you checked if the configs have changed between versions? (Perhaps a diff between new vs old would be informative.) ---- 2020-06-13 00:34:25 UTC - Devin G. Bost: The SQL workers are different from the Pulsar client. ---- 2020-06-13 01:22:19 UTC - Karthik Ramasamy: yup ---- 2020-06-13 01:23:13 UTC - Karthik Ramasamy: yup ---- 2020-06-13 02:33:01 UTC - Addison Higham: I am like 95% certain this is the issue: <https://github.com/apache/pulsar/pull/6422/files#diff-bb90edad9f69a5d791cb43b04d9a4c82R1008> This creates a new offload policy but doesn't copy all properties into the OffloadPolicies, if you have s3 offloading configured and it falls back to defaults, you don't have enough for s3 to be properly configured (<https://github.com/apache/pulsar/pull/6422/files#diff-bb90edad9f69a5d791cb43b04d9a4c82R1008>) This throws an exception which bubbles all the way up to `getManagedLedgerConfig` prevents us from ever creating that config ----
