2020-08-13 10:14:49 UTC - moshe baruch: @moshe baruch has joined the channel ---- 2020-08-13 10:15:15 UTC - moshe baruch: Hi Pulsar Team,
We use pulsar SQL and we found that the Pulsar SQL is lacking the last message when querying pulsar topics. How can we solve this issue? This is a known issue? Please advise. ---- 2020-08-13 10:15:28 UTC - moshe baruch: we use pulsar-client - 2.6.0V ---- 2020-08-13 14:48:44 UTC - Jim M.: @Jim M. has joined the channel ---- 2020-08-13 14:49:40 UTC - Jim M.: Pulsar java client question, are there metrics the client produces. Use case: I want to harvest consumer metrics to determine if a subscription backlog is too large and horizontally scale the pod in K8s using custom metrics in Kubernetes ---- 2020-08-13 15:44:48 UTC - Addison Higham: This is a known issue but is already fixed, it just needs a configuration change (see <https://github.com/apache/pulsar/issues/3828>). The docs mention the fix at the very bottom, <https://pulsar.apache.org/docs/en/sql-deployment-configurations/>, specifically: > The broker does not advance LAC, so when Pulsar SQL bypass broker to query data, it can only read entries up to the LAC that all the bookies learned. You can enable periodically write LAC on the broker by setting "bookkeeperExplicitLacIntervalInMills" in the broker.conf. ---- 2020-08-13 15:52:38 UTC - Addison Higham: There are metrics that the producer has, see <http://pulsar.apache.org/api/client/org/apache/pulsar/client/api/Producer.html#getStats-->, but it doesn't give backlog data. However, you can enable per-subscription or per-consumer metrics here: <https://pulsar.apache.org/docs/en/reference-metrics/#subscription-metrics> ---- 2020-08-13 15:53:26 UTC - Jim M.: ah so nothing that can be used with K8s per say ---- 2020-08-13 16:00:27 UTC - Addison Higham: Pulsar exposes all it's metrics via prometheus, you can get metrics from Prometheus into custom-metrics for autoscaling using this guide: <https://docs.openshift.com/container-platform/4.1/monitoring/exposing-custom-application-metrics-for-autoscaling.html> ---- 2020-08-14 01:37:41 UTC - aloyszhang: Does anyone know how pulsar compile `*.proto` to java file? In module `pulsar-common` , I didn’t find any plugin like `com.github.igor-petruk.protobuf: protobuf-maven-plugin` or `org.xolstice.maven.plugins: protobuf-maven-plugin` ---- 2020-08-14 03:23:04 UTC - Ryan: <https://github.com/kafkaesque-io/pulsar-beam|https://github.com/kafkaesque-io/pulsar-beam> ----
