consumer group and current offsets

Hello, Normally I know that each consumer group has a current offset for each partition it consumes, and when we delete a group, its current offsets are deleted with it, and when we (re)create the consumer group again, it will have no current offsets (meaning it will go from offset 0 or offset late

How to emit lag into Prometheus?

Hi, what is suggested way to emit lag into Prometheus? Is there a Kafka Streams job for it? Is Burrow or Kafka Lag Exporter? Emitting lag from the consumer is not a great option of course, so looking at independent jobs that emit lag. Do you know any? Thank you

joins & co-location

Hi guys, in case of KTable with KTable joins, is co-location a requirement? Do the 2 topics need to have the same number of partitions? https://forum.confluent.io/t/ktable-ktable-joins-is-colocation-needed/1034 Thank you, Nicolae

linking different topics AVRO schemas with OWL/RDFS ontologies

sociate-data-sources-like-Kafka-topics-or-db-tables-that-they-represent-Entity-X-and-they-are-owned-by-team-Y-Would-OWL-Ontologies-be-ok-to-model-entities-across Thank you very much, Nicu -- Thank you, Nicolae Marasoiu Scala Engineer Orion, OVO Group

logical delete from a log compacted topic; tombstone message? with kafka-streams

perhaps need to emit a tombstone message (is there such a thing?) so that the “record” is effectively / logically deleted from the current state compacted log What would be the manner to achieve this using kafka-streams? Thank you, -- Thank you, Nicolae Marasoiu Scala Engineer Orion, OVO Group

topics metadata, OWL ontologies and AVRO

d enhanced discoverability of resources, data and services along some similar properties like entity types, flows, etc? Please ask me, I think it is still potentially fuzzy, Thank you very much, Nicolae Marasoiu -- Thank you, Nicolae Marasoiu Scala Engineer Orion, OVO Group

Re: could not find implicit kstreams.Consumed[String,Entity]

Hi, In fact an import was missing. It works now: import org.apache.kafka.streams.Topology import org.apache.kafka.streams.scala.ImplicitConversions._ import org.apache.kafka.streams.scala.Serdes._ On Thu, 9 Apr 2020 at 11:49, Nicolae Marasoiu < nicolae.maras...@ovoenergy.com> wrote:

could not find implicit kstreams.Consumed[String,Entity]

nversions._import org.apache.kafka.streams.scala.Serdesimport org.apache.kafka.streams.scala.Serdes._import org.apache.kafka.streams.scala.kstream.Consumed For more (up to date) details: https://stackoverflow.com/questions/61103804/why-do-i-get-this-compilation-error-could-not-find-implicit-value-for-kstream https://www.quora.com/unanswered/Kafka-streams-avro-serde-Is-Scala-Either-type-supported Thank you very much, -- Thank you, Nicolae Marasoiu Scala Engineer Orion, OVO Group

Re: zokeeper.connect no longer available on producer?

gt;when the brokers are up again, sends will succeed again. There is no need >to close the producer in between. Is that not what you are seeing? > >Thanks, > >Jun > > >On Fri, Sep 27, 2013 at 7:15 AM, Nicolae Marasoiu >wrote: > >> Hi, >> >> We have a

zokeeper.connect no longer available on producer?

Hi, We have a blocking async producer. We noticed: 1. when the metadata brokers are down, the client no longer attempts to reconnect to any of them. 2. zookeeper.connect is no longer recognized as a valid configuration, hinting that in fact the zookeeper balancing is no longer supported on