Re: [DISCUSS] KIP-174 - Deprecate and remove internal converter configs in WorkerConfig

Hi Ewen, Sorry, I am bit late in responding this. Thanks for your inputs and I've updated the KIP by adding more details to it. Regards, Umesh On Mon, 31 Jul 2017 at 21:51 Ewen Cheslack-Postava wrote: > On Sun, Jul 30, 2017 at 10:21 PM, UMESH CHAUDHARY > wrote: > >> Hi Ewen, >> Thanks for you

Re: Kafka streams regex match

Hi Shekar, that warning is expected during rebalances and should generally resolve itself. How many threads/app instances are you running? It is impossible to tell what is happening with the full logs. Thanks, Damian On Mon, 7 Aug 2017 at 22:46 Shekar Tippur wrote: > Damien, > > Thanks for poin

[kafka streams] 'null' values in state stores

Hi, I noticed the following: When a kafka streams application starts, it will restore its state in its state-stores (from the log-compacted kafka topic). All good so far, but I noticed that the 'deleted' entries are actually read in into the store as 'key' with value:`null` Is this expected beha

Re: [kafka streams] 'null' values in state stores

Hi, The null values are treated as deletes when they are written to the store. You can see here: https://github.com/apache/kafka/blob/0.11.0/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java#L261 On Tue, 8 Aug 2017 at 11:22 Bart Vercammen wrote: > Hi, > > I noticed

Re: [kafka streams] 'null' values in state stores

That's RocksDB .. I'm using in-memory stores ... here: https://github.com/apache/kafka/blob/0.11.0/streams/src/main/java/org/apache/kafka/streams/state/internals/ChangeLoggingKeyValueBytesStore.java#L56 the 'null' is not checked ... On Tue, Aug 8, 2017 at 12:52 PM, Damian Guy wrote: > Hi, > The

Re: Kafka streams regex match

I am running this on a mac laptop. I am using defaults. Sent from my iPhone > On Aug 8, 2017, at 03:11, Damian Guy wrote: > > Hi Shekar, that warning is expected during rebalances and should generally > resolve itself. > How many threads/app instances are you running? > It is impossible to tell

Kafka Connect on DC/OS

I am running Confluent Connect HdfsSinkConnector on some Avro data generated from a Kafka Streams application in a Kafka topic. The whole stuff Confluent Connect, Confluent Kafka and HDFS run on Mesos DC/OS. The Kafka part is of version Confluent Kafka 3.2.2, which is Apache Kafka 0.10.2.1. The pr

Re: [kafka streams] 'null' values in state stores

The change logger is not used during restoration of the in-memory-store. Restoration is handled https://github.com/apache/kafka/blob/0.11.0/streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryKeyValueStore.java#L79 But, even then it is just putting `null` when it should be delet

Re: [DISCUSS] Streams DSL/StateStore Refactoring

Hello Jan, Thanks for your feedback. Trying to explain them a bit more here since I think there are still a bit mis-communication here: Here are a few things I need to clarify for KIP-182 first: 1. KIP-182 is mainly about refactoring the public APIs, NOT for making any optimizations on the inter

Way to check if custom SMT has been added to the classpath or even if it i working.

Hi All, i have created a custom SMT and have deployed. I would like to know if there is a way to check if the transform is working or not.(def not working as the messages are not getting transformed) I am also trying to remote debug using intellij and nothing seam working, as i do not see any con

Re: KafkaProducer (0.9): Single producer multiple topics.

Hello Guys, Any update on this? Here’s what we’ve tried so far: We briefly looked at the code and here's what we understand: Kafka server accepts a a batch of messages for each topic partition. Map[TopicAndPartition, MessageSet] It tries to process the entire batch within the configured timeout

Apache Kafka support for Oracle Sparc 10/11 server

HI Team, For one of my customer, who currently have Oracle Sparc servers with Solaris 10/11 OS on it, Wanted to check whether Kafka with all of its components(specifically as a stream) are supported? -- Thank You!!! Regards Siddayya

Unable to start kafka server, 0.0.0.0:9092: Address already in use. Please help

Hello all, I am trying to use kafka locally on my Mac. When I am trying to start the kafka server I get the following error. Please help. Thanks. [2017-08-08 22:16:37,319] FATAL [Kafka Server 0], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.common.K

Re: [DISCUSS] KIP-174 - Deprecate and remove internal converter configs in WorkerConfig

Great, looking good. I'd probably be a bit more concrete about the Proposed Changes (e.g., "will log an warning if the config is specified" and "since the JsonConverter is the default, the configs will be removed immediately from the example worker configuration files"). Other than that this LGTM

Re: Unable to start kafka server, 0.0.0.0:9092: Address already in use. Please help

I was successfully using it for many months, and now I am unable to. I have used lsof -i tcp:9092 to see any process using 9092 port and I don't see any process using 9092. On Tue, Aug 8, 2017 at 10:28 PM, shyla deshpande wrote: > Hello all, > > I am trying to use kafka locally on my Mac. When I

Re: Adding partitons | Unaffected producers

Thanks. Any possible reasons why we didnt made consumers partition aware. do we have a similar property for consumers. I found that in streams application, if i increase partitions i need to restart my stream application, same would hold for consumers as well. -Sameer. On Fri, Aug 4, 2017 at 8:2

Re: Unable to start kafka server, 0.0.0.0:9092: Address already in use. Please help

I have even tried deleting the contents of log.dirs and dataDir before starting the zookeeper and kafka server, still no success. I appreciate any help. On Tue, Aug 8, 2017 at 10:43 PM, shyla deshpande wrote: > I was successfully using it for many months, and now I am unable to. > I have used ls