Re: Kafka Streams

Very interesting, looks a lot like many operations from Spark were brought across. Any plans to integrate with the reactive-stream protocol for interoperability with libraries akka-stream and RxJava? Thanks, DaVe. David Buschman d...@timeli.io > On Mar 10, 2016, at 2:26 PM, Jay Kr

Re: Kafka Streams

processing "1-at-a-time”, is it auto magic on commit handling at the beginning/end of the processing or can we specify where in the processing an offset commit happens? Thanks, DaVe. David Buschman d...@timeli.io > On Mar 11, 2016, at 7:21 AM, Dick Davies wrote: > > Ni

Re: Low-level Consumer Example (Scala)

ew API ( v0.11.X) is unproven, you might start with 0.10.X first, which is the version I am using now. Thanks, DaVe. David Buschman d...@timeli.io > On Apr 5, 2016, at 1:58 PM, Afshartous, Nick wrote: > > > Hi, > > I'm looking for a complete low-level consumer e

Re: Encryption at Rest

Kafka handles messages which are compose of an array of bytes. Kafka does not care what is in those byte arrays. You could use a custom Serializer and Deserializer to encrypt and decrypt the data from with your application(s) easily enough. This give the benefit of having encryption at rest a

Re: Receiving ILLEGAL_GENERATION, but I can't find information on the exception.

To add to what Dana said, we fixed this issue on AWS with setting the “max.partition.fetch.bytes” to a smaller setting so out consumer would poll more frequently. Try setting “max.partition.fetch.bytes” to “75”, then “50”, then “25”, … until the error stop occurring. The default i