Re: Table a KStream

2017-02-10 Thread Nick DeCoursin
QaLMqiR2e1?subj= > Re+DISCUSS+KIP+114+KTable+materialization+and+improved+semantics > > > However for now, you can only choose from the two options as described. > > > -Matthias > > > > On 2/10/17 1:49 PM, Nick DeCoursin wrote: > > To be hones

Re: Table a KStream

2017-02-10 Thread Nick DeCoursin
he key of the KStream is the key of the KTable. // Any latter key overwrites the former. someStream.table( Serde, Serde, topicName, tableName ); or maybe the Serdes can be inferred? Either way, this would be a nice clean approach to a (maybe?) common use case. Thank you, Nick On 2

Re: Kafka Connect - Unknown magic byte

2017-02-10 Thread Nick DeCoursin
(the `kafka-avro-console-consumer` works because it doesn't deserialize the key.) Nick On 10 February 2017 at 19:25, Nick DeCoursin wrote: > Finally, here's the problem: > > $ curl -X GET localhost:8081/subjects > ["test-value","test-key","tes

Re: Kafka Connect - Unknown magic byte

2017-02-10 Thread Nick DeCoursin
;d like to have both the key and the value as Avro, not just the value. Thank you, Nick On 10 February 2017 at 12:55, Nick DeCoursin wrote: > It seems like a bug. > > Thanks, > Nick > > On 9 February 2017 at 14:57, Nick DeCoursin > wrote: > >> Hello, >> >

Re: Kafka Connect - Unknown magic byte

2017-02-10 Thread Nick DeCoursin
It seems like a bug. Thanks, Nick On 9 February 2017 at 14:57, Nick DeCoursin wrote: > Hello, > > Here is a github repo with the failing case: https://github.com/decoursin/ > kafka-connect-test. > > I've tried other similar things and nothing seems to work. > > Tha

Re: Kafka Connect - Unknown magic byte

2017-02-09 Thread Nick DeCoursin
Hello, Here is a github repo with the failing case: https://github.com/decoursin/kafka-connect-test. I've tried other similar things and nothing seems to work. Thanks, Nick On 9 February 2017 at 04:40, Nick DeCoursin wrote: > Any help here? I can create a git repo with the code, if

Re: Kafka Connect - Unknown magic byte

2017-02-08 Thread Nick DeCoursin
Any help here? I can create a git repo with the code, if somebody assures me they'll have a look. Thank you, Nick On 8 February 2017 at 10:39, Nick DeCoursin wrote: > Below's the rest of my consumer, which includes the serde code. It's worth > noting that when I run the

Re: Kafka Connect - Unknown magic byte

2017-02-08 Thread Nick DeCoursin
println(e); } })); } } Thank you very much, Nick On 8 February 2017 at 01:26, Gwen Shapira wrote: > Since the data that goes through your Streams app is the one with the > bad magic byte, I suspect your streams Serde is not serializing Avro > correctly (i.e. in the

Kafka Connect - Unknown magic byte

2017-02-07 Thread Nick DeCoursin
streamsConfiguration.put(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://localhost:8081";); final KStream tests = builder.stream(Serdes.String(), testSpecificAvroSerde, "test"); tests.map((id, command) -> { System.out.println("test id=" + id + " c

Re: Track Latest Count - Unique Key

2017-01-25 Thread Nick DeCoursin
e semantics for Kafka haven't been > completed, so this would be at-least-once. > > On Wed, 25 Jan 2017 at 08:43 Nick DeCoursin > wrote: > > > From the documentation > > <http://docs.confluent.io/3.1.1/streams/developer-guide.html#id3>: > > > > The c

Track Latest Count - Unique Key

2017-01-25 Thread Nick DeCoursin
ord, update the corresponding key by > incrementing its count by one. How? Is there any examples of this online? To me, it doesn't seem so trivial because there's no such thing as a transaction in Kafka, Thank you, Nick -- Nick DeCoursin Software Engineer foodpanda Tel | +1 9

Re: Table a KStream

2017-01-24 Thread Nick DeCoursin
I have a Kafka Stream, and I > want to > >> create a table from it backed by a state store. The key of the stream > could > >> be the same as the table. > >> > >> I've tried following examples, but it seems all examples use `groupBy` > or > >

Table a KStream

2017-01-24 Thread Nick DeCoursin
into `KTable`s. Is there any other way? Thank you very much, Nick DeCoursin -- Nick DeCoursin Software Engineer foodpanda Tel | +1 920 450 5434 Mail | n.decour...@foodpanda.com Skype | nick.foodpanda Foodpanda GmbH | Schreiberhauer Str. 30 | 10317 Berlin | Germany Sitz der Gesellschaft | Berli

Process KTable on Predicate

2016-11-12 Thread Nick DeCoursin
ed on any new event. Thank you, Nick DeCoursin