Hi,
currently, in DSL only primary-key joins are supported. However in you
case, you have a non-primary-key join. There is already a JIRA to add
support for this: https://issues.apache.org/jira/browse/KAFKA-3705
Currently, you will need to use Processor API. For the non-time-based
input (if not t
Hi Andy,
One option would be to take the original transaction data and enrich it
with the customer ID and then send the result to a new topic, partitioned
appropriately. That would then allow you to do a join between that topic
and the ledger data (otherwise I don't see how you can join given that
This sounds like Kafka not being entirely robust to disk corruption, which
seems entirely possible and normal. I'd simply delete that log file and let
a replica replay catch it up at broker bootup.
Trying to guard against all possible disk corruption bugs sounds very
difficult to me, it seems bett
Hi there,
If you're only shutting down a single broker of many and you have a
replication factor more than 1, those consumer and producer configs should
handle it. However, if you have only a single broker, I'd recommend getting
some replication in before doing any maintenance work - Kafka is real
inline
On Mon, Sep 5, 2016 at 11:58 PM, F21 wrote:
> Hi Tom,
>
> Thank you so much for your response. I had a feeling that approach would
> run into scalability problems, so thank you for confirming that.
>
> Another approach would be to have each service request a subscription from
> the event
Hey Tom,
Thanks for your help :)
In terms of the number of topics, there would be 1 topic per service.
Each service would be running multiple instances, but they would be in
the same consumer group consuming the same topic.
I am expecting around 30ish microservices at the moment, so around 3
Also, another upcoming feature is (slightly simplified description follows)
"global" state/KTable. Today, a KTable is always partitioned/sharded. One
advantage of global state/tables will be for use cases where you need to
perform non-key joins, similar to what Guillermo described previously in
t
Done: https://issues.apache.org/jira/browse/KAFKA-4131
Thanks Guozhang for also looking at this. BTW, do you have any idea when
0.10.1 will be released?
-David
On 9/5/16, 12:14 AM, "Guozhang Wang" wrote:
Hello David,
Thanks for reporting this issue, and after some look through
Definitely worth putting it there. I will find some time soon to do it.
This is the least I can do!
Thanks guys for the quick feedback.
Derar
On Mon, Sep 5, 2016 at 1:43 PM, Ismael Juma wrote:
> Hi Derar,
>
> The support for wildcards is limited to `*` at this point. Sorry for the
> confusion.
Is there a ticket to extend wildcard topic ACL support to include the use
case Derar originally mentioned? And/or are there plans to?
On Mon, Sep 5, 2016 at 3:43 PM, Ismael Juma wrote:
> Hi Derar,
>
> The support for wildcards is limited to `*` at this point. Sorry for the
> confusion. If you're
We had a hardware failure on broker 1 of a 3 broker cluster over the weekend.
The broker was replaced, and when the replacement broker came up it started to
replicate partitions from the other 2 brokers as you'd expect. But while broker
1 (the replacement) was able to fetch properly from broker
Hi Samuel,
I am not aware of a JIRA for it. This has been discussed a few times, but I
don't think anyone is actively working on it.
Ismael
On Tue, Sep 6, 2016 at 7:22 PM, Samuel Taylor
wrote:
> Is there a ticket to extend wildcard topic ACL support to include the use
> case Derar originally m
Yifan,
Did you look into the producer metrics and check the avg. batch size /
request size? Did they change significantly compared with 0.8.2.1 clients
(there should be similar metrics but with different metrics name)?
Guozhang
On Mon, Sep 5, 2016 at 11:17 PM, Yifan Ying wrote:
> Hi Ismael,
>
I have a very simple Metrics Reporter that implements the Kafka 0.10.0
MetricsReporter interface, that takes in the metrics as a parameter and logs
them. When a producer starts to send messages, I can see that it is correctly
configured to use the Metrics Reporter I created because it starts to
The PMC for Apache Kafka has invited Jason Gustafson to join as a committer and
we are pleased to announce that he has accepted!
Jason has contributed numerous patches to a wide range of areas, notably
within the new consumer and the Kafka Connect layers. He has displayed
great taste and judgement
Michael,
One thing to be aware is that if the producer stops sending messages, some
of the metrics will be reset to 0 after the metric window elapses. Do you
see metric values reported in the reporter different from that in jconsole?
Thanks,
Jun
On Tue, Sep 6, 2016 at 3:03 PM, Michael Ross wro
Welcome, and really happy to have you onboard Jason!
Guozhang
On Tue, Sep 6, 2016 at 3:25 PM, Neha Narkhede wrote:
> The PMC for Apache Kafka has invited Jason Gustafson to join as a
> committer and
> we are pleased to announce that he has accepted!
>
> Jason has contributed numerous patches t
Well deserved, Jason. Looking forward to your future contributions :)
On Tue, Sep 6, 2016 at 3:29 PM, Guozhang Wang wrote:
> Welcome, and really happy to have you onboard Jason!
>
>
> Guozhang
>
> On Tue, Sep 6, 2016 at 3:25 PM, Neha Narkhede wrote:
>
>> The PMC for Apache Kafka has invited Jaso
Congratulations Jason! Well-deserved. :)
Ismael
On Tue, Sep 6, 2016 at 11:25 PM, Neha Narkhede wrote:
> The PMC for Apache Kafka has invited Jason Gustafson to join as a
> committer and
> we are pleased to announce that he has accepted!
>
> Jason has contributed numerous patches to a wide range
Jan,
For the time rolling issue, Jiangjie has committed a fix (
https://issues.apache.org/jira/browse/KAFKA-4099) to trunk. Perhaps you can
help test out trunk and see if there are any other issues related to
time-based index?
Thanks,
Jun
On Mon, Sep 5, 2016 at 11:52 PM, Jan Filipiak
wrote:
>
Congratulations Jason on this very well deserved recognition.
--Vahid
From: Neha Narkhede
To: "d...@kafka.apache.org" ,
"users@kafka.apache.org"
Cc: "priv...@kafka.apache.org"
Date: 09/06/2016 03:26 PM
Subject:[ANNOUNCE] New committer: Jason Gustafson
The PMC for Apac
Congratulations Jason!
On Tue, Sep 6, 2016 at 3:40 PM, Vahid S Hashemian wrote:
> Congratulations Jason on this very well deserved recognition.
>
> --Vahid
>
>
>
> From: Neha Narkhede
> To: "d...@kafka.apache.org" ,
> "users@kafka.apache.org"
> Cc: "priv...@kafka.apache.org"
> Date:
Hi,
I'm trying to send java object using kryo object serializer . The
producer is able to send the payload to the queue, but I'm having issues
reading the data in consumer. I'm using consumer group using KafkaStream.
The consumer code is based out of the example in kafka documentation.
Here's th
congrats jason!
On Tue, Sep 6, 2016 at 4:12 PM, Sriram Subramanian wrote:
> Congratulations Jason!
>
> On Tue, Sep 6, 2016 at 3:40 PM, Vahid S Hashemian <
> vahidhashem...@us.ibm.com
> > wrote:
>
> > Congratulations Jason on this very well deserved recognition.
> >
> > --Vahid
> >
> >
> >
> > Fr
Congrats, Jason!
On Tue, Sep 6, 2016 at 5:09 PM, Onur Karaman
wrote:
> congrats jason!
>
> On Tue, Sep 6, 2016 at 4:12 PM, Sriram Subramanian
> wrote:
>
> > Congratulations Jason!
> >
> > On Tue, Sep 6, 2016 at 3:40 PM, Vahid S Hashemian <
> > vahidhashem...@us.ibm.com
> > > wrote:
> >
> > > Co
Thanks all!
On Tue, Sep 6, 2016 at 5:13 PM, Becket Qin wrote:
> Congrats, Jason!
>
> On Tue, Sep 6, 2016 at 5:09 PM, Onur Karaman >
> wrote:
>
> > congrats jason!
> >
> > On Tue, Sep 6, 2016 at 4:12 PM, Sriram Subramanian
> > wrote:
> >
> > > Congratulations Jason!
> > >
> > > On Tue, Sep 6, 2
Congrats, Jason!
On Tuesday, September 6, 2016, Jason Gustafson wrote:
> Thanks all!
>
> On Tue, Sep 6, 2016 at 5:13 PM, Becket Qin > wrote:
>
> > Congrats, Jason!
> >
> > On Tue, Sep 6, 2016 at 5:09 PM, Onur Karaman
> > >
> > wrote:
> >
> > > congrats jason!
> > >
> > > On Tue, Sep 6, 2016 at
congrats, Jason!
On Wed, Sep 7, 2016 at 9:28 AM, Ashish Singh wrote:
> Congrats, Jason!
>
> On Tuesday, September 6, 2016, Jason Gustafson wrote:
>
> > Thanks all!
> >
> > On Tue, Sep 6, 2016 at 5:13 PM, Becket Qin > > wrote:
> >
> > > Congrats, Jason!
> > >
> > > On Tue, Sep 6, 2016 at 5:09
Hi Guozhang,
batch-size-avg and request-size-avg looked similar after the upgrade.
Yifan
On Tue, Sep 6, 2016 at 2:19 PM, Guozhang Wang wrote:
> Yifan,
>
> Did you look into the producer metrics and check the avg. batch size /
> request size? Did they change significantly compared with 0.8.2.1
Thanks David,
0.10.1 is planned to be released in Oct, we are following a time-based
release cycle moving forward to have more predictable releases.
Guozhang
On Tue, Sep 6, 2016 at 8:12 AM, David Garcia wrote:
> Done: https://issues.apache.org/jira/browse/KAFKA-4131
>
> Thanks Guozhang for als
30 matches
Mail list logo
