But this will reduce the throughput in a good scenario. May be we need to
enhance the Callback interface appropriately.
On Mon, Aug 17, 2015 at 7:15 PM, sunil kalva <kalva.ka...@gmail.com> wrote:
> tx jeff,
> Actually we need to set "buffer.memory" to minimum (default is ~35 MB) and
> "block.on.buffer.full" to "true" so that the sender will block as soon as
> these conditions met. And then release once the cluster is healthy.
>
> --
> SunilKalva
>
> On Mon, Aug 17, 2015 at 11:20 PM, Jeff Holoman <jholo...@cloudera.com>
> wrote:
>
> > Actually this won't work. The problem is if the producer loses
> connectivity
> > to the broker, then messages will continue to queue up until batch.size
> is
> > exhausted. Then the send will block. At this point, if you gain
> > connectivity again, then the messages will be resent.
> >
> > If all brokers die, you should get Not Enough Replicas if you have the
> > min.isr.set first. This should be an indication that you are having a
> > problem, before you get to the error in the actual prodcuer thread. Keep
> in
> > mind that as of right now you need to manually set the min.isr's with
> alter
> > topic due to KAFKA-2114
> > Either way, you will eventually get an error in the producer thread that
> > isn't passed to the callback.
> >
> > And I think that's the point.
> >
> > Thanks
> >
> > Jeff
> >
> >
> >
> >
> >
> >
> > On Mon, Aug 17, 2015 at 12:39 PM, Madhukar Bharti <
> > bhartimadhu...@gmail.com>
> > wrote:
> >
> > > Hi Sunil,
> > >
> > > Producer will throw an Exception in callback if there is problem while
> > > sending data. You can check like:
> > >
> > > public void onCompletion(RecordMetadata arg0, Exception arg1) {
> > > if (arg1 != null) {
> > > System.out.println("exception occured");
> > > }
> > > System.out.println("sent")
> > >
> > >
> > >
> > >
> > > On Mon, Aug 17, 2015 at 9:20 PM, sunil kalva <kalva.ka...@gmail.com>
> > > wrote:
> > >
> > > > Hi all
> > > > I am using new java producer in async mode, when my entire cluster is
> > > down
> > > > i am loosing all my messages. How do we get notification when the
> > cluster
> > > > is down so that i can send messages to another cluster. The callback
> is
> > > > only triggered when the cluster is reachable .
> > > >
> > > > --SK
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks and Regards,
> > > Madhukar Bharti
> > > Mob: 7845755539
> > >
> >
> >
> >
> > --
> > Jeff Holoman
> > Systems Engineer
> >
>
