kishore
How is going to reduce throughput if we have enough memory for batch size
?, could you please explain
On Tue, Aug 18, 2015 at 11:47 AM, Kishore Senji <kse...@gmail.com> wrote:
> But this will reduce the throughput in a good scenario. May be we need to
> enhance the Callback interface appropriately.
>
> On Mon, Aug 17, 2015 at 7:15 PM, sunil kalva <kalva.ka...@gmail.com>
> wrote:
>
> > tx jeff,
> > Actually we need to set "buffer.memory" to minimum (default is ~35 MB)
> and
> > "block.on.buffer.full" to "true" so that the sender will block as soon
> as
> > these conditions met. And then release once the cluster is healthy.
> >
> > --
> > SunilKalva
> >
> > On Mon, Aug 17, 2015 at 11:20 PM, Jeff Holoman <jholo...@cloudera.com>
> > wrote:
> >
> > > Actually this won't work. The problem is if the producer loses
> > connectivity
> > > to the broker, then messages will continue to queue up until batch.size
> > is
> > > exhausted. Then the send will block. At this point, if you gain
> > > connectivity again, then the messages will be resent.
> > >
> > > If all brokers die, you should get Not Enough Replicas if you have the
> > > min.isr.set first. This should be an indication that you are having a
> > > problem, before you get to the error in the actual prodcuer thread.
> Keep
> > in
> > > mind that as of right now you need to manually set the min.isr's with
> > alter
> > > topic due to KAFKA-2114
> > > Either way, you will eventually get an error in the producer thread
> that
> > > isn't passed to the callback.
> > >
> > > And I think that's the point.
> > >
> > > Thanks
> > >
> > > Jeff
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Aug 17, 2015 at 12:39 PM, Madhukar Bharti <
> > > bhartimadhu...@gmail.com>
> > > wrote:
> > >
> > > > Hi Sunil,
> > > >
> > > > Producer will throw an Exception in callback if there is problem
> while
> > > > sending data. You can check like:
> > > >
> > > > public void onCompletion(RecordMetadata arg0, Exception arg1) {
> > > > if (arg1 != null) {
> > > > System.out.println("exception occured");
> > > > }
> > > > System.out.println("sent")
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Aug 17, 2015 at 9:20 PM, sunil kalva <kalva.ka...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi all
> > > > > I am using new java producer in async mode, when my entire cluster
> is
> > > > down
> > > > > i am loosing all my messages. How do we get notification when the
> > > cluster
> > > > > is down so that i can send messages to another cluster. The
> callback
> > is
> > > > > only triggered when the cluster is reachable .
> > > > >
> > > > > --SK
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks and Regards,
> > > > Madhukar Bharti
> > > > Mob: 7845755539
> > > >
> > >
> > >
> > >
> > > --
> > > Jeff Holoman
> > > Systems Engineer
> > >
> >
>
