it sounds like there's an incorrect assumption
>> that
>> > > > > >> responses
>> > > > > >> >> from
>> > > >
d
> > > the
> > > > > >> >> >> coordinator:
> > > > > >> >> >> >> >> {JoinGroup, SyncGroup, LeaveGroup, Heartbeat,
> > > > OffsetCommit,
> > > > > >> >> >> OffsetFetch,
&
gt; > > IBM MessageHub
> > > > > eco...@uk.ibm.com
> > > > > IBM UK Ltd, Hursley Park, SO21 2JN
> > > > >
> > > > > IBM United Kingdom Limited Registered in England and Wales with
> > number
> > > > > 74159
> > > > IBM MessageHub
> > > > eco...@uk.ibm.com
> > > > IBM UK Ltd, Hursley Park, SO21 2JN
> > > >
> > > > IBM United Kingdom Limited Registered in England and Wales with
> number
> > > > 741598 Registered office: PO Box 41
stered in England and Wales with number
> > > 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants.
> > PO6
> > > 3AU
> > >
> > >
> > >
> > > From: Guozhang Wang
> > > To: "dev@kafka.apache.org"
> >
> > IBM United Kingdom Limited Registered in England and Wales with number
> > 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants.
> PO6
> > 3AU
> >
> >
> >
> > From: Guozhang Wang
> > To: "dev@kafka.ap
ng Wang
> To: "dev@kafka.apache.org"
> Date: 28/03/2017 19:02
> Subject:Re: [VOTE] KIP-81: Bound Fetch memory usage in the
> consumer
>
>
>
> 1) Makes sense.
> 2) Makes sense. Thanks!
>
> On Tue, Mar 28, 2017 at 10:11 AM, Mickael Maison
>
mited Registered in England and Wales with number
741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6
3AU
From: Guozhang Wang
To: "dev@kafka.apache.org"
Date: 28/03/2017 19:02
Subject: Re: [VOTE] KIP-81: Bound Fetch memory usage in the
consumer
1)
gt; > the
> >> >> >> >> > > moment, coordinator shares the same NetworkClient (and
> hence
> >> the
> >> >> >> same
> >> >> >> >> > > Selector) with consumer connections used for fetching
> records.
> >> &g
> >> >> >> it
>> >> >> >> > > will be good to ensure that coordinator is not blocked for
>> read
>> >> by
>> >> >> >> fetch
>> >> >> >> > > responses. This may
ser,
> >> messages
> >> >> are
> >> >> >> > > deleted from the MemoryPool so new messages can be stored.*
> >> >> >> > > Can you expand that a bit? I am assuming that partial buffers
> >> never
> >> >>
> >> consumer is
>> >> >> > > still holding a reference to the buffer. Would buffers be freed
>> when
>> >> >> > > fetches for all the partitions in a response are parsed, but
>> perhaps
>> >> >> not
>> >> >
re parsed, but
> perhaps
> >> >> not
> >> >> > > yet returned to the user (i.e., is the memory freed when a
> >> reference to
> >> >> > the
> >> >> > > response buffer is no longer required)? It wi
e good to document
>> the
>> >> > > (approximate) maximum memory requirement for the non-compressed
>> case.
>> >> > There
>> >> > > is data read from the socket, cached in the Fetcher and (as Radai
>> has
>> >> &g
;> > > On Tue, Dec 6, 2016 at 2:04 AM, radai
> >> > wrote:
> >> > >
> >> > >> +1 (non-binding).
> >> > >>
> >> > >> small nit pick - just because you returned a response to user
> doesnt
> >> &
no longer used. for some cases the actual "point of
>> > >> termination" may be the deserializer (really impl-dependant), but
>> > >> generally, wouldnt it be "nice" to have an explicit dispose() call on
>> > >> responses
t; responses (with the addition that getting the next batch of data from
> a
> > >> consumer automatically disposes the previous results)
> > >>
> > >> On Mon, Dec 5, 2016 at 6:53 AM, Edoardo Comar
> > wrote:
> > >>
> > >> > +1 (non bind
--
> >> > Edoardo Comar
> >> > IBM MessageHub
> >> > eco...@uk.ibm.com
> >> > IBM UK Ltd, Hursley Park, SO21 2JN
> >> >
> >> > IBM United Kingdom Limited Registered in England and Wales with number
> &g
Edoardo Comar
>> > IBM MessageHub
>> > eco...@uk.ibm.com
>> > IBM UK Ltd, Hursley Park, SO21 2JN
>> >
>> > IBM United Kingdom Limited Registered in England and Wales with number
>> > 741598 Registered office: PO Box 41, North Harbour, Portsmouth,
t; > 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants.
> PO6
> > 3AU
> >
> >
> >
> > From: Mickael Maison
> > To: dev@kafka.apache.org
> > Date: 05/12/2016 14:38
> > Subject:[VOTE] KIP-81: Bound Fetch memory usa
To: dev@kafka.apache.org
> Date: 05/12/2016 14:38
> Subject:[VOTE] KIP-81: Bound Fetch memory usage in the consumer
>
>
>
> Hi all,
>
> I'd like to start the vote for KIP-81:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 81%3A+Bound+Fetc
3AU
From: Mickael Maison
To: dev@kafka.apache.org
Date: 05/12/2016 14:38
Subject:[VOTE] KIP-81: Bound Fetch memory usage in the consumer
Hi all,
I'd like to start the vote for KIP-81:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-81%3A+Bound+Fetch+memory+usage+i
Hi all,
I'd like to start the vote for KIP-81:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-81%3A+Bound+Fetch+memory+usage+in+the+consumer
Thank you
23 matches
Mail list logo
