18 11:08 am, "Lucas Wang" wrote:
> >
> > Thanks Jun. I've changed the KIP with the suggested 2 step upgrade.
> > Please take a look again when you have time.
> >
> > Regards,
> > Lucas
> >
> >
> > On Thu, Oct 4, 2018 at 10:06 AM Jun
Hi, Mani,
Thanks for the KIP. +1 from me.
Jun
On Wed, Sep 19, 2018 at 5:19 AM, Manikumar
wrote:
> Hi All,
>
> I would like to start voting on KIP-371, which adds a configuration option
> for building custom SSL principal names.
>
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
Congratulations, Mani!
Jun
On Thu, Oct 11, 2018 at 10:39 AM, Jason Gustafson
wrote:
> Hi all,
>
> The PMC for Apache Kafka has invited Manikumar Reddy as a committer and we
> are
> pleased to announce that he has accepted!
>
> Manikumar has contributed 134 commits including significant work to
Hi, Patrick,
Thanks for the KIP. Looks good to me overall and very useful. A few
comments below.
1. "will reject the requests with smaller broker generation than its
current generation." Is czxid monotonically increasing?
2. To clarify on the issue of the controller missing a ZK watcher. ZK
watc
Clement,
Thanks for your interest. Just gave you the permission.
Jun
On Mon, Oct 22, 2018 at 10:00 AM, Pellerin, Clement <
clement_pelle...@ibi.com> wrote:
> Can you give wiki permissions to wiki ID cpellerin?
> I was able to create and edit KIP-383 last week but it no longer works for
> me.
>
; We can further optimize the flow by avoiding sending requests to a
> broker if its broker generation is larger than the one in the controller
> context.
>
> I will also update the KIP to clarify how it works for BrokerChangeEvent
> processing in more detail.
>
> Thanks,
&g
vent between
> 6) and 7). In this case, controller already updates the cached czxid to the
> up-to-date ones so the bounced broker will not reject control requests in
> 8), which cause a correctness problem.
>
>
> Best,
> Zhanxiang (Patrick) Huang
>
> -
Hi, Patrick,
Could you update the KIP with the changes to ControlledShutdownRequest
based on the discussion thread?
Thanks,
Jun
On Sun, Oct 21, 2018 at 2:25 PM, Mickael Maison
wrote:
> +1( non-binding)
> Thanks for the KIP!
>
> On Sun, Oct 21, 2018, 03:31 Harsha Chintalapani wrote:
>
> > +1
Hi, Patrick,
Thanks for the updated KIP. +1
Jun
On Wed, Oct 24, 2018 at 4:52 PM, Patrick Huang wrote:
> Hi Jun,
>
> Sure. I already updated the KIP. Thanks!
>
> Best,
> Zhanxiang (Patrick) Huang
>
> ----------
> *From:* Jun Rao
> *Sent:* Wed
Hi, Everyone,
The follow is the preview of a blog on Kafka supporting more partitions.
https://drive.google.com/file/d/122TK0oCoforc2cBWfW_yaEBjTMoX6yMt
Please let me know if you have any comments by Tuesday.
Thanks,
Jun
,
> Harsha
> On Nov 2, 2018, 11:50 AM -0700, Jun Rao , wrote:
> > Hi, Everyone,
> >
> > The follow is the preview of a blog on Kafka supporting more partitions.
> >
> > https://drive.google.com/file/d/122TK0oCoforc2cBWfW_yaEBjTMoX6yMt
> >
> > Please let me know if you have any comments by Tuesday.
> >
> > Thanks,
> >
> > Jun
>
and performance?
> > It looks like we are suggesting with 200k partitions and 4k per broker
> max
> > size of a cluster should be around 50 brokers?
> >
> > Thanks,
> > Harsha
> > On Nov 2, 2018, 11:50 AM -0700, Jun Rao , wrote:
> > > Hi, Everyone
Hi, Mani,
Thanks for running the release. Verified quickstart on 2.12 binary. +1
Jun
On Thu, Oct 25, 2018 at 7:28 PM, Manikumar
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 2.0.1.
>
> This is a bug fix release closing
Hi, Yulin,
I assume the performance issue that you mentioned is for writes instead of
reads. By default, Kafka flushes data to disks asynchronously in batches.
So, even when there are multiple files to write, the batching can amortize
the HDD seek overhead. It would be useful to understand the num
The blog is now published to
https://blogs.apache.org/kafka/entry/apache-kafka-supports-more-partitions
Thanks,
Jun
On Fri, Nov 2, 2018 at 2:36 PM, Jun Rao wrote:
> Hi, Mayuresh,
>
> Most of the controlled shutdown time is in leader election. The controller
> currently does
Hi, Stanislav,
Thanks for the KIP. +1. I guess this only covers the Kafka metrics (not the
Yammer metrics). It would be useful to make this clear.
Jun
On Tue, Nov 6, 2018 at 1:00 AM, Stanislav Kozlovski
wrote:
> Hey everybody,
>
> I'm starting a vote thread on KIP-386: Standardize on Min/Avg/M
Thanks for running the release, Mani!
Jun
On Fri, Nov 9, 2018 at 11:42 AM, Manikumar wrote:
> The Apache Kafka community is pleased to announce the release for Apache
> Kafka 2.0.1
>
> This is a bug fix release and it includes fixes and improvements from 51
> JIRAs.
>
> All of the changes in th
Hi, Jason,
Thanks for the KIP. Looks good overall. A few minor comments below.
1. The section on handling FETCH_OFFSET_TOO_LARGE error says "Use the
OffsetForLeaderEpoch API to verify the current position with the leader".
The OffsetForLeaderEpoch request returns log end offset if the request
lea
Hi, Everyone,
We have two upcoming Kafka Summits, one in NYC and another in London. The
deadline for summiting proposals is Dec 20 for both events. Please consider
submitting a proposal if you are interested. The Links to submit abstracts
are
Kafka Summit NYC - https://myeventi.events/kafka19/ny/
Hi, Ryanne,
Thanks for the KIP. At the high level, this looks like a reasonable
proposal. A few comments below.
1. About using a single connector cluster to manage connectors accessing
multiple Kafka clusters. It's good that you brought this up. The following
are the tradeoffs that I see. The be
Hi, Konstandin,
Thanks for the KIP. I agree with Magnus on the protocol version changes. As
for the sentinel value, currently ConsumerRecord.NO_TIMESTAMP (-1) is used
for V0 message format. For compatibility, it seems that we still need to
preserve that.
Jun
On Thu, Dec 6, 2018 at 2:32 AM Magnus
t;
> Sort of. I'm using this to construct an AdminClient for topic ACL and
> configuration sync, since the Connect framework doesn't expose it. I intend
> to follow-up KIP-382 with a proposal to expose this info to Connectors.
> There's also KIP-158, but it deals with top
ertise.
> >
> > Ryanne's main concern with this approach was that there are additional
> > worker setting that apply to all connectors and that no truly universal
> > approach would be feasible while running a single worker per Connect
> node.
> > A
Hi, Everyone,
This is a reminder about the deadline for proposal this Thursday.
Thanks,
Jun
On Tue, Dec 4, 2018 at 1:49 PM Jun Rao wrote:
> Hi, Everyone,
>
> We have two upcoming Kafka Summits, one in NYC and another in London. The
> deadline for summiting proposals is Dec 20 for
ingful
> semantics now, and hence its protocol version would need bump as we change
> its field. And I think a single byte indicating the type as (EARLEST,
> LATEST, ACTUAL_TIMESTAMP_VALUE) should be sufficient, but I'll leave it to
> Konstandin to decide if he wants to do this KIP o
Is as Sönke suggested.
>
> What do you think?
>
> Ryanne
>
>
> On Mon, Dec 17, 2018 at 3:40 PM Jun Rao wrote:
>
>> Hi, Sonke, Ryanne,
>>
>> Thanks for the explanation. To me, the single connect cluster model could
>> be useful for any connector,
Hi, Ryanne,
Thanks for KIP. Still have a few more comments below.
100. "This is not possible with MirrorMaker today -- records would be
replicated back and forth indefinitely, and the topics in either cluster
would be merged inconsistently between clusters. " This is not 100% true
since MM can d
Hi, Radwa.
Thanks for your interest. Added you to the contributors list.
Jun
On Mon, Jan 7, 2019 at 1:49 PM Radwa Osama wrote:
> Hello All,
>
> Would you please add me to contributors list?
>
> My jira username is: RO86
>
> Thanks
> Radwa
>
> ConnectDistributed, but which runs multiple DistributedHerders. Each
> DistributedHerder uses a different Kafka cluster for coordination -- they
> are completely separate apart from running in the same process.
>
> Thanks for helping improve the doc!
> Ryanne
>
> On Fri,
> prefixes, because you could be migrating from e.g. A's C.topic1 to B's
> C.topic1, i.e. migrating "laterally", if you will.
>
> I suppose we could break this out into multiple methods (upstream,
> downstream, lateral etc), but I think that would add a lot more complex
has
> both
> >> a
> >> > > producer and a consumer (e.g., Kafka streams) produces a record,
> then
> >> > > attempts to consume it back and the consume() comes back with
> "record
> >> > does
> >> > > not exist"? That's fi
Hi, Taras,
Thanks for your interest. Just gave you the wiki permission.
Jun
On Tue, Jan 15, 2019 at 9:47 AM Taras Danylchuk
wrote:
> Hello,
>
> I would like to have a permission to create KIP (
>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementPr
Congratulations, Vahid.
Thanks,
Jun
On Tue, Jan 15, 2019 at 2:45 PM Jason Gustafson wrote:
> Hi All,
>
> The PMC for Apache Kafka has invited Vahid Hashemian as a project
> committer and
> we are
> pleased to announce that he has accepted!
>
> Vahid has made numerous contributions to the Kafka
Hi, Everyone,
Just want to let you know that Kafka Summit San Francisco, 2018 is open for
submissions. The deadline for submission is Jun. 16. The conference itself
will be held on Oct. 16 - Oct. 17.
More details can be found at
https://kafka-summit.org/kafka-summit-san-francisco-2018/speakers/
Hi, Rajini,
Thanks for the KIP. +1
Could you document in the wiki how to
set ssl.endpoint.identification.algorithm to empty in the server property
file and through dynamic config? It's not obvious how to do that.
Jun
On Wed, May 9, 2018 at 8:28 AM, Rajini Sivaram
wrote:
> Hi all,
>
> Since th
Hi, Ron,
Thanks for the KIP. +1 from me. Just a few minor comments below.
1. It seems that we can translate an OAuth token to a principle name
through the claim name configured in JASS. However, it's not clear to me
how an OAuth token is mapped to a claim. Could you clarify that?
2. The wiki has
Hi, Mickael,
Thanks for the KIP. +1
Jun
On Wed, Apr 25, 2018 at 1:51 AM, Mickael Maison
wrote:
> Hi,
>
> There has been no objections in the DISCUSS thread so I'd like to
> start a vote on KIP-282:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 282%3A+Add+the+listener+name+to+the+a
and OAuthBearerValidatorCallback, respectively.
>
> Regarding the third question, yes, I see your point that the way the spec
> is worded could be taken to imply that the error code is a single
> character: "A single ASCII..." (
> https://tools.ietf.org/html/rfc6749#secti
sub' claim
> value in some way, etc. I write the callback handler code, so I complete
> flexibility to do whatever my OAuth 2 installation requires me to do.
>
> Ron
>
> On Tue, May 15, 2018 at 1:39 PM, Jun Rao wrote:
>
> > Hi, Ron,
> >
> > Thanks f
Hi, Lucas,
Thanks for the KIP. A few comments below.
1. As Eno mentioned in the discussion thread, I am wondering how much of
this is still an issue after KAFKA-5642. With that fix, the requests from
the controller to the brokers are batched in all the common cases. Have you
deployed Kafka 1.1? W
Hi, Matthias,
Thanks for running the release. Verified quickstart on scala 2.12 binary. +1
Jun
On Fri, Jun 22, 2018 at 6:43 PM, Matthias J. Sax
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the second candidate for release of Apache Kafka 0.10.2.2.
>
> Note, that RC
Hi, Matthias,
Thanks for running the release. Verified quickstart on scala 2.12 binary. +1
Jun
On Fri, Jun 22, 2018 at 3:14 PM, Matthias J. Sax
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 0.11.0.3.
>
> This is a bug
Hi, Dong,
Thanks for running the release. Verified quickstart on scala 2.12 binary. +1
Jun
On Thu, Jun 28, 2018 at 6:12 PM, Dong Lin wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the second candidate for release of Apache Kafka 1.1.1.
>
> Apache Kafka 1.1.1 is a bug
Hi, Matthias,
Thanks for the running the release. Verified quickstart on scala 2.12
binary. +1
Jun
On Fri, Jun 29, 2018 at 10:02 PM, Matthias J. Sax
wrote:
> Hello Kafka users, developers and client-developers,
>
> This is the second candidate for release of Apache Kafka 1.0.2.
>
> This is a b
Hi, Lucas, Dong,
If all disks on a broker are slow, one probably should just kill the
broker. In that case, this KIP may not help. If only one of the disks on a
broker is slow, one may want to fail that disk and move the leaders on that
disk to other brokers. In that case, being able to process th
Hi, Luis,
Thanks for the KIP. Overall, this seems a useful KIP. A few comments below.
1. I guess both new configurations will be at the topic level?
2. Since the log cleaner now needs to keep both the offset and another long
(say timestamp) in the de-dup map, it reduces the number of keys that we
e compaction would only affected the “tail”,
> while the “head” remained untouched. Now that you say its possible that the
> last message actually gets overwritten somehow, I guess a new bullet point
> will have to be added to the KIP for this (after I’ve found the time to
> review the po
"log.cleaner.delete.retention.ms", WDYT?
>
>
> Guozhang
>
>
> On Fri, Jul 6, 2018 at 11:51 AM, Jun Rao wrote:
>
> > Hi, Luis,
> >
> > 1. The cleaning policy is configurable at both global and topic level.
> The
> > global one has the nam
Hi, Jason,
Thanks for the KIP. Looks good overall. Just a few minor comments below.
1. "As the consumer is fetching from a partition, it will keep a small
cache of the recent epochs that were fetched for each partition. " Do we
need to cache more than one leader epoch? Also, during consumer failo
ues and the meaning of "Idle" still
> holds:
>
>
>
>
>
>
> *Before this KIPAfter this KIPNetworkProcessorAvgIdlePercentidle = blocked
> on selectnot idle includes being blocked on requestQueueidle = blocked on
> selectnot idle includes being blocked on either control
t; such as read-only partition or repartition epoch.
> >
> > Thanks,
> > Dong
> >
> > On Wed, Feb 7, 2018 at 5:28 PM, Jun Rao wrote:
> >
> >> Hi, Dong,
> >>
> >> Thanks for the reply. The general idea that you had for adding
> partitio
> Either approach we choose, it does not affect public interfaces or the
> write up in the KIP,
> and we can discuss further in the PR.
>
> Thanks,
> Lucas
>
>
>
> On Wed, Jul 11, 2018 at 8:46 AM, Jun Rao wrote:
>
> > Hi, Lucas,
> >
> > 2. Good p
> > > >> > > > > > testing scenario
> > > >> > > > > > > > >> > > > > > with a backlog of produce requests. In
> > > >> production,
> > > >> > > > other
> > > >
er side, with a single thread controller request handler, the
> logic should be:
> 1. Process what ever request seen in the controller request queue
> 2. For the given epoch, drop request if its correlation id is smaller than
> that of the last processed request.
>
> Thanks,
>
Hi, Stanislav,
Thanks for the KIP. +1
Just one minor comment. Since the JWT token supports customizable claim
fields, it would be useful to clarify when to use the SASL extension vs the
customized fields in JWT.
Jun
On Wed, Jul 25, 2018 at 10:03 AM, Stanislav Kozlovski <
stanis...@confluent.io>
Hi, Jason,
Thanks for the updated KIP. Well thought-through. Just a few minor comments
below.
50. For seek(TopicPartition partition, OffsetAndMetadata offset), I guess
under the cover, it will make OffsetsForLeaderEpoch request to determine if
the seeked offset is still valid before fetching? If
secret, we can decrypt and get the original values.
>
> Thank you,
>
> Rajini
>
> On Fri, Jan 5, 2018 at 12:11 AM, Jun Rao wrote:
>
> > Hi, Rajin,
> >
> > Does providing the old-secret help? My understanding is that the encoded
> > passwd is the re
on. The password configs encoded are
> > > keystore/truststore passwords and JAAS configuration. We need to be
> able
> > to
> > > extract the actual values for these, so we cannot use 1-way hash. So if
> > we
> > > have the old secret, we can decrypt and get the
t; > >
> > > > > On Thu, Jan 4, 2018 at 7:04 PM, Dong Lin
> > wrote:
> > > > >
> > > > > > Hey Jun, Jason,
> > > > > >
> > > > > > Thanks much for all the feedback. I have updated the KIP based on
> &
Hi, Jiangjie,
Sorry for the late response. The proposal sounds good overall. A couple of
minor comments below.
1. For throttling a fetch request, we could potentially just send an empty
response. We can return a throttle time calculated from a full response,
but only mute the channel on the serve
Hi, Ivan,
Thanks for the PR. Sorry for the delay. Just merged it.
Jun
On Wed, Jan 10, 2018 at 12:56 PM, Ivan Babrou wrote:
> What does it take to merge a documentation change?
>
> * https://github.com/apache/kafka/pull/3567
>
> Seeing zero activity is very discouraging.
>
> not very sure whether it is still worthwhile to have a per-partition
> partition_epoch if the metadata already has the global epoch.
>
>
> On Tue, Jan 9, 2018 at 6:58 PM, Dong Lin wrote:
>
> > Hey Jun,
> >
> > Thanks so much. These comments very useful. Please
ed the KIP to reflect the changes.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
>
> On Mon, Jan 8, 2018 at 6:30 PM, Jun Rao wrote:
>
> > Hi, Jiangjie,
> >
> > Sorry for the late response. The proposal sounds good overall. A couple
> of
> > minor comment
will honor the throttle time,
> they will back off until throttle time is reached. If that throttle time is
> longer than connection.max.idle.ms, it seems not a big overhead because
> there will only be one connection re-establishment in quite a few minutes.
> Compared with such overhea
Hi, Chris,
Thanks for your interest. Just added you to the contributor list.
Jun
On Tue, Jan 16, 2018 at 3:48 PM, Chris Egerton wrote:
> Hello Kafka dev community,
>
> I'd like to become an official Kafka contributor. I've only made one small
> patch to Kafka so far but would like to become mo
sponse.
>
> Can you take another look at the latest KIP?
>
> Thanks!
> Dong
>
>
>
> On Wed, Jan 10, 2018 at 2:24 PM, Jun Rao wrote:
>
> > Hi, Dong,
> >
> > My replies are the following.
> >
> > 60. What you described could also work. The drawb
sponse.
>
> Can you take another look at the latest KIP?
>
> Thanks!
> Dong
>
>
>
> On Wed, Jan 10, 2018 at 2:24 PM, Jun Rao wrote:
>
> > Hi, Dong,
> >
> > My replies are the following.
> >
> > 60. What you described could also work. The drawb
Hi, Mani,
Thanks for the KIP. Looks good to me overhead. Just a couple of minor
comments below.
1. Should hmac be of type ByteBuffer? We return hmac as byte[] in
DelegationToken.
So, it seems it's more consistent to pass in hmac as byte[] too.
2. Does describeDelegationToken() return all tokens?
> handle the broker failure between zookeeper writes for a topic creation. I
> have updated the KIP to use the suggested approach.
>
>
> On Wed, Jan 17, 2018 at 9:57 AM, Dong Lin wrote:
>
> > Hey Jun,
> >
> > Thanks much for the comments. Please see my comments i
Hi, Jorge,
Thanks for the KIP. Looks good to me overall. A few comments below.
1. It seems that ConsumerDescription should be MemberDescription?
2. Each offset can have an optional metadata. So, in
ListGroupOffsetsResult, perhaps it's better to have
KafkaFuture>, where
OffsetAndMetadata contains
hanks,
Jun
On Thu, Jan 18, 2018 at 6:30 PM, Dong Lin wrote:
> Hey Jun,
>
> I agree. I have updated the KIP to remove the class OffetEpoch and replace
> OffsetEpoch with byte[] in APIs that use it. Can you see if it looks good?
>
> Thanks!
> Dong
>
> On Thu, Jan 18,
Hi, Colin,
Thanks for the KIP. Looks good to me overall. Just a couple of more
comments.
1. As I mentioned earlier, it might be useful to add some metrics for
monitoring the usage of the session cache. For example, it would be useful
to know how many slots are being used (or unused), # of total p
2018 at 3:19 PM, Jorge Esteban Quilcate Otoya <
quilcate.jo...@gmail.com> wrote:
> Hi all,
>
> Thanks all for your votes and approving this KIP :)
>
> @Jun Rao:
>
> 1. Yes, KIP is updated with MemberDescription.
> 2. Changed:
> ```
> public class ListGroupOffset
> OK.
> >
> > > 4. I think the word "incremental" is redundant in the config names.
> Maybe
> > > it could just be "max.fetch.session.cache.slots" for example?
> >
> > What if we want to have fetch sessions for non-increme
Hi, Litao,
Thanks for your interest. Just gave you the wiki permission.
Jun
On Tue, Jan 23, 2018 at 9:33 AM, Litao Deng
wrote:
> Hey.
>
> I would like to add a KIP; however, I even can NOT see the 'Create' button
> on the header of the KIP home page.
>
> Could you please grant me the permissio
Hi, Rajini,
Thanks for the KIP. +1 from me.
Jun
On Thu, Jan 18, 2018 at 8:58 AM, tao xiao wrote:
> +1 (non-binding)
>
> On Fri, 19 Jan 2018 at 00:47 Rajini Sivaram
> wrote:
>
> > Hi all,
> >
> > I would like to restart the vote for KIP-86:
> >https://cwiki.apache.org/confluence/display/K
KIP-222? Is there something you would like me to add in this KIP?
> >
> > Thanks!
> > Dong
> >
> > On Fri, Jan 19, 2018 at 3:00 PM, Jun Rao wrote:
> >
> >> Hi, Dong,
> >>
> >> The issue with using just byte[] for OffsetEpoch is that it
Hi, Mani,
Thanks for the KIP. +1
Jun
On Sun, Jan 21, 2018 at 7:44 AM, Manikumar
wrote:
> Hi All,
>
> I would like to start a vote on KIP-249 which would add delegation token
> operations
> to Java Admin Client.
>
> We have merged DelegationToken API PR recently. We want to include admin
> clie
Hi, Litao,
Thanks for the KIP. Good proposal. A few comments below.
1. The KIP says "select the live replica with the largest LEO". I guess
what you meant is selecting the live replicas in ISR with the largest LEO?
2. I agree that we can probably just reuse the current min.isr
configuration, bu
tive count for rates since we added
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 187+-+Add+cumulative+count+metric+for+all+Kafka+rate+metrics>
>
> Oh, good point.
>
> C.
>
>
> > Ismael
> >
> > On Tue, Jan 23, 2018 at 4:21 PM, Col
Just some clarification on the current fencing logic. Currently, if the
producer uses acks=-1, a write will only succeed if the write is received
by all in-sync replicas (i.e., committed). This is true even when min.isr
is set since we first wait for a message to be committed and then check the
min
Hi, Dong,
Thanks for the KIP. It looks good overall. We are working on a separate KIP
for adding partitions while preserving the ordering guarantees. That may
require another flavor of partition epoch. It's not very clear whether that
partition epoch can be merged with the partition epoch in this
isting consumers, waiting for the committed offset to go beyond this
> offset for partition 3, and trigger rebalance again so that partition 3 can
> be reassigned to some consumer.
>
>
> Thanks,
> Dong
>
>
> On Tue, Feb 6, 2018 at 10:10 AM, Jun Rao wrote:
>
> > Hi, Do
Hi, Dong,
Thanks for the KIP. At the high level, this makes sense. A few comments
below.
1. It would be useful to support removing partitions as well. The general
idea could be bumping the leader epoch for the remaining partitions. For
the partitions to be removed, we can make them read-only and
, Feb 21, 2018 at 10:17 AM, Jun Rao wrote:
>
> > Hi, Dong,
> >
> > Thanks for the KIP. At the high level, this makes sense. A few comments
> > below.
> >
> > 1. It would be useful to support removing partitions as well. The general
> > idea could be b
Hi, Damian,
It would also be useful to include KAFKA-6111, which prevents
deleteLogDirEventNotifications
path to be deleted correctly from Zookeeper. The patch should be committed
later today.
Thanks,
Jun
On Thu, Mar 1, 2018 at 1:47 PM, Damian Guy wrote:
> Thanks Jason. Assuming the system te
KAFKA-6111 is now merged to 1.1 branch.
Thanks,
Jun
On Thu, Mar 1, 2018 at 2:50 PM, Jun Rao wrote:
> Hi, Damian,
>
> It would also be useful to include KAFKA-6111, which prevents
> deleteLogDirEventNotifications path to be deleted correctly from
> Zookeeper. The patch shou
linear hashing (or partition splitting) vs rehashing every key: The benefit
of the former is that it reduces the # of partitions to which keys from an
existing partition are re-distributed, which potentially reduces the
overhead of rebuilding the state in a consumer. The downside is that the
load m
Hi, Mattias,
Regarding your comment "If it would be time-delay based, it might be
problematic
for Kafka Streams: if we get the information that the new input partitions
are available for producing, we need to enable the new changelog partitions
for producing, too. If those would not be available y
t; scenario is actually two concurrent failures (both A and B are
> > > > considered
> > > > >>>> wedged), this is still a regression to the current version.
> > > > >>>>
> > > > >>>> So to resolve this issue, I&#x
Hi, Jan,
Thanks for the feedback. Just some comments on the earlier points that you
mentioned.
50. You brought up the question of whether existing data needs to be copied
during partition expansion. My understand of your view is that avoid
copying existing data will be more efficient, but it does
t; would be unclear when/if we can delete the old topic. Thus, it moves the
> "problem" into the application layer. It's hard to judge for me atm what
> the impact would be, but it's something we should pay attention to.
>
>
> -Matthias
>
> On 3/6/18 3:45 P
; https://cwiki.apache.org/confluence/display/SAMZA/SEP-5%3A+
> Enable+partition+expansion+of+input+streams
> https://issues.apache.org/jira/browse/SAMZA-1293
>
> To be clear. I think SEP-5 (state of last week, dont know if it adapted to
> this discussion) is on a way better path th
handled and you should be able to understand me
> and inch better.
>
> Please do not do A to the project, it deserves better!
>
> Best Jan
>
>
>
> On 13.03.2018 02:40, Jun Rao wrote:
>
>> Hi, Jan,
>>
>> Thanks for the reply. A few more comments bel
Hi, Jimin,
Thanks for your interest. Added you to the contributor list.
Jun
On Wed, Mar 14, 2018 at 3:17 AM, Jimin Hsieh
wrote:
> Hi,
>
> Would it be possible to add me to the contributor list? My jira username:
> JiminHsieh
>
> Thanks,
> Jimin Hsieh
>
Hi, Sirisha,
Thanks for your interest. Just added you to the contributor list.
Jun
On Wed, Mar 14, 2018 at 12:05 PM, Sirisha Sindiri wrote:
> Hi
>
> Could you please add me to the contributor list?
> My jira name is Sindiri.
>
> Thanks,
> Sirisha Sindiri
>
/SPARK-23685 to
> myself but I could not.
> Will you be able to help me out?
>
> Thanks,
> Sirisha
>
> On Wed, Mar 14, 2018 at 3:20 PM, Jun Rao wrote:
>
> > Hi, Sirisha,
> >
> > Thanks for your interest. Just added you to the contributor list.
> >
>
Hi, Rajini,
Thanks for the KIP. Looks good overall. A few comments below.
10. "If quota config is used, *user* tag is set to user principal of
the session and *client-id* tag is set to empty string. " Could we just
omit such a tag if the value is empty?
11. I think Viktor has a valid point on h
tePartitionMetadata`, we may need to update quota bound for one or
> > more existing metrics. I didn't want to expose metrics to the callback.
> So `
> > quotaLimit` was providing the new quotas corresponding to existing
> > metrics. But perhaps a neater way to do this is to re
ing consumers.
> (newer consumers with support for this). But with new topic i just meant
> the topic that now has a different partition count. Plenty of ways to
> handle that (versions, aliases)
>
> Hope I can further get my idea across.
>
> Best Jan
>
>
>
>
>
1 - 100 of 5922 matches
Mail list logo
