Bravo!
On Thu, Oct 11, 2018 at 1:48 PM Ismael Juma wrote:
> Congratulations Manikumar! Thanks for your continued contributions.
>
> Ismael
>
> On Thu, Oct 11, 2018 at 10:39 AM Jason Gustafson
> wrote:
>
> > Hi all,
> >
> > The PMC for Apache Kafka has invited Manikumar Reddy as a committer and
Hey y'all!
Please take a look at KIP-382:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0
Thanks for your feedback and support.
Ryanne
this.
>
> Excited to see the discussion on this one.
>
> Rhys
>
> > On Oct 15, 2018, at 9:16 AM, Ryanne Dolan wrote:
> >
> > Hey y'all!
> >
> > Please take a look at KIP-382:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0
> >
> > Thanks for your feedback and support.
> >
> > Ryanne
>
>
ne
> right?
>
> Other than that. It feels like the KIP has too much features where many
> of them are not really wanted and counter productive but I will just
> wait and see how the discussion goes.
>
> Best Jan
>
>
> On 15.10.2018 18:16, Ryanne Dolan wrote:
> >
; https://engineering.salesforce.com/open-sourcing-mirus-3ec2c8a38537). I
> strongly believe Mirrormaker itself needs an upgrade, so I'm not
> questioning that, but more on the technical side of the solution.
>
> Thanks
> Eno
>
> On Mon, Oct 15, 2018 at 11:19 PM Ryanne D
Steff,
> Guess people have used it, assuming that all records that have been
polled > at the time of callback to "commit", have also had their offsets
committed. > But that is not true.
(excerpt from KIP)
The documentation for SourceTask.commit() reads:
> Commit the offsets, up to the offsets t
Konstantine, thanks for the explanation, makes sense.
Ryanne
On Tue, Oct 16, 2018, 1:51 PM Konstantine Karantasis <
konstant...@confluent.io> wrote:
> Matthias, Ryanne, Rhys, Guozhang, thank you all for your comments!
>
> Ryanne, to try to address your specific comments, let me start by saying
>
"cycle detection" -- cyclical replication does not result in infinite
> > recursion.
>
> Oh - got it, it checks the entire prefix, which seems obvious to me in
> retrospect :)
>
> Rhys
>
>
> > On Oct 15, 2018, at 3:18 PM, Ryanne Dolan wrote:
> >
>
hange that commit() is given (via argument) a list/collection of
> the records for which it is a guarantee. Thats what my current fix does
> (see PR).
>
>
> On 16/10/2018 19.33, Ryanne Dolan wrote:
>
> Steff,
>
> > Guess people have used it, assuming that all reco
Zahari,
It sounds to me like this problem is due to Akka attempting to implement
additional backpressure on top of the Consumer API. I'd suggest they not do
that, and then this problem goes away.
Ryanne
On Wed, Oct 17, 2018 at 7:35 AM Zahari Dichev
wrote:
> Hi there,
>
> Are there any opinions
e same.
>
> Pretty sure without this _design choice_ you can skip on that exactly
> once already
>
> Best Jan
>
> On 16.10.2018 18:16, Ryanne Dolan wrote:
> > > But one big obstacle in this was
> > always that group coordination happened on the source clu
rly named, but I don't
think there's anything fundamentally missing from the API.
Ryanne
On Wed, Oct 17, 2018 at 10:24 AM Per Steffensen wrote:
> On 17/10/2018 16.43, Ryanne Dolan wrote:
> > I see, thanks.
> > On the other hand, the commitRecord() callback provides the f
hat made you to consider a
> new design from ground up.
>
> Thanks,
> Harsha
>
> On Wed, Oct 17, 2018, at 8:34 AM, Ryanne Dolan wrote:
> > Jan, these are two separate issues.
> >
> > 1) consumer coordination should not, ideally, involve unreliable or slow
> &g
st genuinely attempting to understand what
> can be done on our end to improve the Akka streams integration.. Thanks in
> advance :)
>
> Zahari
>
> On Wed, Oct 17, 2018 at 5:49 PM Ryanne Dolan
> wrote:
>
> > Zahari,
> >
> > It sounds to me like this problem is due
Zahari, that makes sense, thanks for reframing your question. I suspect
that pause/resume was not intended to be called at high frequency like
that, but I agree with you that the current behavior is needlessly
inefficient. I like your idea of making it configurable.
Ryanne
On Thu, Oct 18, 2018, 6
oing away from this another step.
>
> On 17.10.2018 17:34, Ryanne Dolan wrote:
> > Jan, these are two separate issues.
> >
> > 1) consumer coordination should not, ideally, involve unreliable or slow
> > connections. Naively, a KafkaSourceConnector would coordinate via the
livered successfully, which the present
API exposes.
That said, I'm not opposed to your proposed callbacks, and I agree that
commit() and commitRecord() are poorly named. I just don't believe the
present API is incorrect.
Ryanne
On Thu, Oct 18, 2018 at 7:04 AM Per Steffensen wrote:
ement, coordination etc. This also means that existing tooling,
dashboards etc that work with Connectors do not work with uReplicator, and
any future tooling would need to treat uReplicator as a special case.
Ryanne
On Wed, Oct 17, 2018 at 12:30 PM Ryanne Dolan wrote:
> Harsha, yes I can
Thanks Harsha. Done.
On Fri, Oct 19, 2018 at 1:03 AM Harsha Chintalapani wrote:
> Ryanne,
>Makes sense. Can you please add this under rejected alternatives so
> that everyone has context on why it wasn’t picked.
>
> Thanks,
> Harsha
> On Oct 18, 2018, 8:02 A
g.
> MM2="cluster-name-foo") so in case another MM2 instance sees this message
> and it is configured to replicate data into "cluster-name-foo" it would
> just skip it instead of replicating it back.
>
> On Sat, Oct 20, 2018 at 5:48 AM Ryanne Dolan
> wro
Stupid question: do you have transforms=DateConvert as well?
On Fri, Nov 9, 2018, 9:00 PM sanjeev0915 wrote:
> Hi
>
> Please help for the below issue
>
> i am using Kafka connect JDBC Connector and trying to pull the data from
> Oracle Database table. I am using Apache Kafka (not the confluent).
t; or off, to fit for different use cases.
>
> Interesting I was going to support topic messages with extra headers of
> source DC info, for cycle detection…..
>
> Looking forward your reply.
>
> Regards,
>
> Dan
> On 2018/10/23 19:56:02, Ryanne Dolan wrote:
> &
2
You can specify topic whitelists and other connector-level settings here
too, or you can use the REST API to remote-control a running cluster.
I've also updated the KIP with minor changes to bring it in line with the
current implementation.
Looking forward to your feedback, thanks!
Ryanne
O
gt; > > > > I'd have the same questions if e.g. transformations could be
> ignored
> > or
> > > > > overridden by connectors or if there were multiple places to
> specify
> > > what
> > > > > serde to use.
> > > >
>
gt; run an entire separate Connect cluster - I'd much prefer an option to use a
> regular connect cluster from an ops point of view. Is it maybe worth
> spending some time investigating whether we can come up with a change to
> connect that enables what MM would need?
>
>
I think if would be far
> less implementation and maintenance effort.
>
> But again, all of that is based on my, potentially flawed, understanding of
> your proposal, please feel free to correct me :)
>
> Best regards,
> Sönke
>
> On Fri, Nov 30, 2018 at 1:39 AM Ryanne Dolan
sagree, I think
> to a certain extent this is down to a question of personal
> style/preference. And as this is your baby and you have put a lot more
> effort and thought into it than I ever will I'll shut up now :)
>
> Again, thanks for all your good work!
>
> Best regards,
his before, maybe
> you'd like to share the burden :)
>
> Best regards,
> Sönke
>
> On Wed, Dec 5, 2018 at 5:15 AM Ryanne Dolan wrote:
>
> > Sönke,
> >
> > I think so long as we can keep the differences at a very high level (i.e.
> > the &quo
ndlers, today
> people can add handlers to have a little extra custom logic if needed, and
> the handler api is public today so should be supported going forwards so
> people are not on mass re-writing these.
>
> On 12/5/18, 5:36 PM, "Ryanne Dolan" wrote:
>
>
lusterUtils. Since this is part of the public interface, could
> you document the public APIs?
>
> 4. source.cluster.bootstrap.servers/target.cluster.bootstrap.servers: Does
> a Source/Sink connect need both? Currently, the producer URL used in a
> SourceWorker always comes from the W
t;
> On Tue, Dec 11, 2018 at 5:32 AM Michael Pearce
> wrote:
>
> > So this is indeed what using headers with hops avoids is creating lots
> and
> > lots of topics __, so you can have more complex topology setups.
> >
> > I ask why not support having two ways
setups.
> >
> > I ask why not support having two ways of setting up and closing the door?
> >
> > One based on hops using headers, and another based on topic naming. After
> > all flexibility is what we want its for end users how to use right?
> >
> >
&
d also
> the compaction issues where transactional processing is master-master in
> regions, where the processing is sticky to region but of failure or
> planned, processing of certain accounts move regions.
>
> Also I ask that you keep compatibility of the handler api interface in MM
&
u see update A1, A2, B1, A3, B2, E1, E2 and in cluster Y you
> see B1,B2, A1,E1, A2, A3, E2 as the ordering by of the updates account is
> preserved.
>
> With the topic solution your suggesting we would have no way true way of
> replaying and re-constituting the order between X.account
t;
> > Regarding the single connect cluster model, yes, the co-existence of a
> MM2
> > REST API and the nearly identical Connect API is one of my concerns.
> > Implementation wise, my understanding is that the producer URL in a
> > SourceTask is always obtained from the connect
>
> Jun
>
> On Fri, Dec 14, 2018 at 1:25 PM Ryanne Dolan
> wrote:
>
> > Thanks Sönke, you're spot-on. I don't want MM2 to wait for Connect
> features
> > that don't exist yet, especially if MM2 is the primary use case for them.
> > Moreover, I t
heckpoint. Because the consumer may commit at arbitrary offset.
> Does the connector need to keep a mapping between each source offset to
> destination offset? If so how would that be done?
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Thu, Dec 13, 2018 at 8:23 AM Ryanne D
ed upon.
>
> Not sure if that has any actual impact on MM2 but wanted to at least
> mention it.
>
> Best regards,
> Sönke
>
> On Wed, Dec 19, 2018 at 11:00 PM Ryanne Dolan
> wrote:
>
> > Becket, thanks for taking a look.
> >
> > > 1. Only relying on t
Hey y'all, please vote to adopt KIP-382 by replying +1 to this thread.
For your reference, here are the highlights of the proposal:
- Leverages the Kafka Connect framework and ecosystem.
- Includes both source and sink connectors.
- Includes a high-level driver that manages connectors in a dedica
t;
> > > +1
> > >
> > > This looks like a huge project! Wikimedia would be very excited to have
> > > this. Thanks!
> > >
> > > On Thu, Dec 20, 2018 at 9:52 AM Ryanne Dolan
> > > wrote:
> > >
> >
Jun, let's leave the REST API out of the KIP then.
I have been arguing that Connect wouldn't benefit from the
multi-cluster/herder/worker features we need in MM2, and that the effort
would result in a needlessly complex Connect REST API. But certainly two
separate APIs is inherently more complex t
ulti-cluster/herder/worker features" in a
> different KIP at some time? If yes, please feel free to let me know if
> I can provide any help on that front. Otherwise, I am also happy to
> draft a proposal as basis for discussion.
>
> Best regards,
> Sönke
>
> On Fr
source offset that is
> closest but no larger than the committed source offset. (e.g. committed
> offsets 150 will be mapped to the entry (99 -> 199))
> 2. Add a offsets difference because we know since that entry the offsets
> are increasing one by one. (target offsets = 199 + (150 -99) =
original topic. (BTW, I
> assume that MM2 will mirror the timestamps from source to target without
> being changed)
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Thu, Dec 27, 2018 at 1:16 AM Ryanne Dolan
> wrote:
>
> > Becket, this is great feedback, thanks.
> &g
t; +1 (binding). Nice work Ryan.
> > >>> -Harsha
> > >>>
> > >>> On Fri, Dec 21, 2018, at 8:14 AM, Andrew Schofield wrote:
> > >>>> +1 (non-binding)
> > >>>>
> > >>>> Andrew Schofiel
we define new settings, but is an existing MM1
> config guaranteed to continue working? Does that include custom extensions
> like MessageHandlers? I'm not sure I entirely understand the compatibility
> story here (which could also be that we just don't provide one -- just want
>
I'd also prefer to avoid the new configuration property if possible. Seems
like a lighter touch without it.
Ryanne
On Sun, Jan 6, 2019 at 7:25 PM Paul Davidson
wrote:
> Hi Konstantine,
>
> Thanks for your feedback! I think my reply to Ewen covers most of your
> points, and I mostly agree. If
ker.sh is
> implemented? My understanding is that it will start as many as
> separate DistributedHerder as the Kafka clusters? So, essentially it's
> running multiple logical connect clusters on the same shared worker nodes?
>
> Thanks,
>
> Jun
>
>
> On Thu, Dec 20
> To differentiate between the normal Kafka config settings and the
application's log level settings, we will introduce a new resource type -
BROKER_LOGGERS
Stanislav, can you explain why log level wouldn't be a "normal Kafka config
setting"?
Ryanne
On Tue, Jan 8, 2019, 4:26 PM Stanislav Kozlovs
steps, we don't need to know the
> targetClusterAlias B. We just need to know the connection string to
> cluster B, which targetConsumerConfig provides.
>
> 107. Thanks. Could you add that description to the KIP?
>
> Thanks,
>
> Jun
>
> On Mon, Jan 7, 2019 at 3:50 PM Ry
two eases this KIP's
> > implementation and user's implementation of AlterConfigPolicy (e.g deny
> all
> > requests that try to alter log level) significantly. We would also be
> able
> > to introduce a
> >
> > On Wed, Jan 9, 2019 at 1:48 AM Ryanne Dolan
't see this necessarily as a blocker for this KIP. So I am fine
> if people think we should defer the improvements of cross-cluster failover
> to a later decision.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Tue, Jan 8, 2019 at 2:23 AM Ryanne Dolan wrote:
>
Hey y'all,
Please review the following small KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-414%3A+Notify+SourceTask+of+ACK%27d+offsets%2C+metadata
Thanks!
Ryanne
+1 thx
On Mon, Jan 14, 2019, 8:06 AM Gwen Shapira +1. Thanks, that will be very helpful.
>
> On Mon, Jan 14, 2019, 4:43 AM Alex D
> > Hello guys,
> >
> > We need your VOTES for the KIP-379: Multiple Consumer Group Management.
> >
> > KIP-379:
> >
> >
> https://cwiki.apache.org/confluence/display
Thanks Dong.
> 1. Currently if there is topic created with "." in the topic name, would
it cause correctness issue for this KIP?
Yes, RemoteClusterUtils would be confused by existing topics that have a
period, and MM2 might try to send records to existing topics if they happen
to be prefixed with
I had to change the KIP number (concurrency is hard!) so the link is now:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-416%3A+Notify+SourceTask+of+ACK%27d+offsets%2C+metadata
Ryanne
On Fri, Jan 11, 2019 at 2:43 PM Ryanne Dolan wrote:
> Hey y'all,
>
> Please review the f
Hey y'all, please vote for KIP-416 by replying +1 to this thread.
Right now, there is no way for a SourceConnector/Task to know:
- whether a record was successfully sent to Kafka, vs filtered out or
skipped.
- the downstream offsets and metadata of sent records
KIP-416 proposes adding a recordLo
Andrew, do we know whether the SourceTask may be start()ed again? If this
is the last call to a SourceTask I suggest we call it close(). I can't tell
from the documentation.
Also, do we need this if a SourceTask can keep track of whether it was
start()ed since the last stop()?
Ryanne
On Fri, Ja
Randall,
I have some concerns with this proposal.
Firstly, I don't believe it is the job of a connector to configure topics,
generally, nor for topic-specific settings to hang out in connector
configurations. Automatic creation of topics with default settings is an
established pattern elsewhere,
> single partition to maintain consistency) and if there is a chance the
> connector preference won't be used, connectors will have to force it via
> admin client which brings us back to the terrible config situation we
> currently have with Admin client.
>
> Gwen
>
>
> On T
c.
>
> I still think that the proposed KIP provides a simple way for most source
> connector users to control (via configuration) the settings of the topics
> that will be created by Connect for that connector, which works with all
> existing source connectors out of the box and
I would like to create some KIPs.
JIRA user: ryannedolan
wiki user: ryannedolan
Ryanne Dolan
ryannedo...@gmail.com
Thanks!
Ryanne
thanks!!
On Sun, Sep 30, 2018, 1:23 PM Matthias J. Sax wrote:
> Done
>
> On 9/27/18 11:51 AM, Ryanne Dolan wrote:
> > I would like to create some KIPs.
> >
> > JIRA user: ryannedolan
> > wiki user: ryannedolan
> > Ryanne Dolan
> > ryannedo...@gmail.com
> >
> > Thanks!
> > Ryanne
> >
>
>
Konstantine, this is exciting work! Couple questions:
I understand that, overall, rebalances would require less work and less
time acquiring and releasing resources. But OTOH I believe individual
consumers might see successive revokes/assigns while a rebalance settles.
Is that right? And if so, wo
+1 (non-binding)
But I suggest:
- drop "get" from getOffset, getTimestamp.
- add to the motivation section why this is better than constructing a
KafkaConsumer and using seek(), commit() etc.
- add some rejected alternatives.
Ryanne
On Mon, Jan 21, 2019, 7:57 AM Dongjin Lee We have +4 non-
r in intent to commitRecord() in my view.
>
> Just my 2 cents.
>
> Andrew Schofield
> IBM Event Streams
>
>
> On 17/01/2019, 23:54, "Ryanne Dolan" wrote:
>
> Hey y'all, please vote for KIP-416 by replying +1 to this thread.
>
> Right
Andrew Schofield suggested we overload the commitRecord method instead of
adding a new one. Thoughts?
Ryanne
On Thu, Jan 17, 2019, 5:34 PM Ryanne Dolan I had to change the KIP number (concurrency is hard!) so the link is now:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/K
oposal is something we can all agree to. Let's start simple and learn
> from our users whether or not they need more flexibility and control.
>
> Please respond with your thoughts. Thanks!
>
> Best regards,
>
> Randall
>
> On Tue, Nov 27, 2018 at 7:36 PM Ryanne Dolan
>
>
> Thanks for the KIP and patient discussion. +1 from me as well.
>
> Jiangjie (Becket) Qin
>
> On Fri, Jan 11, 2019 at 1:11 AM Jun Rao wrote:
>>
>> Hi, Ryanne,
>>
>> Thanks for the explanation. All make sense to me now. +1 on the KIP from me.
>&g
+1 non-binding, thanks!
Ryanne
On Tue, Jan 22, 2019 at 11:38 AM te...@confluent.io wrote:
>
> Hi all,
>
> We would like to start vote on KIP-421 to to enhance the AbstractConfig base
> class to support replacing variables in configurations just prior to parsing
> and validation.
>
> Link for t
sk that
> it's actually finished. One of the purposes of the KC framework is to make
> life easy for a connector developer and a nice clean "all done now" method
> would help.
>
> I think I'll add a diagram to illustrate to the KIP.
>
> Andrew Schofi
em is that stop() is a signal to stop polling. It's basically a
>> request from the framework to the task and it doesn't tell the task that
>> it's actually finished. One of the purposes of the KC framework is to make
>> life easy for a connector developer and a n
Pippin, thanks for your interest. I will publish a PR soon (several
days?) which you'll be able to build and play with. Watch this space
:)
Ryanne
On Thu, Jan 24, 2019 at 5:19 PM Pippin Wallace wrote:
>
> I see that the Current state of KIP-382 recently changed from Voting to
> Accepted on Conf
I've updated the KIP and PR to overload commitRecord instead of adding a
new method. Here's the PR:
https://github.com/apache/kafka/pull/6171
Ryanne
On Mon, Jan 21, 2019 at 6:29 PM Ryanne Dolan wrote:
> Andrew Schofield suggested we overload the commitRecord method instead of
&
ethod is only
> called when a record is ACKed and I would have thought that the connector
> implementor would want to provide only a single variant of commitRecord().
>
> Andrew Schofield
> IBM Event Streams
>
> On 31/01/2019, 03:00, "Ryanne Dolan" wrote:
>
>
Harsha, Sriharsha, Suresh, a couple thoughts:
- How could this be used to leverage fast key-value stores, e.g. Couchbase,
which can serve individual records but maybe not entire segments? Or is the
idea to only support writing and fetching entire segments? Would it make
sense to support both?
- I
> I suspect less broker logic would change in that case -- the broker
> wouldn't necessarily care if it reads from file:// or s3:// to load a given
> segment."
>
> Yes, this is what we are discussing in KIP. We are leaving the details of
> loading segments to RLM read part i
Main objective here not to change the existing protocol and still be able
> to write and read logs from remote storage.
>
>
> -Harsha
>
> On Feb 4, 2019, 2:53 PM -0800, Ryanne Dolan ,
> wrote:
> > Thanks Harsha, makes sense for the most part.
> >
> > > tie
5061), leaving out the
> >> group
> >> > ID
> >> > could lead to naming conflicts if multiple clusters run the same Kafka
> >> > cluster. This would probably not be a problem for many (including us)
> as
> >> > metrics exporters can usually be c
Hey y'all, I'm happy to announce that I've created a PR for MM2:
https://github.com/apache/kafka/pull/6295
Please take a look!
Ryanne
On Tue, Jan 22, 2019 at 11:43 AM Ryanne Dolan wrote:
> Thanks all, this is a large KIP and has sparked a lot of great
> discussion, bo
't use these variables would
> behave
> > as-is, but this way the users could define their own client IDs yet still
> > get the benefit of uniquely identifying each of the clients. For example,
> > if my worker config contained the following:
> >
> > producer.cl
Hey y'all, please consider the following small KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-438%3A+Expose+task%2C+connector+IDs+in+Connect+API
Thanks!
Ryanne
+1 (non-binding)
Thanks!
Ryanne
On Wed, Mar 6, 2019, 4:28 PM Konstantine Karantasis <
konstant...@confluent.io> wrote:
> I'd like to open the vote on KIP-415: Incremental Cooperative Rebalancing
> in Kafka Connect
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Coo
Hello! When using exactly-once semantics, uncommitted or aborted records
are skipped by the consumer as if they don't exist.
When inspecting the topic manually, use isolation.level=read_committed to
get the same behavior.
Ryanne
On Fri, Mar 15, 2019, 6:08 AM Федор Чернилин
wrote:
> I also noti
gn.
Ryanne
On Fri, Mar 15, 2019 at 10:36 AM Федор Чернилин
wrote:
> Thanks, I understand how consumer works. The main question is related to
> why the join did not work and how it happened that only one message
> remained uncommitted.
>
> пт, 15 мар. 2019 г. в 16:29, Ryanne D
Maybe mark it abandoned?
fwiw, I think both KIPs break the worker-vs-connector boundary that, for
better or worse, is fundamental to Connect's architecture and APIs.
An alternative came up a few times during discussions for KIP-382: support
multiple "herders" within the same Connect cluster. Each
ouple of basic questions.
> Am I getting that right that you basically want to expose the task id from
> ConnectorTaskId? And if so, then I guess you'll provide the implementation
> too?
>
> Thanks,
> Viktor
>
> On Tue, Mar 5, 2019 at 6:49 PM Ryanne Dolan wrote:
Jason, awesome KIP.
I'm wondering how this change would affect availability of the cluster when
a rack is unreachable. Is there a scenario where availability is improved
or impaired due to the proposed changes?
Ryanne
On Tue, Mar 19, 2019 at 4:32 PM Jason Gustafson wrote:
> Hi Jun,
>
> Yes, th
efer to wait some time for it
> to be restored before routing traffic elsewhere. Does that make sense?
>
> Best,
> Jason
>
> On Tue, Mar 19, 2019 at 3:43 PM Ryanne Dolan
> wrote:
>
> > Jason, awesome KIP.
> >
> > I'm wondering how this change would a
ection.policy` config for example could provide an
> additional option to implement the policy you are suggesting or perhaps
> even something custom. I will mention this in the KIP and leave this open
> as a potential future improvement. As it stands, users can still choose to
> keep cu
pretty
> closely with your suggestion. Can you review and confirm?
>
> Best regards,
>
> Randall
>
> On Fri, Mar 1, 2019 at 3:04 PM Ryanne Dolan wrote:
>
> > Paul, Randall, I don't think most people will care to exercise so much
> > control over the cl
+1 (non-binding)
Great stuff, thanks.
Ryanne
On Mon, Mar 25, 2019, 11:08 AM Jason Gustafson wrote:
> Hi All, discussion on the KIP seems to have died down, so I'd like to go
> ahead and start a vote. Here is a link to the KIP:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Al
Hey Paul, thanks for the kind words re MM2.
I'm not a Streams expert first off, but I think I understand your question:
if a Streams app can switch between topics with and without a cluster alias
prefix when you migrate between prod and pre-prod, while preserving state.
Streams supports regexes an
> > > > > user A does not have access to a Kafka topic, when that topic is
> > moved
> > > > to
> > > > > > S3 or HDFS there needs to be a way to prevent access to the S3
> > bucket
> > > > for
> > > > > > that us
oday it's up to the operator to make sure the HD space is
> monitored and take necessary actions to mitigate that before it becomes
> fatal failure for broker. We don't stop users to configure the retention
> period to infinite and they can easily run out of the space.
>
> Th
>> explain.)
> >>
> >> Looking forward to a vote.
> >>
> >> Best regards,
> >>
> >> Randall
> >>
> >> On Wed, Apr 3, 2019 at 6:49 PM pdavidson >> .invalid>
> >> wrote:
> >>
> &g
+1 (non binding)
Thanks
Ryanne
On Fri, Apr 12, 2019, 11:11 AM Paul Davidson
wrote:
> Just a reminder that KIP-411 is open for voting. No votes received yet!
>
> On Fri, Apr 5, 2019 at 9:07 AM Paul Davidson
> wrote:
>
> > Hi all,
> >
> > Since we seem to have agreement in the discussion I would
Sönke, I'd find this very helpful. It's annoying to keep track of which
commands use which form -- I always seem to guess wrong.
Though I don't think there is any reason to deprecate existing forms, e.g.
consumer.config vs consumer-config. I think it's perfectly reasonable to
have multiple spellin
Is that a fair statement?
>- You mentioned the "stop()" interface can be called multiple times.
> Would the same thing be true for the proposed interface? Does it
> matter? Or
>there is a guard against that?
>- I also agree with Ryan th
Hey y'all, I'm happy to announce that the PR for "MirrorMaker 2.0" is ready
for review, after a long spell in "draft".
https://github.com/apache/kafka/pull/6295
MirrorMaker 2.0 is in the Kafka 2.3.0 release plan. Please take a look so
we can get this merged.
Also, shameless plug: I'm giving a ta
1 - 100 of 283 matches
Mail list logo
