Build failed in Jenkins: kafka_system_tests #112

2015-10-19 Thread ewen
See -- [...truncated 1703 lines...] test_id: 2015-10-19--001.kafkatest.tests.benchmark_test.Benchmark.test_produce

Jenkins build is back to normal : kafka_system_tests #113

2015-10-20 Thread ewen
See

Build failed in Jenkins: kafka_system_tests #116

2015-10-23 Thread ewen
See -- Started by timer Building in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson3879296377443603922.sh Running command: git pull && .

Jenkins build is back to normal : kafka_system_tests #117

2015-10-23 Thread ewen
See

Build failed in Jenkins: kafka_system_tests #118

2015-10-24 Thread ewen
See -- [...truncated 1671 lines...] test_id: 2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_produce

Jenkins build is back to normal : kafka_system_tests #119

2015-10-24 Thread ewen
See

Build failed in Jenkins: kafka_system_tests #121

2015-10-26 Thread ewen
See -- [...truncated 1671 lines...] test_id: 2015-10-26--001.kafkatest.tests.benchmark_test.Benchmark.test_produce

Jenkins build is back to normal : kafka_system_tests #122

2015-10-27 Thread ewen
See

Build failed in Jenkins: kafka_system_tests #125

2015-10-30 Thread ewen
See -- [...truncated 845 lines...] test_id: 2015-10-30--001.kafkatest.tests.replication_test.ReplicationTest.test_replication_with_broker_failure.interbroker_security_protocol=PLAINTEXT.failure_mo

Build failed in Jenkins: kafka_system_tests #126

2015-10-31 Thread ewen
See -- Started by timer Building on master in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson6104015082044430119.sh Running command: gi

Build failed in Jenkins: kafka_system_tests #127

2015-10-31 Thread ewen
See -- Started by user Geoff Building in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson4612275777450431607.sh Running command: git pull

Build failed in Jenkins: kafka_system_tests #130

2015-11-01 Thread ewen
See -- [...truncated 153 lines...] :copycat:api:processResources UP-TO-DATE :copycat:api:classes :copycat:api:copyDependantLibs :copycat:api:jar :copycat:file:compileJava :copycat:file:processResourc

Build failed in Jenkins: kafka_system_tests #136

2015-11-04 Thread ewen
See -- Started by timer Building on master in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson5709241350268370481.sh Running command: wh

Build failed in Jenkins: kafka_system_tests #137

2015-11-05 Thread ewen
See -- [...truncated 39 lines...] :contrib:clean :copycat:clean :core:clean :examples:clean :log4j-appender:clean :streams:clean :tools:clean :contrib:hadoop-consumer:clean :contrib:hadoop-producer:cl

Build failed in Jenkins: kafka_system_tests #140

2015-11-07 Thread ewen
See -- Started by timer Building on master in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson12500547725881.sh Running command: whi

Build failed in Jenkins: kafka_system_tests #141

2015-11-08 Thread ewen
See -- Started by timer Building on master in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson4524790496297318972.sh Running command: wh

Build failed in Jenkins: kafka_system_tests #142

2015-11-09 Thread ewen
See -- Started by user benstopford Building on master in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson8577356696053981129.sh Running

Build failed in Jenkins: kafka_system_tests #143

2015-11-09 Thread ewen
See -- Started by user Gwen (Chen) Shapira Building on master in workspace [kafka_system_tests] $ /bin/bash /tmp/hudson84854094863440935.sh Ru

Build failed in Jenkins: _kafka_system_tests #144

2015-11-09 Thread ewen
See -- Started by timer Building on master in workspace [_kafka_system_tests] $ /bin/bash /tmp/hudson3075256827416348305.sh /tmp/hudson30752

Compatibility between 0.9 and 0.8.x

2015-11-17 Thread Stephan Ewen
Hi! Sorry if this is question has been asked before, but I have not found a complete answer so far. I found that a 0.9.0 broker will be able to work with 0.8.x consumer clients. Does the opposite also hold? Will a new 0.9.0 consumer (new consumer API) be able to work with 0.8.x brokers? Thanks,

Re: Compatibility between 0.9 and 0.8.x

2015-11-17 Thread Stephan Ewen
/jira/browse/KAFKA-2845 > > best regards > > Stephan Ewen schrieb am Di., 17. Nov. 2015 um 11:13 > Uhr: > > > Hi! > > > > Sorry if this is question has been asked before, but I have not found a > > complete answer so far. > > > > I found that a

Re: [VOTE] 2.0.1 RC0

2018-11-02 Thread Ewen Cheslack-Postava
+1 -Ewen On Thu, Nov 1, 2018 at 10:10 AM Manikumar wrote: > We were waiting for the system test results. There were few failures: > KAFKA-7579, KAFKA-7559, KAFKA-7561 > they are not blockers for 2.0.1 release. We need more votes from > PMC/committers :) > > Thanks Stanisl

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

2018-11-27 Thread Ewen Cheslack-Postava
lines, is there actually a need for TopicSettings class? We already have NewTopic in the AdminClient APIs. Does that not suffice? -Ewen On Mon, Sep 24, 2018 at 11:56 AM Andrew Otto wrote: > FWIW, I’d find this feature useful. > > On Mon, Sep 24, 2018 at 2:42 PM Randall Hauch wrote: > > > R

Re: [DISCUSS] KIP-411: Add option to make Kafka Connect task client ID values unique

2019-01-04 Thread Ewen Cheslack-Postava
f interpolation were supported in the config value (as we now do with managed secrets). For example, it could support something like client.id=connector-${taskId} and the task ID would be substituted automatically into the string. I don't necessarily like that solution (seems complicated and n

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

2019-01-04 Thread Ewen Cheslack-Postava
rstand the compatibility story here (which could also be that we just don't provide one -- just want to make sure it is clear). I may have missed something in this proposal since it's pretty long, let me know if there was something obvious I overlooked. Thanks, -Ewen On Mon, Dec 31, 201

Re: [DISCUSS] KIP-285: Connect Rest Extension Plugin

2018-05-16 Thread Ewen Cheslack-Postava
r metrics. * Currently we have the State classes nested inside ConnectorHealth class. I think this makes those classes more annoying to use. Is there a reason for them to be nested or can we just pull them out to the same level as ConnectorHealth? -Ewen On Tue, May 15, 2018 at 9:30 AM Magesh Nandaku

Re: [DISCUSS] KIP-305: Add Connect primitive number converters

2018-05-17 Thread Ewen Cheslack-Postava
spend time and effort building and maintaining them. Otherwise, this seems simple and straightforward. Generally +1 on the proposal. -Ewen On Thu, May 17, 2018 at 6:04 PM Magesh Nandakumar wrote: > Thanks Randall for the KIP. I think it will be super useful and looks > pretty straightforw

Re: [VOTE] KIP-297: Externalizing Secrets for Connect Configurations

2018-05-17 Thread Ewen Cheslack-Postava
hink this is fine as I don't think there are truly common secrets provider formats/apis/protocols, just want to make sure it is clear. Thanks, Ewen On Thu, May 17, 2018 at 6:19 PM Ted Yu wrote: > +1 > Original message From: Magesh Nandakumar < > mage...@co

Re: [DISCUSS] KIP-285: Connect Rest Extension Plugin

2018-05-17 Thread Ewen Cheslack-Postava
Yup, thanks for the changes. The 'health' package in particular feels like a nice fit given the way we expect it to be used. -Ewen On Wed, May 16, 2018 at 7:02 PM Randall Hauch wrote: > Looks good to me. Thanks for quickly making the changes! Great work! > > Best regards, &

Re: [VOTE] KIP-285: Connect Rest Extension Plugin

2018-05-17 Thread Ewen Cheslack-Postava
+1 (binding) Thanks, Ewen On Thu, May 17, 2018 at 12:16 PM Ted Yu wrote: > +1 > Original message From: Gwen Shapira > Date: 5/17/18 12:02 PM (GMT-08:00) To: dev > Subject: Re: [VOTE] KIP-285: Connect Rest Extension Plugin > LGTM. +1. > > On Wed, Ma

Re: [DISCUSS] KIP-298: Error Handling in Connect

2018-05-17 Thread Ewen Cheslack-Postava
s -- instead of delivering the raw data, we potentially lose raw data & schema info because we're rendering it as JSON. Not sure that's a good idea... I think that last item might be the biggest concern to me -- DLQ formats and control over content & reprocessing seems a bit unclea

Re: [DISCUSS] KIP-305: Add Connect primitive number converters

2018-05-18 Thread Ewen Cheslack-Postava
ebody finds them useful for something if they exist. -Ewen On Fri, May 18, 2018 at 11:55 AM Randall Hauch wrote: > Thanks, Ewen. > > You make several good points, and I've updated the KIP to hopefully address > your comments. I think the symmetry with the Kafka serdes is useful

Re: [DISCUSS] KIP-298: Error Handling in Connect

2018-05-21 Thread Ewen Cheslack-Postava
olute value or something like a percentage? Given the current way of setting things, I'm not sure I'd ever set it to anything but -1 or 0, with maybe 1 as an easy option for restarting a connector to get it past one bad message, then reverting back to -1 or 0. -Ewen On Mon, May 21, 2018 at

Re: [VOTE] KIP-298: Error Handling in Connect kafka

2018-05-21 Thread Ewen Cheslack-Postava
+1 binding. I had one last comment in the DISCUSS thread, but not really a blocker. -Ewen On Mon, May 21, 2018 at 9:48 AM Matthias J. Sax wrote: > +1 (binding) > > > > On 5/21/18 9:30 AM, Randall Hauch wrote: > > Thanks, Arjun. +1 (non-binding) > > > > Regards

Re: [DISCUSS] KIP-298: Error Handling in Connect

2018-05-21 Thread Ewen Cheslack-Postava
On Mon, May 21, 2018 at 12:39 PM Arjun Satish wrote: > Thanks a lot, Ewen! I'll make sure the documentation is clear on the > differences between retries an tolerance. > > Do you think percentage would have the same problem as the one you brought > up? Also, if we say 10% t

Re: [DISCUSS] KIP-304: Connect runtime mode improvements for container platforms

2018-05-21 Thread Ewen Cheslack-Postava
ere fundamentally it does not fit into your k8s/mesos/container + CI/CD workflow to do anything other than pass configs in via command line flags? Or is it mostly the pain of too many topics? Or something else? -Ewen On Fri, May 18, 2018 at 4:47 AM Saulius Valatka wrote: > Hi, > > thanks

Re: [DISCUSS] KIP-305: Add Connect primitive number converters

2018-05-21 Thread Ewen Cheslack-Postava
Sorry, probably not ideal to just be seeing this now given KIP deadline, but in general straightforward ones where we expect things to be uncontroversial, I think its fine to kick off a vote thread. Worst case we have people re-file votes if something substantial changes in the proposal. -Ewen

Re: [VOTE] KIP-305: Add Connect primitive number converters

2018-05-22 Thread Ewen Cheslack-Postava
+1 (binding) -Ewen On Tue, May 22, 2018 at 9:29 AM Ted Yu wrote: > +1 > > On Tue, May 22, 2018 at 9:19 AM, Randall Hauch wrote: > > > I'd like to start a vote of a very straightforward proposal for Connect > to > > add converters for the basic primitive numb

Re: [EXTERNAL] Kafka Connect: New Kafka Source Connector

2018-05-22 Thread Ewen Cheslack-Postava
Sorry for the delay, didn't see this until now. I've given you edit permissions on the wiki. -Ewen On Tue, May 15, 2018 at 2:21 PM McCaig, Rhys wrote: > Hi Team, > > Would someone be able to provide me with Confluence permission in order to > write a KIP for the below

Re: [VOTE] KIP-176: Remove deprecated new-consumer option for tools

2018-05-25 Thread Ewen Cheslack-Postava
+1 (binding) Just follow up on the existing version of the KIP, so nothing new here. Possibly a bit disruptive given how quick the 1.0 -> 2.0 jump happened, but it's the right time to remove it. -Ewen On Thu, May 24, 2018 at 8:13 AM Viktor Somogyi wrote: > +1 (non-binding) > >

Re: [Vote] KIP-321: Update TopologyDescription to better represent Source and Sink Nodes

2018-07-31 Thread Ewen Cheslack-Postava
x27;t the methods on Source also be Optional types? -Ewen On Mon, Jul 30, 2018 at 11:13 PM Damian Guy wrote: > Hi Nishanth, > > I have one nit on the KIP. I think the topicNameExtractor method should > return Optional rather than null. > Sorry I'm late here. > > Than

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

2018-01-08 Thread Ewen Cheslack-Postava
value change out from under you. Having the defaults explicitly exposed can also be helpful when writing tests sometimes. -Ewen On Wed, Jan 3, 2018 at 9:30 AM, Colin McCabe wrote: > On Thu, Dec 21, 2017, at 10:28, Jason Gustafson wrote: > > Hey Matthias, > > > > Let me suggest a

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

2018-01-08 Thread Ewen Cheslack-Postava
On Mon, Jan 8, 2018 at 11:39 AM, Randall Hauch wrote: > Nice feedback, Ewen. Thanks! > > On Thu, Jan 4, 2018 at 5:11 PM, Ewen Cheslack-Postava > wrote: > > > Hey Jakub, > > > > Sorry for not getting to this sooner. Overall the proposal looks good to > &g

Re: [VOTE] KIP-174 Deprecate and remove internal converter configs in WorkerConfig

2018-01-08 Thread Ewen Cheslack-Postava
+1 binding. Thanks for the KIP! -Ewen On Mon, Jan 8, 2018 at 8:34 AM, Ted Yu wrote: > +1 > > On Mon, Jan 8, 2018 at 4:27 AM, UMESH CHAUDHARY > wrote: > > > Hello All, > > Since there are no outstanding comments on this, so I'd like to start a > >

Re: [DISCUSS] KIP-212: Enforce set of legal characters for connector names

2018-01-09 Thread Ewen Cheslack-Postava
think it would be good to add them here or in a subsequent KIP! -Ewen On Tue, Jan 9, 2018 at 1:30 PM, Colin McCabe wrote: > On Sat, Jan 6, 2018, at 16:00, Ewen Cheslack-Postava wrote: > > re: whitespace characters, I'm fine with the restriction since I don't > see > >

[DISCUSS] Release Plan for 1.0.1

2018-01-16 Thread Ewen Cheslack-Postava
nd a) make sure anything they think should go into 1.0.1 is marked as such, with the appropriate priority, and b) make sure anything marked for 1.0.1 should really be there. Once people have taken a pass we can work on a VOTE thread and getting any outstanding PRs reviewed. -Ewen

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

2018-01-16 Thread Ewen Cheslack-Postava
On Sun, Jan 14, 2018 at 1:20 PM, Jakub Scholz wrote: > Hi Ewen, > > Thanks for your comments / questions. > > re 1) I was using the valuesWithPrefixOverride(...) method from > AbstractConfig class. That takes the overrides setting by setting. It > should not be hard to c

Re: [DISCUSS] KIP-212: Enforce set of legal characters for connector names

2018-01-16 Thread Ewen Cheslack-Postava
ol characters). I'll start reviewing the PR, do you want to make any last updates about control characters in the KIP and kick off a VOTE thread? -Ewen On Fri, Jan 12, 2018 at 1:43 PM, Colin McCabe wrote: > On Fri, Jan 12, 2018, at 08:03, Sönke Liebau wrote: > > Hi everybody, >

Re: [DISCUSS] KIP-242: Mask password fields in Kafka Connect REST response

2018-01-16 Thread Ewen Cheslack-Postava
if we can get the KIP voted through. Thanks, Ewen On Fri, Jan 12, 2018 at 10:18 AM, Vincent Meng wrote: > @Ted: The issue is kinda hard to reproduce. It's just something we observe > over time. > > @Ewen: I agree. Opt-in seems to be a good solution to me. To your question, > if

Re: [VOTE] KIP-145: Expose Record Headers in Kafka Connect

2018-01-21 Thread Ewen Cheslack-Postava
+1 (binding) Thanks for the work on this -- not a small upgrade to the Connect APIs! -Ewen On Fri, Jan 19, 2018 at 3:37 PM, Randall Hauch wrote: > Hi everyone, > > I'd like to start the voting on this KIP to add support for headers in > Connect.: > > *https://cwik

Re: [DISCUSS] Release Plan for 1.0.1

2018-01-22 Thread Ewen Cheslack-Postava
s to get them either moved to a later release or upgraded to blockers with folks working on resolving them. We'll probably do an RC either late this week or early next week. Since this is a bug fix release, I am guessing it will only require 1 RC. -Ewen On Mon, Jan 22, 2018 at 11:23 AM, J

Re: [VOTE] KIP-212: Enforce set of legal characters for connector names

2018-01-23 Thread Ewen Cheslack-Postava
+1 (binding) On Tue, Jan 23, 2018 at 7:28 AM, Matt Farmer wrote: > +1 from me (non-binding) =) > > > On Jan 22, 2018, at 7:35 PM, Sönke Liebau > > > wrote: > > > > All, > > > > this KIP has been discussed for quite some time now and I believe we > > addressed all major concerns in the current

Re: [VOTE] KIP-206: Add support for UUID serialization and deserialization

2018-01-29 Thread Ewen Cheslack-Postava
+1 (binding) On Fri, Jan 26, 2018 at 9:16 AM, Colin McCabe wrote: > +1 (non-binding) > > > > On Fri, Jan 26, 2018, at 08:29, Ted Yu wrote: > > +1 > > > > On Fri, Jan 26, 2018 at 7:00 AM, Brandon Kirchner < > > brandon.kirch...@gmail.com> wrote: > > > > > Hi all, > > > > > > I would like to (re)s

[VOTE] 1.0.1 RC0

2018-02-05 Thread Ewen Cheslack-Postava
d from everyone. Thanks, Ewen

Re: [VOTE] 1.0.1 RC0

2018-02-05 Thread Ewen Cheslack-Postava
doesn't show up for too long or we need a follow up RC, I'll just extend/restart the process. -Ewen On Mon, Feb 5, 2018 at 8:21 PM, Ted Yu wrote: > Ewen: > Do you know how long it takes for maven repo to be populated ? > > Looking at: > https://repository.apache.org/con

Re: [VOTE] 1.0.1 RC0

2018-02-05 Thread Ewen Cheslack-Postava
. This requires "closing" two separate staging repos to get everything released. If things look good now, I can update the release script/instructions to make this clearer. -Ewen On Mon, Feb 5, 2018 at 9:59 PM, Ewen Cheslack-Postava wrote: > Not sure, seems to be variable. I "clo

Documentation build system

2018-02-06 Thread Ewen Cheslack-Postava
affects everyone contributing to the project, I figured I'd send this quick note to increase the likelihood people see it and have a chance to weigh in. Thanks, -Ewen

Re: [VOTE] 1.0.1 RC0

2018-02-09 Thread Ewen Cheslack-Postava
Just a heads up that we had a fix for KAFKA-6529 land to fix a file descriptor leak. So this RC is dead and I'll be generating RC1 soon. Thanks, Ewen On Wed, Feb 7, 2018 at 11:06 AM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Ewen, > > +1 > > Building f

[VOTE] 1.0.1 RC1

2018-02-12 Thread Ewen Cheslack-Postava
che.org/10/documentation.html * Protocol: http://kafka.apache.org/10/protocol.html Thanks, Ewen Cheslack-Postava

Re: [VOTE] 1.0.1 RC1

2018-02-12 Thread Ewen Cheslack-Postava
And of course I'm +1 since I've already done normal release validation before posting this. -Ewen On Mon, Feb 12, 2018 at 10:15 AM, Ewen Cheslack-Postava wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Ka

Re: [VOTE] 1.0.1 RC1

2018-02-20 Thread Ewen Cheslack-Postava
ter the PR is merged. -Ewen On Fri, Feb 16, 2018 at 3:53 AM, Mickael Maison wrote: > Ran tests from source and quickstart with binaries > > +1 (non-binding) > > On Fri, Feb 16, 2018 at 6:05 AM, Jason Gustafson > wrote: > > +1. Verified artifacts and ran quickstart. Than

[VOTE] 1.0.1 RC2

2018-02-21 Thread Ewen Cheslack-Postava
com/apache/kafka/tree/1.0.1-rc2 * Documentation: http://kafka.apache.org/10/documentation.html * Protocol: http://kafka.apache.org/10/protocol.html /** Thanks, Ewen Cheslack-Postava

Re: Question about developer documentation

2018-02-27 Thread Ewen Cheslack-Postava
The web page is more about general project info and might be of interest to people beyond just developers. But I agree the wiki landing page could use some updating. Even more than just the developer section as we're missing several releases, the oldest ones are listed at the top, etc. -Ewe

Re: Kafka 0.11.0.1 and filebeat 6.1 compatibility

2018-02-27 Thread Ewen Cheslack-Postava
d by folks from Elastic or the sarama project. -Ewen On Fri, Feb 9, 2018 at 7:11 AM, Sandeep Sarkar wrote: > Hi All, > > > > I am using filebeat 6.1 and kafka 0.11.0.1 to push logs. From filebeat > logs I could see that communication is getting established but then logs > are

Re: [VOTE] 1.0.1 RC2

2018-03-02 Thread Ewen Cheslack-Postava
Thanks everyone for voting. This passes with 3 binding +1, 5 non-binding +1, and no dissenting votes. I'll work on getting the release finalized and send out an announcement when it is ready. -Ewen On Tue, Feb 27, 2018 at 11:18 PM, Jason Gustafson wrote: > +1. Verified artifacts and

Re: [DISCUSS] KIP-186: Increase offsets retention default to 7 days

2018-03-05 Thread Ewen Cheslack-Postava
he vote thread so we can get this fixed for the next version. -Ewen On Fri, Oct 6, 2017 at 11:00 AM, Manikumar wrote: > looks like VOTE thread is *NOT* started for this KIP. > > On Fri, Oct 6, 2017 at 11:23 PM, Manikumar > wrote: > > > looks like VOTE thread is started

[VOTE] KIP-186: Increase offsets retention default to 7 days

2018-03-05 Thread Ewen Cheslack-Postava
ow up with those in subsequent KIPs, see the discussion thread for details. Also note this is related, but complementary, to https://cwiki.apache.org/confluence/display/KAFKA/KIP-211%3A+Revise+Expiration+Semantics+of+Consumer+Group+Offsets . And of course +1 (binding) from me. Thanks, Ewen

[ANNOUNCE] Apache Kafka 1.0.1 Released

2018-03-06 Thread Ewen Cheslack-Postava
Comar, Ewen Cheslack-Postava, Filipe Agapito, fredfp, Guozhang Wang, huxihx, Ismael Juma, Jason Gustafson, Jeremy Custenborder, Jiangjie (Becket) Qin, Joel Hamill, Konstantine Karantasis, lisa2lisa, Logan Buckley, Manjula K, Matthias J. Sax, Nick Chiu, parafiend, Rajini Sivaram, Randall Hauch, R

Re: Exposing additional metadata in Kafka Connect schema parameters

2018-03-06 Thread Ewen Cheslack-Postava
erters that fail to preserve that information. -Ewen On Tue, Mar 6, 2018 at 3:29 AM, Gunnar Morling wrote: > Hi, > > A user of the Debezium CDC Kafka Connect connectors has asked whether we > could provide information about the original source type of captured table > columns. >

Re: [VOTE] KIP-186: Increase offsets retention default to 7 days

2018-03-09 Thread Ewen Cheslack-Postava
Thanks for voting everyone, KIP is accepted with 5 binding +1 and 5 non-binding +1s. PR is already merged and there are upgrade notes about the potential memory impact. -Ewen On Wed, Mar 7, 2018 at 9:25 PM, Ted Yu wrote: > +1 > Original message From: Guozhang Wang >

Re: Seeking Feedback on Kafka Connect Issues

2018-03-19 Thread Ewen Cheslack-Postava
investigate. > Interesting. I think you're right that we're probably just reporting *assigned* tasks with the task-count metric rather than active tasks. I think an active tasks metric would be reasonable, though since you really need to look at the aggregate across workers, I'm not sure it's the best for alerting. Maybe an unhealthy/dead tasks count metric would be better? You can alert on that directly without having to aggregate across workers. -Ewen > > --- > > I would love thoughts on all of the above from anyone on this list. > > Thanks, > > Matt Farmer

Re: [DISCUSS] KIP-242: Mask password fields in Kafka Connect REST response

2018-03-19 Thread Ewen Cheslack-Postava
ST API, we'd have to address these issues anyway. -Ewen On Mon, Mar 19, 2018 at 2:33 PM, Matt Farmer wrote: > What’s the status of this? This is a pretty hard blocker for us to meet > requirements internally to deploy connect in a distributed fashion. > > @Ewen - Regarding

Re: Kafka Connect task re-balance repeatedly

2018-03-22 Thread Ewen Cheslack-Postava
y start running multiple Connect clusters but forget to make some of the default settings unique. I'd probably start by looking into that possibility to debug this issue. -Ewen On Wed, Mar 21, 2018 at 10:15 PM, Ziliang Chen wrote: > Hi, > > I have 2 Kafka Connect instances runs i

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

2018-03-23 Thread Ewen Cheslack-Postava
shouldn't revisit, just that it might also be a faster way to get to a full understanding of the options, concerns, and tradeoffs). -Ewen On Thu, Mar 22, 2018 at 7:19 AM, Richard Yu wrote: > I do have one question though: in the current KIP, throwing > TimeoutException to mark > that

Re: [VOTE] KIP-272: Add API version tag to broker's RequestsPerSec metric

2018-03-30 Thread Ewen Cheslack-Postava
+1 (binding) The incompatibility is unfortunate, but seems unlikely to cause a problem in practice. Let's just make sure there's a note in the upgrade notes about the incompatibility when we have a PR for this. -Ewen On Fri, Mar 30, 2018 at 10:22 AM, Jun Rao wrote: > Hi, Allen, &

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

2018-04-04 Thread Ewen Cheslack-Postava
enges in Connect don't appear here (e.g. conflicts in producer/consumer config naming, since this is a Consumer-only KIP), but similar problems arise. -Ewen On Wed, Apr 4, 2018 at 10:56 PM, Boyang Chen wrote: > Thanks Guozhang! I already updated the pull request and KIP to deprecate &g

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

2018-04-11 Thread Ewen Cheslack-Postava
On Thu, Apr 5, 2018 at 3:28 PM, Matthias J. Sax wrote: > Ewen, > > I cannot completely follow your argument. Can you elaborate a little > bit? After reading you mail, I am not sure if you prefer config > inheritance or not? And if, to what extend? > So we've had issues wi

Re: [DISCUSS] add connect-related packages to "What is considered a "major change" that needs a KIP?"

2018-08-07 Thread Ewen Cheslack-Postava
nnect modules are considered internal implementation details. -Ewen On Fri, Jul 27, 2018 at 7:15 PM Chia-Ping Tsai wrote: > hi Kafka > > There is a section[1] listing the packages which have public interfaces. > However, it doesn't include any connect-related packages. It would

Re: Apache Kafka project charter

2018-09-30 Thread Ewen Cheslack-Postava
n immediate compatibility commitment (compatibility across versions of AK broker & Connect, as well as upgrade & API & config compatibility for the connector itself. -Ewen On Sun, Sep 30, 2018 at 1:44 PM Matthias J. Sax wrote: > I am not aware of anything like this. And I also t

Re: Anyone interested in helping out a little brother Apache project with their Kafka integration?

2018-09-30 Thread Ewen Cheslack-Postava
ka Connect already has a couple of Connectors for. Could you clarify what format/serialization/use case PLC4X has? -Ewen On Wed, Aug 15, 2018 at 7:43 AM Christofer Dutz wrote: > Hi all, > > I am one of the Apache PLC4X (incubating) committers and am looking for > people willing to

Re: [DISCUSS] KIP-190: Handle client-ids consistently between clients and brokers

2017-09-22 Thread Ewen Cheslack-Postava
ened here recently I wanted to raise it here and make sure there would be agreement on this direction. (Long term I'd also like to get the required instantiation of JmxReporter removed as well, but that requires its own KIP.) Thanks, Ewen On Thu, Sep 14, 2017 at 2:09 AM, Tom Bentley wrote: >

Re: [DISCUSS] KIP-190: Handle client-ids consistently between clients and brokers

2017-09-25 Thread Ewen Cheslack-Postava
On Mon, Sep 25, 2017 at 3:35 AM, Mickael Maison wrote: > Hi Ewen, > > I understand your point of view and ideally we'd have one convention > for handling all user provided strings. This KIP reused the > sanitization mechanism we had in place for user principals. > > I

Re: [DISCUSS] KIP-190: Handle client-ids consistently between clients and brokers

2017-09-26 Thread Ewen Cheslack-Postava
up with another KIP to get all the metrics unsanitized when they hit the metrics reporter, but for Jmx sanitized as they are today when characters are encountered that Jmx cannot support. -Ewen On Tue, Sep 26, 2017 at 2:24 AM, Mickael Maison wrote: > Hi Ewen, > > By consistency, I meant

Re: [DISCUSS] KIP-215: Add topic regex support for Connect sinks

2017-10-26 Thread Ewen Cheslack-Postava
while Pattern is used, we only guarantee support for common regular expression syntax. Not sure if there's a good way of defining what "common" syntax is. Otherwise LGTM, and thanks for helping fill in a longstanding gap! -Ewen On Thu, Oct 26, 2017 at 7:56 PM, Ted Yu wrote: >

Re: Permission to create a KIP

2017-10-30 Thread Ewen Cheslack-Postava
Not sure if someone added you and just forgot to reply, but it looks like you already have permissions on the wiki. Just follow the process described here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals -Ewen On Thu, Oct 26, 2017 at 8:40 AM, Elizabeth Bennett

Re: [DISCUSS] KIP-215: Add topic regex support for Connect sinks

2017-10-30 Thread Ewen Cheslack-Postava
re I don't feel strongly about which to use since the error message is clear anyway and will just end up in logs / the REST API for the user to sort out. -Ewen On Fri, Oct 27, 2017 at 6:39 PM, Jeff Klukas wrote: > I've updated the KIP to use the topics.regex name and opened a WIP PR w

Re: [VOTE] KIP-215: Add topic regex support for Connect sinks

2017-11-03 Thread Ewen Cheslack-Postava
+1 binding Thanks Jeff! On Wed, Nov 1, 2017 at 5:21 PM, Randall Hauch wrote: > +1 (non-binding) > > Thanks for pushing this through. Great work! > > Randall Hauch > > On Wed, Nov 1, 2017 at 9:40 AM, Jeff Klukas wrote: > > > I haven't heard any additional concerns over the proposal, so I'd like

Re: [VOTE] KIP-215: Add topic regex support for Connect sinks

2017-11-03 Thread Ewen Cheslack-Postava
final discussion over to the PR! -Ewen On Fri, Nov 3, 2017 at 4:43 PM, Jeff Klukas wrote: > Looks like we've achieved lazy majority, so I'll move the KIP to approved. > > Thanks all for looking this over. > > On Fri, Nov 3, 2017 at 7:31 PM, Jason Gustafson > wrote: > &g

Re: [VOTE] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-12-11 Thread Ewen Cheslack-Postava
+1 (binding) -Ewen On Mon, Dec 11, 2017 at 12:40 PM, Gwen Shapira wrote: > +1 (binding) - nice API improvement, thanks for driving it! > > On Mon, Dec 11, 2017 at 11:52 AM Xavier Léauté > wrote: > > > Thanks Steven, I believe I addressed all the comments. If the it lo

[DISCUSS] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-11 Thread Ewen Cheslack-Postava
artup behavior that might warrant discussion. -Ewen

Re: [DISCUSS] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-11 Thread Ewen Cheslack-Postava
e not going to be able to write new connector configs, asking for connector state will give you empty data since you wouldn't be able to load the configs or status topics, etc). -Ewen On Mon, Dec 11, 2017 at 4:35 PM, Ted Yu wrote: > Looks good overall. > > Currently lookupK

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

2017-12-11 Thread Ewen Cheslack-Postava
old AdminClients talking to new brokers will see a change in behavior (or do they do client-side validation such that old clients simply wouldn't have access to this new functionality)? -Ewen On Mon, Dec 11, 2017 at 2:11 PM, dan wrote: > Dong, > > I agree that it *may* be better f

Re: [DISCUSS] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-11 Thread Ewen Cheslack-Postava
On Mon, Dec 11, 2017 at 4:01 PM, Gwen Shapira wrote: > Thanks, Ewen :) > > One thing that wasn't clear to me from the wiki: Will standalone connect > also have a Kafka cluster ID? While it is true that only tasks have > producers and consumers, I think we assumed that all

Re: [DISCUSS] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-11 Thread Ewen Cheslack-Postava
ded. We haven't been explicit about this before, but unless someone objects, I don't think it is unreasonable. Happy to update the KIP w/ these details if someone feels they would be valuable. -Ewen On Mon, Dec 11, 2017 at 8:21 PM, Ewen Cheslack-Postava wrote: > > On Mon, Dec 11,

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

2017-12-12 Thread Ewen Cheslack-Postava
ing it directly into AdminClient.createTopics since we don't want them explicitly overriding all the defaults. -Ewen On Tue, Dec 12, 2017 at 2:32 PM, dan wrote: > Colin/Ewen, > > i will add changes to bump the API version. > > any preferences on the return type for the new method? tbh it

[VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-15 Thread Ewen Cheslack-Postava
Discussion seems to have tapered off, so I'd like to start the vote on https://cwiki.apache.org/confluence/display/KAFKA/KIP-238%3A+Expose+Kafka+cluster+ID+in+Connect+REST+API Obviously +1 (binding) from me :) -Ewen

Re: [VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-18 Thread Ewen Cheslack-Postava
Seems we've passed 72h and have the necessary 3 +1 binding votes, 2 non-binding, and 0 otherwise, so this KIP passes. I'll follow up w/ another committer about landing the patch. Thanks everyone! -Ewen On Mon, Dec 18, 2017 at 11:27 AM, Jason Gustafson wrote: > +1 (binding) >

Re: [DISCUSS] KIP-228 Negative record timestamp support

2017-12-18 Thread Ewen Cheslack-Postava
ial values relative to .MIN_VALUE instead of relative to 0. -Ewen On Tue, Dec 12, 2017 at 11:12 AM, Dong Lin wrote: > Hey Konstantin, > > Thanks for updating the KIP. > > If we were to support negative timestamp in the message, we probably also > want to support negative ti

Re: [DISCUSS] KIP 145 - Expose Record Headers in Kafka Connect

2018-01-02 Thread Ewen Cheslack-Postava
instead of DropHeader? In general I think this is the right direction for making headers work both flexibly but also easily in the default case. -Ewen On Tue, Jan 2, 2018 at 8:42 AM, Gwen Shapira wrote: > I got the impression that use of Strings in headers is really common, so > the

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

2018-01-02 Thread Ewen Cheslack-Postava
instead can resolve that. -Ewen On Mon, Jan 1, 2018 at 6:22 PM, Ted Yu wrote: > Gentle reminder: one more binding vote is needed for the KIP to pass. > > Cheers > > On Thu, Dec 21, 2017 at 4:13 AM, Damian Guy wrote: > > > +1 > > > > On Wed, 20 Dec 2017 at 21:

  1   2   3   4   5   6   7   8   9   10   >