[GitHub] kafka pull request: KAFKA-2935: Remove vestigial WorkerConfig.CLUS...

2016-05-18 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1404 KAFKA-2935: Remove vestigial WorkerConfig.CLUSTER_CONFIG You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka kafka-2935

[GitHub] kafka pull request #1537: KAFKA-3846: include timestamp in Connect record ty...

2016-06-21 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1537 KAFKA-3846: include timestamp in Connect record types KIP to come You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka kafka-3846

[GitHub] kafka pull request #1567: Minor: fix Bash shebang on vagrant/ scripts

2016-06-28 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1567 Minor: fix Bash shebang on vagrant/ scripts You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka vagrant-scripts-shebang

[GitHub] kafka pull request #1968: MINOR: missing whitespace in doc for `ssl.cipher.s...

2016-11-04 Thread shikhar
Github user shikhar closed the pull request at: https://github.com/apache/kafka/pull/1968 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2131: Remove failing ConnectDistributedTest.test_bad_con...

2016-11-14 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2131 Remove failing ConnectDistributedTest.test_bad_connector_class Since #1911 was merged it is hard to externally test a connector transitioning to FAILED state due to an initialization failure, which

[GitHub] kafka pull request #2139: KAFKA-4161: KIP-89: Allow sink connectors to decou...

2016-11-15 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2139 KAFKA-4161: KIP-89: Allow sink connectors to decouple flush and offset commit You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka

[GitHub] kafka pull request #2182: ConfigDef experimentation - support List and Ma...

2016-11-28 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2182 ConfigDef experimentation - support List and Map You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka configdef-experimentation

[GitHub] kafka pull request #2196: KAFKA-3910: prototype of another approach to cycli...

2016-11-30 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2196 KAFKA-3910: prototype of another approach to cyclic schemas You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka KAFKA-3910

[GitHub] kafka pull request #2232: HOTFIX: Fix HerderRequest.compareTo()

2016-12-08 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2232 HOTFIX: Fix HerderRequest.compareTo() With KAFKA-3008 (#1788), the implementation does not respect the contract that 'sgn(x.compareTo(y)) == -sgn(y.compareTo(x))' This fix add

[GitHub] kafka pull request #2277: KAFKA-4527: task status was being updated before a...

2016-12-19 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2277 KAFKA-4527: task status was being updated before actual pause/resume h/t @ewencp for pointing out the issue You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] kafka pull request #2299: KAFKA-3209: KIP-66: single message transforms

2017-01-03 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2299 KAFKA-3209: KIP-66: single message transforms *WIP* particularly around testing You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka

[GitHub] kafka pull request #2313: KAFKA-4575: ensure topic created before starting s...

2017-01-04 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2313 KAFKA-4575: ensure topic created before starting sink for ConnectDistributedTest.test_pause_resume_sink Otherwise in this test the sink task goes through the pause/resume cycle with 0 assigned

[GitHub] kafka pull request #2182: ConfigDef experimentation - support List and Ma...

2017-01-11 Thread shikhar
Github user shikhar closed the pull request at: https://github.com/apache/kafka/pull/2182 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2365: MINOR: avoid closing over both pre & post-transfor...

2017-01-12 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2365 MINOR: avoid closing over both pre & post-transform record in WorkerSourceTask Followup to #2299 for KAFKA-3209 You can merge this pull request into a Git repository by running: $ git

[GitHub] kafka pull request #2374: KAFKA-3209: KIP-66: more single message transforms

2017-01-13 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2374 KAFKA-3209: KIP-66: more single message transforms WIP, in this PR I'd also like to add doc generation for transformations. You can merge this pull request into a Git repository by ru

[GitHub] kafka pull request #1729: Assign ConfigDef.NO_DEFAULT_VALUE as a literal, us...

2016-08-12 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1729 Assign ConfigDef.NO_DEFAULT_VALUE as a literal, use equals() for comp… …arison rather than == You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] kafka pull request #1729: Assign ConfigDef.NO_DEFAULT_VALUE as a literal, us...

2016-08-12 Thread shikhar
Github user shikhar closed the pull request at: https://github.com/apache/kafka/pull/1729 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1745: WIP: KAFKA-4042: prevent DistributedHerder thread ...

2016-08-16 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1745 WIP: KAFKA-4042: prevent DistributedHerder thread from dying from connector/task lifecycle exceptions You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] kafka pull request #1745: KAFKA-4042: prevent DistributedHerder thread from ...

2016-08-18 Thread shikhar
Github user shikhar closed the pull request at: https://github.com/apache/kafka/pull/1745 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1745: KAFKA-4042: prevent DistributedHerder thread from ...

2016-08-18 Thread shikhar
GitHub user shikhar reopened a pull request: https://github.com/apache/kafka/pull/1745 KAFKA-4042: prevent DistributedHerder thread from dying from connector/task lifecycle exceptions - `worker.startConnector()` and `worker.startTask()` can throw (e.g. `ClassNotFoundException

[GitHub] kafka pull request #1778: KAFKA-4042: Contain connector & task start/stop fa...

2016-08-24 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1778 KAFKA-4042: Contain connector & task start/stop failures within the Worker Invoke the statusListener.onFailure() callback on start failures so that the statusBackingStore is updated. This invo

[GitHub] kafka pull request #1745: KAFKA-4042: prevent DistributedHerder thread from ...

2016-08-24 Thread shikhar
Github user shikhar closed the pull request at: https://github.com/apache/kafka/pull/1745 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1790: KAFKA-4070: implement Connect Struct.toString()

2016-08-25 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1790 KAFKA-4070: implement Connect Struct.toString() You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka add-struct-tostring

[GitHub] kafka pull request #1800: KAFKA-4100: ensure 'fields' and 'fieldsByName' are...

2016-08-29 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1800 KAFKA-4100: ensure 'fields' and 'fieldsByName' are not null for Struct schemas You can merge this pull request into a Git repository by running: $ git pull https://gith

[GitHub] kafka pull request #1815: KAFKA-4115: grow default heap size for connect-dis...

2016-09-01 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1815 KAFKA-4115: grow default heap size for connect-distributed.sh to 1G You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka connect-heap

[GitHub] kafka pull request #1865: KAFKA-4173: SchemaProjector should successfully pr...

2016-09-16 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1865 KAFKA-4173: SchemaProjector should successfully project missing Struct field when target field is optional You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] kafka pull request #1872: KAFKA-4183: centralize checking for optional and d...

2016-09-16 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1872 KAFKA-4183: centralize checking for optional and default values to avoid bugs Cleaner to just check once for optional & default value from the `convertToConnect()` function. It also h

[GitHub] kafka pull request #1964: KAFKA-4010: add ConfigDef toEnrichedRst() for addi...

2016-10-04 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1964 KAFKA-4010: add ConfigDef toEnrichedRst() for additional fields in output followup on https://github.com/apache/kafka/pull/1696 cc @rekhajoshm You can merge this pull request into a Git

[GitHub] kafka pull request #1968: MINOR: missing whitespace in doc for `ssl.cipher.s...

2016-10-04 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1968 MINOR: missing whitespace in doc for `ssl.cipher.suites` You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka patch-1 Alternatively

[GitHub] kafka pull request #1969: MINOR: missing fullstop in doc for `max.partition....

2016-10-04 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1969 MINOR: missing fullstop in doc for `max.partition.fetch.bytes` You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka patch-2

[GitHub] kafka pull request #2040: KAFKA-4161: prototype for exploring API change

2016-10-18 Thread shikhar
GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/2040 KAFKA-4161: prototype for exploring API change You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka kafka-4161 Alternatively you can

[GitHub] kafka pull request #2040: KAFKA-4161: prototype for exploring API change

2016-10-26 Thread shikhar
Github user shikhar closed the pull request at: https://github.com/apache/kafka/pull/2040 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[DISCUSS] KIP-65 Expose timestamps to Connect

2016-06-23 Thread Shikhar Bhushan
Kafkarati, Here is a pretty straightforward proposal, for exposing timestamps that were added in Kafka 0.10 to the connect framework so connectors can make use of them: https://cwiki.apache.org/confluence/display/KAFKA/KIP-65%3A+Expose+timestamps+to+Connect Appreciate your thoughts! Shikhar

[VOTE] KIP-65 Expose timestamps to Connect

2016-06-24 Thread Shikhar Bhushan
Since there isn't much to discuss with this KIP, bringing it to a vote KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-65%3A+Expose+timestamps+to+Connect Pull request: https://github.com/apache/kafka/pull/1537 Thanks, Shikhar

Re: [DISCUSS] KIP-65 Expose timestamps to Connect

2016-06-24 Thread Shikhar Bhushan
). It probably makes sense to be consistent and use either Long everywhere or the primitive long and default values. Feel free to add the comment on the PR <https://github.com/apache/kafka/pull/1537/files> as well and I can follow up there :-) Thanks, Shikhar On Fri, Jun 24, 2016 at 3:52 PM

Re: [DISCUSS] KIP-65 Expose timestamps to Connect

2016-06-26 Thread Shikhar Bhushan
with what you decide. > > Ismael > > On Sat, Jun 25, 2016 at 1:29 AM, Shikhar Bhushan > wrote: > > > Hi Ismael, > > > > Good point. This is down to an implementation detail, the getter was > added > > to the base class for `SourceRecord` and `SinkR

Re: [VOTE] KIP-65 Expose timestamps to Connect

2016-06-28 Thread Shikhar Bhushan
g), provided that we make the usage of `Long/null` versus > > `long/-1` consistent. > > > > Ismael > > > > On Sat, Jun 25, 2016 at 12:42 AM, Gwen Shapira > wrote: > > > > > +1 > > > > > > On Fri, Jun 24, 2016 at 10:59 AM, Shikhar Bhushan

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-07-22 Thread Shikhar Bhushan
flatMap() / supporting 1->n feels nice and general since filtering is just the case of going from 1->0 I'm not sure why we'd need to do any more granular offset tracking (like sub-offsets) for source connectors: after transformation of a given record to n records, all of those n should map to same

Re: Changing hash algorithm to LogCleaner offset map

2016-07-22 Thread Shikhar Bhushan
Not sure I understand the motivation to use a FIPS-compliant hash function for log compaction -- what are the security ramifications? On Fri, Jul 22, 2016 at 2:56 PM Luciano Afranllie wrote: > A little bit of background first. > > We are trying to make a deployment of Kafka that is FIPS 140-2 (

Re: Changing hash algorithm to LogCleaner offset map

2016-07-24 Thread Shikhar Bhushan
we want > to deploy with FIPS 140-2 mode enabled using only RSA security providers. > With this settings it is not possible to use MD5. > > On Fri, Jul 22, 2016 at 8:49 PM, Shikhar Bhushan > wrote: > > > Not sure I understand the motivation to use a FIPS-compliant

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-07-28 Thread Shikhar Bhushan
me is just an example)? > > On Sat, Jul 23, 2016 at 11:13 PM, Ewen Cheslack-Postava > > wrote: > > > On Fri, Jul 22, 2016 at 12:58 AM, Shikhar Bhushan > > wrote: > > > > > flatMap() / supporting 1->n feels nice and general since filtering is > > j

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-07-28 Thread Shikhar Bhushan
> > > Hmm, operating on ConnectRecords probably doesn't work since you need to > emit the right type of record, which might mean instantiating a new one. I > think that means we either need 2 methods, one for SourceRecord, one for > SinkRecord, or we'd need to limit what parts of the message you ca

[DISCUSS] KIP-89: Allow sink connectors to decouple flush and offset commit

2016-11-04 Thread Shikhar Bhushan
Hi all, I created KIP-89 for making a Connect API change that allows for sink connectors to decouple flush and offset commits. https://cwiki.apache.org/confluence/display/KAFKA/KIP-89%3A+Allow+sink+connectors+to+decouple+flush+and+offset+commit I'd welcome your input. Best, Shikhar

[VOTE] KIP-89: Allow sink connectors to decouple flush and offset commit

2016-11-09 Thread Shikhar Bhushan
Hi, I would like to initiate a vote on KIP-89 https://cwiki.apache.org/confluence/display/KAFKA/KIP-89%3A+Allow+sink+connectors+to+decouple+flush+and+offset+commit Best, Shikhar

Re: [VOTE] KIP-89: Allow sink connectors to decouple flush and offset commit

2016-11-14 Thread Shikhar Bhushan
The vote passed with +3 binding votes. Thanks all! On Sun, Nov 13, 2016 at 1:42 PM Gwen Shapira wrote: +1 (binding) On Nov 9, 2016 2:17 PM, "Shikhar Bhushan" wrote: > Hi, > > I would like to initiate a vote on KIP-89 > > https://cwiki.apache.org/confluence/display

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-12-07 Thread Shikhar Bhushan
Hi all, I have another iteration at a proposal for this feature here: https://cwiki.apache.org/confluence/display/KAFKA/Connect+Transforms+-+Proposed+Design I'd welcome your feedback and comments. Thanks, Shikhar On Tue, Aug 2, 2016 at 7:21 PM Ewen Cheslack-Postava wrote: On Thu, J

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-12-14 Thread Shikhar Bhushan
e to do data enrichment by querying external systems, so building a bunch of transformations in could potentially open the floodgates, or at least make decisions about what is included vs what should be 3rd party muddy. -Ewen On Wed, Dec 7, 2016 at 11:46 AM, Shikhar Bhushan wrote: > Hi

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-12-15 Thread Shikhar Bhushan
in Apache Kafka. What's the downside of keeping it out? Or at least keeping the built-in set super minimal (Flume has like 3 built-in interceptors)? Gwen On Wed, Dec 14, 2016 at 1:36 PM, Shikhar Bhushan wrote: > With regard to a), just using `ConnectRecord` with `newRecord` as a new >

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-12-15 Thread Shikhar Bhushan
ct later than to remove functionality. On Thu, Dec 15, 2016 at 11:59 AM, Shikhar Bhushan wrote: > I have updated KIP-66 > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 66%3A+Single+Message+Transforms+for+Kafka+Connect > with > the changes I proposed in the design. >

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-12-15 Thread Shikhar Bhushan
at we have ExtractAvroMetadata that may fit? and ExtractEmailHeaders doesn't sound totally outlandish either... Nothing in the baked-in list by Shikhar looks out of place. I am concerned about slipperly slope. Or the arbitrariness of the decision if we say that this list is final and nothing else will ev

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2017-01-03 Thread Shikhar Bhushan
Makes sense Ewen, I edited the KIP to include this criteria. I'd like to start a voting thread soon unless anyone has additional points for discussion. On Fri, Dec 30, 2016 at 12:14 PM Ewen Cheslack-Postava wrote: On Thu, Dec 15, 2016 at 7:41 PM, Shikhar Bhushan wrote: > There is no

[VOTE] KIP-66: Single Message Transforms for Kafka Connect

2017-01-04 Thread Shikhar Bhushan
Hi all, I'd like to start voting on KIP-66: https://cwiki.apache.org/confluence/display/KAFKA/KIP-66%3A+Single+Message+Transforms+for+Kafka+Connect Best, Shikhar

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

2017-01-04 Thread Shikhar Bhushan
On Wed, Jan 4, 2017 at 1:29 PM, Gwen Shapira wrote: > > > > > I would have preferred not to bundle transformations, but since SMT > > > capability is a much needed feature, I'll take it in its current form. > > > > > > +1 > > > > > >

Re: KafkaConnect SinkTask::put

2017-01-05 Thread Shikhar Bhushan
Hi David, You can override the underlying consumer's `max.poll.records` setting for this. E.g. consumer.max.poll.records=500 Best, Shikhar On Thu, Jan 5, 2017 at 3:59 AM wrote: > Is there any way of limiting the number of events that are passed into the > call to the put

Re: KafkaConnect SinkTask::put

2017-01-06 Thread Shikhar Bhushan
Sorry I forgot to specify, this needs to go into your Connect worker configuration. On Fri, Jan 6, 2017 at 02:57 wrote: > Hi Shikhar, > > I've just added this to ~config/consumer.properties in the Kafka folder > but it doesn't appear to have made any difference. Have I

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

2017-01-06 Thread Shikhar Bhushan
e the `init` method? > > On Thu, Jan 5, 2017 at 5:48 PM, Neha Narkhede wrote: > > > +1 (binding) > > > > On Wed, Jan 4, 2017 at 2:36 PM Shikhar Bhushan > > wrote: > > > > > I do plan on introducing a new `connect:transforms` module (which > > >

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

2017-01-09 Thread Shikhar Bhushan
Thanks all. The vote passed with +5 (binding). On Fri, Jan 6, 2017 at 11:37 AM Shikhar Bhushan wrote: That makes sense to me, I'll fold that into the PR and update the KIP if it gets committed in that form. On Fri, Jan 6, 2017 at 9:44 AM Jason Gustafson wrote: +1 One minor comment: wou

Re: [VOTE] KIP-75 - Add per-connector Converters

2016-08-15 Thread Shikhar Bhushan
+1 (non-binding) On Mon, Aug 15, 2016 at 1:20 PM Ismael Juma wrote: > +1 (binding) > > On 15 Aug 2016 7:21 pm, "Ewen Cheslack-Postava" wrote: > > > I would like to initiate the voting process for KIP-75: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 75+-+Add+per-connector+Conve

[jira] [Commented] (KAFKA-2935) Remove vestigial CLUSTER_CONFIG in WorkerConfig

2016-05-18 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290294#comment-15290294 ] Shikhar Bhushan commented on KAFKA-2935: I could not find any documenta

[jira] [Commented] (KAFKA-3335) Kafka Connect hangs in shutdown hook

2016-05-18 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290333#comment-15290333 ] Shikhar Bhushan commented on KAFKA-3335: Currently {{start()}} looks

[jira] [Assigned] (KAFKA-3846) Connect record types should include timestamps

2016-06-21 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-3846: -- Assignee: Shikhar Bhushan (was: Ewen Cheslack-Postava) > Connect record types sho

[jira] [Work started] (KAFKA-3846) Connect record types should include timestamps

2016-06-23 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3846 started by Shikhar Bhushan. -- > Connect record types should include timesta

[jira] [Commented] (KAFKA-3846) Connect record types should include timestamps

2016-06-23 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15346891#comment-15346891 ] Shikhar Bhushan commented on KAFKA-3846: https://cwiki.apache.org/conflu

[jira] [Created] (KAFKA-3962) ConfigDef support for resource-specific configuration

2016-07-13 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-3962: -- Summary: ConfigDef support for resource-specific configuration Key: KAFKA-3962 URL: https://issues.apache.org/jira/browse/KAFKA-3962 Project: Kafka

[jira] [Updated] (KAFKA-3962) ConfigDef support for resource-specific configuration

2016-07-13 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-3962: --- Description: It often comes up with connectors that you want some piece of configuration

[jira] [Created] (KAFKA-4010) ConfigDef.toRst() should create sections for each group

2016-08-01 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4010: -- Summary: ConfigDef.toRst() should create sections for each group Key: KAFKA-4010 URL: https://issues.apache.org/jira/browse/KAFKA-4010 Project: Kafka

[jira] [Commented] (KAFKA-4375) Kafka consumer may swallow some interrupts meant for the calling thread

2016-11-03 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634044#comment-15634044 ] Shikhar Bhushan commented on KAFKA-4375: Good to have a report of this bei

[jira] [Resolved] (KAFKA-4356) o.a.k.common.utils.SystemTime.sleep() swallows InterruptedException

2016-11-03 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan resolved KAFKA-4356. Resolution: Duplicate > o.a.k.common.utils.SystemTime.sleep() swallows InterruptedExcept

[jira] [Commented] (KAFKA-4161) Decouple flush and offset commits

2016-11-04 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637718#comment-15637718 ] Shikhar Bhushan commented on KAFKA-4161: Created KIP-89 for this h

[jira] [Assigned] (KAFKA-3910) Cyclic schema support in ConnectSchema and SchemaBuilder

2016-11-07 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-3910: -- Assignee: Shikhar Bhushan (was: Ewen Cheslack-Postava) > Cyclic schema support

[jira] [Work started] (KAFKA-4161) Decouple flush and offset commits

2016-11-09 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4161 started by Shikhar Bhushan. -- > Decouple flush and offset comm

[jira] [Updated] (KAFKA-3462) Allow SinkTasks to disable consumer offset commit

2016-11-14 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-3462: --- Resolution: Fixed Status: Resolved (was: Patch Available) This will be handled with

[jira] [Commented] (KAFKA-3209) Support single message transforms in Kafka Connect

2016-12-07 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729737#comment-15729737 ] Shikhar Bhushan commented on KAFKA-3209: [~snisarg] and [~jjchorrobe], I rev

[jira] [Assigned] (KAFKA-3209) Support single message transforms in Kafka Connect

2016-12-09 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-3209: -- Assignee: Shikhar Bhushan > Support single message transforms in Kafka Conn

[jira] [Created] (KAFKA-4524) ConfigDef.Type.LIST does not handle escaping

2016-12-12 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4524: -- Summary: ConfigDef.Type.LIST does not handle escaping Key: KAFKA-4524 URL: https://issues.apache.org/jira/browse/KAFKA-4524 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-3209) Support single message transforms in Kafka Connect

2016-12-12 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743300#comment-15743300 ] Shikhar Bhushan commented on KAFKA-3209: Thanks [~snisarg]. I self-assigned i

[jira] [Created] (KAFKA-4574) Transient failure in ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade with security_protocol = SASL_PLAINTEXT, SSL

2016-12-29 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4574: -- Summary: Transient failure in ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade with security_protocol = SASL_PLAINTEXT, SSL Key: KAFKA-4574 URL: https

[jira] [Created] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

2016-12-29 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4575: -- Summary: Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector Key: KAFKA-4575 URL: https

[jira] [Updated] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

2016-12-29 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4575: --- Component/s: system tests KafkaConnect > Transient failure

[jira] [Assigned] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

2016-12-29 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-4575: -- Assignee: Shikhar Bhushan > Transient failure

[jira] [Commented] (KAFKA-3513) Transient failure of OffsetValidationTest

2017-01-01 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791790#comment-15791790 ] Shikhar Bhushan commented on KAFKA-3513: There was a failure in the last

[jira] [Work started] (KAFKA-3209) Support single message transforms in Kafka Connect

2017-01-02 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3209 started by Shikhar Bhushan. -- > Support single message transforms in Kafka Conn

[jira] [Work started] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

2017-01-04 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4575 started by Shikhar Bhushan. -- > Transient failure in ConnectDistributedTest.test_pause_and_resume_sink

[jira] [Updated] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming sink connector

2017-01-04 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4575: --- Summary: Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming

[jira] [Commented] (KAFKA-4575) Transient failure in ConnectDistributedTest.test_pause_and_resume_sink in consuming messages after resuming source connector

2017-01-04 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799807#comment-15799807 ] Shikhar Bhushan commented on KAFKA-4575: By the way, the error messag

[jira] [Created] (KAFKA-4598) Create new SourceTask commit callback method that takes offsets param

2017-01-05 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4598: -- Summary: Create new SourceTask commit callback method that takes offsets param Key: KAFKA-4598 URL: https://issues.apache.org/jira/browse/KAFKA-4598 Project

[jira] [Commented] (KAFKA-4598) Create new SourceTask commit callback method that takes offsets param

2017-01-05 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802110#comment-15802110 ] Shikhar Bhushan commented on KAFKA-4598: In the meantime the workaround is to

[jira] [Commented] (KAFKA-4598) Create new SourceTask commit callback method that takes offsets param

2017-01-06 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15805460#comment-15805460 ] Shikhar Bhushan commented on KAFKA-4598: Yeah, that's a reasonable al

[jira] [Created] (KAFKA-4678) Create separate page for Connect docs

2017-01-20 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4678: -- Summary: Create separate page for Connect docs Key: KAFKA-4678 URL: https://issues.apache.org/jira/browse/KAFKA-4678 Project: Kafka Issue Type

[jira] [Created] (KAFKA-4042) Missing error handling in Worker.startConnector() can cause Herder thread to die

2016-08-15 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4042: -- Summary: Missing error handling in Worker.startConnector() can cause Herder thread to die Key: KAFKA-4042 URL: https://issues.apache.org/jira/browse/KAFKA-4042

[jira] [Work started] (KAFKA-4042) Missing error handling in Worker.startConnector() can cause Herder thread to die

2016-08-15 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4042 started by Shikhar Bhushan. -- > Missing error handling in Worker.startConnector() can cause Herder thr

[jira] [Assigned] (KAFKA-3054) Connect Herder fail forever if sent a wrong connector config or task config

2016-08-15 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-3054: -- Assignee: Shikhar Bhushan (was: jin xing) > Connect Herder fail forever if sen

[jira] [Work started] (KAFKA-3054) Connect Herder fail forever if sent a wrong connector config or task config

2016-08-15 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3054 started by Shikhar Bhushan. -- > Connect Herder fail forever if sent a wrong connector config or t

[jira] [Updated] (KAFKA-4042) DistributedHerder thread can die because of connector & task lifecycle exceptions

2016-08-15 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4042: --- Summary: DistributedHerder thread can die because of connector & task lifecycle except

[jira] [Updated] (KAFKA-4042) DistributedHerder thread can die because of connector & task lifecycle exceptions

2016-08-15 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4042: --- Description: As one example, there isn't exception handli

[jira] [Created] (KAFKA-4048) Connect does not support RetriableException consistently for sources & sinks

2016-08-16 Thread Shikhar Bhushan (JIRA)
Shikhar Bhushan created KAFKA-4048: -- Summary: Connect does not support RetriableException consistently for sources & sinks Key: KAFKA-4048 URL: https://issues.apache.org/jira/browse/KAFKA-

[jira] [Updated] (KAFKA-4042) DistributedHerder thread can die because of connector & task lifecycle exceptions

2016-08-16 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4042: --- Component/s: KafkaConnect > DistributedHerder thread can die because of connector &am

[jira] [Updated] (KAFKA-4048) Connect does not support RetriableException consistently for sinks

2016-08-16 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4048: --- Summary: Connect does not support RetriableException consistently for sinks (was: Connect

[jira] [Updated] (KAFKA-4048) Connect does not support RetriableException consistently for sinks

2016-08-16 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4048: --- Description: We only allow for handling {{RetriableException}} from calls to {{SinkTask.put

[jira] [Commented] (KAFKA-3054) Connect Herder fail forever if sent a wrong connector config or task config

2016-08-18 Thread Shikhar Bhushan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427362#comment-15427362 ] Shikhar Bhushan commented on KAFKA-3054: Addressing this in KAFKA-4042, w

  1   2   >