Re: Kafka Issue

2021-08-20 Thread Soumyajit Sahu
Microsoft Azure Event Hubs are not Kafka (although they are compatible with Kafka protocol and clients). You may find better answers on the Azure support. On Fri, Aug 20, 2021, 1:06 AM Luke Chen wrote: > Sorry, I don't have much knowledge on how event Hubs get all the active > topics. > But at

Re: [DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

2021-07-06 Thread Soumyajit Sahu
y > particular use case. Both of the KIPs are at the Record-level. > > Thanks, > -James > > > On Jun 30, 2021, at 10:05 AM, Soumyajit Sahu > wrote: > > > > Hi Nikolay, > > Great to hear that. I'm ok with either one too. > > I had missed noticing t

Re: [DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

2021-06-30 Thread Soumyajit Sahu
his feature very useful as well as adding custom > validation > >> to incoming records would be nice to prevent bad data from making it to > the > >> topic. > >> > >> On Wed, Apr 7, 2021 at 7:03 PM Soumyajit Sahu > > >> wrote: > >> &g

RE: Re: [DISCUSS] KIP-679 Producer will enable the strongest delivery guarantee by default

2021-04-23 Thread Soumyajit Sahu
I would like to raise an issue with making enable.idempotence=true by default. This has the ability to crash a broker with OutOfMemory exception whenever there are too many producerIds. This can take many current deployments by surprise. I would suggest to hold on to this change until we have fi

Re: [DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

2021-04-07 Thread Soumyajit Sahu
o this, we would just catch > an exception and use the exception text as the validation error. > > best, > Colin > > > On Tue, Apr 6, 2021, at 15:57, Soumyajit Sahu wrote: > > Hi Tom, > > > > Makes sense. Thanks for the explanation. I get what Colin had mean

Re: [DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

2021-04-06 Thread Soumyajit Sahu
n prevent, innovation in > the rest of Kafka. > > Kind regards, > > Tom > > > > On Sun, Apr 4, 2021 at 7:31 PM Soumyajit Sahu > wrote: > > > Hi Colin, > > I see that both the interface "Record" and the implementation > > "Default

Re: [DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

2021-04-04 Thread Soumyajit Sahu
ussions about proposals similar to this before, > although I'm having trouble finding one right now. The issue here is that > Record is a private class -- it is not part of any public API, and may > change at any time. So we can't expose it in public APIs. > > best, > C

[DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

2021-04-01 Thread Soumyajit Sahu
Hello All, I would like to start a discussion on the KIP-729. https://cwiki.apache.org/confluence/display/KAFKA/KIP-729%3A+Custom+validation+of+records+on+the+broker+prior+to+log+append Thanks! Soumyajit

Request to Create KIP

2021-03-22 Thread Soumyajit Sahu
Hello, Please allow my wiki id (soumyajit-sahu) to create Kafka Improvement Proposals. Thank you. Regards Soumyajit Sahu

[GitHub] kafka pull request #1716: KAFKA-3123: Make abortAndPauseCleaning() a no-op w...

2017-06-14 Thread soumyajit-sahu
Github user soumyajit-sahu closed the pull request at: https://github.com/apache/kafka/pull/1716 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time

2016-11-30 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709138#comment-15709138 ] Soumyajit Sahu commented on KAFKA-1194: --- [~haraldk] could you please provide al

[jira] [Commented] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range

2016-11-29 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15706415#comment-15706415 ] Soumyajit Sahu commented on KAFKA-3123: --- I am closing this previously opened

[GitHub] kafka pull request #1028: KAFKA-3123: Follower Broker cannot start if offset...

2016-11-29 Thread soumyajit-sahu
Github user soumyajit-sahu closed the pull request at: https://github.com/apache/kafka/pull/1028 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time

2016-11-29 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15706222#comment-15706222 ] Soumyajit Sahu commented on KAFKA-1194: --- Thanks for confirming that [~har

[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time

2016-11-28 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702991#comment-15702991 ] Soumyajit Sahu commented on KAFKA-1194: --- [~abhit011], [~bradvido] Could you pl

[jira] [Commented] (KAFKA-2170) 10 LogTest cases failed for file.renameTo failed under windows

2016-09-22 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515023#comment-15515023 ] Soumyajit Sahu commented on KAFKA-2170: --- [~haraldk], I have updated/fixed the h

[jira] [Commented] (KAFKA-2170) 10 LogTest cases failed for file.renameTo failed under windows

2016-08-18 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427096#comment-15427096 ] Soumyajit Sahu commented on KAFKA-2170: --- [~haraldk] Thanks for checking it ou

[GitHub] kafka pull request #1757: KAFKA-2170: Fixes for Windows

2016-08-17 Thread soumyajit-sahu
GitHub user soumyajit-sahu opened a pull request: https://github.com/apache/kafka/pull/1757 KAFKA-2170: Fixes for Windows This PR is to revive the older PR https://github.com/apache/kafka/pull/154 made by @mpoindexter (Mike Poindexter). I have replicated his work over the new code

[GitHub] kafka pull request #1718: KAFKA-4031: Check if buffer cleaner is null before...

2016-08-10 Thread soumyajit-sahu
GitHub user soumyajit-sahu opened a pull request: https://github.com/apache/kafka/pull/1718 KAFKA-4031: Check if buffer cleaner is null before using it A small fix to check null before using the reference You can merge this pull request into a Git repository by running: $ git

[jira] [Assigned] (KAFKA-4031) Check DirectBuffer's cleaner to be not null before using

2016-08-10 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu reassigned KAFKA-4031: - Assignee: Soumyajit Sahu > Check DirectBuffer's cleaner to be not null befo

[jira] [Created] (KAFKA-4031) Check DirectBuffer's cleaner to be not null before using

2016-08-10 Thread Soumyajit Sahu (JIRA)
Soumyajit Sahu created KAFKA-4031: - Summary: Check DirectBuffer's cleaner to be not null before using Key: KAFKA-4031 URL: https://issues.apache.org/jira/browse/KAFKA-4031 Project:

[jira] [Assigned] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range

2016-08-09 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu reassigned KAFKA-3123: - Assignee: Soumyajit Sahu (was: Neha Narkhede) > Follower Broker cannot start if offs

[GitHub] kafka pull request #1716: KAFKA-3123: Make abortAndPauseCleaning() a no-op w...

2016-08-09 Thread soumyajit-sahu
GitHub user soumyajit-sahu opened a pull request: https://github.com/apache/kafka/pull/1716 KAFKA-3123: Make abortAndPauseCleaning() a no-op when state is already LogCleaningPaused The function of LogCleanerManager.abortAndPauseCleaning() is to mark a TopicAndPartition as

[jira] [Commented] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range

2016-08-09 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414500#comment-15414500 ] Soumyajit Sahu commented on KAFKA-3123: --- I hit another (similar) exception w

[jira] [Closed] (KAFKA-3448) Support zone index in IPv6 regex

2016-08-09 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu closed KAFKA-3448. - > Support zone index in IPv6 regex > > >

[jira] [Closed] (KAFKA-2305) Cluster Monitoring and Management UI

2016-08-09 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu closed KAFKA-2305. - > Cluster Monitoring and Management UI > > >

[jira] [Resolved] (KAFKA-2305) Cluster Monitoring and Management UI

2016-06-02 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu resolved KAFKA-2305. --- Resolution: Invalid > Cluster Monitoring and Management

[jira] [Assigned] (KAFKA-2305) Cluster Monitoring and Management UI

2016-06-02 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu reassigned KAFKA-2305: - Assignee: Soumyajit Sahu (was: Neha Narkhede) > Cluster Monitoring and Management

[jira] [Updated] (KAFKA-3448) Support zone index in IPv6 regex

2016-05-03 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3448: -- Summary: Support zone index in IPv6 regex (was: IPV6 Regex is missing % character) > Supp

[jira] [Updated] (KAFKA-3448) IPV6 Regex is missing % character

2016-05-03 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3448: -- Description: When an address is written textually, the zone index is appended to the address

[GitHub] kafka pull request: KAFKA-3448: Fix ipv6 regex pattern trunk

2016-05-02 Thread soumyajit-sahu
GitHub user soumyajit-sahu opened a pull request: https://github.com/apache/kafka/pull/1305 KAFKA-3448: Fix ipv6 regex pattern trunk IPV6 address can contain % character. This PR is in lieu of the now abandoned PR 1120 You can merge this pull request into a Git repository by

[GitHub] kafka pull request: KAFKA-3448: add % character to ipv6 regex

2016-05-02 Thread soumyajit-sahu
Github user soumyajit-sahu closed the pull request at: https://github.com/apache/kafka/pull/1120 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Updated] (KAFKA-3448) IPV6 Regex is missing % character

2016-03-22 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3448: -- Fix Version/s: 0.10.1.0 Status: Patch Available (was: Open) > IPV6 Regex is miss

[GitHub] kafka pull request: KAFKA-3448: add % character to ipv6 regex

2016-03-22 Thread soumyajit-sahu
GitHub user soumyajit-sahu opened a pull request: https://github.com/apache/kafka/pull/1120 KAFKA-3448: add % character to ipv6 regex IPV6 address can contain % character You can merge this pull request into a Git repository by running: $ git pull https://github.com/Microsoft

[GitHub] kafka pull request: KAFKA-3448: Include % character in IPV6 Regex

2016-03-22 Thread soumyajit-sahu
Github user soumyajit-sahu closed the pull request at: https://github.com/apache/kafka/pull/1119 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] kafka pull request: KAFKA-3448: Include % character in IPV6 Regex

2016-03-22 Thread soumyajit-sahu
GitHub user soumyajit-sahu opened a pull request: https://github.com/apache/kafka/pull/1119 KAFKA-3448: Include % character in IPV6 Regex IPV6 addresses could have the % character. When an address is written textually, the zone index is appended to the address, separated by a

[jira] [Created] (KAFKA-3448) IPV6 Regex is missing % character

2016-03-22 Thread Soumyajit Sahu (JIRA)
Soumyajit Sahu created KAFKA-3448: - Summary: IPV6 Regex is missing % character Key: KAFKA-3448 URL: https://issues.apache.org/jira/browse/KAFKA-3448 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request: KAFKA-3123: Follower Broker cannot start if of...

2016-03-07 Thread soumyajit-sahu
GitHub user soumyajit-sahu opened a pull request: https://github.com/apache/kafka/pull/1028 KAFKA-3123: Follower Broker cannot start if offsets are already out of range Function abortAndPauseCleaning() doesn't have to throw exception if the state is already LogCleaningAbort

[jira] [Updated] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range

2016-01-19 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3123: -- Attachment: 0001-Fix-Follower-crashes-when-offset-out-of-range-during.patch Submitting a patch

[jira] [Updated] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range

2016-01-19 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3123: -- Labels: patch (was: ) Status: Patch Available (was: Open) Submitting a patch for the code

[jira] [Created] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range

2016-01-19 Thread Soumyajit Sahu (JIRA)
Soumyajit Sahu created KAFKA-3123: - Summary: Follower Broker cannot start if offsets are already out of range Key: KAFKA-3123 URL: https://issues.apache.org/jira/browse/KAFKA-3123 Project: Kafka

[jira] [Created] (KAFKA-2305) Cluster Monitoring and Management UI

2015-06-29 Thread Soumyajit Sahu (JIRA)
Soumyajit Sahu created KAFKA-2305: - Summary: Cluster Monitoring and Management UI Key: KAFKA-2305 URL: https://issues.apache.org/jira/browse/KAFKA-2305 Project: Kafka Issue Type: Wish