Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2527

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-16073) Tiered Storage Bug: Incorrect Handling of Offset Ranges During Segment Deletion

[hzh0425 (Jira)]

hzh0425 created KAFKA-16073:
---

 Summary: Tiered Storage Bug: Incorrect Handling of Offset Ranges 
During Segment Deletion
 Key: KAFKA-16073
 URL: https://issues.apache.org/jira/browse/KAFKA-16073
 Project: Kafka
  Issue Type: Bug
  Components: core, Tiered-Storage
Affects Versions: 3.6.1
Reporter: hzh0425
Assignee: hzh0425
 Fix For: 3.6.1


This bug pertains to Apache Kafka's tiered storage functionality. Specifically, 
it involves a timing issue in the {{UnifiedLog.deleteSegments}} method. The 
method first deletes segments from memory but delays updating the 
{{{}localLogStartOffset{}}}. Meanwhile, in 
{{{}ReplicaManager.handleOffsetOutOfRangeError{}}}, if the fetch offset is less 
than {{{}localLogStartOffset{}}}, it triggers the read remote process. However, 
if it's greater, an {{OffsetOutOfRangeException}} is sent to the client.

Consider a scenario with concurrent operations, where {{{}offset1 < offset2 < 
offset3{}}}. A client requests {{offset2}} while a background thread is 
deleting segments. The segments are deleted in memory, but 
{{LocalLogStartOffset}} is still at {{offset1}} and not yet updated to 
{{{}offset3{}}}. In this state, since {{offset2}} is greater than 
{{{}offset1{}}}, {{ReplicaManager.handleOffsetOutOfRangeError}} erroneously 
returns an {{OffsetOutOfRangeException}} to the client. This happens because 
the system has not yet recognized the new starting offset ({{{}offset3{}}}), 
leading to incorrect handling of fetch requests.
 
 
 
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Contributor permissions

[Justine Olshan]

Hey Aadithya,

I saw your request and tried to approve it but there was a conflict with
the name of the account. It was asking me to contact Infra.

I can try to do that.

Thanks,
Justine

On Sun, Dec 31, 2023 at 2:12 PM Aadithya C  wrote:

> I created a new username - *aadithya*. Please let me know if this one works
> for you.
>
> Adithya
>
>
> On Sun, Dec 31, 2023 at 2:00 PM Aadithya C  wrote:
>
> > Thanks! I submitted a request to retrieve my username but I havent
> > received any email from the app. Is there a different way to retrieve
> > my user-id?
> >
> >
> > On Fri, Dec 29, 2023 at 9:11 AM Greg Harris
> >  wrote:
> > >
> > > Hey Adithya,
> > >
> > > Thanks for your interest in Apache Kafka!
> > > I added permissions for your Wiki account, so you should be ready to
> > > contribute KIPs.
> > >
> > > I could not find your ASF JIRA account though. Could you verify that
> > > is the username shown in the Jira dashboard?
> > > https://issues.apache.org/jira/secure/ViewProfile.jspa
> > >
> > > Thanks!
> > > Greg
> > >
> > > On Thu, Dec 28, 2023 at 1:21 PM Aadithya C 
> wrote:
> > > >
> > > > Requesting contributor permissions.
> > > >
> > > > JIRA username: ditac
> > > > GitHub username: ditac
> > > > Wiki username: ditac
> > > >
> > > >
> > > > Thanks,
> > > > Adithya
> >
>

Re: Contributor permissions

[Justine Olshan]

It is also possible the name you chose was already taken. I have emailed
infra to clarify.

On Mon, Jan 1, 2024 at 8:19 AM Justine Olshan  wrote:

> Hey Aadithya,
>
> I saw your request and tried to approve it but there was a conflict with
> the name of the account. It was asking me to contact Infra.
>
> I can try to do that.
>
> Thanks,
> Justine
>
> On Sun, Dec 31, 2023 at 2:12 PM Aadithya C  wrote:
>
>> I created a new username - *aadithya*. Please let me know if this one
>> works
>> for you.
>>
>> Adithya
>>
>>
>> On Sun, Dec 31, 2023 at 2:00 PM Aadithya C  wrote:
>>
>> > Thanks! I submitted a request to retrieve my username but I havent
>> > received any email from the app. Is there a different way to retrieve
>> > my user-id?
>> >
>> >
>> > On Fri, Dec 29, 2023 at 9:11 AM Greg Harris
>> >  wrote:
>> > >
>> > > Hey Adithya,
>> > >
>> > > Thanks for your interest in Apache Kafka!
>> > > I added permissions for your Wiki account, so you should be ready to
>> > > contribute KIPs.
>> > >
>> > > I could not find your ASF JIRA account though. Could you verify that
>> > > is the username shown in the Jira dashboard?
>> > > https://issues.apache.org/jira/secure/ViewProfile.jspa
>> > >
>> > > Thanks!
>> > > Greg
>> > >
>> > > On Thu, Dec 28, 2023 at 1:21 PM Aadithya C 
>> wrote:
>> > > >
>> > > > Requesting contributor permissions.
>> > > >
>> > > > JIRA username: ditac
>> > > > GitHub username: ditac
>> > > > Wiki username: ditac
>> > > >
>> > > >
>> > > > Thanks,
>> > > > Adithya
>> >
>>
>

[DISCUSS] Kafka distributed transaction support

[Rowland Smith]

Hi All,

I am new to the Kafka developer community. After learning more about
Kafka's transactional capabilities recently, I became interested in
exploring what would be required to provide support for the XA interface
specified in the X/ Open Distributed Processing Model in the Kafka producer
client. I have put together a proof of concept to satisfy my curiosity, and
based on that work, I think that extending the Kafka producer with XA
support is doable with reasonable effort.

As I understand the Kafka development team's process, the first step in the
process would be to produce a KIP describing the feature's goals and
design. My question in this email is whether XA support has ever been
considered previously by the PMC and if so, with what result. I don't want
to spend time working on a KIP if XA support is not something that the PMC
sees value in including and supporting in the Kafka codebase.

Any feedback would be appreciated. I am excited to work on this feature if
there is interest in the community.

Regards,
Rowland