[DISCUSS] KIP-476: Add Java AdminClient Interface

[Andy Coates]

Hi folks,

I'd like to start a discussion thread for KIP-476:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-476%3A+Add+Java+AdminClient+Interface

Thanks,

Andy

[jira] [Created] (KAFKA-8454) Add Java AdminClient interface

[Andy Coates (JIRA)]

Andy Coates created KAFKA-8454:
--

 Summary: Add Java AdminClient interface
 Key: KAFKA-8454
 URL: https://issues.apache.org/jira/browse/KAFKA-8454
 Project: Kafka
  Issue Type: Bug
  Components: admin, clients, core, streams
Reporter: Andy Coates
Assignee: Andy Coates


Task to track the work of [KIP-476: Add Java AdminClient 
Interface|https://cwiki.apache.org/confluence/display/KAFKA/KIP-476%3A+Add+Java+AdminClient+Interface]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-8455) Add NothingSerde to Serdes

[John Roesler (JIRA)]

John Roesler created KAFKA-8455:
---

 Summary: Add NothingSerde to Serdes
 Key: KAFKA-8455
 URL: https://issues.apache.org/jira/browse/KAFKA-8455
 Project: Kafka
  Issue Type: Improvement
  Components: streams
Reporter: John Roesler


Often, when reading an input topic, the key is expected to be null, but there's 
actually no way to represent this fact in Consumed, leading to confusing type 
signatures down the topology.

For example, you might use the BytesSerde, but then you have a 
KStream. When maintaining a large application, this becomes a 
hazard, since you'd need to "be really careful" not to try and dereference the 
key at any point, since you actually know it's always null.

Much better would be to actually represent the fact that the key is null, using 
the Void type. One such example of this is the NothingSerde I wrote here: 
https://github.com/confluentinc/kafka-streams-examples/blob/5.2.1-post/src/test/java/io/confluent/examples/streams/IntegrationTestUtils.java#L465

After some conversations, I've come to believe this would actually be a useful 
addition to the main Serdes collection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] 2.2.1 RC1

[Mickael Maison]

+1 non-binding

We've been running it for a few days in a few clusters, so far no
issues. I also ran unit tests and checked signatures.
Thanks Vahid for running this release

On Fri, May 31, 2019 at 9:01 AM Andrew Schofield
 wrote:
>
> +1 (non-binding)
>
> Built and ran source and sink connectors.
>
> Andrew Schofield - IBM
>
> On 31/05/2019, 08:55, "Viktor Somogyi-Vass"  wrote:
>
> +1 (non-binding)
>
> 1. Ran unit tests
> 2. Ran some basic automatic end-to-end tests over plaintext and SSL too
> 3. Ran systests sanity checks
>
> Viktor
>
> On Thu, May 23, 2019 at 6:04 PM Harsha  wrote:
>
> > +1 (binding)
> >
> > 1. Ran unit tests
> > 2. System tests
> > 3. 3 node cluster with few manual tests.
> >
> > Thanks,
> > Harsha
> >
> > On Wed, May 22, 2019, at 8:09 PM, Vahid Hashemian wrote:
> > > Bumping this thread to get some more votes, especially from 
> committers,
> > so
> > > we can hopefully make a decision on this RC by the end of the week.
> > >
> > > Thanks,
> > > --Vahid
> > >
> > > On Mon, May 13, 2019 at 8:15 PM Vahid Hashemian <
> > vahid.hashem...@gmail.com>
> > > wrote:
> > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the second candidate for release of Apache Kafka 2.2.1.
> > > >
> > > > Compared to RC0, this release candidate also fixes the following
> > issues:
> > > >
> > > >- [KAFKA-6789] - Add retry logic in AdminClient requests
> > > >- [KAFKA-8348] - Document of kafkaStreams improvement
> > > >- [KAFKA-7633] - Kafka Connect requires permission to create
> > internal
> > > >topics even if they exist
> > > >- [KAFKA-8240] - Source.equals() can fail with NPE
> > > >- [KAFKA-8335] - Log cleaner skips Transactional mark and batch
> > > >record, causing unlimited growth of __consumer_offsets
> > > >- [KAFKA-8352] - Connect System Tests are failing with 404
> > > >
> > > > Release notes for the 2.2.1 release:
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~vahid%2Fkafka-2.2.1-rc1%2FRELEASE_NOTES.html&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=PFuHh6XL6d20YKU%2B6V2qZa5qDVL0ET%2BR0QOFjVjqmho%3D&reserved=0
> > > >
> > > > *** Please download, test and vote by Thursday, May 16, 9:00 pm PT.
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2FKEYS&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=nB8hxA70E7kbIUd0IHPOXZ1i%2F4TgqatRrakwtqsCDOE%3D&reserved=0
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~vahid%2Fkafka-2.2.1-rc1%2F&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=6t0AGO3glzUF8ip9lcsMep9Y4Am2k6RuMthH9FuU4s4%3D&reserved=0
> > > >
> > > > * Maven artifacts to be voted upon:
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fkafka%2F&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=usesbkDDEUiZLHBjkkQVBFecT1vB437tUnFx%2B4glIlo%3D&reserved=0
> > > >
> > > > * Javadoc:
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~vahid%2Fkafka-2.2.1-rc1%2Fjavadoc%2F&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037988014&sdata=B8QrkKiUQL9yPtH8KDKDVqyPR9ZncCHfPIg1BUbxL18%3D&reserved=0
> > > >
> > > > * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fkafka%2Freleases%2Ftag%2F2.2.1-rc1&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037988014&sdata=CYy7aOpeU8j38G6HsXUr1dc0iJ538MJXKi1Uew%2BI8%2BU%3D&reserved=0
> > > >
> > > > * Documentation:
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2F22%2Fdocumentation.html&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037988014&sdata=HKpGmS%2FkOndS5y4lZAQ1104eRte1T2tpLVoddePfWDE%3D&reserved=0
> > > >
> > > > * Protocol:
> > > > 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2F22%2Fprotocol.html&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7

[jira] [Created] (KAFKA-8456) Flaky Test StoreUpgradeIntegrationTest#shouldMigratePersistentWindowStoreToTimestampedWindowStoreUsingPapi

[Boyang Chen (JIRA)]

Boyang Chen created KAFKA-8456:
--

 Summary: Flaky Test  
StoreUpgradeIntegrationTest#shouldMigratePersistentWindowStoreToTimestampedWindowStoreUsingPapi
 Key: KAFKA-8456
 URL: https://issues.apache.org/jira/browse/KAFKA-8456
 Project: Kafka
  Issue Type: Bug
  Components: streams
Reporter: Boyang Chen


[https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/22331/console]
*01:20:07* 
org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest.shouldMigratePersistentWindowStoreToTimestampedWindowStoreUsingPapi
 failed, log available in 
/home/jenkins/jenkins-slave/workspace/kafka-pr-jdk8-scala2.11/streams/build/reports/testOutput/org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest.shouldMigratePersistentWindowStoreToTimestampedWindowStoreUsingPapi.test.stdout*01:20:07*
 *01:20:07* org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest > 
shouldMigratePersistentWindowStoreToTimestampedWindowStoreUsingPapi 
FAILED*01:20:07* java.lang.AssertionError: Condition not met within timeout 
15000. Could not get expected result in time.*01:20:07* at 
org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:375)*01:20:07*  
   at 
org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:335)*01:20:07*  
   at 
org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest.verifyWindowedCountWithTimestamp(StoreUpgradeIntegrationTest.java:830)*01:20:07*
 at 
org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest.shouldMigrateWindowStoreToTimestampedWindowStoreUsingPapi(StoreUpgradeIntegrationTest.java:573)*01:20:07*
 at 
org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest.shouldMigratePersistentWindowStoreToTimestampedWindowStoreUsingPapi(StoreUpgradeIntegrationTest.java:517)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-8457) Remove Log dependency from Replica

[Vikas Singh (JIRA)]

Vikas Singh created KAFKA-8457:
--

 Summary: Remove Log dependency from Replica
 Key: KAFKA-8457
 URL: https://issues.apache.org/jira/browse/KAFKA-8457
 Project: Kafka
  Issue Type: Bug
  Components: core
Reporter: Vikas Singh
Assignee: Vikas Singh


A partition can have one log but many replicas. Putting log in replica meant 
that we have to have if-else each time we need to access log. Moving the log 
out of replica and in partition will make code simpler and it will also help in 
testing where mocks will get simplified.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : kafka-trunk-jdk11 #591

[Apache Jenkins Server]

See 

Re: [VOTE] KIP-440: Extend Connect Converter to support headers

[sapiensy]

Hey everyone, just bumping this thread again. We need one more vote from the 
committers. Thanks! :)

On 2019/05/19 14:31:15, Kamal Chandraprakash  
wrote: 
> +1 (non-binding). Thanks for the KIP!
> 
> On Sun, May 19, 2019 at 6:36 PM Dongjin Lee  wrote:
> 
> > +1 (non-binding).
> >
> > Binding: +2 (Randall, Gwen)
> > Non-binding: +2 (Andrew, Dongjin)
> >
> > We need one more +1 from the committers. Is there anyone else?
> >
> > Thanks,
> > Dongjin
> >
> > On Fri, May 10, 2019 at 12:16 AM Andrew Schofield <
> > andrew_schofi...@live.com>
> > wrote:
> >
> > > +1 (non-binding).
> > >
> > > Looks good.
> > >
> > > On 09/05/2019, 15:55, "Gwen Shapira"  wrote:
> > >
> > > +1 (binding)
> > > Thank you!
> > >
> > > On Mon, May 6, 2019, 11:25 PM Yaroslav Tkachenko  > >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > I'd like to start a vote for KIP-440: Extend Connect Converter to
> > > support
> > > > headers (
> > > >
> > > >
> > >
> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-440%253A%2BExtend%2BConnect%2BConverter%2Bto%2Bsupport%2Bheaders&data=02%7C01%7C%7C82864a9a5f3049e8ca1d08d6d48e5147%7C84df9e7fe9f640afb435%7C1%7C0%7C636930105039335723&sdata=FyQT5pfTwtT2NTMStgSl6zRjiaWFVJqG3%2B4vt5nRYmI%3D&reserved=0
> > > > )
> > > >
> > > > Discussion:
> > > >
> > > >
> > >
> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread.html%2F1fc1e3d2cddd8311d3db7c98f0d09a1a137ca4b20d1f3c8ab203a855%40%253Cdev.kafka.apache.org%253E&data=02%7C01%7C%7C82864a9a5f3049e8ca1d08d6d48e5147%7C84df9e7fe9f640afb435%7C1%7C0%7C636930105039335723&sdata=B2a4Mx9ScWaO3HEEw0LoIRX0ajETAcwUmDxt5Ir5FIs%3D&reserved=0
> > > >
> > > > Thanks!
> > > >
> > >
> > > --
> > > *Dongjin Lee*
> > >
> > > *A hitchhiker in the mathematical world.*
> > > *github:  github.com/dongjinleekr
> > > linkedin:
> > kr.linkedin.com/in/dongjinleekr
> > > speakerdeck:
> > speakerdeck.com/dongjin
> > > *
> > >
> >
> 

[jira] [Created] (KAFKA-8458) Flaky Test AdminClientIntegrationTest#testElectPreferredLeaders

[Matthias J. Sax (JIRA)]

Matthias J. Sax created KAFKA-8458:
--

 Summary: Flaky Test 
AdminClientIntegrationTest#testElectPreferredLeaders
 Key: KAFKA-8458
 URL: https://issues.apache.org/jira/browse/KAFKA-8458
 Project: Kafka
  Issue Type: Bug
  Components: admin, unit tests
Affects Versions: 2.2.0
Reporter: Matthias J. Sax


Failed locally:
{code:java}
java.util.concurrent.ExecutionException: 
org.apache.kafka.common.errors.TimeoutException: Aborted due to timeout.
at 
org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at 
org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at 
org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at 
org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at 
kafka.api.AdminClientIntegrationTest.testElectPreferredLeaders(AdminClientIntegrationTest.scala:1282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.errors.TimeoutException: Aborted due to 
timeout.
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] KIP-476: Add Java AdminClient Interface

[Matthias J. Sax]

Thanks for the KIP Andy!

Using an interface instead of an abstract class makes totally sense to me.


-Matthias

On 5/31/19 2:59 AM, Andy Coates wrote:
> Hi folks,
> 
> I'd like to start a discussion thread for KIP-476:
> 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-476%3A+Add+Java+AdminClient+Interface
> 
> Thanks,
> 
> Andy
> 



signature.asc
Description: OpenPGP digital signature

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

[Matthias J. Sax]

(1) The current PR suggests to always instantiate an `ArrayList` --
however, if a user wants to use any other list implementation, they have
no way to specify this. It might be good to either allow users to
specify the list-type on the deserializer, or encode the list type
directly in the bytes, and hence, whatever type the serialized list was,
the same type will be used on deserialization (might only work for Java
build-it list types).

Personally, I thinks its better/more flexible to specify the list-type
on the deserializer, as it also allows to plug-in any custom list types.

This could of course be opt-in and for the case users don't care, we
just default to `ArrayList`.


(2) For Java built-in types, we could check the type via `instanceof` --
if the type is unknown, we fall back to per-element length encoding. As
an alternative, we could also add a constructor taking an `enum` with
two values `fixed-size` and `variable-size`, or a config instead of a
constructor element.


Just bounding off ideas -- maybe there are good reasons (too
complicated?) to not support either of them.


-Matthias


On 5/24/19 11:09 AM, Development wrote:
> Hey,
> 
> - did we consider to make the return type (ie, ArrayList, vs
> LinkesList) configurable or encode it the serialized bytes?
> 
> Not sure about this one. Could you elaborate?
> 
> - atm the size of each element is encoded individually; did we consider
> an optimization for fixed size elements (like Long) to avoid this overhead?
> 
> I cannot think of any clean way to do so. How would you see it?
> 
> Btw I resolved all your comments under PR
> 
> Best,
> Daniyar Yeralin
> 
>> On May 24, 2019, at 12:01 AM, Matthias J. Sax  wrote:
>>
>> Thanks for the KIP. I also had a look into the PR and have two follow up
>> question:
>>
>>
>> - did we consider to make the return type (ie, ArrayList, vs
>> LinkesList) configurable or encode it the serialized bytes?
>>
>> - atm the size of each element is encoded individually; did we consider
>> an optimization for fixed size elements (like Long) to avoid this overhead?
>>
>>
>>
>> -Matthias
>>
>> On 5/15/19 6:05 PM, John Roesler wrote:
>>> Sounds good!
>>>
>>> On Tue, May 14, 2019 at 9:21 AM Development  wrote:

 Hey,

 I think it the proposal is finalized, no one raised any concerns. Shall we 
 call it for a [VOTE]?

 Best,
 Daniyar Yeralin

> On May 10, 2019, at 10:17 AM, John Roesler  wrote:
>
> Good observation, Daniyar.
>
> Maybe we should just not implement support for serdeFrom.
>
> We can always add it later, but I think you're right, we need some
> kind of more sophisticated support, or at least a second argument for
> the inner class.
>
> For now, it seems like most use cases would be satisfied without
> serdeFrom(...List...)
>
> -John
>
> On Fri, May 10, 2019 at 8:57 AM Development  wrote:
>>
>> Hi,
>>
>> I was trying to add some test cases for the list serde, and it led me to 
>> this class `org.apache.kafka.common.serialization.SerializationTest`. I 
>> saw that it relies on method 
>> `org.apache.kafka.common.serialization.serdeFrom(Class type)`
>>
>> Now, I’m not sure how to adapt List serde for this method, since it 
>> will be a “nested class”. What is the best approach in this case?
>>
>> I remember that in Jackson for example, one uses a TypeFactory, and 
>> constructs “collectionType” of two classes. For example, 
>> `constructCollectionType(List.class, String.class).getClass()`. I don’t 
>> think it applies here.
>>
>> Any ideas?
>>
>> Best,
>> Daniyar Yeralin
>>
>>> On May 9, 2019, at 2:10 PM, Development  wrote:
>>>
>>> Hey Sophie,
>>>
>>> Thank you for your input. I think I’d rather finish this KIP as is, and 
>>> then open a new one for the Collections (if everyone agrees). I don’t 
>>> want to extend the current KIP-466, since most of the work is already 
>>> done for it.
>>>
>>> Meanwhile, I’ll start adding some test cases for this new list serde 
>>> since this discussion seems to be approaching its logical end.
>>>
>>> Best,
>>> Daniyar Yeralin
>>>
 On May 9, 2019, at 1:35 PM, Sophie Blee-Goldman  
 wrote:

 Good point about serdes for other Collections. On the one hand I'd 
 guess
 that non-List Collections are probably relatively rare in practice (if
 anyone disagrees please correct me!) but on the other hand, a) even if 
 just
 a small number of people benefit I think it's worth the extra effort 
 and b)
 if we do end up needing/wanting them in the future it would save us a 
 KIP
 to just add them now. Personally I feel it would make sense to expand 
 the
 scope of this KIP a bit to include all Collections as a logical unit, 
 but
>

[jira] [Created] (KAFKA-8459) Flakey test BaseQuotaTest#testThrottledRequest

[Boyang Chen (JIRA)]

Boyang Chen created KAFKA-8459:
--

 Summary: Flakey test BaseQuotaTest#testThrottledRequest
 Key: KAFKA-8459
 URL: https://issues.apache.org/jira/browse/KAFKA-8459
 Project: Kafka
  Issue Type: Bug
Reporter: Boyang Chen


[https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/5151/consoleFull]
*12:14:29* kafka.api.UserQuotaTest > testThrottledRequest STARTED*12:14:57* 
kafka.api.UserQuotaTest.testThrottledRequest failed, log available in 
/home/jenkins/jenkins-slave/workspace/kafka-pr-jdk11-scala2.12/core/build/reports/testOutput/kafka.api.UserQuotaTest.testThrottledRequest.test.stdout*12:14:57*
 *12:14:57* kafka.api.UserQuotaTest > testThrottledRequest FAILED*12:14:57* 
org.scalatest.exceptions.TestFailedException: Consumer throttle metric not 
updated: avg=0.0 max=0.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] KIP-472: Add header to RecordContext

[Matthias J. Sax]

Thanks for the KIP.

However, I have some doubts that this would work.

In the example, you mention 4 records

> r1(2), r2(3), r3(7), and r4(9)

and that if r3 and r4 would be filtered, the downstream task would not
advance partition time from 3 to 9.

However, if r3 and r4 are filtered, no record will be sent downstream at
all -- hence, there is no record to which partition-time could be
piggy-bagged onto its header.

When we sent r2, we don't know anything about r3 and r4 yet. And if
there is an r5 that is not filtered, r5 would advance partition time
anyway based on its own timestamp, hence no header is required.

Also, I am not a big fan of adding headers in general, as they "leak"
internal implementation details.


Overall, my personal opinion is, that we should change Kafka's message
format and allow for "heartbeat" messages, that don't carry any data,
but only a timestamp. By default, those messages would not be exposed to
an application but would be considered "internal" similar to transaction
markers. However, changing the message format is a mayor change and
hence, I am not sure if it worth doing at all atm.


-Matthias



On 5/20/19 7:20 PM, Richard Yu wrote:
> Hello,
> 
> I wish to introduce a minor addition present in RecordContext (a public
> facing API). This addition works to both provide the user with
> more information regarding the processing state of the partition, but also
> help resolve a bug which Kafka is currently experiencing.
> Here is the KIP Link:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-472%3A+%5BSTREAMS%5D+Add+partition+time+field+to+RecordContext
> 
> Cheers,
> Richard Yu
> 



signature.asc
Description: OpenPGP digital signature

Re: [DISCUSS] KIP-472: Add header to RecordContext

[Richard Yu]

Hi Matthias,

Thanks for responding. :)

I suppose from the scope of the change that is needed to fix the timestamp
propagation bug is too complex (in other words, probably not worth it).
So should this KIP be closed since it might be a little excessive?

There probably is no need for too big of a change like this one.

On Fri, May 31, 2019 at 3:17 PM Matthias J. Sax 
wrote:

> Thanks for the KIP.
>
> However, I have some doubts that this would work.
>
> In the example, you mention 4 records
>
> > r1(2), r2(3), r3(7), and r4(9)
>
> and that if r3 and r4 would be filtered, the downstream task would not
> advance partition time from 3 to 9.
>
> However, if r3 and r4 are filtered, no record will be sent downstream at
> all -- hence, there is no record to which partition-time could be
> piggy-bagged onto its header.
>
> When we sent r2, we don't know anything about r3 and r4 yet. And if
> there is an r5 that is not filtered, r5 would advance partition time
> anyway based on its own timestamp, hence no header is required.
>
> Also, I am not a big fan of adding headers in general, as they "leak"
> internal implementation details.
>
>
> Overall, my personal opinion is, that we should change Kafka's message
> format and allow for "heartbeat" messages, that don't carry any data,
> but only a timestamp. By default, those messages would not be exposed to
> an application but would be considered "internal" similar to transaction
> markers. However, changing the message format is a mayor change and
> hence, I am not sure if it worth doing at all atm.
>
>
> -Matthias
>
>
>
> On 5/20/19 7:20 PM, Richard Yu wrote:
> > Hello,
> >
> > I wish to introduce a minor addition present in RecordContext (a public
> > facing API). This addition works to both provide the user with
> > more information regarding the processing state of the partition, but
> also
> > help resolve a bug which Kafka is currently experiencing.
> > Here is the KIP Link:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-472%3A+%5BSTREAMS%5D+Add+partition+time+field+to+RecordContext
> >
> > Cheers,
> > Richard Yu
> >
>
>

Re: [VOTE] 2.2.1 RC1

[Matthias J. Sax]

Thanks for running the release Harsha!

- downloaded source code, built, and run tests locally;
  required 3 runs to get tests passing

Failing test tracked as:
 - https://issues.apache.org/jira/browse/KAFKA-8260
 - https://issues.apache.org/jira/browse/KAFKA-8458


- run quickstart with Scala 2.11 binaries

- verified signatures for source and both binaries



+1 (binding)



-Matthias


On 5/31/19 8:30 AM, Mickael Maison wrote:
> +1 non-binding
> 
> We've been running it for a few days in a few clusters, so far no
> issues. I also ran unit tests and checked signatures.
> Thanks Vahid for running this release
> 
> On Fri, May 31, 2019 at 9:01 AM Andrew Schofield
>  wrote:
>>
>> +1 (non-binding)
>>
>> Built and ran source and sink connectors.
>>
>> Andrew Schofield - IBM
>>
>> On 31/05/2019, 08:55, "Viktor Somogyi-Vass"  wrote:
>>
>> +1 (non-binding)
>>
>> 1. Ran unit tests
>> 2. Ran some basic automatic end-to-end tests over plaintext and SSL too
>> 3. Ran systests sanity checks
>>
>> Viktor
>>
>> On Thu, May 23, 2019 at 6:04 PM Harsha  wrote:
>>
>> > +1 (binding)
>> >
>> > 1. Ran unit tests
>> > 2. System tests
>> > 3. 3 node cluster with few manual tests.
>> >
>> > Thanks,
>> > Harsha
>> >
>> > On Wed, May 22, 2019, at 8:09 PM, Vahid Hashemian wrote:
>> > > Bumping this thread to get some more votes, especially from 
>> committers,
>> > so
>> > > we can hopefully make a decision on this RC by the end of the week.
>> > >
>> > > Thanks,
>> > > --Vahid
>> > >
>> > > On Mon, May 13, 2019 at 8:15 PM Vahid Hashemian <
>> > vahid.hashem...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hello Kafka users, developers and client-developers,
>> > > >
>> > > > This is the second candidate for release of Apache Kafka 2.2.1.
>> > > >
>> > > > Compared to RC0, this release candidate also fixes the following
>> > issues:
>> > > >
>> > > >- [KAFKA-6789] - Add retry logic in AdminClient requests
>> > > >- [KAFKA-8348] - Document of kafkaStreams improvement
>> > > >- [KAFKA-7633] - Kafka Connect requires permission to create
>> > internal
>> > > >topics even if they exist
>> > > >- [KAFKA-8240] - Source.equals() can fail with NPE
>> > > >- [KAFKA-8335] - Log cleaner skips Transactional mark and batch
>> > > >record, causing unlimited growth of __consumer_offsets
>> > > >- [KAFKA-8352] - Connect System Tests are failing with 404
>> > > >
>> > > > Release notes for the 2.2.1 release:
>> > > > 
>> https://nam04.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~vahid%2Fkafka-2.2.1-rc1%2FRELEASE_NOTES.html&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=PFuHh6XL6d20YKU%2B6V2qZa5qDVL0ET%2BR0QOFjVjqmho%3D&reserved=0
>> > > >
>> > > > *** Please download, test and vote by Thursday, May 16, 9:00 pm PT.
>> > > >
>> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > > > 
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2FKEYS&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=nB8hxA70E7kbIUd0IHPOXZ1i%2F4TgqatRrakwtqsCDOE%3D&reserved=0
>> > > >
>> > > > * Release artifacts to be voted upon (source and binary):
>> > > > 
>> https://nam04.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~vahid%2Fkafka-2.2.1-rc1%2F&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=6t0AGO3glzUF8ip9lcsMep9Y4Am2k6RuMthH9FuU4s4%3D&reserved=0
>> > > >
>> > > > * Maven artifacts to be voted upon:
>> > > > 
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fkafka%2F&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037978009&sdata=usesbkDDEUiZLHBjkkQVBFecT1vB437tUnFx%2B4glIlo%3D&reserved=0
>> > > >
>> > > > * Javadoc:
>> > > > 
>> https://nam04.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~vahid%2Fkafka-2.2.1-rc1%2Fjavadoc%2F&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037988014&sdata=B8QrkKiUQL9yPtH8KDKDVqyPR9ZncCHfPIg1BUbxL18%3D&reserved=0
>> > > >
>> > > > * Tag to be voted upon (off 2.2 branch) is the 2.2.1 tag:
>> > > > 
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fkafka%2Freleases%2Ftag%2F2.2.1-rc1&data=02%7C01%7C%7C139a19a5204441d9a17908d6e59d49bf%7C84df9e7fe9f640afb435%7C1%7C0%7C636948861037988014&sdata=CYy7aOpeU8j38G6HsXUr1dc0iJ538MJXKi1Uew%2BI8%2BU%3D&reserved=0
>> > > >
>> > > > * 

Re: [DISCUSS] KIP-475: New Metric to Measure Number of Tasks on a Connector

[Oleksandr Diachenko]

On 2019/05/30 06:06:12, Cyrus Vafadari  wrote: 
> Hello Dev,
> 
> I'd like to start the discussion of KIP-475: New Metric to Measure Number
> of Tasks on a Connector.
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-475%3A+New+Metric+to+Measure+Number+of+Tasks+on+a+Connector
> 
> The proposal is pretty straightforward -- to add a new metric to Connect to
> measure the number of tasks on a Connector. Currently, we support this on
> Worker level, so this KIP just adds another metric to support this
> per-connector.
> 
> There is also a PR:
> https://github.com/apache/kafka/pull/6843
> 
> Thanks,
> 
> Cyrus
> 

Hi Cyrus,

That sounds like a useful addition.

Regards, Alex.

Jenkins build is back to normal : kafka-2.3-jdk8 #30

[Apache Jenkins Server]

See 

Re: [DISCUSS] KIP-472: Add header to RecordContext

[Matthias J. Sax]

Sure thing :)

For the original issue to preserve partition-time within a task across
restarts, we don't need this KIP.

And I agree that downstream time propagation via heartbeats is nothing
critical atm but may require a big change. Hence, maybe not worth the
effort right now.

If you are fine with it, we can discard this KIP for now. Maybe we can
revisit this idea at a later point when we really need it.

Thanks a lot!


-Matthias

On 5/31/19 4:19 PM, Richard Yu wrote:
> Hi Matthias,
> 
> Thanks for responding. :)
> 
> I suppose from the scope of the change that is needed to fix the timestamp
> propagation bug is too complex (in other words, probably not worth it).
> So should this KIP be closed since it might be a little excessive?
> 
> There probably is no need for too big of a change like this one.
> 
> On Fri, May 31, 2019 at 3:17 PM Matthias J. Sax 
> wrote:
> 
>> Thanks for the KIP.
>>
>> However, I have some doubts that this would work.
>>
>> In the example, you mention 4 records
>>
>>> r1(2), r2(3), r3(7), and r4(9)
>>
>> and that if r3 and r4 would be filtered, the downstream task would not
>> advance partition time from 3 to 9.
>>
>> However, if r3 and r4 are filtered, no record will be sent downstream at
>> all -- hence, there is no record to which partition-time could be
>> piggy-bagged onto its header.
>>
>> When we sent r2, we don't know anything about r3 and r4 yet. And if
>> there is an r5 that is not filtered, r5 would advance partition time
>> anyway based on its own timestamp, hence no header is required.
>>
>> Also, I am not a big fan of adding headers in general, as they "leak"
>> internal implementation details.
>>
>>
>> Overall, my personal opinion is, that we should change Kafka's message
>> format and allow for "heartbeat" messages, that don't carry any data,
>> but only a timestamp. By default, those messages would not be exposed to
>> an application but would be considered "internal" similar to transaction
>> markers. However, changing the message format is a mayor change and
>> hence, I am not sure if it worth doing at all atm.
>>
>>
>> -Matthias
>>
>>
>>
>> On 5/20/19 7:20 PM, Richard Yu wrote:
>>> Hello,
>>>
>>> I wish to introduce a minor addition present in RecordContext (a public
>>> facing API). This addition works to both provide the user with
>>> more information regarding the processing state of the partition, but
>> also
>>> help resolve a bug which Kafka is currently experiencing.
>>> Here is the KIP Link:
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-472%3A+%5BSTREAMS%5D+Add+partition+time+field+to+RecordContext
>>>
>>> Cheers,
>>> Richard Yu
>>>
>>
>>
> 



signature.asc
Description: OpenPGP digital signature

[jira] [Created] (KAFKA-8460) Flaky Test PlaintextConsumerTest#testLowMaxFetchSizeForRequestAndPartition

[Boyang Chen (JIRA)]

Boyang Chen created KAFKA-8460:
--

 Summary: Flaky Test  
PlaintextConsumerTest#testLowMaxFetchSizeForRequestAndPartition
 Key: KAFKA-8460
 URL: https://issues.apache.org/jira/browse/KAFKA-8460
 Project: Kafka
  Issue Type: Bug
Reporter: Boyang Chen


 
*16:17:04* kafka.api.PlaintextConsumerTest > 
testLowMaxFetchSizeForRequestAndPartition FAILED*16:17:04* 
org.scalatest.exceptions.TestFailedException: Timed out before consuming 
expected 2700 records. The number consumed was 1980.*16:17:04* at 
org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)*16:17:04*
 at 
org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:529)*16:17:04*
 at 
org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389)*16:17:04*
 at org.scalatest.Assertions.fail(Assertions.scala:1091)*16:17:04*  
   at org.scalatest.Assertions.fail$(Assertions.scala:1087)*16:17:04* 
at org.scalatest.Assertions$.fail(Assertions.scala:1389)*16:17:04* at 
kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:789)*16:17:04* at 
kafka.utils.TestUtils$.pollRecordsUntilTrue(TestUtils.scala:765)*16:17:04*  
   at 
kafka.api.AbstractConsumerTest.consumeRecords(AbstractConsumerTest.scala:156)*16:17:04*
 at 
kafka.api.PlaintextConsumerTest.testLowMaxFetchSizeForRequestAndPartition(PlaintextConsumerTest.scala:801)*16:17:04*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-8461) Flakey test UncleanLeaderElectionTest#testUncleanLeaderElectionDisabledByTopicOverride

[Boyang Chen (JIRA)]

Boyang Chen created KAFKA-8461:
--

 Summary: Flakey test 
UncleanLeaderElectionTest#testUncleanLeaderElectionDisabledByTopicOverride
 Key: KAFKA-8461
 URL: https://issues.apache.org/jira/browse/KAFKA-8461
 Project: Kafka
  Issue Type: Bug
  Components: core
Reporter: Boyang Chen


[https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/5168/consoleFull]
*15:47:56* kafka.integration.UncleanLeaderElectionTest > 
testUncleanLeaderElectionDisabledByTopicOverride FAILED*15:47:56* 
org.scalatest.exceptions.TestFailedException: Timing out after 3 ms since 
expected new leader 1 was not elected for partition 
topic-9147891452427084986-0, leader is Some(-1)*15:47:56* at 
org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:530)*15:47:56*
 at 
org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:529)*15:47:56*
 at 
org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389)*15:47:56*
 at org.scalatest.Assertions.fail(Assertions.scala:1091)*15:47:56*  
   at org.scalatest.Assertions.fail$(Assertions.scala:1087)*15:47:56* 
at org.scalatest.Assertions$.fail(Assertions.scala:1389)*15:47:56* at 
kafka.utils.TestUtils$.$anonfun$waitUntilLeaderIsElectedOrChanged$8(TestUtils.scala:722)*15:47:56*
 at scala.Option.getOrElse(Option.scala:138)*15:47:56* at 
kafka.utils.TestUtils$.waitUntilLeaderIsElectedOrChanged(TestUtils.scala:712)*15:47:56*
 at 
kafka.integration.UncleanLeaderElectionTest.verifyUncleanLeaderElectionDisabled(UncleanLeaderElectionTest.scala:258)*15:47:56*
 at 
kafka.integration.UncleanLeaderElectionTest.testUncleanLeaderElectionDisabledByTopicOverride(UncleanLeaderElectionTest.scala:153)*15:47:56*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KAFKA-8199) ClassCastException when trying to groupBy after suppress

[Guozhang Wang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-8199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guozhang Wang resolved KAFKA-8199.
--
Resolution: Fixed

> ClassCastException when trying to groupBy after suppress
> 
>
> Key: KAFKA-8199
> URL: https://issues.apache.org/jira/browse/KAFKA-8199
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 2.1.0
>Reporter: Bill Bejeck
>Assignee: Jose Lopez
>Priority: Blocker
> Fix For: 2.3.0
>
>
> A topology with a groupBy after a suppress operation results in a 
> ClassCastException
>  The following sample topology
> {noformat}
> Properties properties = new Properties(); 
> properties.put(StreamsConfig.APPLICATION_ID_CONFIG, "appid"); 
> properties.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG,"localhost");
> StreamsBuilder builder = new StreamsBuilder();
>  builder.stream("topic")
> .groupByKey().windowedBy(TimeWindows.of(Duration.ofSeconds(30))).count() 
> .suppress(Suppressed.untilTimeLimit(Duration.ofHours(1), 
> BufferConfig.unbounded())) 
> .groupBy((k, v) -> KeyValue.pair(k,v)).count().toStream(); 
> builder.build(properties);
> {noformat}
> results in this exception:
> {noformat}
> java.lang.ClassCastException: 
> org.apache.kafka.streams.kstream.internals.KTableImpl$$Lambda$4/2084435065 
> cannot be cast to 
> org.apache.kafka.streams.kstream.internals.KTableProcessorSupplier{noformat}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[VOTE] KIP-474: To deprecate WindowStore#put(key, value)

[omkar mestry]

Hi all,

Since we seem to have an agreement in the discussion I would like to
start the vote on KIP-474.

KIP 474 :-
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=115526545

Thanks & Regards
Omkar Mestry

Re: [VOTE] KIP-474: To deprecate WindowStore#put(key, value)

[Boyang Chen]

Thanks omkar for taking the initiative, +1 (non-binding).


From: omkar mestry 
Sent: Saturday, June 1, 2019 1:40 PM
To: dev@kafka.apache.org
Subject: [VOTE] KIP-474: To deprecate WindowStore#put(key, value)

Hi all,

Since we seem to have an agreement in the discussion I would like to
start the vote on KIP-474.

KIP 474 :-
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=115526545

Thanks & Regards
Omkar Mestry

Re: [VOTE] KIP-474: To deprecate WindowStore#put(key, value)

[Dongjin Lee]

+1 (non-binding).

Thanks,
Dongjin


Virus-free.
www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Sat, Jun 1, 2019 at 2:45 PM Boyang Chen  wrote:

> Thanks omkar for taking the initiative, +1 (non-binding).
>
> 
> From: omkar mestry 
> Sent: Saturday, June 1, 2019 1:40 PM
> To: dev@kafka.apache.org
> Subject: [VOTE] KIP-474: To deprecate WindowStore#put(key, value)
>
> Hi all,
>
> Since we seem to have an agreement in the discussion I would like to
> start the vote on KIP-474.
>
> KIP 474 :-
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=115526545
>
> Thanks & Regards
> Omkar Mestry
>


-- 
*Dongjin Lee*

*A hitchhiker in the mathematical world.*
*github:  github.com/dongjinleekr
linkedin: kr.linkedin.com/in/dongjinleekr
speakerdeck: speakerdeck.com/dongjin
*

Build failed in Jenkins: kafka-trunk-jdk8 #3694

[Apache Jenkins Server]

See 


Changes:

[github] MINOR: Reordering the props modification with configs construction

--
[...truncated 4.07 MB...]

kafka.log.LogCleanerParameterizedIntegrationTest > 
testCleaningNestedMessagesWithMultipleVersions[4] STARTED

kafka.log.LogCleanerParameterizedIntegrationTest > 
testCleaningNestedMessagesWithMultipleVersions[4] PASSED

kafka.log.LogCleanerParameterizedIntegrationTest > cleanerTest[4] STARTED

kafka.log.LogCleanerParameterizedIntegrationTest > cleanerTest[4] PASSED

kafka.log.LogCleanerParameterizedIntegrationTest > 
testCleanerWithMessageFormatV0[4] STARTED

kafka.log.LogCleanerParameterizedIntegrationTest > 
testCleanerWithMessageFormatV0[4] PASSED

kafka.log.TimeIndexTest > testTruncate STARTED

kafka.log.TimeIndexTest > testTruncate PASSED

kafka.log.TimeIndexTest > testEntry STARTED

kafka.log.TimeIndexTest > testEntry PASSED

kafka.log.TimeIndexTest > testAppend STARTED

kafka.log.TimeIndexTest > testAppend PASSED

kafka.log.TimeIndexTest > testEntryOverflow STARTED

kafka.log.TimeIndexTest > testEntryOverflow PASSED

kafka.log.TimeIndexTest > testLookUp STARTED

kafka.log.TimeIndexTest > testLookUp PASSED

kafka.log.TimeIndexTest > testSanityCheck STARTED

kafka.log.TimeIndexTest > testSanityCheck PASSED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[0] STARTED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[0] PASSED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[1] STARTED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[1] PASSED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[2] STARTED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[2] PASSED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[3] STARTED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[3] PASSED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[4] STARTED

kafka.log.LogCleanerLagIntegrationTest > cleanerTest[4] PASSED

kafka.log.ProducerStateManagerTest > 
testProducerSequenceWithWrapAroundBatchRecord STARTED

kafka.log.ProducerStateManagerTest > 
testProducerSequenceWithWrapAroundBatchRecord PASSED

kafka.log.ProducerStateManagerTest > testCoordinatorFencing STARTED

kafka.log.ProducerStateManagerTest > testCoordinatorFencing PASSED

kafka.log.ProducerStateManagerTest > testTruncate STARTED

kafka.log.ProducerStateManagerTest > testTruncate PASSED

kafka.log.ProducerStateManagerTest > testLoadFromTruncatedSnapshotFile STARTED

kafka.log.ProducerStateManagerTest > testLoadFromTruncatedSnapshotFile PASSED

kafka.log.ProducerStateManagerTest > testRemoveExpiredPidsOnReload STARTED

kafka.log.ProducerStateManagerTest > testRemoveExpiredPidsOnReload PASSED

kafka.log.ProducerStateManagerTest > 
testOutOfSequenceAfterControlRecordEpochBump STARTED

kafka.log.ProducerStateManagerTest > 
testOutOfSequenceAfterControlRecordEpochBump PASSED

kafka.log.ProducerStateManagerTest > testFirstUnstableOffsetAfterTruncation 
STARTED

kafka.log.ProducerStateManagerTest > testFirstUnstableOffsetAfterTruncation 
PASSED

kafka.log.ProducerStateManagerTest > testTakeSnapshot STARTED

kafka.log.ProducerStateManagerTest > testTakeSnapshot PASSED

kafka.log.ProducerStateManagerTest > testDeleteSnapshotsBefore STARTED

kafka.log.ProducerStateManagerTest > testDeleteSnapshotsBefore PASSED

kafka.log.ProducerStateManagerTest > 
testNonMatchingTxnFirstOffsetMetadataNotCached STARTED

kafka.log.ProducerStateManagerTest > 
testNonMatchingTxnFirstOffsetMetadataNotCached PASSED

kafka.log.ProducerStateManagerTest > testAppendEmptyControlBatch STARTED

kafka.log.ProducerStateManagerTest > testAppendEmptyControlBatch PASSED

kafka.log.ProducerStateManagerTest > testFirstUnstableOffsetAfterEviction 
STARTED

kafka.log.ProducerStateManagerTest > testFirstUnstableOffsetAfterEviction PASSED

kafka.log.ProducerStateManagerTest > testNoValidationOnFirstEntryWhenLoadingLog 
STARTED

kafka.log.ProducerStateManagerTest > testNoValidationOnFirstEntryWhenLoadingLog 
PASSED

kafka.log.ProducerStateManagerTest > testLoadFromEmptySnapshotFile STARTED

kafka.log.ProducerStateManagerTest > testLoadFromEmptySnapshotFile PASSED

kafka.log.ProducerStateManagerTest > 
testProducersWithOngoingTransactionsDontExpire STARTED

kafka.log.ProducerStateManagerTest > 
testProducersWithOngoingTransactionsDontExpire PASSED

kafka.log.ProducerStateManagerTest > testBasicIdMapping STARTED

kafka.log.ProducerStateManagerTest > testBasicIdMapping PASSED

kafka.log.ProducerStateManagerTest > updateProducerTransactionState STARTED

kafka.log.ProducerStateManagerTest > updateProducerTransactionState PASSED

kafka.log.ProducerStateManagerTest > testRecoverFromSnapshot STARTED

kafka.log.ProducerStateManagerTest > testRecoverFromSnapshot PASSED

kafka.log.ProducerStateManagerTest > testPrepareUpdateDoesNotMutate STARTED

kafka.log.ProducerStateManagerTest > testPrepareUpdateDoesNotMutate PASSED

kafka.log.Prod