[
https://issues.apache.org/jira/browse/KAFKA-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080229#comment-17080229
]
Lovro Pandžić commented on KAFKA-9603:
--------------------------------------
Thanks, it seems I've been able to reproduce the issue on our test environment.
I'll try to reproduce it locally and isolate the cause as much as possible,
ideally I'll be able to share a project that reproduces the issue.
> Number of open files keeps increasing in Streams application
> ------------------------------------------------------------
>
> Key: KAFKA-9603
> URL: https://issues.apache.org/jira/browse/KAFKA-9603
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.4.0, 2.3.1
> Environment: Spring Boot 2.2.4, OpenJDK 13, Centos image
> Reporter: Bruno Iljazovic
> Priority: Major
>
> Problem appeared when upgrading from *2.0.1* to *2.3.1*.
> Relevant Kafka Streams code:
> {code:java}
> KStream<String, Event1> events1 =
> builder.stream(FIRST_TOPIC_NAME, Consumed.with(stringSerde, event1Serde,
> event1TimestampExtractor(), null))
> .mapValues(...);
> KStream<String, Event2> events2 =
> builder.stream(SECOND_TOPIC_NAME, Consumed.with(stringSerde, event2Serde,
> event2TimestampExtractor(), null))
> .mapValues(...);
> var joinWindows = JoinWindows.of(Duration.of(1, MINUTES).toMillis())
> .until(Duration.of(1, HOURS).toMillis());
> events2.join(events1, this::join, joinWindows, Joined.with(stringSerde,
> event2Serde, event1Serde))
> .foreach(...);
> {code}
> Number of open *.sst files keeps increasing until eventually it hits the os
> limit (65536) and causes this exception:
> {code:java}
> Caused by: org.rocksdb.RocksDBException: While open a file for appending:
> /.../0_8/KSTREAM-JOINOTHER-0000000010-store/KSTREAM-JOINOTHER-0000000010-store.1579435200000/001354.sst:
> Too many open files
> at org.rocksdb.RocksDB.flush(Native Method)
> at org.rocksdb.RocksDB.flush(RocksDB.java:2394)
> {code}
> Here are example files that are opened and never closed:
> {code:java}
> /.../0_27/KSTREAM-JOINTHIS-0000000009-store/KSTREAM-JOINTHIS-0000000009-store.1582459200000/000114.sst
> /.../0_27/KSTREAM-JOINOTHER-0000000010-store/KSTREAM-JOINOTHER-0000000010-store.1582459200000/000065.sst
> /.../0_29/KSTREAM-JOINTHIS-0000000009-store/KSTREAM-JOINTHIS-0000000009-store.1582156800000/000115.sst
> /.../0_29/KSTREAM-JOINTHIS-0000000009-store/KSTREAM-JOINTHIS-0000000009-store.1582459200000/000112.sst
> /.../0_31/KSTREAM-JOINTHIS-0000000009-store/KSTREAM-JOINTHIS-0000000009-store.1581854400000/000051.sst
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
