Re: Flink RocksDB logs filling up disk space

2020-01-28 Thread Ahmad Hassan
iour-configuration > > Best > Yun Tang > -- > *From:* Ahmad Hassan > *Sent:* Tuesday, January 28, 2020 17:43 > *To:* user > *Subject:* Re: Flink RocksDB logs filling up disk space > > Hi Yun, > > Thank you for pointing that out. In our production landscapes with

Re: Flink RocksDB logs filling up disk space

2020-01-28 Thread Yun Tang
https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-offset-committing-behaviour-configuration Best Yun Tang From: Ahmad Hassan Sent: Tuesday, January 28, 2020 17:43 To: user Subject: Re: Flink RocksDB logs filling up disk

Re: Flink RocksDB logs filling up disk space

2020-01-28 Thread Ahmad Hassan
al of > checkpoint, even you could avoid to record too many logs, and I don't think > current checkpoint configuration is appropriate. > > Best > Yun Tang > -- > *From:* Ahmad Hassan > *Sent:* Monday, January 27, 2020 20:22 > *To:* user &

Re: Flink RocksDB logs filling up disk space

2020-01-27 Thread Yun Tang
checkpoint, even you could avoid to record too many logs, and I don't think current checkpoint configuration is appropriate. Best Yun Tang From: Ahmad Hassan Sent: Monday, January 27, 2020 20:22 To: user Subject: Re: Flink RocksDB logs filling up disk

Re: Flink RocksDB logs filling up disk space

2020-01-27 Thread Ahmad Hassan
Thanks Chesnay! On Mon, 27 Jan 2020 at 11:29, Chesnay Schepler wrote: > Please see https://issues.apache.org/jira/browse/FLINK-15068 > > On 27/01/2020 12:22, Ahmad Hassan wrote: > > Hello, > > In our production systems, we see that flink rocksdb checkpoint IO logs > are filling up disk space ver

Re: Flink RocksDB logs filling up disk space

2020-01-27 Thread Chesnay Schepler
Please see https://issues.apache.org/jira/browse/FLINK-15068 On 27/01/2020 12:22, Ahmad Hassan wrote: Hello, In our production systems, we see that flink rocksdb checkpoint IO logs are filling up disk space very very quickly in the order of GB's as the logging is very verbose. How do we disab

Flink RocksDB logs filling up disk space

2020-01-27 Thread Ahmad Hassan
Hello, In our production systems, we see that flink rocksdb checkpoint IO logs are filling up disk space very very quickly in the order of GB's as the logging is very verbose. How do we disable or suppress these logs please ? The rocksdb file checkpoint.cc is dumping huge amount of checkpoint logs