Not sure if you have looked at FLINK-8707 FYI
On Tue, Mar 20, 2018 at 2:13 PM, Govindarajan Srinivasaraghavan < govindragh...@gmail.com> wrote: > Hi, > > We have a streaming job that runs on flink in docker and checkpointing > happens every 10 seconds. After several starts and cancellations we are > facing this issue with file handles. > > The job reads data from kafka, processes it and writes it back to kafka and > we are using RocksDB state backend. For now we have increased the number > file handles to resolve the problem but would like to know if this is > expected or is it an issue. Thanks. > > java.io.FileNotFoundException: > /tmp/flink-io-b3043cd6-50c8-446a-8c25-fade1b1862c0/ > cb317fc2578db72b3046468948fa00f2f17039b6104e72fb8c58938e5869cfbc.0.buffer > (Too many open files) > > at java.io.RandomAccessFile.open0(Native Method) > > at java.io.RandomAccessFile.open(RandomAccessFile.java:316) > > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) > > at > org.apache.flink.streaming.runtime.io.BufferSpiller.createSpillingChannel( > BufferSpiller.java:259) > > at > org.apache.flink.streaming.runtime.io.BufferSpiller.< > init>(BufferSpiller.java:120) > > at > org.apache.flink.streaming.runtime.io.BarrierBuffer.< > init>(BarrierBuffer.java:149) > > at > org.apache.flink.streaming.runtime.io.StreamTwoInputProcessor.<init> > (StreamTwoInputProcessor.java:147) > > at > org.apache.flink.streaming.runtime.tasks.TwoInputStreamTask.init( > TwoInputStreamTask.java:79) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask. > invoke(StreamTask.java:235) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) > > at java.lang.Thread.run(Thread.java:748) > > Regards, > Govind >
