Agree it is definitely not intuitive trying to figure out what to do based on this message. I think the message should be changed to "please consider increasing maximum permitted memory size, increase task manager parallelism, or using a non-memory-based state backend".
Could you please open a JIRA ticket in: https://issues.apache.org/jira/projects/FLINK/issues/ Thanks, Rong On Thu, May 3, 2018 at 7:56 PM, Ken Krugler <kkrugler_li...@transpac.com> wrote: > Currently in the MemCheckpointStreamFactory.checkSize() method, it can > throw an IOException with: > > throw new IOException( > "Size of the state is larger than > the maximum permitted memory-backed state. Size=" > + size + " , > maxSize=" + maxSize > + " . Consider > using a different state backend, like the File System State backend."); > > But this will happen even if you’re using the File System State backend, > right? > > This came up here: https://stackoverflow.com/questions/50149005/ > ioexception-size-of-the-state-is-larger-than-the-maximum- > permitted-memory-backe <https://stackoverflow.com/questions/50149005/ > ioexception-size-of-the-state-is-larger-than-the-maximum- > permitted-memory-backe> > > So shouldn’t it suggest using the RocksDB state backend instead? > > — Ken > > -------------------------- > Ken Krugler > +1 530-210-6378 > http://www.scaleunlimited.com > Custom big data solutions & training > Flink, Solr, Hadoop, Cascading & Cassandra > >
