The FSStatebackend uses the heap to keep the data, only the state snapshots are stored in the file system.
On Thu, Feb 23, 2017 at 6:13 PM, vinay patil <vinay18.pa...@gmail.com> wrote: > Hi, > > When I disabled checkpointing the memory usage is similar for all nodes, > this means that for checkpointing enabled case the data is first flushed > to memory of CORE nodes (DataNode daemon is running here in case of EMR ) . > > I am going to run with FSStatebackend on a high end cluster with 122GB > RAM, in case of FSStatebackend does it use TM heap memory or physical > memory to store the state ? > > Regards, > Vinay Patil > > On Thu, Feb 23, 2017 at 7:50 PM, vinay patil [via Apache Flink User > Mailing List archive.] <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=11845&i=0>> wrote: > >> Hi Stephan, >> >> Anyways the Async exception is gone. >> >> I have increased my instance type to r3.2xlarge having 60GB of memory. >> BUt what I have observed here is that for two task managers the memory >> usage is close to 30GB but for other two it goes up to 55GB, the load is >> equally distributed among all TM's. >> Why does this happen ? >> >> ------------------------------ >> If you reply to this email, your message will be added to the discussion >> below: >> http://apache-flink-user-mailing-list-archive.2336050.n4. >> nabble.com/Re-Checkpointing-with-RocksDB-as-statebackend- >> tp11752p11831.html >> To start a new topic under Apache Flink User Mailing List archive., email >> [hidden >> email] <http:///user/SendEmail.jtp?type=node&node=11845&i=1> >> To unsubscribe from Apache Flink User Mailing List archive., click here. >> NAML >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > > ------------------------------ > View this message in context: Re: Checkpointing with RocksDB as > statebackend > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Checkpointing-with-RocksDB-as-statebackend-tp11752p11845.html> > > Sent from the Apache Flink User Mailing List archive. mailing list archive > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at > Nabble.com. >
