Hi,

When using IGFS with a secondary file system, with write behind configured
by using DUAL_ASYNC IgfsMode, is there any way to force the flush of the
data from the Ignite caches into the secondary file system? A possible
scenario here might be a temporary cluster with Ignite installed, that uses
IGFS with DUAL_ASYNC to write to an HDFS cluster running in a permanent
cluster that is configured as the secondary file system. In order to be
able to shutdown this cluster we need to know that all the data has been
flushed to HDFS or we might have data loss. For what I see in
http://apache-ignite-users.70518.x6.nabble.com/Flush-the-cache-into-the-persistence-store-manually-td5077.html
this wasn't available at the time that question was answered. The solution
proposed there seems to be traversing the cache writing each cached entry
to the data store that is cached. But for IGFS I understand that is not so
straightforward, because the dataCache and metadataCache used by IGFS don't
store the HDFS files directly, but the result of splitting them into
pieces.

Is there any way to flush the data from IGFS into HDFS? If not, is there
any recommendation about how we could traverse the dataCache and
metadataCache used by IGFS to manually write the data into HDFS? If we do
that traversal, is there any way to avoid the async writes of IGFS and the
write done in that traversal to interfere with each other, or lead to
duplicate writes?

Thanks a lot for your help!

Juan Rodriguez Hortala

Reply via email to