Dmitriy, A few questions regarding the user cases for the utility: 1) Would I be able to read the extracted data from the dumped file without Ignite node binary/marshaller metadata? In other words, will I be able to move only the dumped file to another grid or will I need to move the metadata as well? 2) Are you planning to add a public API version of this utility as a part of Ignite? For example, if I am planning to run some statistics on a checkpointed data, will I be able to get some sort of an iterator to process this data? 3) How a user will choose which caches (cache groups) to process? Will the user need to provide a cache or cache ID (or either of them)? Will the utility be able to extract a single cache data from a cache group? 4) I think the upload part of the utility is missing some input parameters - for example, what cluster to connect to, what caches to upload to, etc.
сб, 30 июн. 2018 г. в 22:38, Dmitriy Govorukhin < dmitriy.govoruk...@gmail.com>: > Igniters, > > I am working on IGNITE-7644 > <https://issues.apache.org/jira/browse/IGNITE-7644> (export all key-value > data from a persisted partition), > it will be command line tool for extracting data from Ignite partition > file without the need to start node. > The main motivation is to have a lifebuoy in case if a file has damage for > some reason. > > I suggest simple API and two commands for the first implementation: > > -c > --CRC [srcPath] - check CRC for all(or by type) pages in partition > > -e > --extract [srcPath] [outPath] - dump all survey data from partition to > another file with raw key/value pair format > (required graceful stop for a node, not necessary after --restore will be > implemented) > > Output file format see in attached, this format does not contain any index > inside but it is very simple and > flexible for future works with raw key/value data. > > Future features: > -u > --upload - reload raw key/value pairs to node > > -s > --status - check current node file status, need binary recovery or not > (node crash on the middle of a checkpoint) > > -r > --restore - restore binary consistency (finish checkpoint, required WAL > file for recovery) > > Let's start a discussion, any comments are welcome. > >
