Re: Read or save specific blocks of a file

2018-05-03 Thread ayan guha
Is this a recommended way of reading data in the long run? I think it might be better to write or look for an InputFormat which supports the need Btw Block is designed to be hdfs internal representation to enable certain features. It would be interesting to understand the usecase where client app

Re: Read or save specific blocks of a file

2018-05-03 Thread Thodoris Zois
Hello Madhav, What I did is pretty straight-forward. Let's say that your HDFS block is 128 MB and you store a file of 256 MBs in HDFS, named Test.csv. First use the command: `hdfs fsck Test.csv -locations -blocks -files`. It will return you some very useful information including the list of blocks.