Does "copyToLocal" not consider the block locality?

2015-03-21 Thread 남윤민
Hello everyone. I have experienced a very strange situation about HDFS operation. I have a 1 master and 10 slaves cluster environment. When I put a file A into HDFS with dfs.replication=10, I can see every block of the file A is replicated in every node. So, it is reasonable to think tha

How to retrieve cached data in datanode? (Centralized cache management)

2014-08-29 Thread 남윤민
Hello, I have a question about using the cached data in memory via centralized cache management. I cached the data what I want to use through the CLI (hdfs cacheadmin -addDirectives ...). Then, when I write my mapreduce application, how can I read the cached data in memory? Here is the sour

Re: Re: About debugging HDFS and MapReduce...

2013-10-06 Thread 남윤민
Subject : Re: About debugging HDFS and MapReduce... Your image didn't go through. You may want to upload it to image sharing site. What version of Hadoop are you using ? Thanks On Oct 6, 2013, at 8:56 PM, 남윤민 // Yoonmin Nam

About debugging HDFS and MapReduce...

2013-10-06 Thread 남윤민
Hello!! My name is Yoonmin Nam from South Korea.Currently, I am trying to debug namenode and datanode using eclipse and fully-distributed environemnt in my lab. (1 master and 10 nodes)However, I can do debug but not fully, just some portion of source code of hadoop.This means I cannot see how so