What's different of 1)namenode metadata 2)image and 3)edit logs?

2009-11-16 Thread Steve Gao
What are the differences between the 3 concepts: (1) namenode metadata, (2) namenode image, and (3) edit logs? BTW,I know the secondary namenode backups namenode data. I was told that there is another daemon which also backups meta data . Is that true? What is the daemon? Thanks.

What if an XML file cross boundary of HDFS chunks?

2009-10-29 Thread Steve Gao
Does anybody have the similar issue? If you store XML files in HDFS, how can you make sure a chunk reads by a mapper does not contain partial data of an XML segment? For example: book1 me ..what if this is the boundary of a chunk?... 2009 book2 me 2009 book3 me

[Help] Why "java.util.zip.ZipOutputStream" need to use /tmp?

2009-08-28 Thread Steve Gao
would someone give us a hint? Thanks. Why "java.util.zip.ZipOutputStream" need to use /tmp? The hadoop version is 0.18.3 . Recently we got "out of space" issue. It's from "java.util.zip.ZipOutputStream". We found that /tmp is full and after cleaning /tmp the problem is solved. However why hadoop

Why "java.util.zip.ZipOutputStream" need to use /tmp?

2009-08-27 Thread Steve Gao
The hadoop version is 0.18.3 . Recently we got "out of space" issue. It's from "java.util.zip.ZipOutputStream". We found that /tmp is full and after cleaning /tmp the problem is solved. However why hadoop needs to use /tmp? We had already configured hadoop tmp to a local disk in: hadoop-site.x