On 3/22/12 5:41 PM, "Eli Collins" <e...@cloudera.com> wrote:
>On Thu, Mar 22, 2012 at 4:27 PM, <milind.bhandar...@emc.com> wrote: >> Eli, >> >> I think by "current definition of visible length", you mean that once a >> client opens a file and gets block list, it will always be able to read >>up >> to the length at open. >> > >I was thinking of the definition from the design doc. See my last >comment on HDFS-2288, part of the confusion is that we're using the >same name for two different things. > >> However, correct me if I am wrong, but this definition is already >> violated, if file is deleted after open. > >I think you're right. Another thing that could be fixed with COW blocks and MVCC principles. If a file was opened, then deleted the blocks on the opened file would still be visible to that client, but no new ones. > >> So, truncate does add some complexity, but not a whole lot. If client >>gets >> an EOF before length at open, it must retry to see if the new visible >> length is different (rather than to see if the file does not exist >> anymore). >> >> Right ? >> > >Makes sense. I was thinking you were talking about truncate on open >files, which be harder. You can already truncate a file on open, you >just can't choose the offset you want to truncate at (the NN >implements this by deleting the file). > >Thanks, >Eli > >> >> --- >> Milind Bhandarkar >> Greenplum Labs, EMC >> (Disclaimer: Opinions expressed in this email are those of the author, >>and >> do not necessarily represent the views of any organization, past or >> present, the author might be affiliated with.) >> >> >> >> On 3/22/12 4:03 PM, "Eli Collins" <e...@cloudera.com> wrote: >> >>>On Thu, Mar 22, 2012 at 3:57 PM, Tsz Wo Sze <szets...@yahoo.com> wrote: >>>>> Do you think having the invariant that blocks are not mutated would >>>>> significantly simply the design? >>>> >>>> No. As mentioned in my previous email and others, the complexity is >>>>in >>>>hflush. Once we have hflush, append is straightforward. >>> >>>I understand that append is a small delta once you have hflush, what >>>I'm saying is that the overall design of the file system is >>>significantly simplified if you can assume blocks are not mutated. Eg >>>see the way truncate is going to interact with the current definition >>>of visible length (it violates it). Resolving issues like that are >>>non-trivial. >>> >>>Thanks, >>>Eli >>> >>
