[
https://issues.apache.org/jira/browse/HDFS-6346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uma Maheswara Rao G resolved HDFS-6346.
---------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
I have just committed this to branch!
> Optimize OP_SET_XATTRS by persisting single Xattr entry per
> setXattr/removeXattr api call
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-6346
> URL: https://issues.apache.org/jira/browse/HDFS-6346
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Uma Maheswara Rao G
> Assignee: Yi Liu
> Attachments: HDFS-6346.1.patch, HDFS-6346.2.patch, HDFS-6346.patch,
> editsStored
>
>
> When a new xattrs set on an Inode, it may add this with OP_SET_XATTRS and
> let's say [user.name1:value1]
> On a next call if we set another xattrs, then it may store along with older
> existing xattrs again. It may be like [user.name1:value1, user.name2:value2]
> So, on adding more xattrs on same Inode, that list may grow and we keep store
> the entries of already existing name, value fairs.
> Right now we defaulted the max Xattrs on an Inode to 32 and configured. If
> user modified it to much larger value and start setting xattrs, then edits
> loading may create issue like my above example.
> But I didn't refer any usecase of having large number of xattrs, this is just
> the assumption to consider a case. My biggest doubt is whether we will have
> such real usecases to have huge xattrs on a single INode.
> So, here is a thought on having OP_SET_XATTR for each setXAttr operation to
> be logged, When we replay them we need to consolidate. This is some initial
> thought we can think more if others also feel we need to consider this case
> to handle.
> Otherwise we endup storing Xattrs entries in editlog file as n(n+1)/2 where n
> is number xattrs for a file/dir. This may be issue only when we have large
> number configured max xattrs for inode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
