On Tue, Jan 12, 2021 at 04:59:51PM +0100, Jean-Pierre André wrote:
> Not exactly : what the documentation says is that reparse points
> and EA's are incompatible. They are both supposed to be returned
> in dwReserved0 by FindFirstFile()... However Microsoft does not
> respect this anymore, as you can see in your post of ntfsinfo
> for 'AppxManifest.xml'

Ah, that is the reason why both are not supposed to be present.

I guess the newest documentation says it is only for the reparse data,
now, but of course it could still be set for EAs as an undocumented
feature.

I also saw some people complaining that chkdsk in Windows 7 would flag
this as an error, so people are not able to dual boot Windows 7 and
Windows 10 on the same system cleanly. (Windows 7 chkdsk will break
these WofCompressed files).

Anyway, I commented out the exclusions related to not allowing ntfs_ea
and ntfs_reparse_data on the same file, and everything seems to be
'okay'.

> Anyway, the issue is that named streams are used both for
> extended attributes and for the WOF compressed data. Historically
> EA's were designed for OS/2 extended attributes, and their
> limitations were dissuasive for representing Linux extended
> attributes, so named streams are used instead. This was consistent
> with Microsoft Office using named streams for a similar purpose.
> 
> The issue is actually telling which named streams are extended
> attributes and which ones are not. And having a generic way to
> do so to avoid having to maintain a sort of black list.

Okay, that makes sense. So normal POSIX xattrs cannot be used with
streams_interface=windows -- and for WofCompressed you have to use this
(if you want to access the data without going through a plugin).

> It would be useful you explain your use case. The WofCompressed files
> are Windows system files which you cannot backup and restore as if
> they were plain files. A system partition is best copied as a whole.
> 
> You are unlikely to have user files being WofCompressed.
> 
> Note : when using "streams_interface=windows" you should not use
> xattr, you can read and write the raw attribute by appending a
> colon and the name of the stream to the file name (for instance :
> AppxManifest.xml:WofCompressedData), and you have a big limit on
> the stream size.

Why is there a limit? I assumed that it was stored like any file data
(due to it being a $DATA node).

So my use case is quite particular and not likely to be common.

I have a small SSD that I am using for the main system drive, and I want
to fit the C: drive on it. I don't use Windows much, so I don't want to
dedicate much space to it. So, what I usually did was move all the
Program Files to another drive, and use a reparse point to redirect it.
This usually works fine, except when it comes do updates, due to the use
of hard links into WinSxS.

So, I wanted to try to do an analysis to do an intelligent move of this
directory -- basically -- anywhere where there are things that are
hardlinked with things in WinSxS, to leave on the SSD (since it won't
save space moving them anyway), and other things move to the drive. I'm
hoping by having reparse points from the second drive back to the C:
drive, it will allow hardlinks to continue to work where they are used,
so updates should also work.

To make that easier, I added an xattr (internal, under system.ntfs_*)
that lists all the possible aliases to the same file as seen by the
filesystem (since that is pretty cheap to do with NTFS it seems --
easier than scanning the whole filesystem and looking for duplicate
inodes)

Finally, to do this completely, I would need to copy WofCompressed files
-- which seems to work fine if I remove the reparse data, copy the file
(making sure to make the main stream sparse and empty and include the
WofCompressed stream), and then apply the reparse data back to both
files. I am not using any plugins. I had to use the
streams_interface=windows to handle cases where the WofCompressed stream
is too large to pass through the xattr interface. It works well, but
seems to make it impossible to make the drive symlinks in the .NTFS-3G
directory to make reparse symlinks work. It might be worth removing the
':' from those anyway, because Windows chkdsk complains about them
anyway.

The idea also is to be able to 'unwind' this if I ever decide to get a
bigger drive.

I did run into one issue playing with these reparse points lacking
plugins where I would sometimes get EIO on any access to the file, where
an initial stat() done by fuse fails. But I was able to fix that with
this small change:

diff --git a/src/ntfs-3g.c b/src/ntfs-3g.c
index 2eb197a8..a15c623e 100644
--- a/src/ntfs-3g.c
+++ b/src/ntfs-3g.c
@@ -147,7 +147,7 @@
 #define CALL_REPARSE_PLUGIN(ni, op_name, ...)                  \
         (reparse = (REPARSE_POINT*)NULL,                        \
         ops = select_reparse_plugin(ctx, ni, &reparse),         \
-        (!ops ? -errno                                          \
+        (!ops ? (errno ? -errno : -EOPNOTSUPP)                  \
                 : (ops->op_name ?                               \
                         ops->op_name(ni, reparse, __VA_ARGS__)  \
                         : -EOPNOTSUPP))),                       \

(The symptom also is that ls -l would not show the files as symlinks to
'unsupported reparse point'.)

This is basically because when we use this from stat, if
select_reparse_plugin doesn't set errno (which I guess it doesn't
always), it would treat it like the call to have the plugin fill the
stat structure succeeded, so it wouldn't be filled with the information
from the dummy symlink, and fuse would mark this as an error.

> > Here is an example of some files I found. They all seem to have the
> > 0x40000 attribute. Sometimes the EA points to a package name.
> 
> This is interesting. Maybe 0x40000 (whose purpose I do not know)
> can be used to tell whether a file is a Windows system file which
> is not eligible for Linux extended attributes.

I found them here: 

https://docs.microsoft.com/en-us/windows/win32/fileio/file-attribute-constants

It says it means FILE_ATTRIBUTE_RECALL_ON_OPEN which seems to mean the
data might need to be fetched from elsewhere? I don't know what that
means at all. Maybe they have repurposed the attribute that is normally
not used directly on the filesystem for some other meaning. Or maybe it
is just a signal that the file's data is provided by some function that
processes the special reparse point, and the main stream is really
empty. I didn't check if it was set on all WofCompressed files.

> ntfs-3g does not use the EA's at all. Microsoft probably use them
> in the update process, so better not to interfere. Consequently they
> are probably not eligible for extended attributes either (but checking
> for EA presence would be heavy for this purpose).

Right now, I'm trying to copy them with the system.ntfs_ea xattr, which
seems to work, but I didn't check if it is correct.

Thanks for the information and clarifications.

-- 
Douglas Paul




_______________________________________________
ntfs-3g-devel mailing list
ntfs-3g-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel

Reply via email to