[zfs-discuss] XATTRs, ZAP and the Mac
Now that the documentation section is expanding, I can make a not-so-ill-informed comment about zfs and the Mac OS. If I'm reading the documentation correctly, the news is unfortunately bad. The "problem" with Mac support is that the Mac stored all of its display-related information in the directory itself. By this I refer to things like the icon's position within its directory window, the "type and creator" flags that are an analog to a file extension, and even things like window sizing, position and scroll location for folders. The idea here was that on a floppy, which the FS was targetted for way back when, a single pass over the flat-file directory would give you EVERYTHING you needed to draw the display. Given the extremely limited bandwidth, this was an extremely practical decision, if potentially limiting. For the interested, here's the info needed: http://developer.apple.com/technotes/tn/tn1150.html#FinderInfo It's a pair of four-byte strings, three 2x16-bit Points and potentially a 32-bit Rect, and a couple of flags, the vast majority of which are no longer used or are duplicated in zfs's directory ZAP anyway. Now generally the Unix nerds would suggest hanging the Mac's extra information in the xattrs.The problem here is that zfs stores all xattrs separately from the ZAP directory entry. So while it is physically possible to make another ACE for the Mac info and hang it off the xattr "inode" (ZAP), this means that drawing a window in the Finder will require a file system walk! This just isn't going to work, Mac users won't accept PC-slow directory displays -- I know I wouldn't. What I don't understand is why this disturbingly inflexible design was chosen. Note that the ACL ACE is built in order to store up to six entries in-line, which likely serves 95% of all cases. Why an identical solution was not used for xattrs absolutely baffles me. An identical six entry with overflow ACE dir would work wonderfully for xattrs, and in this particular case, would store all the needed Mac-related goodness. So can anyone tell me why xattrs weren't handled in the same way as ACLs? It smells of inside-the-box-thinking, but I'm no FS expert and there may very well be a good reason. And if there isn't a good reason, is it simply too late to fix this? If xattr were moved below gid and used up the pad[4], that would at least give us something useful. Maury This message posted from opensolaris.org ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] XATTRs, ZAP and the Mac
I don't disagree with the below, however, you can run your mac on UFS instead of HFS+. Since UFS hasn't been mac-ified, I'm wondering if the below is actually true for all filesystem types. On May 2, 2006, at 2:02 PM, Maury Markowitz wrote: Now that the documentation section is expanding, I can make a not- so-ill-informed comment about zfs and the Mac OS. If I'm reading the documentation correctly, the news is unfortunately bad. The "problem" with Mac support is that the Mac stored all of its display-related information in the directory itself. By this I refer to things like the icon's position within its directory window, the "type and creator" flags that are an analog to a file extension, and even things like window sizing, position and scroll location for folders. The idea here was that on a floppy, which the FS was targetted for way back when, a single pass over the flat- file directory would give you EVERYTHING you needed to draw the display. Given the extremely limited bandwidth, this was an extremely practical decision, if potentially limiting. For the interested, here's the info needed: http://developer.apple.com/technotes/tn/tn1150.html#FinderInfo It's a pair of four-byte strings, three 2x16-bit Points and potentially a 32-bit Rect, and a couple of flags, the vast majority of which are no longer used or are duplicated in zfs's directory ZAP anyway. Now generally the Unix nerds would suggest hanging the Mac's extra information in the xattrs.The problem here is that zfs stores all xattrs separately from the ZAP directory entry. So while it is physically possible to make another ACE for the Mac info and hang it off the xattr "inode" (ZAP), this means that drawing a window in the Finder will require a file system walk! This just isn't going to work, Mac users won't accept PC-slow directory displays -- I know I wouldn't. What I don't understand is why this disturbingly inflexible design was chosen. Note that the ACL ACE is built in order to store up to six entries in-line, which likely serves 95% of all cases. Why an identical solution was not used for xattrs absolutely baffles me. An identical six entry with overflow ACE dir would work wonderfully for xattrs, and in this particular case, would store all the needed Mac-related goodness. So can anyone tell me why xattrs weren't handled in the same way as ACLs? It smells of inside-the-box-thinking, but I'm no FS expert and there may very well be a good reason. And if there isn't a good reason, is it simply too late to fix this? If xattr were moved below gid and used up the pad[4], that would at least give us something useful. Maury This message posted from opensolaris.org ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss - Gregory Shaw, IT Architect Phone: (303) 673-8273Fax: (303) 673-8273 ITCTO Group, Sun Microsystems Inc. 1 StorageTek Drive ULVL4-382 [EMAIL PROTECTED] (work) Louisville, CO 80028-4382 [EMAIL PROTECTED] (home) "When Microsoft writes an application for Linux, I've Won." - Linus Torvalds ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] XATTRs, ZAP and the Mac
On Tue, May 02, 2006 at 01:02:59PM -0700, Maury Markowitz wrote: > The "problem" with Mac support is that the Mac stored all of its > display-related information in the directory itself. As you've mentioned, performance considerations mean that extended attributes may not be the best solution for storing this information. Unfortunately, embedding the extended attributes in the znode_phys_t (as you suggest) is not really practical. The extended attributes are full-fledged files, with all the associated metadata (permissions, dates, etc). There simply isn't enough space in the znode_phys_t to store even a single extended attribute. Even if you invented a new interface for name-value extended properties on files, it would be tough to fit many properties in the unused space in the znode_phys_t. But such an interface might be worth considering, if there are important uses for it (the Mac windowing information may be one). One way to store this information would be to simply add it to the znode_phys_t (by using a couple words of the padding). This would result in very good performance, since you wouldn't even have to find the attribute with the matching name. Keep in mind that the whole idea of storing this windowing information with the file presumes that there are not multiple (hard) links to a file. If there are multiple names for the file (ie. it exists in multiple directories), it probably makes more sense to store the windowing information with the directory entry. That change would be fairly straightforward: use the ZAP to store a larger value in the directory entry, adding words that describe the window information. --matt ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS on FUSE/Linux [Google Summer of Code 2006]
I've made an application draft for the project I'm proposing. This draft was made with the Gentoo Linux mentoring organization in mind, but of course, I'm much more interested in being assigned a mentor from ZFS/OpenSolaris itself :) You can read the draft in PDF format here: http://wizy.org/zfs-on-fuse.pdf Anyway, I'm not planning on being mentored too much, but it's possible I will need some help understanding the ZFS code. Comments? :) On Monday 01 May 2006 05:58, Ricardo Correia wrote: > Hi, I'm a computer engineering student from Portugal, and I would love to > make a ZFS implementation for FUSE (Filesystem in Userspace) for Linux > (and FreeBSD). I know it would be cool to have a kernel-level > implementation, but that would be too big of a project for me to work on > my own. > > However, I'm fairly confident I would be able to do a mostly-complete > implementation of ZFS on FUSE in my spare time. > > Is there a possibility to arrange a mentor for this project from > ZFS/OpenSolaris? > > Thank you. > > ___ > zfs-discuss mailing list > [EMAIL PROTECTED] > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Best. Solaris Tees. Ever.
Sorry for the cross-posting, but I think the union of these lists will be interested in this. Joyent, a recent convert to Solaris (and ZFS in particular), has put together some fantastic T-shirts: http://flickr.com/photos/textdriveinc/139222094/ http://flickr.com/photos/textdriveinc/139222093/ https://store.joyent.com/products/show/3 https://store.joyent.com/products/show/4 They're both great -- but I especially love "Boys are from LAMP, Men are from MARS" (MySQL + Apache + Rails + Solaris). And if you're interested in seeing more of Joyent's rationale for running on Solaris, check out Jason Hoffman's Scaling with Rails presentation: http://www.scalewithrails.com/downloads/ScaleWithRails-April2006.pdf Finally, to confirm the rumors before they get started: it's true that the DTrace team demo'd DTrace on Rails(*) to Joyent explicitly out of the hope of getting a DTrace T-shirt. Yes, we know -- we're pathetic. - Bryan -- Bryan Cantrill, Solaris Kernel Development. http://blogs.sun.com/bmc (*) http://blogs.sun.com/roller/page/bmc?entry=dtrace_on_rails ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS porting help
Folks - Several people have vocalized interest in porting ZFS to operating systems other than solaris. While our 'mentoring' bandwidth may be small, I am hoping to create a common forum where people could share their experiences and ideas about how to effectively port ZFS and ease the ongoing maintenance burden. To that end, I started with this page: http://www.opensolaris.org/os/community/zfs/porting/ Which gives some guidelines about what I think would be a reasonable way of attacking a port of ZFS. While not quite a WiKi, if folks are actively working a port, feel free to email me (or one of the other ZFS community leaders) and we'll make you a leader as well so you can edit this page and share experiences, link to your efforts, etc. If there is enough interest, it may also make sense to create a discussion forum strictly for detailed questions about ZFS code that may not interest the community at large. If '[EMAIL PROTECTED]' were created, would this be beneficial to anyone? - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS porting help
Eric Schrock wrote: Folks - Several people have vocalized interest in porting ZFS to operating systems other than solaris. While our 'mentoring' bandwidth may be small, I am hoping to create a common forum where people could share their experiences and ideas about how to effectively port ZFS and ease the ongoing maintenance burden. To that end, I started with this page: http://www.opensolaris.org/os/community/zfs/porting/ Which gives some guidelines about what I think would be a reasonable way of attacking a port of ZFS. While not quite a WiKi, if folks are actively working a port, feel free to email me (or one of the other ZFS community leaders) and we'll make you a leader as well so you can edit this page and share experiences, link to your efforts, etc. Excellent! If there is enough interest, it may also make sense to create a discussion forum strictly for detailed questions about ZFS code that may not interest the community at large. If '[EMAIL PROTECTED]' were created, would this be beneficial to anyone? Definitely. Please sign me up when you create it. thanks, James C. McPherson -- Solaris Datapath Engineering Data Management Group Sun Microsystems ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] XATTRs, ZAP and the Mac
On Tue, May 02, 2006 at 05:10:09PM -0400, Maury Markowitz wrote: > >Unfortunately, embedding the extended attributes in the znode_phys_t (as > >you suggest) is not really practical. The extended attributes are > >full-fledged files > > I think that's the disconnect. WHY are they "full-fledged files"? Because that's what the specification calls for. If they weren't full-fledged files, they wouldn't be compatable with existing interfaces. That wouldn't necessarily be a bad thing, just a different thing. As I mentioned, a new, lighter-weight interface could be designed in addition to extended attributes. > >windowing information with the directory entry. That change would be > >fairly straightforward: use the ZAP to store a larger value in the > >directory entry, adding words that describe the window information. > > I think I need a little hand holding here. Where is the ZAP in relation to > the directory or file? I read the documentation to imply that the ZAP was > the directory (page 46, bottom) effectively. However, looking at it now, I > see there is no file name for instance. > > Reading over section 5, and I correct that a znode_phys_t is "built up > inside" a ZAP (potentially micro), and the ZAP itself holds the > znode_phys_t and other additional information? If this is correct, is there > a diagram somewhere that illustrates the resulting overall structure? Ah, I see that you've been reading the on-disk format document. Section 6.2 describes the relation between the ZAP, directory entries, and the znode_phys_t: Filesystem directories are implemented as ZAP objects. Each directory holds a set of name-value pairs which contain the names and object numbers for each director entry. Traversing through a directory tree is as simple as looking up the value for an entry and reading that object number. All filesystem objects contain a znode_phys_t structure in the bonus buffer of it's dnode. This structure stores the attributes for the filesystem object. To recap, the ZAP implements a directory, mapping from file name to object number. That object number identifies (ie. refers to) a particular dnode, which contains a znode_phys_t in the bonus buffer. As I mentioned, the directory could potentially map from file name to object number + some additional information, since the values stored in the ZAP can be variable-length. Could elaborate on what makes it seem like there "is no file name" and that the "ZAP itself holds the znode_phys_t"? Then we can change that documentation to make it clear that that is not the case. --matt ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ant tool for migrate from ufs/svm to ZFS and pools
Hi Exists (or It will exists) any metoth or tool for migrate a UFS/SVM filesystems with soft partitions to ZFS filesystems with pools? Any ideas for migrate a instaled base: Solaris 10 UFS/Solaris Volme Manager to Solaris 10 ZFS or only we have a backup-recovery option? Thanks This message posted from opensolaris.org ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ant tool for migrate from ufs/svm to ZFS and pools
fran wrote: Exists (or It will exists) any metoth or tool for migrate a UFS/SVM filesystems with soft partitions to ZFS filesystems with pools? Any ideas for migrate a instaled base: Solaris 10 UFS/Solaris Volme Manager to Solaris 10 ZFS or only we have a backup-recovery option? Hi Fran, no, there is no such tool right now. A few people (including myself) have talked about how such a tool might be designed, but I don't think there's been any activity to date. best regards, James C. McPherson -- Solaris Datapath Engineering Data Management Group Sun Microsystems ___ zfs-discuss mailing list [EMAIL PROTECTED] http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
