Hello can, I haven't been wasting so much time as in this thread... but from time to time it won't hurt :)
More below :) Wednesday, December 12, 2007, 4:46:42 PM, you wrote: >> Hello Bill, >> I know, everyone loves their baby... cyg> No, you don't know: you just assume that everyone is as biased cyg> as you and others here seem to be. Which in turn is just your assumption :) >> I've never said there are not fragmentation problems with ZFS. cyg> Not having made a study of your collected ZFS contributions here cyg> I didn't know that. But some of ZFS's developers are on record cyg> stating that they believe there is no need to defragment (unless cyg> they've changed their views since and not bothered to make us cyg> aware of it), and in the entire discussion in the recent 'ZFS + cyg> DB + "fragments"' thread there were only three contributors cyg> (Roch, Anton, and I) who seemed willing to admit that any problem existed. Which ZFS developer said that there's no need to defragment in ZFS? cyg> So since one of my 'claims' for which you requested cyg> substantiation involved fragmentation problems, it seemed appropriate to address them. I would say that right now there are other more important things to be done in ZFS than addressing fragmentation. While in one environment it looks like lowering fragmentation would help with some issues, in all the other environments I haven't run into fragmentation problem. >> Also you haven't done your work home properly, as one of ZFS >> developers actually stated they are going to work on ZFS >> de-fragmentation and disk removal (pool shrinking). >> See http://www.opensolaris.org/jive/thread.jspa?messageID=139680? cyg> Hmmm - there were at least two Sun ZFS personnel participating cyg> in the database thread, and they never mentioned this. I guess cyg> they didn't do their 'work home' properly either (and unlike me they're paid to do it). Maybe they don't know? Different project, different group? My understanding (I might be wrong) is that actually what they are working on is disk removal from pool (which looks like is much more requested by people than fixing fragmentation 'problem'). In order to accomplish it you need a mechanism to re-arrange data in a pool, which as a side effect could be also used as a de-fragment tool. That doesn't mean the pool won't fragment again in a future - if it's a real problem in given environment. >> The point is, and you as a long time developer (I guess) should know it, >> you can't have everything done at once (lack of resources, and it takes >> some time anyway) so you must prioritize. cyg> The issues here are not issues of prioritization but issues of cyg> denial. Your citation above is the first suggestion that I've cyg> seen (and by all appearances the first that anyone else cyg> participating in these discussions has seen) that the ZFS crew cyg> considers the fragmentation issue important enough to merit active attention in the future. Jeeez... now you need some kind of acknowledge from ZFS developers every time you think you found something? Are you paying their bills or what? While it's fine to talk about theoretical/hypothetical problems, I'm not entirely sure here is a good place to do it. On the other hand you can very often find ZFS developers responding on this list (and not only) to actual user problems. Another problem, I guess, could be - they already spent a lot of their time in projects they have to deliver - do you really expect them to spent still more time on analyzing some loosely statements of yours? Come on, they also have their private lifes and other things to do. Ignoring their customers/users would be unwise, responding to everyone with every problem, especially not a real user experience problem - would be just unpractical. Then there is your attitude - you know, there's a very good reasons why people at interviews are checking if you can actually work with the others people in a group. You're a very good example why. Don't expect people to take you seriously if you behave the way you do. As you put it before - you get what you deserve for. You probably got even more attention here that you deserved. I guess, that you are another good engineer, quite skillful, unfortunately unable to work in a team, and definitely not with customers. I would say some people here recognized it within you and did their best to treat you seriously and actually hear you - it's just that everyone has his limits. Looking thru your posts here, you can find lots words, some technical input but not much actual value - at first it could be entertaining, even intriguing but quickly becomes irritating. Bill, you could be the best engineer in the world, if you can't communicate with it you'll be the only one person who would recognize it. Or perhaps some people here (not only here) are right and for whatever reasons you are just trolling. cyg> Do you by any chance have any similar hint of recognition that cyg> RAID-Z might benefit from revamping as well? Maybe I do :)))) Seriously - is RAID-Z the best RAID humanity can think of? Nope. I doubt one exist for all cases if even for one. Is RAID-Z an improvement in some real workloads comparing to RAID-5 - sure, it is. In other environments RAID-5 delivers better performance. As usual - it's hard to satisfy everyone. Maybe (definitely) what would be useful in ZFS is to actually have RAID-5 like implementation, along with RAID-Z, so write performance will suffer but concurrent random read performance will be much better. Now, perhaps (I haven't analyzed it), one could create RAID-5 like implementation which offers similar logical storage capacity, always on-disk consistency and best performance for concurrent small random reads, concurrent small random writes, sequential reads, mixed workloads, etc. Even if there's one, maybe it's just hard to implement and no-one has done so far - are you volunteering? Users would love you - that one you can be sure. cyg> ZFS is open source and if >> someone thinks that given feature is more important than the other >> he/she should try to fix it or at least voice it here so ZFS >> developers can possibly adjust their priorities if there's good enough >> and justified demand. cyg> That just won't wash, Robert: as I noted above, the problem cyg> here has been denial that these are flaws at all, not just a cyg> debate about how to 'prioritize' addressing them (though in the There's much of denial here - but mostly from you. You just can't understand why so many people are exited about ZFS that you try to persuade all of them they are wrong. Well, mister Bill - you are. Just because you've been mostly ignored here, for a very good reasons, it doesn't mean all the other users are - quite often it's the opposite. You behave like some people who can't understand up-to day why DTrace is so revolutionary, and somehow they claim it's all been before, there's hardly anything new, etc. Of course, like you, those are people who actually don't use it and can't accept the fact that DTrace has actually greatly impacted many environments. The same like ZFS does and will still more in a future I believe. You may not like it, you may disagree with it, at the end it's up to users (sys admins, etc.). cyg> Most environments actually aren't all that cyg> performance-sensitive, so of course they don't complain. Even if cyg> they run into problems, they just buy more hardware - because cyg> that's what they're used to doing: the idea that better software cyg> could eliminate the need to do so either doesn't cross their cyg> minds at all or seems like too much of a pipe dream to take seriously. Try to keep focused pleased. If the environment is not performance-sensitive it won't run into a performance problem one would need to fix by throwing more HW or other way. cyg> Trouble is, ZFS and its fanboys tout it as offering *superior* - cyg> not merely adequate - performance, whereas for some cyg> not-all-that-uncommon situations its performance can be worse *by cyg> over an order of magnitude* due to the fragmention which is cyg> designed into its operation and for which no current relief is cyg> available (nor was any relief apparently generally known to be cyg> projected for the future, until now). The fact that many Mr. Troll - where did that order of magnitude came from? I guess out of your head, again. Will ZFS deliver less performance in some cases comparing to other products - sure. Will it deliver better performance in other cases - sure. But it's not only about performance, and there'll be market for specialized, niche products for many years to come. I can find a corner case for every common file system in a marked and show it's performing badly - not a big deal. Unless, that secret BillFS is somewhere... >> You just >> assuming (correctly for some usage cases). I guess your message has >> been well heard. cyg> But hardly well understood. Let me repeat my self. Bill, you could be the best engineer in the world, if you can't communicate with it you'll be the only one person who would recognize it. So even you can't communicate with other people or there's no much practical value in what you're saying. cyg> Is it any cyg> wonder that my respect for so many of you is close to zero? Judging from your posts so far - no surprise at all. It shouldn't be a surprise for you either that many of us feel the same regarding you. So maybe just leave us alone? cyg> I wouldn't wait for any positive feedback if I were >> you - anyway, what kind of feedback are you waiting for? cyg> I'm waiting for the idiots either to shut up or to shape up. Sorry for being blunt - I suggest to start with yourself. Bill, really - step back, go to the cinema, read some book, forget about ZFS for some time at least, and world will look much better for you. You'll see. >> And again, "under conditions that I've described quite clearly." - >> that's exactly the problem. You've just described something while >> others do have actual and real problems which should be addressed >> first. cyg> Once again, you are confusing the very real problem of cyg> stone-wall denial here with a simple issue of prioritization. As it looks like there is actually some work being done regarding fragmentation in ZFS, you're simply mistaken. There's no stone-wall denial here. And whether you like it or not - it's mostly a matter of priorities. >> Well, is it really so hard to understand that a lot of people use ZFS >> because it actually solves their problems? cyg> Not at all: it's just far from obvious that it solves their cyg> problems any (let alone significantly) better than other existing cyg> open source options. And that would not be any issue if some cyg> people here weren't so zealous in asserting ZFS's alleged cyg> stunning superiority - but if they continue to do so, I'll cyg> continue to challenge them to *substantiate* that claim. Well - I've been using ZFS in a production for years now. It has helped me with data corruption on many occasions. It saved lot of money as it was at least good enough comparing to NetApp for example. Storage management with lot of TBs is much much better experience since then. RAID-Z2 is helpful too. Built-in compression has helped on many occasions too. No need for /etc/vfstab /etc/df/dfstab has made it still simpler to manage. Free snapshots and clones are very helpful feature. ZFS+Zones - that combination just rocks - just don't want to go back to "old ways". Performance - well, in some environments much better performance - especially for writes. Quick resilvering, file system or RAID creation - really cool. Can't stand waiting 24hours for new HW RAID-5 array to build... with ZFS it's just a couple of seconds... In most part it's about entire package ZFS delivers - lot of the stuff you can find here and there, with ZFS you've got it all on a one plate. And it's free, open source, it's working on commodity hardware and on high-end one as well. It's just changing some economics. ZFS is brilliant as a whole package. cyg> "Now I have to find a way to justify myself with my head office cyg> that after spending 100k+ in hw and migrating to "the most cyg> advanced OS" we are running about 8 time slower :)" cyg> Some people might consider such a problem to be 'real' (and cyg> somewhat personal as well); he goes on to observe that "while cyg> that rsync process is running, ZONEX is completely unusable cyg> because of the rsync I/O load" - another 'real-world' indication cyg> of how excessive (and unnecessary) RAID-Z disk loading cyg> compromises other aspects of system performance (though limited cyg> scheduling intelligence may have contributed to this as well). Well, then you can often find people buying some mid-range array, configuring RAID-5 out of 12 or more disks, putting lot of writes and complaining - why on the hell I spent so much money and I get so bad performance. Unfortunately, you've still got to understand technology to make a proper use of it. RAID-Z has its good points and its bad points - small random reads case is its weak point. >> Then, in some environments RAID-Z2 (on JBOD) actually provides better >> performance than RAID-5 (and HW R5 for that matter). And, opposite >> to you, I'm not speculating but I've been working with such >> environment (lot of concurrent writes which are more critical than >> much less reads later). cyg> Don't confuse apples with oranges. As long as it can accumulate cyg> enough dirty data before it has to flush it to disk, COW with cyg> batch write-back can make *any* write strategy work well. So cyg> there's no need to accept the brain-damaged nature of RAID-Z's cyg> performance with small-to-medium-sized random accesses in order cyg> to obtain the good performance that you describe above: a good cyg> ZFS RAID-5-like implementation could do just as well for those cyg> workloads *plus* beat both conventional RAID-5 and RAID-Z at cyg> small-update workloads *plus* cremate RAID-Z in terms of cyg> throughput on small-to-medium read workloads. But still - you're comparing RAID-Z to some non-existant implementation. What I care about is - what can I get from a market to solve my problems and how much would it cost me. Like it or not, ZFS is quite often a best choice. But, yes Bill, I agree with you - that non-existant implementation of your is better than ZFS. And remember, RAID-Z is not ZFS - it's just a small part of it. >> So when you saying that RAID-Z is brain-damaging - well, it's >> mostly positive experience of a lot of people with RAID-Z vs. your statement >> without any >> real-world backing. cyg> I just provided one example above from a participant in this cyg> forum (and it seems unlikely that it's the only one). Does that cyg> mean that I get to accuse you of not having "done your work home cyg> properly", because you were unaware of it? Again, wrong assumption of yours. I was aware of it. That's why I wrote "mostly positive experience". As usual you'll find corner cases and because of lack of knowledge people hurt themselfes - as old problem as humanity exists. I'm also haven't used RAID-Z in all cases because I needed more performance in a specific environment - nothing really hard to understand. cyg> I'm talking about what ZFS *could* have provided to make good on ok, if you're comparing ZFS to a non-existant implementation of your version of ZFS - then maybe you're right, current ZFS would look pale. Go ahead - implement it! Or help make current ZFS better. cyg> their claim that they had eliminated (or at least effectively cyg> hidden) volume-management: a *real* 'storage pool' that just Have you ever worked with storage? cyg> And yes, across a very wide range of disk-size variations it's cyg> possible to utilize 100% of the capacity of each individual disk cyg> in such a pool using relatively simple distribution strategies - cyg> especially if you can perform very minor rearrangements to cover Go ahead and implement it! Or maybe you can point me to some implementation I can use? cyg> That's usually the case with amateurs who have difficulty cyg> understanding in detail how the systems that they use work. But cyg> at least many of them have the sense not to argue interminably cyg> with people who have actually designed and built such systems and cyg> *do* understand them in (excruciating) detail. I'm sorry that the file system you've been working on hasn't induced so much excitement in users - maybe you should try working harder? Since you've got no interest in making ZFS better and you're suggesting you are an expert in file systems - well, just deliver all the stuff you're talking about. Make it open source, available on main platforms and world will love you. You know, it's not about the ideas, at least not only - it's about delivering the actual product which often means some compromises instead of pursuing a holy grail. They did deliver ZFS, which is not perfect but still much much better and promising in many aspects than what we've got in a market. And no surprise NetApp is afraid - Solaris+ZFS )or maybe even FreeBSD +ZFS) is a potential NetApp killer, at least NetApp we know. And you haven't actually delivered anything like it. Since you're not interested in improving ZFS, it's hard to call you an expert. What users care about at the end are not promises but actual product - where's yours? Why no-one cares? cyg> I really don't have much interest in meeting *your* criteria for cyg> being convinced, Robert - at least in part because it's not clear cyg> that *anything* would convince you. So it's more fun to see how cyg> completely committed people like you are to keeping their heads cyg> firmly wedged up where the sun don't shine to avoid actually cyg> facing up to the fact that ZFS just ain't quite what you thought it was. First, I don't want you to convince to anything - I doubt it's possible. You've got your hidden agenda or troll mentality and probably there's not much I can do about it. You are just a guy who feels he needs convince everyone they are wrong, while not even using that technology, who keeps referring to some non existent technology. People are exited about ZFS because they can use it - if it's perfect or not, doesn't matter much. What matters is it offers them often much better experience than other *available* technologies, and for free and in a open source form - which changes economics for some environments. >> Maybe, just maybe, it's possible that all people around you don't >> understand you, that world is wrong and we're all so stupid. Well, >> maybe. Even if it is so, then perhaps it's time to stop being Don Quixote >> and move on? cyg> No, but it might be getting close to it - I'll let you know. Don't bother - I really don't care. Bill - I don't think there's a point in continuing that discussion. At least I see no point. I know you tried your best - unfortunately you haven't convinced me and many others (if anyone). Somehow ZFS is still solving some problems better and cheaper than other solutions and I'm quite happy using it. As many other people are too. -- Best regards, Robert Milkowski mailto:[EMAIL PROTECTED] http://milek.blogspot.com _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
