> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Edward Ned Harvey > > (1) I'll push the recordsize back > up to 128k, and then repeat this test something slightly smaller than 128k. > Say, 120k.
Good news. :-) Changing the recordsize made a world of difference. I've had this benchmark running for the last 2-3 days now, and I'm up to 3.4M blocks in the pool (as reported by zdb -D) ... Over this period, my arc_meta_used climbed from initial 100M to 1500M. Call it 412 bytes per unique block in the pool. Not too far from what was previously said (376 bytes based on sizeof ddt_entry_t). I am repeating a cycle: time write some unique data with dedup off, then time rm the file, and time write the same data again with dedup on (no verify), then time rm the file, and finally time write it again and leave it on disk just to hog more DDT entries. Move on to the next unique file. Each file is about 3G in size, on a single sata disk. No records are ever repeated - every block in the pool is unique. Presently I'm up to about 500G written of a 1T drive, and the performance difference with/without dedup is 100sec vs 75sec to write the same 3G of data. Currently my arc_meta_limit is 7680M (I tweaked it) which theoretically means I should be able to continue this up to around 19M blocks or so. Clearly much more than my little 1T drive can handle on a 128K recordsize. But I don't just want to see it succeed - I want to predict where it will fail, and confirm the hypothesis with a measured result. So here's what I'm going to do. With arc_meta_limit at 7680M, of which 100M was consumed "naturally," that leaves me 7580 to play with. Call it 7500M. Divide by 412 bytes, it means I'll hit a brick wall when I reach a little over 19M blocks. Which means if I set my recordsize to 32K, I'll hit that limit around 582G disk space consumed. That is my hypothesis, and now beginning the test. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
