Hi Ian,
On 26.02.12 23:42, Ian Collins wrote:
I had high hopes of significant performance gains using zfs diff in
Solaris 11 compared to my home-brew stat based version in Solaris 10.
However the results I have seen so far have been disappointing.
Testing on a reasonably sized filesystem (4TB), a diff that listed 41k
changes took 77 minutes. I haven't tried my old tool, but I would
expect the same diff to take a couple of hours.
Size does not matter (at least here).
How many files do you have and do you have enough cache in main memory
(25% of ARC) or cache device (set to metadata only).
If you are able to manage that every dnode (512 Byte) is in the ARC or
the L2ARC then your compare will fly!
When your are doing too much other stuff (do you IO? Do you have
applications running?)
They will move dnode data out of the direct access and compare needs to
read a lot from disk.
You are comparing a measurement with a guess. That is not a valid test.
The box is well specified, an x4270 with 96G of RAM and a FLASH
accelerator card used for log and cache.
Number of files/size of files is missing.
How much of the pool is used (in %)?
Perhaps the recordsize is lowered, then
How much is used for the cache.
Did you set secondarycache=metadata?
When, is your burn in long enough, that all the metadata is on fast devices?
How large is your L2ARC?
What is running in parallel to your test?
What is the disk configuration (you know: disks are slow)?
Do you use de-duplication (does not directly harm the performance, but
needs memory
and slows down zfs diff through that)?
Tell me the hit rates of the cache (metadata and data in ARC and L2ARC).
Good?
Raidz or mirror?
Are there any ways to improve diff performance?
Yes. Mainly memory. Or use less files.
Regards,
Ulrich
--
Ulrich Graef / Principal Sales Consultant / Phone: + 49 6103 752 359
ORACLE Deutschland B.V.& Co. KG / Amperestr. 6 / 63225 Langen
http://www.oracle.com
ORACLE Deutschland B.V.& Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 Muenchen
Registergericht: Amtsgericht Muenchen, HRA 95603
Geschaeftsfhrer: Juergen Kunz
Komplementaerin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschaeftsfhrer: Alexander van der Ven, Astrid Kepper, Val Maher
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
