date:20070420

[zfs-discuss] Re: Preferred backup mechanism for ZFS?

2007-04-20 Thread Anton B. Rang

> Initially I wanted a way to  do a dump to tape like ufsdump.  I
> don't know if this makes sense anymore because the tape market is
> crashing slowly.

It makes sense if you need to keep backups for more than a handful of years 
(think regulatory requirements or scientific data), or if cost is important. 
Storing tape is much cheaper than keeping disks running. (Storing disks isn't 
practical over long periods of time; not only does the signal on the media 
degrade, but so do some components.)

> People just don't backup 300MB per night anymore. We
> are looking at terabytes of data and I don't know how
> to backup a terabyte a night.

If you're actually generating a terabyte per day of data, I'm impressed.  :-)

Tape seems a reasonable way to back that up, in any case. A T1 stores 500 
GB on each tape and runs at 120 MB/sec, so a terabyte would take roughly 2.5 
hours to backup with a single tape drive. LTO-4 is in the same ballpark. Of 
course, that assumes your disk system can keep up.

The SAM-QFS approach of continuous archiving makes a lot of sense here since it 
effectively lets backups run continuously. I don't know how much Sun can say 
about the work going on to add SAM to ZFS.

> Or a really big question that I guess I have to ask, do we even care anymore?

If we're serious about disaster recovery, we do.

In particular, remote replication is NOT a substitute for backups.

Anton
 
 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[zfs-discuss] Re: Bottlenecks in building a system

2007-04-20 Thread Anton B. Rang

If you're using this for multimedia, do some serious testing first. ZFS tends 
to have "bursty" write behaviour, and the worst-case latency can be measured in 
seconds. This has been improved a bit in recent builds but it still seems to 
"stall" periodically.

(QFS works extremely well for streaming, as evidenced in recent Sun press 
releases, but I'm not sure what the cost is these days.)
 
 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[zfs-discuss] Re: Multi-tera, small-file filesystems

2007-04-20 Thread Anton B. Rang

You should definitely worry about the number of files when it comes to backup & 
management. It will also make a big difference in space overhead.

A ZFS filesystem with 2^35 files will have a minimum of 2^44 bytes of overhead 
just for the file nodes, which is about 16 TB.

If it takes about 20 ms for the overhead to backup a file (2 seeks), then 2^35 
files will take 21 years to back up.  ;-)

I'm guessing you didn't really mean 2^35, though. (If you did, you're likely to 
need a system along the lines of DARPA's HPCS program)

Anton
 
 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] ZFS boot: 3 smaller glitches with console, /etc/dfs/sharetab and /dev/random

2007-04-20 Thread Constantin Gonzalez

Hi,

>> 2. After going through the zfs-bootification, Solaris complains on
>> reboot that
>>/etc/dfs/sharetab is missing. Somehow this seems to have been
>> fallen through
>>the cracks of the find command. Well, touching /etc/dfs/sharetab
>> just fixes
>>the issue.
> 
> This is unrelated to ZFS boot issues, and sounds like this bug:
> 
> 6542481 No sharetab after BFU from snv_55
> 
> It's fixed in build 62.

hmm, that doesn't fit what I saw:

- Upgraded from snv_61 to snv_62
- snv_62 booted with not problems (other than the t_optmgmt bug)
- Then migrated to ZFS boot
- Now the sharetab issues shows up.

So why did the sharetab issue only show up after the ZFSification of the
boot process?

Best regards,
Constantin

-- 
Constantin GonzalezSun Microsystems GmbH, Germany
Platform Technology Group, Global Systems Engineering  http://www.sun.de/
Tel.: +49 89/4 60 08-25 91   http://blogs.sun.com/constantin/

Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Preferred backup mechanism for ZFS?

2007-04-20 Thread Tim Thomas

Hi Wee

I run a setup of SAM-FS for our main file server and we loved the
backup/restore parts that you described.

That is great to hear.

The main concerns I have with SAM fronting the entire conversation is
data integrity. Unlike ZFS, SAMFS does not do end to end checksumming.
My initial reaction is that the world has got by without file systems
that can do this for a long time...so I don't see the absence of this as
a big deal. On the other hand, it hard to argue against a feature that
improves data integrity, so I will not. Anyway, SAM-FS has been enhanced
in this respect...in SAM-FS 4.6 you can enable the following...

/If required, you can enable data verification for archive copies. This
feature checks
for data corruption on any data that is copied to secondary and/or
tertiary media.
The data verification process performs a read-after-write verification
test, and
records a confirmation of data validity in the metadata properties for
that file.
An ssum option is used to mark files and directories as needing to be
verified. Child
directories inherit the data verification properties of their parent.
The normal
checksum method is employed to verify copies written to tape or disk
archive.
Use the ssum -e command to set data verification for a file or
directory. This forces
the generation and use of checksums for archiving and staging, and
prevents the
release of the file until all archive copies have been created and their
checksums

verified. Only a superuser can set this attribute on a file or directory./

This is taken from the Sun StorageTek SAM Archive Configuration and
Administration Guide Version 4, Update 6 (SAM-FS 4.6 was released April
6th). You can get all the SAM-FS 4.6 docs from here...

http://www.sun.com/products-n-solutions/hardware/docs/Software/Storage_Software/Sun_SAM-FS_and_Sun_SAM-QFS_Software/index.html

This checksum model is different than ZFS and is more like the way a
backup product verifies its backups.

We have considered the setup you proposed (samfs copy1 -> ZFS) but you
will run into problem with fs-cache. Being only a copy, ZFS probably
do not need much caching but will win the battle for memory due to the
way its cache is managed. Unless there is a visible memory shortfall,
ZFS will starve (sorry guys) samfs from memory it could use as cache.
Also, ZFS's data integrity feature is limited by the use of 2nd hand
data.

I don't know enough about how ZFS manages memory other than what I have
seen on this alias (I just joined a couple of weeks ago) which seems to
indicate it is a memory hog...as is VxFS so we are in good company. I
am not against keeping data in memory so long as it has also been
written to somewhere non-volatile as well so that data is not lost if
the lights go out... and applications don't fight for memory to run. I
recall stories from years ago where VxFS hogged so much memory on a Sun
Cluster node that the Cluster services stalled and the cluster failed over!

I need to go read some white papers on this...but I assume that
something like direct I/O (which UFS, VxFS and QFS all have) is in the
plans for ZFS so we don't end up double buffering data for apps like
databases ? - that is just ugly.

64 matches

Mail list logo