[zfs-discuss] zfs cache behaviour.

2009-02-24 Thread Pascal Fortin
Hi all, I'm looking for a document that explain the working of the zfs cache. Especially the S10 update6. I need to understand why the utilization of the cache is not up to the arc size we set. In respect of the server workload, we expect that the datas are spread across the size allowed for t

Re: [zfs-discuss] Motherboard for home zfs/solaris file server

2009-02-24 Thread Neal Pollack
On 02/23/09 20:24, Ilya Tatar wrote: Hello, I am building a home file server and am looking for an ATX mother board that will be supported well with OpenSolaris (onboard SATA controller, network, graphics if any, audio, etc). I decided to go for Intel based boards (socket LGA 775) since it see

Re: [zfs-discuss] Motherboard for home zfs/solaris file server

2009-02-24 Thread Carson Gaspar
Neal Pollack wrote: On 02/23/09 20:24, Ilya Tatar wrote: ... efficiency is an important factor. After reading several posts about ZFS it looks like I want ECC memory as well. ... Any motherboard for the Core2 or Core i7 Intel processors with the ICH Not. Intel decided we don't need ECC m

Re: [zfs-discuss] zfs cache behaviour.

2009-02-24 Thread Richard Elling
Pascal Fortin wrote: Hi all, I'm looking for a document that explain the working of the zfs cache. Especially the S10 update6. I need to understand why the utilization of the cache is not up to the arc size we set. In respect of the server workload, we expect that the datas are spread across

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Mon, Feb 23, 2009 at 10:05:31AM -0800, Christopher Mera wrote: > I recently read up on Scott Dickson's blog with his solution for > jumpstart/flashless cloning of ZFS root filesystem boxes. I have to say > that it initially looks to work out cleanly, but of course there are > kinks to be worked

Re: [zfs-discuss] Motherboard for home zfs/solaris file server

2009-02-24 Thread Rob Logan
Not. Intel decided we don't need ECC memory on the Core i7 I thought that was a Core i7 vs Xeon E55xx for socket LGA-1366 so that's why this X58 MB claims ECC support: http://supermicro.com/products/motherboard/Xeon3000/X58/X8SAX.cfm ___ zfs-discus

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Mattias Pantzare
On Tue, Feb 24, 2009 at 19:18, Nicolas Williams wrote: > On Mon, Feb 23, 2009 at 10:05:31AM -0800, Christopher Mera wrote: >> I recently read up on Scott Dickson's blog with his solution for >> jumpstart/flashless cloning of ZFS root filesystem boxes.  I have to say >> that it initially looks to w

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Christopher Mera
Either way - it would be ideal to quiesce the system before a snapshot anyway, no? My next question now is what particular steps would be recommended to quiesce a system for the clone/zfs stream that I'm looking to achieve... All your help is appreciated. Regards, Christopher Mera -Origi

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Brent Jones
On Tue, Feb 24, 2009 at 10:41 AM, Christopher Mera wrote: > Either way -  it would be ideal to quiesce the system before a snapshot > anyway, no? > > My next question now is what particular steps would be recommended to quiesce > a system for the clone/zfs stream that I'm looking to achieve... >

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 07:37:39PM +0100, Mattias Pantzare wrote: > On Tue, Feb 24, 2009 at 19:18, Nicolas Williams > wrote: > > When you snapshot a ZFS filesystem you get just that -- a snapshot at > > the filesystem level.  That does not mean you get a snapshot at the > > _application_ level.  N

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Christopher Mera
Thanks for your responses.. Brent: And I'd have to do that for every system that I'd want to clone? There must be a simpler way.. perhaps I'm missing something. Regards, Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.openso

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 10:56:45AM -0800, Brent Jones wrote: > If you are writing a script to handle ZFS snapshots/backups, you could > issue an SMF command to stop the service before taking the snapshot. > Or at the very minimum, perform an SQL dump of the DB so you at least > have a consistent fu

Re: [zfs-discuss] ZFS: unreliable for professional usage?

2009-02-24 Thread Robert Milkowski
Hello Joe, Monday, February 23, 2009, 7:23:39 PM, you wrote: MJ> Mario Goebbels wrote: >> One thing I'd like to see is an _easy_ option to fall back onto older >> uberblocks when the zpool went belly up for a silly reason. Something >> that doesn't involve esoteric parameters supplied to zdb. MJ

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Brent Jones
On Tue, Feb 24, 2009 at 11:32 AM, Christopher Mera wrote: > Thanks for your responses.. > > Brent: > And I'd have to do that for every system that I'd want to clone?  There > must be a simpler way.. perhaps I'm missing something. > > > Regards, > Chris > Well, unless the database software itself

Re: [zfs-discuss] Motherboard for home zfs/solaris file server -- ECC claims

2009-02-24 Thread Miles Nordin
> "rl" == Rob Logan writes: rl> that's why this X58 MB claims ECC support: the claim is worth something. People always say ``AMD supports ECC because the memory controller is in the CPU so they all support it, it cannot be taken away from you by lying idiot motherboard manufacturers or

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Miles Nordin
> "cm" == Christopher Mera writes: cm> it would be ideal to quiesce the system before a snapshot cm> anyway, no? It would be more ideal to find the bug in SQLite2 or ZFS. Training everyone, ``you always have to quiesce the system before proceeding, because it's full of bugs'' is ret

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Miles Nordin
> "bj" == Brent Jones writes: bj> tolerating that your database will not be consistent after a bj> snapshot and have to replay logs / consistency check it ``not be consistent'' != ``have to replay logs'' pgpLNmP6hsO3I.pgp Description: PGP signature _

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Christopher Mera
How is it that flash archives can avoid these headaches? Ultimately I'm doing this to clone ZFS root systems because at the moment Flash Archives are UFS only. -Original Message- From: Brent Jones [mailto:br...@servuhome.net] Sent: Tuesday, February 24, 2009 2:49 PM To: Christopher Me

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Lori Alt
On 02/24/09 12:57, Christopher Mera wrote: How is it that flash archives can avoid these headaches? Are we sure that they do avoid this headache? A flash archive (on ufs root) is created by doing a cpio of the root file system. Could a cpio end up archiving a file that was mid-way through a

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Christopher Mera
Here's what makes me say that: There are over 700 boxes deployed using Flash Archive's on an S10 system with a UFS root. We've been working on basing our platform on a ZFS root and took Scott Dickson's suggestions (http://blogs.sun.com/scottdickson/entry/flashless_system_cloning_with_z fs) fo

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 01:17:47PM -0600, Nicolas Williams wrote: > I don't think there's any way to ask svc.config to pause. Well, IIRC that's not quite right. You can pstop svc.startd, gently kill (i.e., not with SIGKILL) svc.configd, take your snapshot, then prun svc.startd. Nico --

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Tim
On Tue, Feb 24, 2009 at 2:15 PM, Nicolas Williams wrote: > On Tue, Feb 24, 2009 at 01:17:47PM -0600, Nicolas Williams wrote: > > I don't think there's any way to ask svc.config to pause. > > Well, IIRC that's not quite right. You can pstop svc.startd, gently > kill (i.e., not with SIGKILL) svc.co

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 02:53:14PM -0500, Miles Nordin wrote: > > "cm" == Christopher Mera writes: > > cm> it would be ideal to quiesce the system before a snapshot > cm> anyway, no? > > It would be more ideal to find the bug in SQLite2 or ZFS. Training > everyone, ``you always have

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 02:27:18PM -0600, Tim wrote: > On Tue, Feb 24, 2009 at 2:15 PM, Nicolas Williams > wrote: > > > On Tue, Feb 24, 2009 at 01:17:47PM -0600, Nicolas Williams wrote: > > > I don't think there's any way to ask svc.config to pause. > > > > Well, IIRC that's not quite right. You

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 12:19:22PM -0800, Christopher Mera wrote: > There are over 700 boxes deployed using Flash Archive's on an S10 system > with a UFS root. We've been working on basing our platform on a ZFS > root and took Scott Dickson's suggestions > (http://blogs.sun.com/scottdickson/entry

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Miles Nordin
> "la" == Lori Alt writes: la> Could a cpio end up archiving a file that was mid-way la> through an SQLite2 transaction? cpio is actually much worse for a database than a snapshot! I don't know what's going on in this specific case, but the cpio backup is worse for SQLite2-using thi

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Toby Thain
On 24-Feb-09, at 1:37 PM, Mattias Pantzare wrote: On Tue, Feb 24, 2009 at 19:18, Nicolas Williams wrote: On Mon, Feb 23, 2009 at 10:05:31AM -0800, Christopher Mera wrote: I recently read up on Scott Dickson's blog with his solution for jumpstart/flashless cloning of ZFS root filesystem boxes

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Tim
On Tue, Feb 24, 2009 at 2:37 PM, Nicolas Williams wrote: > > > > > > > Hot Backup? > > > > # Connect to the database > > sqlite3 db $dbfile > > # Lock the database, copy and commit or rollback > > if {[catch {db transaction immediate {file copy $dbfile ${dbfile}.bak}} > res]} { > >puts "Ba

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 03:25:53PM -0600, Tim wrote: > On Tue, Feb 24, 2009 at 2:37 PM, Nicolas Williams > wrote: > > > > > > > > > > > > Hot Backup? > > > > > > # Connect to the database > > > sqlite3 db $dbfile > > > # Lock the database, copy and commit or rollback > > > if {[catch {db trans

[zfs-discuss] hardware advice

2009-02-24 Thread Harry Putnam
I'm sorry this is going to be rather lame since I can't even provide a `specs' page for the hardware I want to ask about. Its pretty discouraging trying to get information from Aopen. A specification page seems like the bare minimum to provide. Building up a home zfs server and have specific har

Re: [zfs-discuss] Motherboard for home zfs/solaris file server

2009-02-24 Thread Carson Gaspar
Rob Logan wrote: Not. Intel decided we don't need ECC memory on the Core i7 I thought that was a Core i7 vs Xeon E55xx for socket LGA-1366 so that's why this X58 MB claims ECC support: http://supermicro.com/products/motherboard/Xeon3000/X58/X8SAX.cfm They lie*. Read the Intel Core i7 specs

Re: [zfs-discuss] Motherboard for home zfs/solaris file server

2009-02-24 Thread Tim
On Tue, Feb 24, 2009 at 4:04 PM, Carson Gaspar wrote: > > They lie*. Read the Intel Core i7 specs - no ECC on any of them. > > * They claim "future Nehalem processor families". These mysterious future > CPUs may indeed support ECC. The Core i7-(920|940|965) do not. > Given the current state of

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Mon, Feb 23, 2009 at 02:36:07PM -0800, Christopher Mera wrote: > panic[cpu0]/thread=dacac880: BAD TRAP: type=e (#pf Page fault) > rp=d9f61850 addr=1048c0d occurred in module "zfs" due to an illegal > access to a user address Can you describe what you're doing with your snapshot? Are you zfs se

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Christopher Mera
It's a zfs snapshot that's then sent to a file.. On the new boxes I'm doing a jumpstart install with the SUNWCreq package, and using the finish script to mount an NFS filesystem that contains the *.zfs dump files. Zfs receive is actually importing the data and the boot environment then boots fine

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 03:08:18PM -0800, Christopher Mera wrote: > It's a zfs snapshot that's then sent to a file.. > > On the new boxes I'm doing a jumpstart install with the SUNWCreq > package, and using the finish script to mount an NFS filesystem that > contains the *.zfs dump files. Zfs rec

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Greg Palmer
Miles Nordin wrote: Hope this helps untangle some FUD. Snapshot backups of databases *are* safe, unless the database or application above it is broken in a way that makes cord-yanking unsafe too. Actually Miles, what they were asking for is generally referred to as a checkpoint and they are

Re: [zfs-discuss] zfs streams & data corruption

2009-02-24 Thread Miles Nordin
> "gp" == Greg Palmer writes: gp> Performing a checkpoint will perform such tasks as making sure gp> that all transactions recorded in the log but not yet written gp> to the database are written out and that the system is not in gp> the middle of a write when you grab the data