Matthew,
I really do appreciate this discussion, thank you for taking the time to
go over this with me.
Matthew Ahrens <[EMAIL PROTECTED]> wrote on 01/04/2007 01:49:00 PM:
> [EMAIL PROTECTED] wrote:
> > [9:40am] [/data/test]:test% zfs snapshot data/[EMAIL PROTECTED]
> > [9:41am] [/data/test]:test% zfs snapshot data/[EMAIL PROTECTED]
> ...
> > [9:42am] [/data/test/images/fullres]:test% zfs list
> > NAME USED AVAIL REFER MOUNTPOINT
> > data/test 13.4G 14.2T 6.54G /data/test
> > data/[EMAIL PROTECTED] 61.2K - 66.6K -
> > data/[EMAIL PROTECTED] 0 - 13.4G -
> > data/[EMAIL PROTECTED] 0 - 13.4G -
> > data/[EMAIL PROTECTED] 0 - 6.54G -
>
> When snap3 is deleted, no space will be freed (because it will still be
> referenced by snap2), therefore the space "used" by snap3 is 0.
Where does it show (on any of the snaps) that they are holding 6.8G of disk
space hostage? I understand snap2 and snap3 both share that data, thats
why below I say that snap3, being the most specific owner should list the
6.8G as used -- showing that you need to delete snaps from 0 -> snap3 to
free 6.8G ( more specifically deleing snap1 gets you 61.3k, snap2 0, snap
three 6.8G if you delete them in that order, and only guaranteed if you
delete them all). If I just delete snap3 and leave snap2, I would expect
snap2 to be the most specific owner of that delta data showing 6.8G "usage"
now.
>
> > What I would expect to see is:
> > data/[EMAIL PROTECTED] 6.86G - 13.4G
>
> > This shows me that snap3 now is the most specific "owner" of 6.86G of
> > delta.
>
> I understand that you *want* it to display a different value, but it is
> correctly showing the documented value. How can we make the manpage
> better, to avoid this confusion in the future?
The amount of space consumed by this dataset and all its
descendants. This is the value that is checked against
this dataset's quota and reservation. The space used
does not include this dataset's reservation, but does
take into account the reservations of any descendant
datasets. The amount of space that a dataset consumes
from its parent, as well as the amount of space that
will be freed if this dataset is recursively destroyed,
is the greater of its space used and its reservation.
maybe kill this part below entirely and put the "usage" column for
snapshots is undetermined and may or may not reflect actual disk usage
associated with the snapshot especialy when blocks are freed between
snapshots.
When snapshots (see the "Snapshots" section) are
created, their space is initially shared between the
snapshot and the file system, and possibly with previous
snapshots. As the file system changes, space that was
previously shared becomes unique to the snapshot, and
counted in the snapshot's space used. Additionally,
deleting snapshots can increase the amount of space
unique to (and used by) other snapshots.
>
> > By delta I mean the COW blocks that the snap represents as delta from
the
> > previous snap and the next snap or live -- in other words what makes
this
> > snap different from the previous or next snap.
>
> The best way I could come up with to define a bounded number of stats to
> express space usage for snapshots was the amount of space "born" and the
> amount "killed". Space "born" is the amount of space that is newly
> allocated in this snapshot (ie. not referenced in the prev snap, but
> referenced here). Space "killed" is the amount of space that is newly
> freed in this snapshot (ie. referenced in the prev snap, but not
> referenced here).
For the common case you don't care about born or killed, you only care
about blocks that are referenced on this snap that are not on the next (or
live if you are the last snap) -- these cover both new files/blocks and
deleted files/blocks that this snap now owns.
Assuming you can get an array of COW blocks for a given snapshot, and for
each snapshot in order of oldest to newest show the size_of ( COW blocks )
that are in snapshot N but do not exist in snapshot N+1 (set N-N+1) would
show me enough information to plan destroys. The number here is how much
space would be freed by deleting snapshots from oldest one to this one in
sequence. This would allow us to plan deletions, or even see where peak
deltas happen on a long series of snaps. I understand this is not as nice
as doing a bidirectional difference test to gather more information such as
what would deleting a random snapshot in the series free if anything, but
that seems to require much more overhead. I think this covers the most
common usage, deleting older snaps before deleting random snaps in the
series or the newest snap first.
Even if this was a long operation that was executed via a list flag it
would sure help. I am trying to get up to speed on the zfs source so I can
see if this is possible or to describe this in more relevant terms to the
code, but this will take me a while.
>
> We considered including these numbers, but decided against it, primarily
> because they can't actually answer the important question: how much
> space will be freed if I delete these N snapshots?
>
> You can't answer this question because you don't know *which* blocks
> were born and killed. Consider 2 filesystems, A and B, both of which
> have lots of churn between every snapshot. However, in A every block is
> referenced by exactly 2 snapshots, and in B every block is referenced by
> exactly 3 snapshots (excluding the first/last few). The born/killed
> stats for A and B's snapshots may be the same, so there's no way to tell
> that to free up space in A, you must delete at least 2 adjacent snaps
> vs. for B, you must delete at least 3.
I am not looking for (and I don't believe any vm/fs outputs that I know of)
display of the amount of space that would be freed by deleting 1->N random
snapshot in a series of snapshots -- That seems like an edge use case.
Common case to me is, how much would be freed by deleting the snapshots in
order of age from oldest to newest always starting with the oldest. That
usage seems to be so common that many snapshotting systems just
automatically kill the oldest snapshots as they require more live usable
space unless they are configured not to.
>
> To really answer this question (how much space would be freed if I
> deleted these N snapshots), you need to know for each snapshot, how much
> of the space that it references was first referenced in each of the
> previous snapshots. We're working on a way to compute and graphically
> display these values, which should make them relatively easy to
interpret.
While that would be very nice, it seems much more complex than the
information I am really asking for.
>
> --matt
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
