Ross wrote: > Well, I really like the idea of an automatic service to manage send/receives > to backup devices, so if you guys don't mind, I'm going to share some other > ideas for features I think would be useful. >
cool. > One of the first is that you need some kind of capacity management and > snapshot deletion. Eventually backup media are going to fill and you need to > either prompt the user to remove snapshots, or even better, you need to > manage the media automatically and remove old snapshots to make space for new > ones. > I've implemented something like this for a project I'm working on. Consider this a research project at this time, though I hope to leverage some of the things we learn as we scale up, out, and refine the operating procedures. There is a failure mode lurking here. Suppose you take two sets of snapshots: local and remote. You want to do an incremental send, for efficiency. So you look at the set of snapshots on both machines and find the latest, common snapshot. You will then send the list of incrementals from the latest, common through the latest snapshot. On the remote machine, if there are any other snapshots not in the list you are sending and newer than the latest, common snapshot, then the send/recv will fail. In practice, this means that if you use the zfs-auto-snapshot feature, which will automatically destroy older snapshots as it goes (eg. the default policy for "frequent" is take snapshots every 15 minutes, keep 4). If you never have an interruption in your snapshot schedule, you can merrily cruise along and not worry about this. But if there is an interruption (for maintenance, perhaps) and a snapshot is destroyed on the sender, then you also must make sure it gets destroyed on the receiver. I just polished that code yesterday, and it seems to work fine... though it makes folks a little nervous. Anyone with an operations orientation will recognize that there needs to be a good process wrapped around this, but I haven't worked through all of the scenarios on the receiver yet. > I'm thinking that a setup like time slider would work well, where you specify > how many of each age of snapshot to keep. But I would want to be able to > specify different intervals for different devices. > > eg. I might want just the latest one or two snapshots on a USB disk so I can > take my files around with me. On a removable drive however I'd be more > interested in preserving a lot of daily / weekly backups. I might even have > an archive drive that I just store monthly snapshots on. > > What would be really good would be a GUI that can estimate how much space is > going to be taken up for any configuration. You could use the existing > snapshots on disk as a guide, and take an average size for each interval, > giving you average sizes for hourly, daily, weekly, monthly, etc... > ha ha, I almost blew coffee out my nose ;-) I'm sure that once the forward time-slider functionality is implemented, it will be much easier to manage your storage utilization :-) So, why am I giggling? My wife just remembered that she hadn't taken her photos off the camera lately... 8 GByte SD cards are the vehicle of evil destined to wreck your capacity planning :-) > That could then be used in a GUI (I'm thinking a visual column with colours > for each type of snapshot showing how full the drive would be). You know the > size of the external drive (and that's fixed for each device), you also know > the average sizes of snapshots, so you can show the user how much space they > will have, and let them play around with the numbers. > I think there is some merit for this as a backwards-looking process. -- richard _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
