On Fri, Dec 19, 2008 at 6:47 PM, Richard Elling <richard.ell...@sun.com> wrote: > Ross wrote: >> >> Well, I really like the idea of an automatic service to manage >> send/receives to backup devices, so if you guys don't mind, I'm going to >> share some other ideas for features I think would be useful. >> > > cool. > >> One of the first is that you need some kind of capacity management and >> snapshot deletion. Eventually backup media are going to fill and you need >> to either prompt the user to remove snapshots, or even better, you need to >> manage the media automatically and remove old snapshots to make space for >> new ones. >> > > I've implemented something like this for a project I'm working on. > Consider this a research project at this time, though I hope to > leverage some of the things we learn as we scale up, out, and > refine the operating procedures.
Way cool :D > There is a failure mode lurking here. Suppose you take two sets > of snapshots: local and remote. You want to do an incremental > send, for efficiency. So you look at the set of snapshots on both > machines and find the latest, common snapshot. You will then > send the list of incrementals from the latest, common through the > latest snapshot. On the remote machine, if there are any other > snapshots not in the list you are sending and newer than the latest, > common snapshot, then the send/recv will fail. In practice, this > means that if you use the zfs-auto-snapshot feature, which will > automatically destroy older snapshots as it goes (eg. the default > policy for "frequent" is take snapshots every 15 minutes, keep 4). > > If you never have an interruption in your snapshot schedule, you > can merrily cruise along and not worry about this. But if there is > an interruption (for maintenance, perhaps) and a snapshot is > destroyed on the sender, then you also must make sure it gets > destroyed on the receiver. I just polished that code yesterday, > and it seems to work fine... though it makes folks a little nervous. > Anyone with an operations orientation will recognize that there > needs to be a good process wrapped around this, but I haven't > worked through all of the scenarios on the receiver yet. Very true. In this context I think this would be fine. You would want a warning to pop up saying that a snapshot has been deleted locally and will have to be overwritten on the backup, but I think that would be ok. If necessary you could have a help page explaining why - essentially this is a copy of your pool, not just a backup of your files, and to work it needs an accurate copy of your snapshots. If you wanted to be really fancy, you could have an option for the user to view the affected files, but I think that's probably over complicating things. I don't suppose there's any way the remote snapshot can be cloned / separated from the pool just in case somebody wanted to retain access to the files within it? > >> I'm thinking that a setup like time slider would work well, where you >> specify how many of each age of snapshot to keep. But I would want to be >> able to specify different intervals for different devices. >> >> eg. I might want just the latest one or two snapshots on a USB disk so I >> can take my files around with me. On a removable drive however I'd be more >> interested in preserving a lot of daily / weekly backups. I might even have >> an archive drive that I just store monthly snapshots on. >> >> What would be really good would be a GUI that can estimate how much space >> is going to be taken up for any configuration. You could use the existing >> snapshots on disk as a guide, and take an average size for each interval, >> giving you average sizes for hourly, daily, weekly, monthly, etc... >> > > ha ha, I almost blew coffee out my nose ;-) I'm sure that once > the forward time-slider functionality is implemented, it will be > much easier to manage your storage utilization :-) So, why am > I giggling? My wife just remembered that she hadn't taken her > photos off the camera lately... 8 GByte SD cards are the vehicle > of evil destined to wreck your capacity planning :-) Haha, that's a great image, but I've got some food for thought even with this. If you think about it, even though 8GB sounds a lot, it's barely over 1% of a 500GB drive, so it's not an unmanageable blip as far as storage goes. Also, if you're using the default settings for Tim's backups, you'll be taking snapshots every 15 minutes, hour, day, week and month. Now, when you start you're not going to have any sensible averages for your monthly snapshot sizes, but you're very rapidly going to get a set of figures for your 15 minute snapshots. What I would suggest is to use those to extrapolate forwards to give very rough estimates of usage early on, with warnings as to how rough these are. In time these estimates will improve in accuracy, and your 8GB photo 'blip' should be relatively easily incorporated. What you could maybe do is have a high and low usage estimate shown in the GUI. Early on these will be quite a way apart, but should tend to close up as time goes on and the accuracy of the predictions improve. What you would need is some intelligence that can warn the user if it looks like their usage is going to push their backups over the capacity of their storage device. But if you have a service running and watching for new devices being attached, that service can also keep track of the statistics, and display a warning in the system status area prompting the user to take a look at their backup plan and either keep a few less snapshots, or retire that device and buy a larger one. >> That could then be used in a GUI (I'm thinking a visual column with >> colours for each type of snapshot showing how full the drive would be). You >> know the size of the external drive (and that's fixed for each device), you >> also know the average sizes of snapshots, so you can show the user how much >> space they will have, and let them play around with the numbers. >> > > I think there is some merit for this as a backwards-looking process. > -- richard And there's some potential to use it as a planning tool too I'd say. Certainly I'd kill for a storage visualisation tool like this on my servers. Being able to predict (even with quite a wide range of uncertancy) how much space my pool and my backups are going to need would be a godsend, especially if you could have the software extrapolate out to 12-24 months in the future. In fact, with the right statistics behind it, you could find a growth curve that matches currently sampled data, let the administrator use that for network planning, and even alert them should the predicted growth ever exceed a certain percentage of that. Heh, you could even have a countdown in the status area, letting us know how long we have until our pools fill up :) _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
