Apologies for the low tech suggestion, but if this really is a clone of a previously existing template, could the clone operation just be done ahead of time? I.e., have the build server keep X copies ready for use and generate additional copies as those are consumed, so that the cloning is no longer on the critical path?
On Fri, Mar 1, 2019 at 11:09 AM Jerry Sievers <gsiever...@comcast.net> wrote: > Kenneth Marshall <k...@rice.edu> writes: > > > On Fri, Mar 01, 2019 at 11:57:30AM -0800, Kevin Wilkinson wrote: > > > >> if you are able/willing to use ZFS (rather than ext4, xfs, ...) to > >> store your database, then it might work for you. ZFS is > >> copy-on-write so it can very quickly clone a database. > >> > >> kevin > > > > Hi Arjun > > > > Redhat 7 does have LVM snapshots that does something similar. Kevin is > > correct, COW is the secret. > > Going a bit further... > > Any sort of storage backend that can support *atomic* snapshots across > *all* volumes (in case multiple tablespaces ar involved), can be used to > permit $instantaneous cloning where instantaneous relates to the actual > snapshot time and crash recovery. > > Inability to make *atomic* snaps but perhaps seperate snaps very > quickly, combined with PITR can result in clones of high-churn systems > sized in TBs (as in our use case) to be provisioned in about 1 minute. > > Nothing but the most trivial system can be cloned rapidly and perhaps > any number of times in succession without employment of > thin-provisioning, copy-on-write (as mentioned already), etc. > > Virtual copy is more and more compelling as physical > size, or more precisely, *physical* copy time grow. > > HTH > > > > > > > Regards, > > Ken > > > > > > -- > Jerry Sievers > Postgres DBA/Development Consulting > e: postgres.consult...@comcast.net > >
