Re: [pve-devel] snapshot improvements

2012-09-29 Thread Dietmar Maurer
> > Are your aware of the fact that ssh needs to encrypt/decrypt all data. > > This needs much CPU power and is slow (still no AES support in libs). > > Oh we could also use netcat instead. How exactly? ___ pve-devel mailing list pve-devel@pve.proxmox

Re: [pve-devel] snapshot improvements

2012-09-29 Thread Stefan Priebe - Profihost AG
Am 29.09.2012 um 08:26 schrieb Dietmar Maurer : >> I thought of something like: >> ssh cloud1-1200.de-nserver.de zfs send JBOD01Pool/vm-105-disk- >> 1@testsnap >> | zfs recv tank/abc >> >> or >> >> ssh cloud1-1200.de-nserver.de zfs send >> JBOD01Pool/vm-105-disk-1@spriebetest | gzip >/mnt/pve/n

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Dietmar Maurer
> I thought of something like: > ssh cloud1-1200.de-nserver.de zfs send JBOD01Pool/vm-105-disk- > 1@testsnap > | zfs recv tank/abc > > or > > ssh cloud1-1200.de-nserver.de zfs send > JBOD01Pool/vm-105-disk-1@spriebetest | gzip >/mnt/pve/nfstor/vm-105- > 2012-09-25.gz Are your aware of the fact

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Stefan Priebe
Am 27.09.2012 09:30, schrieb Stefan Priebe - Profihost AG: Am 27.09.2012 09:26, schrieb Dietmar Maurer: The idea is that we do not backup any snapshot data. The vzdump would only include the data of from the running instance. I guess that is OK? But isn't the correct way to make a snapshot and

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Dietmar Maurer
> maybe do you have cloned it ? no - I need to test that again. > > > - Mail original - > > De: "Dietmar Maurer" > À: "Alexandre DERUMIER" > Cc: pve-devel@pve.proxmox.com, "Stefan Priebe" > Envoyé: Vendredi 28 Septembre 201

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Alexandre DERUMIER
maybe do you have cloned it ? - Mail original - De: "Dietmar Maurer" À: "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com, "Stefan Priebe" Envoyé: Vendredi 28 Septembre 2012 11:32:01 Objet: RE: [pve-devel] snapshot improvements > what do you

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Dietmar Maurer
> what do you mean by snapshot used by another snapshot? > ex: > image->snap1-snap2->you are here > > you can delete snap1 without any problem, and without need to merge. Really? - last time I tried I got 'snapshot in use' (or something like that). ___

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Alexandre DERUMIER
"Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com, "Stefan Priebe" Envoyé: Vendredi 28 Septembre 2012 06:40:10 Objet: RE: [pve-devel] snapshot improvements > Subject: Re: [pve-devel] snapshot improvements > > do you mean : > backup the main image > then

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
> Subject: Re: [pve-devel] snapshot improvements > > do you mean : > backup the main image > then > backup each snasphot increment ? Yes, something like that. > zfs send can do incremental backup with "zfs send -I" > > http://docs.oracle.com/cd/E19082-01/8

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Alexandre DERUMIER
DERUMIER" Cc: pve-devel@pve.proxmox.com, "Stefan Priebe" Envoyé: Jeudi 27 Septembre 2012 13:37:59 Objet: RE: [pve-devel] snapshot improvements And if one would consider to backup snapshots, I am quite sure he only wants to backup shared data once. A simple data export would duplicate larg

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
And if one would consider to backup snapshots, I am quite sure he only wants to backup shared data once. A simple data export would duplicate large amounts of data. > Subject: Re: [pve-devel] snapshot improvements > > I think I have already respond to that ;) > > 2 ways : &

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
> ah OK but wouldn't it be nice to be able to backup live snapshots? > compress and store them on a seperate NFS server? Also Proxmox only > allows to schedule beckups not to schedule snapshots ;-) Sure, that would be nice. I already have very detailed plans how to do that (but no budget). _

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Stefan Priebe - Profihost AG
Am 27.09.2012 09:26, schrieb Dietmar Maurer: The idea is that we do not backup any snapshot data. The vzdump would only include the data of from the running instance. I guess that is OK? But isn't the correct way to make a snapshot and then compress the snapshot? How can vzdump verify that the

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
> > The idea is that we do not backup any snapshot data. The vzdump would > > only include the data of from the running instance. I guess that is OK? > > But isn't the correct way to make a snapshot and then compress the > snapshot? How can vzdump verify that the data integrity is fine? I guess I

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
> I think I have already respond to that ;) > > 2 ways : > > - clone the snapshot and export image through iscsi and backup it > > - use zfs send through ssh (zfs send image1@snap1 > /imagefile) Both ways are clumsy. Also, nexenta snapshot support is quite unusable, because you can't delete s

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Alexandre DERUMIER
e-devel@pve.proxmox.com, "Stefan Priebe" Envoyé: Jeudi 27 Septembre 2012 09:04:31 Objet: RE: [pve-devel] snapshot improvements > I also don't think that all storage can export datas with snapshots inside. > > rbd can export an image from a snapshot, nexenta and sheepd

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
> > The idea is that we do not backup any snapshot data. The vzdump would > > only include the data of from the running instance. I guess that is OK? > > But isn't the correct way to make a snapshot and then compress the > snapshot? How can vzdump verify that the data integrity is fine? Sorry, I

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
> I also don't think that all storage can export datas with snapshots inside. > > rbd can export an image from a snapshot, nexenta and sheepdog too. How can you access snapshot data with nexenta? ___ pve-devel mailing list pve-devel@pve.proxmox.com http

Re: [pve-devel] snapshot improvements

2012-09-26 Thread Stefan Priebe - Profihost AG
Am 27.09.2012 06:43, schrieb Dietmar Maurer: I also tried todo backups with snapshots but this doesn't seem to work: INFO: starting new backup job: vzdump 100 --remove 0 --mode snapshot -- compress lzo --storage backuplocal --node serv121 INFO: Starting Backup of VM 100 (qemu) INFO: status = run

Re: [pve-devel] snapshot improvements

2012-09-26 Thread Alexandre DERUMIER
as with snapshots inside. rbd can export an image from a snapshot, nexenta and sheepdog too. - Mail original - De: "Dietmar Maurer" À: "Stefan Priebe" Cc: pve-devel@pve.proxmox.com Envoyé: Jeudi 27 Septembre 2012 06:43:44 Objet: Re: [pve-devel] snapshot imp

Re: [pve-devel] snapshot improvements

2012-09-26 Thread Dietmar Maurer
> I also tried todo backups with snapshots but this doesn't seem to work: > > INFO: starting new backup job: vzdump 100 --remove 0 --mode snapshot -- > compress lzo --storage backuplocal --node serv121 > INFO: Starting Backup of VM 100 (qemu) > INFO: status = running > ERROR: Backup of VM 100 fail

Re: [pve-devel] snapshot improvements

2012-09-26 Thread Stefan Priebe
Am 24.09.2012 10:50, schrieb Dietmar Maurer: I tried to improve snapshot behavior a bit: - take snapshot is now a background task (does not block monitor) - try to save VM state while VM is online So downtime during snapshot should be shorter now, especially if the VM use much RAM. Feel free

Re: [pve-devel] snapshot improvements

2012-09-25 Thread Alexandre DERUMIER
r" À: "Dietmar Maurer" , "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com Envoyé: Mardi 25 Septembre 2012 10:37:42 Objet: RE: [pve-devel] snapshot improvements > > do you think image = size of memory if enough ? (if we have some bytes > > more with incr

Re: [pve-devel] snapshot improvements

2012-09-25 Thread Dietmar Maurer
> > do you think image = size of memory if enough ? (if we have some bytes > > more with incremental ?) > > PVE/QemuServer.pm: > > my $driver_state_size = 32; # assume 32MB is enough to safe all driver > state; > my $size = $conf->{memory} + $driver_state_size; Unfortunately, the qemu co

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Alexandre DERUMIER
quot;Dietmar Maurer" À: "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com Envoyé: Lundi 24 Septembre 2012 12:35:35 Objet: RE: [pve-devel] snapshot improvements > do you think image = size of memory if enough ? (if we have some bytes > more with incremental ?) PVE/Qemu

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Dietmar Maurer
> do you think image = size of memory if enough ? (if we have some bytes > more with incremental ?) PVE/QemuServer.pm: my $driver_state_size = 32; # assume 32MB is enough to safe all driver state; my $size = $conf->{memory} + $driver_state_size; > Also, any problem if some datas in vms

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Alexandre DERUMIER
dre DERUMIER" Cc: pve-devel@pve.proxmox.com Envoyé: Lundi 24 Septembre 2012 11:52:04 Objet: RE: [pve-devel] snapshot improvements > How is it possible to save the vmstate if the vm is not paused ? (with a lot > a > memory write access by example ?) I currently allocate am im

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Dietmar Maurer
> How is it possible to save the vmstate if the vm is not paused ? (with a lot a > memory write access by example ?) I currently allocate am image equal to the size of the VM memory. Then I simply do an incremental state save (like a vm migration), and keep the VM running until (saved_bytes + rem

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Alexandre DERUMIER
I'll test that today. How is it possible to save the vmstate if the vm is not paused ? (with a lot a memory write access by example ?) - Mail original - De: "Dietmar Maurer" À: pve-devel@pve.proxmox.com Envoyé: Lundi 24 Septembre 2012 10:50:30 Objet: [pve-

[pve-devel] snapshot improvements

2012-09-24 Thread Dietmar Maurer
I tried to improve snapshot behavior a bit: - take snapshot is now a background task (does not block monitor) - try to save VM state while VM is online So downtime during snapshot should be shorter now, especially if the VM use much RAM. Feel free to test and report bugs ;-) - Dietmar