On 01/22/2017 11:56 PM, kna...@gmail.com wrote:
Hello all!
I'd like to do some research on how long each stage of OpenVZ CT live
migration takes.
I wonder if there is any single source of such information with
particular timestamps? So far as I understood "-v" flag in vzmigrate
command can't help me much with what I'd like to achieve. "-t" flag
show some suspend statistics.
When doing live migration, a few timings might be interesting:
1. Frozen time (time for which the container was frozen)
2. Total migration time (with or without data migration)
From the availability point of view, frozen time is important and total
migration time is not,
so we try to minimize the frozen time even sacrificing the total
migration time. One example
of the strategy is using two rsyncs, before and after the freeze. If we
would want to minimize
the total migration time, we'd do only once rsync after the freeze.
This is also the reason why vzmigrate -t shows various timings related
to the freeze. Timings
of preparation and first rsync are not shown as this is not something we
optimize for.
Anyway, as pointed out by Scott, it is easy to augment the script with
showing more timestamps,
as it's a shell script.
Kir
The most relevant source is servers log both on source and destination
ones. But it looks like I have to parse both of them to get the
complete picture of the CT migration process.
I've tried different values for LOG_LEVEL in /etc/vz/vz.conf (1, 5, 10
with following 'service vz restart') but it seems like an amount of
details appearing in the log is not changed despite of what is written
in 'man vz.conf':
LOG_LEVEL=number
Set the logging level for the log file (does not affect
console output). The greater the number is, the more information will
be logged to the LOGFILE. Default is 0, which means to log normal
messages and errors. If set to -1, only errors will be logged.
From that description it's not clear what the maximum reasonable value
for that parameter is).
Part of the /var/log/vzctl.log on source:
2017-01-23T10:28:23+0300 vzctl : CT 100 : Setting up checkpoint...
2017-01-23T10:28:23+0300 vzctl : CT 100 : set CPU flags..
2017-01-23T02:28:23-0500 vzctl : CT 100 : suspend...
2017-01-23T02:28:23-0500 vzctl : CT 100 : get context...
2017-01-23T10:28:23+0300 vzctl : CT 100 : Checkpointing completed
successfully
2017-01-23T10:28:23+0300 vzctl : CT 100 : Setting up checkpoint...
2017-01-23T10:28:23+0300 vzctl : CT 100 : join context..
2017-01-23T02:28:23-0500 vzctl : CT 100 : dump...
2017-01-23T10:28:23+0300 vzctl : CT 100 : Checkpointing completed
successfully
2017-01-23T10:28:24+0300 vzctl : CT 100 : Killing...
2017-01-23T10:28:24+0300 vzctl : CT 100 : put context
2017-01-23T10:28:24+0300 vzctl : CT 100 : The ploop library has been
loaded successfully
2017-01-23T10:28:24+0300 : Unmounting file system at /vz/root/100
2017-01-23T10:28:24+0300 : Unmounting device /dev/ploop39934
2017-01-23T10:28:25+0300 vzctl : CT 100 : Container is unmounted
2017-01-23T10:28:25+0300 vzctl : CT 100 : Destroying container private
area: /vz/private/100
2017-01-23T10:28:26+0300 vzctl : CT 100 : Container private area was
destroyed
2017-01-23T10:28:45+0300 vzctl : CT 100 : stat(/vz/root/100): No such
file or directory
2017-01-23T10:28:46+0300 vzctl : CT 100 : CT configuration saved to
/etc/vz/conf/100.conf
2017-01-23T10:28:46+0300 vzctl : CT 100 : CT configuration saved to
/etc/vz/conf/100.conf
2017-01-23T10:29:01+0300 vzctl : CT 100 : Restoring container ...
2017-01-23T10:29:01+0300 vzctl : CT 100 : The ploop library has been
loaded successfully
2017-01-23T10:29:01+0300 : Opening delta
/vz/private/100/root.hdd/root.hdd
2017-01-23T10:29:01+0300 : Adding delta dev=/dev/ploop39934
img=/vz/private/100/root.hdd/root.hdd (rw)
2017-01-23T10:29:01+0300 : Running: fsck.ext4 -p /dev/ploop39934p1
2017-01-23T10:29:01+0300 : Mounting /dev/ploop39934p1 at /vz/root/100
fstype=ext4 data='balloon_ino=12,'
2017-01-23T10:29:01+0300 vzctl : CT 100 : Container is mounted
2017-01-23T10:29:01+0300 vzctl : CT 100 : Running:
/usr/libexec/vzctl/scripts/vps-prestart
2017-01-23T10:29:01+0300 vzctl : CT 100 : Setting CPU units: 1000
2017-01-23T10:29:01+0300 vzctl : CT 100 : Setting CPUs: 2
2017-01-23T10:29:01+0300 vzctl : CT 100 : Setting devices
2017-01-23T10:29:01+0300 vzctl : CT 100 : Container start in progress...
2017-01-23T10:29:01+0300 vzctl : CT 100 : Restoring completed
successfully
2017-01-23T10:29:01+0300 vzctl : CT 100 : Resuming...
2017-01-23T10:29:01+0300 vzctl : CT 100 : put context
A part of the /var/log/vzctl.log on destination (lines with '
Executing command: /bin/cat /proc/net/dev' and 'Executing command:
free -k ' are removed):
2017-01-23T10:28:08+0300 vzctl : CT 100 : stat(/vz/root/100): No such
file or directory
2017-01-23T10:28:09+0300 vzctl : CT 100 : CT configuration saved to
/etc/vz/conf/100.conf
2017-01-23T10:28:09+0300 vzctl : CT 100 : CT configuration saved to
/etc/vz/conf/100.conf
2017-01-23T10:28:24+0300 vzctl : CT 100 : Restoring container ...
2017-01-23T10:28:24+0300 vzctl : CT 100 : The ploop library has been
loaded successfully
2017-01-23T10:28:24+0300 : Opening delta
/vz/private/100/root.hdd/root.hdd
2017-01-23T10:28:24+0300 : Adding delta dev=/dev/ploop38224
img=/vz/private/100/root.hdd/root.hdd (rw)
2017-01-23T10:28:24+0300 : Running: fsck.ext4 -p /dev/ploop38224p1
2017-01-23T10:28:24+0300 : Mounting /dev/ploop38224p1 at /vz/root/100
fstype=ext4 data='balloon_ino=12,'
2017-01-23T10:28:24+0300 vzctl : CT 100 : Container is mounted
2017-01-23T10:28:24+0300 vzctl : CT 100 : Running:
/usr/libexec/vzctl/scripts/vps-prestart
2017-01-23T10:28:24+0300 vzctl : CT 100 : Setting CPU units: 1000
2017-01-23T10:28:24+0300 vzctl : CT 100 : Setting CPUs: 2
2017-01-23T10:28:24+0300 vzctl : CT 100 : Setting devices
2017-01-23T10:28:24+0300 vzctl : CT 100 : Container start in progress...
2017-01-23T10:28:24+0300 vzctl : CT 100 : Restoring completed
successfully
2017-01-23T10:28:24+0300 vzctl : CT 100 : Resuming...
2017-01-23T10:28:24+0300 vzctl : CT 100 : put context
<....skipped....>
2017-01-23T10:29:00+0300 vzctl : CT 100 : Setting up checkpoint...
2017-01-23T10:29:00+0300 vzctl : CT 100 : set CPU flags..
2017-01-23T02:29:00-0500 vzctl : CT 100 : suspend...
2017-01-23T02:29:00-0500 vzctl : CT 100 : get context...
2017-01-23T10:29:00+0300 vzctl : CT 100 : Checkpointing completed
successfully
2017-01-23T10:29:00+0300 vzctl : CT 100 : Setting up checkpoint...
2017-01-23T10:29:00+0300 vzctl : CT 100 : join context..
2017-01-23T02:29:00-0500 vzctl : CT 100 : dump...
2017-01-23T10:29:00+0300 vzctl : CT 100 : Checkpointing completed
successfully
2017-01-23T10:29:01+0300 vzctl : CT 100 : Killing...
2017-01-23T10:29:02+0300 vzctl : CT 100 : put context
2017-01-23T10:29:02+0300 vzctl : CT 100 : The ploop library has been
loaded successfully
2017-01-23T10:29:02+0300 : Unmounting file system at /vz/root/100
2017-01-23T10:29:02+0300 : Unmounting device /dev/ploop38224
2017-01-23T10:29:02+0300 vzctl : CT 100 : Container is unmounted
2017-01-23T10:29:02+0300 vzctl : CT 100 : Destroying container private
area: /vz/private/100
2017-01-23T10:29:03+0300 vzctl : CT 100 : Container private area was
destroyed
So as far as I understand in order to evaluate a duration of each live
migration stage I have to parse logs on both source and destination
servers. But before I will try to write some script doing that I'd
like to make sure there is no another way of extracting such info
unless to parse logs of both servers.
I have installed the following versions of vz* packages on both servers:
vzctl-4.9.4-1.x86_64
vzctl-core-4.9.4-1.x86_64
vzkernel-2.6.32-042stab120.16.x86_64
vzquota-3.1-1.x86_64
vzstats-0.5.3-1.noarch
Best regards,
Nikolay.
_______________________________________________
Users mailing list
Users@openvz.org
https://lists.openvz.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@openvz.org
https://lists.openvz.org/mailman/listinfo/users
