No, the bug is about libvirt pausing the vm's for live snapshot

I am not sure about your question on clock skew... we have to take a lot of snapshot to surpass the 5 minutes skew though :)


On 02/04/2014 01:45 PM, Steve Dainard wrote:
Just for clarity on that bug report; are you suggesting that the guest must go into a paused state to take a snapshot and that the admin shouldn't be aware of this state?

Although not horribly concerning for most tasks, wouldn't this lead to a 10 second clock skew every time there is a snapshot? Or does the guest sync from host hw clock on resume?

Thanks,

*Steve Dainard *
IT Infrastructure Manager
Miovision <http://miovision.com/> | /Rethink Traffic/
519-513-2407 ex.250
877-646-8476 (toll-free)

*Blog <http://miovision.com/blog> | **LinkedIn <https://www.linkedin.com/company/miovision-technologies> | Twitter <https://twitter.com/miovision> | Facebook <https://www.facebook.com/miovision>*
------------------------------------------------------------------------
Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, Canada | N2C 1L3 This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.


On Tue, Feb 4, 2014 at 3:44 AM, Dafna Ron <[email protected] <mailto:[email protected]>> wrote:

    On 02/04/2014 08:20 AM, Elad Ben Aharon wrote:

         From what I saw in the thread, libvirt pauses the VM, which
        effects the continuity of its operation.
        I checked it also in one of the latest builds of 3.3 and I
        observed the same behaviour:
        2014-02-04 10:18:29,441 INFO
         [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
        (DefaultQuartzScheduler_Worker-22) [5cbc7ab5] VM nfs2-1
        86728f5c-3583-420f-9536-bfabaf11b235 moved from Up --> Paused

        Dafna, I saw you've already opened a bug on that:
        https://bugzilla.redhat.com/show_bug.cgi?id=1057587


    just proves I'm getting senile :)


        ----- Original Message -----
        From: "Dafna Ron" <[email protected] <mailto:[email protected]>>
        To: "Maor Lipchuk" <[email protected]
        <mailto:[email protected]>>
        Cc: "Steve Dainard" <[email protected]
        <mailto:[email protected]>>, "Elad Ben Aharon"
        <[email protected] <mailto:[email protected]>>, "Karli
        Sjöberg" <[email protected] <mailto:[email protected]>>,
        [email protected] <mailto:[email protected]>
        Sent: Tuesday, February 4, 2014 12:15:42 AM
        Subject: Re: [Users] oVirt 3.3.3 RC EL6 Live Snapshot

        Maor I am not saying that we are not doing a live snapshot :) I am
        saying that we need a print in the log that states live
        snapshot command
        was called i.e: Print in the log: LiveSnapshotCommand -> this
        can call
        to the rest of snapshotVDSCreateCommand.


        On 02/03/2014 07:38 PM, Maor Lipchuk wrote:

            On 02/03/2014 07:46 PM, Dafna Ron wrote:

                On 02/03/2014 05:34 PM, Maor Lipchuk wrote:

                    On 02/03/2014 07:18 PM, Dafna Ron wrote:

                        Maor,

                        If snapshotVDSCommand is for live snapshot,
                        what is the offline create
                        snapshot command?

                    It is the CreateSnapshotVdsCommand which calls
                    createVolume in VDSM

                but we need to be able to know that a live snapshot
                was sent and not an
                offline snapshot.

            Yes, at the logs we can see the all process :

            First a request to create a snapshot (new volume) sent to
            VDSM:
            2014-02-02 09:41:09,557 INFO
            [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand]
            (pool-6-thread-49) [67ea047a] START, CreateSnapshotVDSCommand(
            storagePoolId = fcb89071-6cdb-4972-94d1-c9324cebf814,
            ignoreFailoverLimit = false, storageDomainId =
            a52938f7-2cf4-4771-acb2-0c78d14999e5, imageGroupId =
            c1cb6b66-655e-48c3-8568-4975295eb037, imageSizeInBytes =
            21474836480,
            volumeFormat = COW, newImageId =
            6d8c80a4-328f-4a53-86a2-a4080a2662ce,
            newImageDescription = , imageId =
            5085422e-6592-415a-9da3-9e43dac9374b,
            sourceImageGroupId =
            c1cb6b66-655e-48c3-8568-4975295eb037), log id: 7875f3f5

            after the snapshot gets created :
            2014-02-02 09:41:20,553 INFO
            [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand]
            (pool-6-thread-49) Ending command successfully:
            org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand

            then the engine calls the live snapshot (see also [1])
            2014-02-02 09:41:30,234 INFO
            [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
            (pool-6-thread-49) FINISH, SnapshotVDSCommand, log id:
            7e0d7872

                Elad, somewhere in this flow we need to know that the
                snapshot was taken
                on a running vm :) this seems like a bug to me.

                        we did not say that live snapshot did not
                        succeed :)  we said that the
                        vm is paused and restarted - which is
                        something that should not happen
                        for live snapshot (or at least never did before).

                    It's not sure that the restart is related to the
                    live snapshot. but that
                    should be observed in the libvirt/vdsm logs.

                yes, I am sure because the user is reporting it and
                the logs show it...

                        as I wrote before, we know that vdsm is
                        reporting the vm as paused, that
                        is because libvirt is reporting the vm as
                        paused and I think that its
                        happening because libvirt is not doing a live
                        snapshot and so pauses the
                        vm while taking the snapshot.

                    That sounds logic to me, it's need to be checked
                    with libvirt, if that
                    kind of behaviour could happen.

                Elad, can you please try to reproduce and open a bug
                to libvirt?

                        Dafna


                        On 02/03/2014 05:08 PM, Maor Lipchuk wrote:

                                From the engine logs it seems that
                            indeed live snapshot is called
                            (The
                            command is snapshotVDSCommand see [1]).
                            This is done right after the snapshot has
                            been created in the VM and it
                            signals the qemu process to start using
                            the new volume created.

                            When live snapshot does not succeed we
                            should see in the log something
                            like "Wasn't able to live snapshot due to
                            error:...", but it does not
                            appear so it seems that this worked out fine.

                            At some point I can see in the logs that
                            VDSM reports to the engine
                            that
                            the VM is paused.


                            [1]
                            2014-02-02 09:41:20,564 INFO
                            
[org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
                            (pool-6-thread-49) START,
                            SnapshotVDSCommand(HostName = ovirt002,
                            HostId
                            = 3080fb61-2d03-4008-b47f-9b66276a4257,
                            vmId=e261e707-a21f-4ae8-9cff-f535f4430446), log
                            id: 7e0d7872
                            2014-02-02 09:41:21,119 INFO
                            
[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
                            (DefaultQuartzScheduler_Worker-93) VM
                            snapshot-test
                            e261e707-a21f-4ae8-9cff-f535f4430446 moved
                            from Up --> Paused
                            2014-02-02 09:41:30,234 INFO
                            
[org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand]
                            (pool-6-thread-49) FINISH,
                            SnapshotVDSCommand, log id: 7e0d7872
                            2014-02-02 09:41:30,238 INFO
                            [org.ovirt.engine.core.bll.CreateSnapshotCommand]
                            (pool-6-thread-49)
                            [67ea047a] Ending command successfully:
                            org.ovirt.engine.core.bll.CreateSnapshotCommand
                            ...

                            Regards,
                            Maor

                            On 02/03/2014 06:24 PM, Dafna Ron wrote:

                                Thanks Steve.

                                from the logs I can see that the
                                create snapshot succeeds and that the
                                vm is resumed.
                                the vm moves to pause as part of
                                libvirt flows:

                                2014-02-02 14:41:20.872+0000: 5843:
                                debug :
                                qemuProcessHandleStop:728 :
                                Transitioned guest snapshot-test to
                                paused state
                                2014-02-02 14:41:30.031+0000: 5843:
                                debug :
                                qemuProcessHandleResume:776
                                : Transitioned guest snapshot-test out
                                of paused into resumed state

                                There are bugs here but I am not sure
                                yet if this is libvirt
                                regression
                                or engine.

                                I'm adding Elad and Maor since in
                                engine logs I can't see anything
                                calling for live snapshot (only for
                                snapshot) - Maor, shouldn't live
                                snapshot command be logged somewhere
                                in the logs?
                                Is it possible that engine is calling
                                to create snapshot and not
                                create
                                live snapshot which is why the vm pauses?

                                Elad, if engine is not logging live
                                snapshot anywhere I would open
                                a bug
                                for engine (to print that in the logs).
                                Also, there is a bug in vdsm log for
                                sdc where the below is logged as
                                ERROR and not INFO:

                                Thread-23::ERROR::2014-02-02
                                
09:51:19,497::sdc::137::Storage.StorageDomainCache::(_findDomain)
                                looking for unfetched domain
                                a52938f7-2cf4-4771-acb2-0c78d14999e5
                                Thread-23::ERROR::2014-02-02
                                
09:51:19,497::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain)


                                looking for domain
                                a52938f7-2cf4-4771-acb2-0c78d14999e5

                                If the engine was sending live
                                snapshot or if there is no
                                difference in
                                the two commands in engine side than I
                                would open a bug for libvirt
                                for
                                pausing the vm during live snapshot.

                                Dafna

                                On 02/03/2014 02:41 PM, Steve Dainard
                                wrote:

                                    [root@ovirt002 ~]# vdsClient -s 0
                                    getStorageDomainInfo
                                    a52938f7-2cf4-4771-acb2-0c78d14999e5
                                    uuid =
                                    a52938f7-2cf4-4771-acb2-0c78d14999e5
                                    pool =
                                    ['fcb89071-6cdb-4972-94d1-c9324cebf814']
                                    lver = 5
                                    version = 3
                                    role = Master
                                    remotePath = gluster-store-vip:/rep1
                                    spm_id = 2
                                    type = NFS
                                    class = Data
                                    master_ver = 1
                                    name = gluster-store-rep1


                                    *Steve Dainard *
                                    IT Infrastructure Manager
                                    Miovision <http://miovision.com/>
                                    | /Rethink Traffic/
                                    519-513-2407 <tel:519-513-2407> ex.250
                                    877-646-8476
                                    <tel:877-646-8476> (toll-free)

                                    *Blog <http://miovision.com/blog>
                                    | **LinkedIn
                                    
<https://www.linkedin.com/company/miovision-technologies>
                                     | Twitter
                                    <https://twitter.com/miovision>  |
                                    Facebook
                                    <https://www.facebook.com/miovision>*
                                    
------------------------------------------------------------------------


                                    Miovision Technologies Inc. | 148
                                    Manitou Drive, Suite 101,
                                    Kitchener,
                                    ON, Canada | N2C 1L3
                                    This e-mail may contain
                                    information that is privileged or
                                    confidential. If you are not the
                                    intended recipient, please delete
                                    the
                                    e-mail and any attachments and
                                    notify us immediately.


                                    On Sun, Feb 2, 2014 at 2:55 PM,
                                    Dafna Ron <[email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>> wrote:

                                            please run vdsClient -s 0
                                    getStorageDomainInfo
a52938f7-2cf4-4771-acb2-0c78d14999e5

                                            Thanks,

                                            Dafna



                                            On 02/02/2014 03:02 PM,
                                    Steve Dainard wrote:

                                                Logs attached with VM
                                    running on qemu-kvm-rhev packages
                                    installed.

                                                *Steve Dainard *
                                                IT Infrastructure Manager
                                                Miovision
                                    <http://miovision.com/> | /Rethink
                                    Traffic/
                                    519-513-2407 <tel:519-513-2407>
                                    <tel:519-513-2407
                                    <tel:519-513-2407>> ex.250

                                    877-646-8476 <tel:877-646-8476>
                                    <tel:877-646-8476
                                    <tel:877-646-8476>> (toll-free)

                                                *Blog
                                    <http://miovision.com/blog> |
                                    **LinkedIn
<https://www.linkedin.com/company/miovision-technologies>
                                     |
                                                Twitter
                                    <https://twitter.com/miovision>  |
                                    Facebook
<https://www.facebook.com/miovision>* ------------------------------------------------------------------------


                                                Miovision Technologies
                                    Inc. | 148 Manitou Drive, Suite 101,
                                                Kitchener, ON, Canada
                                    | N2C 1L3
                                                This e-mail may
                                    contain information that is
                                    privileged or
                                                confidential. If you
                                    are not the intended recipient, please
                                                delete the e-mail and
                                    any attachments and notify us
                                    immediately.


                                                On Sun, Feb 2, 2014 at
                                    5:05 AM, Dafna Ron
                                    <[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>>> wrote:

                                                    can you please
                                    upload full engine, vdsm, libvirt and
                                    vm's
                                                qemu logs?


                                                    On 02/02/2014
                                    02:08 AM, Steve Dainard wrote:

                                                        I have two
                                    CentOS 6.5 Ovirt hosts (ovirt001,
                                    ovirt002)

                                                        I've installed
                                    the applicable qemu-kvm-rhev
                                    packages
                                                from this
                                                        site:
                                    http://www.dreyou.org/ovirt/vdsm32/Packages/
                                    on
                                                ovirt002.

                                                        On ovirt001 if
                                    I take a live snapshot:

                                                        Snapshot 'test
                                    qemu-kvm' creation for VM
                                                'snapshot-test' was
                                                        initiated by
                                    admin@internal.
                                                        The VM is paused
                                                        Failed to
                                    create live snapshot 'test qemu-kvm'
                                    for VM
'snapshot-test'. VM restart is
                                    recommended.
                                                        Failed to
                                    complete snapshot 'test qemu-kvm'
                                    creation
                                                for VM
                                                        'snapshot-test'.
                                                        The VM is then
                                    started, and the status for the
                                    snapshot
                                                        changes to OK.

                                                        On ovirt002
                                    (with the packages from dreyou) I
                                    don't
                                                get any
                                                        messages about
                                    a snapshot failing, but my VM is
                                    still
                                                paused
                                                        to complete
                                    the snapshot. Is there something else
                                                other than
                                                        the
                                    qemu-kvm-rhev packages that would
                                    enable this
                                                functionality?

                                                        I've looked
                                    for some information on when the
                                    packages
                                                would be
                                                        built as
                                    required in the CentOS repos, but I
                                    don't see
                                                        anything
                                    definitive.

                                    
http://lists.ovirt.org/pipermail/users/2013-December/019126.html
                                                        Looks like one
                                    of the maintainers is waiting for
                                                someone to
                                                        tell him what
                                    flags need to be set.

                                                        Also, another
                                    thread here:
                                    
http://comments.gmane.org/gmane.comp.emulators.ovirt.arch/1618
                                                        same
                                    maintainer, mentioning that he
                                    hasn't seen
                                                anything in
                                                        the bug tracker.

                                                        There is a bug
                                    here:
                                    
https://bugzilla.redhat.com/show_bug.cgi?id=1009100
                                    that
                                    seems
                                                        to have ended
                                    in finding a way for qemu to expose
                                                whether it
                                                        supports live
                                    snapshots, rather than figuring
                                    out how
                                                to get
                                                        the CentOS
                                    team the info they need to build the
                                                packages with
                                                        the proper
                                    flags set.

                                                        I have bcc'd
                                    both dreyou (packaged the
                                    qemu-kvm-rhev
                                                packages
                                                        listed above)
                                    and Russ (CentOS maintainer
                                    mentioned in
                                    the
                                                        other threads)
                                    if they wish to chime in and perhaps
                                                        collaborate on
                                    which flags, if any, should be set
                                    for the
                                                        qemu-kvm
                                    builds so we can get a CentOS bug
                                    report
                                                going and
                                                        hammer this out.

                                                        Thanks everyone.

                                                        **crosses
                                    fingers and hopes for live snapshots
                                    soon**



                                                        *Steve Dainard *
                                                        IT
                                    Infrastructure Manager
                                                        Miovision
                                    <http://miovision.com/> | /Rethink
                                    Traffic/
                                    519-513-2407 <tel:519-513-2407>
                                    <tel:519-513-2407
                                    <tel:519-513-2407>>
                                    <tel:519-513-2407 <tel:519-513-2407>
                                                <tel:519-513-2407
                                    <tel:519-513-2407>>>
                                    <tel:519-513-2407
                                    <tel:519-513-2407>
                                    <tel:519-513-2407 <tel:519-513-2407>>
<tel:519-513-2407
                                    <tel:519-513-2407>
                                    <tel:519-513-2407
                                    <tel:519-513-2407>>>> ex.250
                                    877-646-8476 <tel:877-646-8476>
                                    <tel:877-646-8476
                                    <tel:877-646-8476>>
                                    <tel:877-646-8476 <tel:877-646-8476>
                                                <tel:877-646-8476
                                    <tel:877-646-8476>>>
                                    <tel:877-646-8476
                                    <tel:877-646-8476>
                                    <tel:877-646-8476 <tel:877-646-8476>>

<tel:877-646-8476
                                    <tel:877-646-8476>
                                    <tel:877-646-8476
                                    <tel:877-646-8476>>>> (toll-free)

                                                        *Blog
                                    <http://miovision.com/blog> |
                                    **LinkedIn
<https://www.linkedin.com/company/miovision-technologies>
                                     |
                                                        Twitter
                                    <https://twitter.com/miovision>  |
                                    Facebook
<https://www.facebook.com/miovision>*
                                    
------------------------------------------------------------------------




                                                        Miovision
                                    Technologies Inc. | 148 Manitou Drive,
                                    Suite
                                                101,
                                                        Kitchener, ON,
                                    Canada | N2C 1L3
                                                        This e-mail
                                    may contain information that is
                                    privileged or
                                                        confidential.
                                    If you are not the intended
                                    recipient,
                                                please
                                                        delete the
                                    e-mail and any attachments and
                                    notify us
                                                immediately.


                                                        On Fri, Jan
                                    31, 2014 at 1:26 PM, Steve Dainard
<[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>>
<mailto:[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>

<mailto:[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>>>> wrote:


                                                                How
                                    would you developers, speaking for the
                                                        oVirt-community,
                                                                propose to
                                                                solve
                                    this for CentOS _now_ ?

                                                                I
                                    would imagine that the easiest way is
                                    that
                                                you build and
                                                                host
                                    this one
                                    package(qemu-kvm-rhev), since you´ve
                                    basically
                                                already
                                                        have
                                                                the source
                                                                and
                                    recipe (since you´re already
                                    providing it
                                                for RHEV
anyway). Then,
                                                                once
                                    that´s in place, it´s more a
                                    question of
                                                where to
                                                        host the
packages, in what repository. Be
                                    it your
                                    own,
                                                or some
                                                        other
                                                                repo
                                    set up
                                                                for
                                    the SIG.

                                                                This
                                    is my view, how I as a user view this
                                    issue.



                                                            I think
                                    this is a pretty valid view.

                                                            What would
                                    it take to get the correct qemu
                                    package
                                                hosted
                                                        in the
                                                            ovirt repo?

                                                                --

                                                                Med
                                    Vänliga Hälsningar

                                     
-------------------------------------------------------------------------------



                                                                Karli
                                    Sjöberg
Swedish University of Agricultural
                                    Sciences
                                                Box 7079
                                                        (Visiting
                                                                Address
Kronåsvägen 8)
                                                                S-750
                                    07 Uppsala, Sweden
                                                                Phone:
                                    +46-(0)18-67 15 66
                                    <tel:%2B46-%280%2918-67%2015%2066>
                                    <tel:%2B46-%280%2918-67%2015%2066>
                                    <tel:%2B46-%280%2918-67%2015%2066>
                                    <tel:%2B46-%280%2918-67%2015%2066>
                                    [email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>
<mailto:[email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>>
<mailto:[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>
<mailto:[email protected]
                                    <mailto:[email protected]>>>>

                                    
_______________________________________________
                                                                Users
                                    mailing list
                                    [email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>
<mailto:[email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>>
<mailto:[email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>
<mailto:[email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>>>

                                    
http://lists.ovirt.org/mailman/listinfo/users






                                    
_______________________________________________
                                                        Users mailing list
                                    [email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>
<mailto:[email protected]
                                    <mailto:[email protected]>
                                    <mailto:[email protected]
                                    <mailto:[email protected]>>>

                                    
http://lists.ovirt.org/mailman/listinfo/users



                                                    --     Dafna Ron




                                            --     Dafna Ron





-- Dafna Ron




--
Dafna Ron
_______________________________________________
Users mailing list
[email protected]
http://lists.ovirt.org/mailman/listinfo/users

Reply via email to