Chiradeep, The issue I am experiencing is that the system VMs are not syncing to dom0 on devcloud (i.e. the dom0 clock and the SSVM clock are different). As I mentioned earlier in this thread, the syncing was working previously which seems to jibe with your findings. What mechanism is used to sync the dom0 and domU clocks (e.g. NTP, kernel driver, etc)? It may be a situation where the pieces are present, but they aren't configured properly or simply not running.
As an aside, we can not run VirtualBox Additions on devcloud due a conflict with the Xen kernel. Therefore, I hard execute "ntpdate pool.ntp.org" periodically on the devcloud host to keep the clock synced with the "real world". Another approach is to configure NTP with a very large drift and increase check frequency to accomodate the large clock swings that can occur. Thanks, -John On Wed, May 15, 2013 at 1:21 PM, Chiradeep Vittal < chiradeep.vit...@citrix.com> wrote: > Perhaps this is a problem with DevCloud? > http://nerdboys.com/2011/03/15/how-to-fix-virtualbox-time-synchonization-pr > oblems/ > > > > On 5/15/13 10:17 AM, "Chiradeep Vittal" <chiradeep.vit...@citrix.com> > wrote: > > >According to our resident Xen expert, any PV kernel automatically syncs to > >the hardware clock on dom0. > > > >On 5/15/13 9:50 AM, "John Burwell" <jburw...@basho.com> wrote: > > > >>Marcus, > >> > >>Agreed. I think we need to add a set of hypervisor agnostic time > >>keeping guidelines to the documentation. I just wanted to make sure > >>there wasn't anything KVM specific that should be added as well. > >> > >>Thanks, > >>-John > >> > >>On May 15, 2013, at 12:48 PM, Marcus Sorensen <shadow...@gmail.com> > >>wrote: > >> > >>> Just the general one that system vms sync their time to the > >>> hypervisor, thus admins need to keep the hypervisor time correct. It > >>> sounds like that will be the case for all three, if we can manage it. > >>> > >>> On Wed, May 15, 2013 at 10:44 AM, John Burwell <jburw...@basho.com> > >>>wrote: > >>>> Marcus, > >>>> > >>>> Excellent. So, it looks like we have KVM resolved. We just need to > >>>>address Xen and VMWare now. Do you think we need to any guidance to > >>>>the documentation regarding KVM time keeping (e.g. environmental > >>>>prerequisites)? > >>>> > >>>> Thanks, > >>>> -John > >>>> > >>>> > >>>> > >>>> On May 15, 2013, at 12:39 PM, Marcus Sorensen <shadow...@gmail.com> > >>>>wrote: > >>>> > >>>>> KVM LibvirtComputingResource has been patched in master. Tested on > >>>>> master, 4.1, and both the acton and current system vm templates. This > >>>>> patch makes system vms use 'kvmclock' for their timer, which is a vm > >>>>> driver that gets it's time from the hypervisor. No change to the > >>>>> system vm template itself. > >>>>> > >>>>> bfc5887a1bf6b41e88dd7a8f9987fcee8d3d9175 > >>>>> > >>>>> On Wed, May 15, 2013 at 9:08 AM, Chip Childers > >>>>> <chip.child...@sungard.com> wrote: > >>>>>> On Wed, May 15, 2013 at 11:03:16AM -0400, John Burwell wrote: > >>>>>>> Chip, > >>>>>>> > >>>>>>> One other item I neglected to mention was that clock sync, at least > >>>>>>>for Xen system VMs, wasn't an issue in the Jan-Feb timeframe. > >>>>>>>Previously when I encountered these issues, syncing the host's clock > >>>>>>>and rebuilding the system VMs addressed the issue. I assumed, but > >>>>>>>never verified, that the SSVM was syncing back against the host's > >>>>>>>clock through hypervisor. During my testing yesterday, aside from > >>>>>>>hard setting the clock, I was unable to force clock sync on the > >>>>>>>SSVM. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> -John > >>>>>> > >>>>>> I think that's our issue right now... answering the question: Why > >>>>>>is > >>>>>> this only an issue now? Did we just get lucky up to this point? > >>>>>>Since > >>>>>> the SSVMs are the same template as the timeframe you mention, I tend > >>>>>>to > >>>>>> believe that you / we were just lucky. > >>>>>> > >>>>>> Anyone else have thoughts? > >>>>>> > >>>>>>> > >>>>>>> On May 15, 2013, at 10:18 AM, Chip Childers > >>>>>>><chip.child...@sungard.com> wrote: > >>>>>>> > >>>>>>>> Starting a thread on this specific issue. > >>>>>>>> > >>>>>>>> CLOUDSTACK-2492 was opened, which is basically the fact that the > >>>>>>>>System > >>>>>>>> VMs aren't syncing time to the host or to an NTP server. The S3 > >>>>>>>> integration is broken because of this problem, and therefore could > >>>>>>>>not > >>>>>>>> be considered a function available in 4.1 if we release as is. > >>>>>>>> > >>>>>>>> We need input from people that know about the current system VMs > >>>>>>>>(the > >>>>>>>> 3.x VMs), as well as the possibility of using the newer ones that > >>>>>>>>we > >>>>>>>> have been considering experimental for 4.1.0. > >>>>>>>> > >>>>>>>> What should we do? > >>>>>>>> > >>>>>>>> -chip > >>>>>>> > >>>>>>> > >>>> > >> > > > >
