A question on vm migrations when hosts are set into a maintenance mode.
I checked the vm migration when their host is set to a maintenance mode and found that even if the orchestration layer fires the each vm migration at the same time using a ha_worker thread, the actual migration seems to be executed serially. Is this what we expect? And if so, any chance to make the actual migrations in parallel? Thanks Alex Ough
Re: A question on vm migrations when hosts are set into a maintenance mode.
All, After a little more investigation, I found that the 'MigrateCommand' defined its 'executeInSequence' method to return 'FALSE', which seems to make the vm migrations as serial even if the migration requests are dispatched to ha_worker in parallel. You can confirm this in line 56 of '/cloudstack/core/src/com/cloud/agent/api/MigrateCommand.java' So my question is if there is any reason why the method have been defined to return 'FALSE' always? And do we expect any side effects and/or malfunctioning if we change it to returning 'TRUE'? Any answers/comments will be very appreciated. Thanks Alex Ough On Wed, Sep 18, 2013 at 10:22 AM, Alex Ough wrote: > I checked the vm migration when their host is set to a maintenance mode > and found that even if the orchestration layer fires the each vm migration > at the same time using a ha_worker thread, the actual migration seems to be > executed serially. > > Is this what we expect? And if so, any chance to make the actual > migrations in parallel? > > Thanks > Alex Ough >
Re: A question on vm migrations when hosts are set into a maintenance mode.
For a resolution without breaking possible flows, I'd like to add the value of 'executeInSequence' to the global setting. Is there any reason not to do this? Thanks Alex Ough On Mon, Sep 23, 2013 at 12:57 PM, Alex Ough wrote: > All, > > After a little more investigation, I found that the 'MigrateCommand' > defined its 'executeInSequence' method to return 'FALSE', which seems to > make the vm migrations as serial even if the migration requests are > dispatched to ha_worker in parallel. > You can confirm this in line 56 of > '/cloudstack/core/src/com/cloud/agent/api/MigrateCommand.java' > > So my question is if there is any reason why the method have been > defined to return 'FALSE' always? > And do we expect any side effects and/or malfunctioning if we change it to > returning 'TRUE'? > > Any answers/comments will be very appreciated. > Thanks > Alex Ough > > > On Wed, Sep 18, 2013 at 10:22 AM, Alex Ough wrote: > >> I checked the vm migration when their host is set to a maintenance mode >> and found that even if the orchestration layer fires the each vm migration >> at the same time using a ha_worker thread, the actual migration seems to be >> executed serially. >> >> Is this what we expect? And if so, any chance to make the actual >> migrations in parallel? >> >> Thanks >> Alex Ough >> > >
Re: A question on vm migrations when hosts are set into a maintenance mode.
Oh, sorry for the confusion. I must have reversed the flags.
As Kelven pointed, it is set as 'TRUE', which makes the process as
sequential.
So my questions are
1. If there is any reason why the method have been defined to return '*TRUE*'
always?
2. Do we expect any side effects and/or malfunctioning if we change it to
returning *'FALSE*'?
3. For a resolution without breaking possible flows, can we add the value
of 'executeInSequence' to the global setting if #2 answers YES?
On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen wrote:
> I thought executeInSequence of 'true' made it go serially, or
> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
> August of 2010:
>
> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends Command {
> 2010-08-11 09:13:29 -0700 20) String vmName;
> 2010-08-11 09:13:29 -0700 21) String destIp;
> 2011-08-10 10:26:04 -0700 22) String hostGuid;
> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
> 2010-08-11 09:13:29 -0700 24)
> 2010-08-11 09:13:29 -0700 25)
> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
> 2010-08-11 09:13:29 -0700 27) }
> 2012-12-03 22:06:41 -0800 28)
> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
> String destIp, boolean isWindows)
> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
> 2010-08-11 09:13:29 -0700 33) }
> 2012-12-03 22:06:41 -0800 34)
> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
> 2010-08-11 09:13:29 -0700 36) return isWindows;
> 2010-08-11 09:13:29 -0700 37) }
> 2012-12-03 22:06:41 -0800 38)
> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
> 2010-08-11 09:13:29 -0700 40) return destIp;
> 2010-08-11 09:13:29 -0700 41) }
> 2012-12-03 22:06:41 -0800 42)
> 2010-08-11 09:13:29 -0700 43) public String getVmName() {
> 2010-08-11 09:13:29 -0700 44) return vmName;
> 2010-08-11 09:13:29 -0700 45) }
> 2012-12-03 22:06:41 -0800 46)
> 2011-08-10 10:26:04 -0700 47) public void setHostGuid(String guid)
> {
> 2011-08-10 10:26:04 -0700 48) this.hostGuid = guid;
> 2011-08-10 10:26:04 -0700 49) }
> 2012-12-03 22:06:41 -0800 50)
> 2011-08-10 10:26:04 -0700 51) public String getHostGuid() {
> 2011-08-10 10:26:04 -0700 52) return this.hostGuid;
> 2011-08-10 10:26:04 -0700 53) }
> 2010-08-11 09:13:29 -0700 54)
> 2010-08-11 09:13:29 -0700 55) @Override
> 2010-08-11 09:13:29 -0700 56) public boolean executeInSequence() {
> 2010-08-11 09:13:29 -0700 57) return true;
> 2010-08-11 09:13:29 -0700 58) }
> 2010-08-11 09:13:29 -0700 59) }
>
> On Tue, Sep 24, 2013 at 9:58 AM, Chip Childers
> wrote:
> > Hey Kelven - This topic was discussed briefly in the past [1]. Are you
> > able to provide any thoughts on Alex's ideas below?
> >
> > -chip
> >
> >
> > [1] http://markmail.org/message/fznrszaswruvlmuy
> >
> >
> >
> > On Tue, Sep 24, 2013 at 10:53:04AM -0500, Alex Ough wrote:
> >> For a resolution without breaking possible flows, I'd like to add the
> value
> >> of 'executeInSequence' to the global setting.
> >> Is there any reason not to do this?
> >>
> >> Thanks
> >> Alex Ough
> >>
> >>
> >> On Mon, Sep 23, 2013 at 12:57 PM, Alex Ough
> wrote:
> >>
> >> > All,
> >> >
> >> > After a little more investigation, I found that the 'MigrateCommand'
> >> > defined its 'executeInSequence' method to return 'FALSE', which seems
> to
> >> > make the vm migrations as serial even if the migration requests are
> >> > dispatched to ha_worker in parallel.
> >> > You can confirm this in line 56 of
> >> > '/cloudstack/core/src/com/cloud/agent/api/MigrateCommand.java'
> >> >
> >> > So my question is if there is any reason why the method have been
> >> > defined to return 'FALSE' always?
> >> > And do we expect any side effects and/or malfunctioning if we change
> it to
> >> > returning 'TRUE'?
> >> >
> >> > Any answers/comments will be very appreciated.
> >> > Thanks
> >> > Alex Ough
> >> >
> >> >
> >> > On Wed, Sep 18, 2013 at 10:22 AM, Alex Ough
> wrote:
> >> >
> >> >> I checked the vm migration when their host is set to a maintenance
> mode
> >> >> and found that even if the orchestration layer fires the each vm
> migration
> >> >> at the same time using a ha_worker thread, the actual migration
> seems to be
> >> >> executed serially.
> >> >>
> >> >> Is this what we expect? And if so, any chance to make the actual
> >> >> migrations in parallel?
> >> >>
> >> >> Thanks
> >> >> Alex Ough
> >> >>
> >> >
> >> >
>
>
Re: A question on vm migrations when hosts are set into a maintenance mode.
Marcus/Kelven,
Any thoughts on my suggestions?
Thanks
Alex Ough
On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough wrote:
> Oh, sorry for the confusion. I must have reversed the flags.
> As Kelven pointed, it is set as 'TRUE', which makes the process as
> sequential.
>
> So my questions are
> 1. If there is any reason why the method have been defined to return '*
> TRUE*' always?
> 2. Do we expect any side effects and/or malfunctioning if we change it to
> returning *'FALSE*'?
> 3. For a resolution without breaking possible flows, can we add the value
> of 'executeInSequence' to the global setting if #2 answers YES?
>
>
> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen wrote:
>
>> I thought executeInSequence of 'true' made it go serially, or
>> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
>> August of 2010:
>>
>> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends Command
>> {
>> 2010-08-11 09:13:29 -0700 20) String vmName;
>> 2010-08-11 09:13:29 -0700 21) String destIp;
>> 2011-08-10 10:26:04 -0700 22) String hostGuid;
>> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
>> 2010-08-11 09:13:29 -0700 24)
>> 2010-08-11 09:13:29 -0700 25)
>> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
>> 2010-08-11 09:13:29 -0700 27) }
>> 2012-12-03 22:06:41 -0800 28)
>> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
>> String destIp, boolean isWindows)
>> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
>> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
>> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
>> 2010-08-11 09:13:29 -0700 33) }
>> 2012-12-03 22:06:41 -0800 34)
>> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
>> 2010-08-11 09:13:29 -0700 36) return isWindows;
>> 2010-08-11 09:13:29 -0700 37) }
>> 2012-12-03 22:06:41 -0800 38)
>> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
>> 2010-08-11 09:13:29 -0700 40) return destIp;
>> 2010-08-11 09:13:29 -0700 41) }
>> 2012-12-03 22:06:41 -0800 42)
>> 2010-08-11 09:13:29 -0700 43) public String getVmName() {
>> 2010-08-11 09:13:29 -0700 44) return vmName;
>> 2010-08-11 09:13:29 -0700 45) }
>> 2012-12-03 22:06:41 -0800 46)
>> 2011-08-10 10:26:04 -0700 47) public void setHostGuid(String guid)
>> {
>> 2011-08-10 10:26:04 -0700 48) this.hostGuid = guid;
>> 2011-08-10 10:26:04 -0700 49) }
>> 2012-12-03 22:06:41 -0800 50)
>> 2011-08-10 10:26:04 -0700 51) public String getHostGuid() {
>> 2011-08-10 10:26:04 -0700 52) return this.hostGuid;
>> 2011-08-10 10:26:04 -0700 53) }
>> 2010-08-11 09:13:29 -0700 54)
>> 2010-08-11 09:13:29 -0700 55) @Override
>> 2010-08-11 09:13:29 -0700 56) public boolean executeInSequence() {
>> 2010-08-11 09:13:29 -0700 57) return true;
>> 2010-08-11 09:13:29 -0700 58) }
>> 2010-08-11 09:13:29 -0700 59) }
>>
>> On Tue, Sep 24, 2013 at 9:58 AM, Chip Childers
>> wrote:
>> > Hey Kelven - This topic was discussed briefly in the past [1]. Are you
>> > able to provide any thoughts on Alex's ideas below?
>> >
>> > -chip
>> >
>> >
>> > [1] http://markmail.org/message/fznrszaswruvlmuy
>> >
>> >
>> >
>> > On Tue, Sep 24, 2013 at 10:53:04AM -0500, Alex Ough wrote:
>> >> For a resolution without breaking possible flows, I'd like to add the
>> value
>> >> of 'executeInSequence' to the global setting.
>> >> Is there any reason not to do this?
>> >>
>> >> Thanks
>> >> Alex Ough
>> >>
>> >>
>> >> On Mon, Sep 23, 2013 at 12:57 PM, Alex Ough
>> wrote:
>> >>
>> >> > All,
>> >> >
>> >> > After a little more investigation, I found that the 'MigrateCommand'
>> >> > defined its 'executeInSequence' method to return 'FALSE', which
>> seems to
>> >> > make the vm migrations as serial even if the migration requests are
>> >> > dispatched to ha_worker in parallel.
>> >> > You can confirm this in line 56 of
>> >> > '/cloudstack/core/src/com/cloud/agent/api/MigrateCommand.java'
>> >> >
>> >> > So my question is if there is any reason why the method have been
>> >> > defined to return 'FALSE' always?
>> >> > And do we expect any side effects and/or malfunctioning if we change
>> it to
>> >> > returning 'TRUE'?
>> >> >
>> >> > Any answers/comments will be very appreciated.
>> >> > Thanks
>> >> > Alex Ough
>> >> >
>> >> >
>> >> > On Wed, Sep 18, 2013 at 10:22 AM, Alex Ough
>> wrote:
>> >> >
>> >> >> I checked the vm migration when their host is set to a maintenance
>> mode
>> >> >> and found that even if the orchestration layer fires the each vm
>> migration
>> >> >> at the same time using a ha_worker thread, the actual migration
>> seems to be
>> >> >> executed serially.
>> >> >>
>> >> >> Is this what we expect? And if so, any chance to make the actual
>> >> >> migrations in parallel?
>> >> >>
>> >> >> Thanks
>> >> >> Alex Ough
>> >> >>
>> >> >
>> >> >
>>
>>
>
Re: A question on vm migrations when hosts are set into a maintenance mode.
Thanks for the reply, Marcus.
What about the option #3, which is to make it as a global setting?
I think it can prevent side effects if exist.
Alex Ough
On Wed, Oct 2, 2013 at 11:37 AM, Marcus Sorensen wrote:
> Not sure. I don't know the history well enough to know if there were
> issues in the past, it might be that some hypervisors were fine and
> others weren't.
>
> On Wed, Oct 2, 2013 at 9:21 AM, Alex Ough wrote:
> > Marcus/Kelven,
> >
> > Any thoughts on my suggestions?
> >
> > Thanks
> > Alex Ough
> >
> >
> > On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough
> wrote:
> >>
> >> Oh, sorry for the confusion. I must have reversed the flags.
> >> As Kelven pointed, it is set as 'TRUE', which makes the process as
> >> sequential.
> >>
> >> So my questions are
> >> 1. If there is any reason why the method have been defined to return
> >> 'TRUE' always?
> >> 2. Do we expect any side effects and/or malfunctioning if we change it
> to
> >> returning 'FALSE'?
> >> 3. For a resolution without breaking possible flows, can we add the
> value
> >> of 'executeInSequence' to the global setting if #2 answers YES?
> >>
> >>
> >> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen
> >> wrote:
> >>>
> >>> I thought executeInSequence of 'true' made it go serially, or
> >>> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
> >>> August of 2010:
> >>>
> >>> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends
> Command
> >>> {
> >>> 2010-08-11 09:13:29 -0700 20) String vmName;
> >>> 2010-08-11 09:13:29 -0700 21) String destIp;
> >>> 2011-08-10 10:26:04 -0700 22) String hostGuid;
> >>> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
> >>> 2010-08-11 09:13:29 -0700 24)
> >>> 2010-08-11 09:13:29 -0700 25)
> >>> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
> >>> 2010-08-11 09:13:29 -0700 27) }
> >>> 2012-12-03 22:06:41 -0800 28)
> >>> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
> >>> String destIp, boolean isWindows)
> >>> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
> >>> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
> >>> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
> >>> 2010-08-11 09:13:29 -0700 33) }
> >>> 2012-12-03 22:06:41 -0800 34)
> >>> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
> >>> 2010-08-11 09:13:29 -0700 36) return isWindows;
> >>> 2010-08-11 09:13:29 -0700 37) }
> >>> 2012-12-03 22:06:41 -0800 38)
> >>> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
> >>> 2010-08-11 09:13:29 -0700 40) return destIp;
> >>> 2010-08-11 09:13:29 -0700 41) }
> >>> 2012-12-03 22:06:41 -0800 42)
> >>> 2010-08-11 09:13:29 -0700 43) public String getVmName() {
> >>> 2010-08-11 09:13:29 -0700 44) return vmName;
> >>> 2010-08-11 09:13:29 -0700 45) }
> >>> 2012-12-03 22:06:41 -0800 46)
> >>> 2011-08-10 10:26:04 -0700 47) public void setHostGuid(String guid)
> >>> {
> >>> 2011-08-10 10:26:04 -0700 48) this.hostGuid = guid;
> >>> 2011-08-10 10:26:04 -0700 49) }
> >>> 2012-12-03 22:06:41 -0800 50)
> >>> 2011-08-10 10:26:04 -0700 51) public String getHostGuid() {
> >>> 2011-08-10 10:26:04 -0700 52) return this.hostGuid;
> >>> 2011-08-10 10:26:04 -0700 53) }
> >>> 2010-08-11 09:13:29 -0700 54)
> >>> 2010-08-11 09:13:29 -0700 55) @Override
> >>> 2010-08-11 09:13:29 -0700 56) public boolean executeInSequence() {
> >>> 2010-08-11 09:13:29 -0700 57) return true;
> >>> 2010-08-11 09:13:29 -0700 58) }
> >>> 2010-08-11 09:13:29 -0700 59) }
> >>>
> >>> On Tue, Sep 24, 2013 at 9:58 AM, Chip Childers
> >>> wrote:
> >>> > Hey Kelven - This topic was discussed briefly in the past [1]. Are
> you
> >>> > able to provide any thoughts on Alex's ideas below?
> >>> >
> >>> > -chip
> >>> >
> >>> >
> >>> > [1] http://markmail.org/message/fznrszaswruvlmuy
>
Re: A question on vm migrations when hosts are set into a maintenance mode.
Hi Koushik,
Thanks for your reply, but unfortunately, this setting does NOT cover
'MigrateCommand'.
As you specified, it seems to be effective only in Start/Stop/CopyCommand.
So can we include 'MigrateCommand' in that setting?
Thanks
Alex Ough
On Wed, Oct 2, 2013 at 11:44 PM, Koushik Das wrote:
> It is already a global setting in 4.2. The name of the setting is
> "execute.in.sequence.hypervisor.commands".
>
> Check out Config.java
>
> ExecuteInSequence("Advanced", ManagementServer.class, Boolean.class,
> "execute.in.sequence.hypervisor.commands", "true", "If set to true,
> StartCommand, StopCommand, CopyCommand will be synchronized on the agent
> side." +
> " If set to false, these commands become asynchronous. Default value
> is true.", null),
>
>
> -Koushik
>
> On 02-Oct-2013, at 10:43 PM, Alex Ough alex.o...@sungard.com>> wrote:
>
> Thanks for the reply, Marcus.
>
> What about the option #3, which is to make it as a global setting?
> I think it can prevent side effects if exist.
>
> Alex Ough
>
>
> On Wed, Oct 2, 2013 at 11:37 AM, Marcus Sorensen <mailto:shadow...@gmail.com>>wrote:
>
> Not sure. I don't know the history well enough to know if there were
> issues in the past, it might be that some hypervisors were fine and
> others weren't.
>
> On Wed, Oct 2, 2013 at 9:21 AM, Alex Ough alex.o...@sungard.com>> wrote:
> Marcus/Kelven,
>
> Any thoughts on my suggestions?
>
> Thanks
> Alex Ough
>
>
> On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough alex.o...@sungard.com>>
> wrote:
>
> Oh, sorry for the confusion. I must have reversed the flags.
> As Kelven pointed, it is set as 'TRUE', which makes the process as
> sequential.
>
> So my questions are
> 1. If there is any reason why the method have been defined to return
> 'TRUE' always?
> 2. Do we expect any side effects and/or malfunctioning if we change it
> to
> returning 'FALSE'?
> 3. For a resolution without breaking possible flows, can we add the
> value
> of 'executeInSequence' to the global setting if #2 answers YES?
>
>
> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen <mailto:shadow...@gmail.com>>
> wrote:
>
> I thought executeInSequence of 'true' made it go serially, or
> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
> August of 2010:
>
> 2010-08-11 09:13:29 -0700 19) public class MigrateCommand extends
> Command
> {
> 2010-08-11 09:13:29 -0700 20) String vmName;
> 2010-08-11 09:13:29 -0700 21) String destIp;
> 2011-08-10 10:26:04 -0700 22) String hostGuid;
> 2010-08-11 09:13:29 -0700 23) boolean isWindows;
> 2010-08-11 09:13:29 -0700 24)
> 2010-08-11 09:13:29 -0700 25)
> 2010-08-11 09:13:29 -0700 26) protected MigrateCommand() {
> 2010-08-11 09:13:29 -0700 27) }
> 2012-12-03 22:06:41 -0800 28)
> 2010-08-11 09:13:29 -0700 29) public MigrateCommand(String vmName,
> String destIp, boolean isWindows)
> 2010-08-11 09:13:29 -0700 30) this.vmName = vmName;
> 2010-08-11 09:13:29 -0700 31) this.destIp = destIp;
> 2010-08-11 09:13:29 -0700 32) this.isWindows = isWindows;
> 2010-08-11 09:13:29 -0700 33) }
> 2012-12-03 22:06:41 -0800 34)
> 2010-08-11 09:13:29 -0700 35) public boolean isWindows() {
> 2010-08-11 09:13:29 -0700 36) return isWindows;
> 2010-08-11 09:13:29 -0700 37) }
> 2012-12-03 22:06:41 -0800 38)
> 2010-08-11 09:13:29 -0700 39) public String getDestinationIp() {
> 2010-08-11 09:13:29 -0700 40) return destIp;
> 2010-08-11 09:13:29 -0700 41) }
> 2012-12-03 22:06:41 -0800 42)
> 2010-08-11 09:13:29 -0700 43) public String getVmName() {
> 2010-08-11 09:13:29 -0700 44) return vmName;
> 2010-08-11 09:13:29 -0700 45) }
> 2012-12-03 22:06:41 -0800 46)
> 2011-08-10 10:26:04 -0700 47) public void setHostGuid(String guid)
> {
> 2011-08-10 10:26:04 -0700 48) this.hostGuid = guid;
> 2011-08-10 10:26:04 -0700 49) }
> 2012-12-03 22:06:41 -0800 50)
> 2011-08-10 10:26:04 -0700 51) public String getHostGuid() {
> 2011-08-10 10:26:04 -0700 52) return this.hostGuid;
> 2011-08-10 10:26:04 -0700 53) }
> 2010-08-11 09:13:29 -0700 54)
> 2010-08-11 09:13:29 -0700 55) @Override
> 2010-08-11 09:13:29 -0700 56) public boolean executeInSequence() {
> 2010-08-11 09:13:29 -0700 57) return true;
> 2010-08-11 09:13:29 -0700 58) }
> 2010-08-11 09:13:29 -0700 59) }
>
> On Tue, Sep 24, 2013 at 9:58 AM, Chip Childers
> mailto:chip.child...@sungard.com>> wrote:
Re: A question on vm migrations when hosts are set into a maintenance mode.
I'm not sure what else commands 'MigrateCommand' actually execute in
addition to 'Start/Stop/CopyCommand', but can we include 'MigrateCommand'
if it consists of only those 3 commands?
Thanks
Alex Ough
On Thu, Oct 3, 2013 at 11:31 AM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:
> On 10/3/13 9:25 AM, "Koushik Das" wrote:
>
> >Alena,
> >I see that you had added this config setting. Any specific reason that
> >this only for Start/Stop/Copy and not for all the commands as Alex
> >mentioned? The name of the setting looks generic.
>
> Because these are the commands for which hypervisors support parallel
> execution for sure (tested by QA on Xen/KVM/VmWare).
>
> As all our commands are generic, and not hypervisor specific, you have to
> be very careful if decide to enable parallel execution for MigrateCommand
> and test it on all the hypervisors CS supports.
>
> -Alena.
>
> >
> >-Koushik
> >
> >On 03-Oct-2013, at 8:10 PM, Alex Ough wrote:
> >
> >> Hi Koushik,
> >>
> >> Thanks for your reply, but unfortunately, this setting does NOT cover
> >> 'MigrateCommand'.
> >> As you specified, it seems to be effective only in
> >>Start/Stop/CopyCommand.
> >>
> >> So can we include 'MigrateCommand' in that setting?
> >>
> >> Thanks
> >> Alex Ough
> >>
> >>
> >> On Wed, Oct 2, 2013 at 11:44 PM, Koushik Das
> >>wrote:
> >>
> >>> It is already a global setting in 4.2. The name of the setting is
> >>> "execute.in.sequence.hypervisor.commands".
> >>>
> >>> Check out Config.java
> >>>
> >>>ExecuteInSequence("Advanced", ManagementServer.class, Boolean.class,
> >>> "execute.in.sequence.hypervisor.commands", "true", "If set to true,
> >>> StartCommand, StopCommand, CopyCommand will be synchronized on the
> >>>agent
> >>> side." +
> >>>" If set to false, these commands become asynchronous. Default value
> >>> is true.", null),
> >>>
> >>>
> >>> -Koushik
> >>>
> >>> On 02-Oct-2013, at 10:43 PM, Alex Ough >>> alex.o...@sungard.com>> wrote:
> >>>
> >>> Thanks for the reply, Marcus.
> >>>
> >>> What about the option #3, which is to make it as a global setting?
> >>> I think it can prevent side effects if exist.
> >>>
> >>> Alex Ough
> >>>
> >>>
> >>> On Wed, Oct 2, 2013 at 11:37 AM, Marcus Sorensen >>> <mailto:shadow...@gmail.com>>wrote:
> >>>
> >>> Not sure. I don't know the history well enough to know if there were
> >>> issues in the past, it might be that some hypervisors were fine and
> >>> others weren't.
> >>>
> >>> On Wed, Oct 2, 2013 at 9:21 AM, Alex Ough
> >>> >>> alex.o...@sungard.com>> wrote:
> >>> Marcus/Kelven,
> >>>
> >>> Any thoughts on my suggestions?
> >>>
> >>> Thanks
> >>> Alex Ough
> >>>
> >>>
> >>> On Tue, Sep 24, 2013 at 12:09 PM, Alex Ough
> >>> >>> alex.o...@sungard.com>>
> >>> wrote:
> >>>
> >>> Oh, sorry for the confusion. I must have reversed the flags.
> >>> As Kelven pointed, it is set as 'TRUE', which makes the process as
> >>> sequential.
> >>>
> >>> So my questions are
> >>> 1. If there is any reason why the method have been defined to return
> >>> 'TRUE' always?
> >>> 2. Do we expect any side effects and/or malfunctioning if we change it
> >>> to
> >>> returning 'FALSE'?
> >>> 3. For a resolution without breaking possible flows, can we add the
> >>> value
> >>> of 'executeInSequence' to the global setting if #2 answers YES?
> >>>
> >>>
> >>> On Tue, Sep 24, 2013 at 11:19 AM, Marcus Sorensen >>> <mailto:shadow...@gmail.com>>
> >>> wrote:
> >>>
> >>> I thought executeInSequence of 'true' made it go serially, or
> >>> sequentially. In my codebase for 4.1,4.2,master it's been 'true' since
> >>> August of 2010:
> >>>
> >>> 2010-08-11 09:13:29 -0700 19)
Resource Configuration Failure in Simulator mode (v4.2)
All, After getting the latest codes of v4.2, I'm trying to configure the resources using 'setup/dev/advanced.cfg', but it seems to be failing with this error. "Could not find the selected project in the reactor: :cloud-marvin" I'm using this command and it was working ok before getting the latest. "mvn -Pdeveloper,marvin.setup -Dmarvin.config=setup/dev/advanced.cfg -pl :cloud-marvin integration-test" I'm wondering if this is happening only to me or something is missing. Your help will be very appreciated. Thanks Alex Ough
Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
All, Currently, under the environment of cloudstack with multiple regions, each region has its own management server running with a separate database. So if we want to support multiple regions and provide one point of entry for a customer, we need to duplicate domain/account/user information of that customer to all of the databases of regions the customer accesses, which will cause data discrepancies when users update those data independently in each management server. So I'd like to provide a way to sync up the data using the messaging system introduced in 4.1.0. Using the events from each management server, updates from each region can be propagated to the rest regions and they can be executed accordingly. I hope you guys have a chance to think about this and give some feedbacks if interested. Thanks in advance. Alex Ough
Re: Resource Configuration Failure in Simulator mode (v4.2)
Thanks Amogh, that solved the problem. Btw, don't we need to update the repository to fix this? Thanks Alex Ough On Thu, Oct 17, 2013 at 10:21 PM, Amogh Vasekar wrote: > Can you check the main pom.xml to ensure "tools" is not commented out as a > module from developer profile? This should be near the end of pom.xml > > Thanks, > Amogh > > On 10/17/13 9:34 AM, "Alex Ough" wrote: > > >All, > > > >After getting the latest codes of v4.2, I'm trying to configure the > >resources using 'setup/dev/advanced.cfg', but it seems to be failing with > >this error. > >"Could not find the selected project in the reactor: :cloud-marvin" > > > >I'm using this command and it was working ok before getting the latest. > >"mvn -Pdeveloper,marvin.setup -Dmarvin.config=setup/dev/advanced.cfg -pl > >:cloud-marvin integration-test" > > > >I'm wondering if this is happening only to me or something is missing. > >Your help will be very appreciated. > >Thanks > >Alex Ough > > >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
I created a jira for this feature. https://issues.apache.org/jira/browse/CLOUDSTACK-4992 But it doesn't allow for me to assign it to myself, so any permission do I need for this? If so, can anyone give me this permission? If there is anything missing, let me know. Thanks Alex Ough On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala wrote: > > -Original Message- > > From: Alex Ough [mailto:alex.o...@sungard.com] > > Sent: Thursday, 17 October 2013 11:25 PM > > To: dev@cloudstack.apache.org; u...@cloudstack.apache.org > > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple > > Regions > > > > All, > > > > Currently, under the environment of cloudstack with multiple regions, > each > > region has its own management server running with a separate database. > So if > > we want to support multiple regions and provide one point of entry for a > > customer, we need to duplicate domain/account/user information of that > > customer to all of the databases of regions the customer accesses, which > will > > cause data discrepancies when users update those data independently in > each > > management server. > > > > So I'd like to provide a way to sync up the data using the messaging > system > > introduced in 4.1.0. Using the events from each management server, > updates > > from each region can be propagated to the rest regions and they can be > > executed accordingly. > > > > I hope you guys have a chance to think about this and give some > feedbacks if > > interested. > > Thanks in advance. > > Alex Ough > > [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up > using messaging system is the right way to go. > > > [1] > http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html > >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
And I'd like to write the design document in the wiki page, but I don't seem to have a permission to create pages. So can anyone give me the permission? My account in the wiki is alex.o...@sungard.com Thanks in advance. Alex Ough On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough wrote: > I created a jira for this feature. > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992 > > But it doesn't allow for me to assign it to myself, so any permission do I > need for this? > If so, can anyone give me this permission? > > If there is anything missing, let me know. > Thanks > Alex Ough > > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala > wrote: > >> > -Original Message- >> > From: Alex Ough [mailto:alex.o...@sungard.com] >> > Sent: Thursday, 17 October 2013 11:25 PM >> > To: dev@cloudstack.apache.org; u...@cloudstack.apache.org >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple >> > Regions >> > >> > All, >> > >> > Currently, under the environment of cloudstack with multiple regions, >> each >> > region has its own management server running with a separate database. >> So if >> > we want to support multiple regions and provide one point of entry for a >> > customer, we need to duplicate domain/account/user information of that >> > customer to all of the databases of regions the customer accesses, >> which will >> > cause data discrepancies when users update those data independently in >> each >> > management server. >> > >> > So I'd like to provide a way to sync up the data using the messaging >> system >> > introduced in 4.1.0. Using the events from each management server, >> updates >> > from each region can be propagated to the rest regions and they can be >> > executed accordingly. >> > >> > I hope you guys have a chance to think about this and give some >> feedbacks if >> > interested. >> > Thanks in advance. >> > Alex Ough >> >> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up >> using messaging system is the right way to go. >> >> >> [1] >> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html >> >> >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
All, While I'm waiting for the permissions, I'd like to bring up a discussion on the overall architecture to support this feature. There can be 2 different approaches as below. 1. master - slave architecture : the manual changes are allowed only in one master management server, and those in other servers are either prohibited or discarded. 2. multiple source architecture : all management servers allow manual changes and any change in any server will be propagated to the rest of servers. I'm not sure if we can impose the restriction of #1. If not, #2 is a little more complicated, but the only issue is to distinguish the events from either manual jobs or triggered automatic processing, which I'm currently working on. Your comments/recommendations will be very appreciated. Thanks Alex Ough On Wed, Oct 30, 2013 at 12:19 PM, Alex Ough wrote: > And I'd like to write the design document in the wiki page, but I don't > seem to have a permission to create pages. > So can anyone give me the permission? > > My account in the wiki is alex.o...@sungard.com > > Thanks in advance. > Alex Ough > > > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough wrote: > >> I created a jira for this feature. >> >> https://issues.apache.org/jira/browse/CLOUDSTACK-4992 >> >> But it doesn't allow for me to assign it to myself, so any permission do >> I need for this? >> If so, can anyone give me this permission? >> >> If there is anything missing, let me know. >> Thanks >> Alex Ough >> >> >> On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala >> wrote: >> >>> > -Original Message- >>> > From: Alex Ough [mailto:alex.o...@sungard.com] >>> > Sent: Thursday, 17 October 2013 11:25 PM >>> > To: dev@cloudstack.apache.org; u...@cloudstack.apache.org >>> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple >>> > Regions >>> > >>> > All, >>> > >>> > Currently, under the environment of cloudstack with multiple regions, >>> each >>> > region has its own management server running with a separate database. >>> So if >>> > we want to support multiple regions and provide one point of entry for >>> a >>> > customer, we need to duplicate domain/account/user information of that >>> > customer to all of the databases of regions the customer accesses, >>> which will >>> > cause data discrepancies when users update those data independently in >>> each >>> > management server. >>> > >>> > So I'd like to provide a way to sync up the data using the messaging >>> system >>> > introduced in 4.1.0. Using the events from each management server, >>> updates >>> > from each region can be propagated to the rest regions and they can be >>> > executed accordingly. >>> > >>> > I hope you guys have a chance to think about this and give some >>> feedbacks if >>> > interested. >>> > Thanks in advance. >>> > Alex Ough >>> >>> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up >>> using messaging system is the right way to go. >>> >>> >>> [1] >>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html >>> >>> >> >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
Thanks Chip, and can you also give a permission in Jira so that I can assign myself in its jira? Alex Ough On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers wrote: > Permission added. > > On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote: > > And I'd like to write the design document in the wiki page, but I don't > > seem to have a permission to create pages. > > So can anyone give me the permission? > > > > My account in the wiki is alex.o...@sungard.com > > > > Thanks in advance. > > Alex Ough > > > > > > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough > wrote: > > > > > I created a jira for this feature. > > > > > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992 > > > > > > But it doesn't allow for me to assign it to myself, so any permission > do I > > > need for this? > > > If so, can anyone give me this permission? > > > > > > If there is anything missing, let me know. > > > Thanks > > > Alex Ough > > > > > > > > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala < > kishan.kav...@citrix.com>wrote: > > > > > >> > -Original Message- > > >> > From: Alex Ough [mailto:alex.o...@sungard.com] > > >> > Sent: Thursday, 17 October 2013 11:25 PM > > >> > To: dev@cloudstack.apache.org; u...@cloudstack.apache.org > > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple > > >> > Regions > > >> > > > >> > All, > > >> > > > >> > Currently, under the environment of cloudstack with multiple > regions, > > >> each > > >> > region has its own management server running with a separate > database. > > >> So if > > >> > we want to support multiple regions and provide one point of entry > for a > > >> > customer, we need to duplicate domain/account/user information of > that > > >> > customer to all of the databases of regions the customer accesses, > > >> which will > > >> > cause data discrepancies when users update those data independently > in > > >> each > > >> > management server. > > >> > > > >> > So I'd like to provide a way to sync up the data using the messaging > > >> system > > >> > introduced in 4.1.0. Using the events from each management server, > > >> updates > > >> > from each region can be propagated to the rest regions and they can > be > > >> > executed accordingly. > > >> > > > >> > I hope you guys have a chance to think about this and give some > > >> feedbacks if > > >> > interested. > > >> > Thanks in advance. > > >> > Alex Ough > > >> > > >> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up > > >> using messaging system is the right way to go. > > >> > > >> > > >> [1] > > >> > http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html > > >> > > >> > > > > >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
Great! Thanks a lot, Daan. On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland wrote: > you are added to jira, Alex > > On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough wrote: > > Thanks Chip, and can you also give a permission in Jira so that I can > > assign myself in its jira? > > > > Alex Ough > > > > > > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers >wrote: > > > >> Permission added. > >> > >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote: > >> > And I'd like to write the design document in the wiki page, but I > don't > >> > seem to have a permission to create pages. > >> > So can anyone give me the permission? > >> > > >> > My account in the wiki is alex.o...@sungard.com > >> > > >> > Thanks in advance. > >> > Alex Ough > >> > > >> > > >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough > >> wrote: > >> > > >> > > I created a jira for this feature. > >> > > > >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992 > >> > > > >> > > But it doesn't allow for me to assign it to myself, so any > permission > >> do I > >> > > need for this? > >> > > If so, can anyone give me this permission? > >> > > > >> > > If there is anything missing, let me know. > >> > > Thanks > >> > > Alex Ough > >> > > > >> > > > >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala < > >> kishan.kav...@citrix.com>wrote: > >> > > > >> > >> > -Original Message- > >> > >> > From: Alex Ough [mailto:alex.o...@sungard.com] > >> > >> > Sent: Thursday, 17 October 2013 11:25 PM > >> > >> > To: dev@cloudstack.apache.org; u...@cloudstack.apache.org > >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among > Multiple > >> > >> > Regions > >> > >> > > >> > >> > All, > >> > >> > > >> > >> > Currently, under the environment of cloudstack with multiple > >> regions, > >> > >> each > >> > >> > region has its own management server running with a separate > >> database. > >> > >> So if > >> > >> > we want to support multiple regions and provide one point of > entry > >> for a > >> > >> > customer, we need to duplicate domain/account/user information of > >> that > >> > >> > customer to all of the databases of regions the customer > accesses, > >> > >> which will > >> > >> > cause data discrepancies when users update those data > independently > >> in > >> > >> each > >> > >> > management server. > >> > >> > > >> > >> > So I'd like to provide a way to sync up the data using the > messaging > >> > >> system > >> > >> > introduced in 4.1.0. Using the events from each management > server, > >> > >> updates > >> > >> > from each region can be propagated to the rest regions and they > can > >> be > >> > >> > executed accordingly. > >> > >> > > >> > >> > I hope you guys have a chance to think about this and give some > >> > >> feedbacks if > >> > >> > interested. > >> > >> > Thanks in advance. > >> > >> > Alex Ough > >> > >> > >> > >> [KK] Alex, it was discussed sometime back. Related thread [1]. > Sync up > >> > >> using messaging system is the right way to go. > >> > >> > >> > >> > >> > >> [1] > >> > >> > >> > http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html > >> > >> > >> > >> > >> > > > >> > >> > >
Re: haPlanners error in fresh 4.2 checkout
I still have this issue Any update on this? Thanks Alex Ough On Fri, Nov 1, 2013 at 4:41 AM, sebgoa wrote: > I deleted my local 4.2 branch and did a brand new checkout, same result. > > latest 4.2 does not work for me -same error as below - > > On Oct 31, 2013, at 8:34 PM, Srikanteswararao Talluri < > srikanteswararao.tall...@citrix.com> wrote: > > > I too faced this issue today. Is there any resolution for this? > > > > Thanks, > > ~Talluri > > > > On 31/10/13 2:35 pm, "sebgoa" wrote: > > > >> > >> On Oct 29, 2013, at 7:56 PM, Nitin Mehta > wrote: > >> > >>> Yeah, that should have cleaned the workspace. Did you see what Prachi > >>> recommended below ? > >> > >> Yes and it's there. I do a fresh 4.2 checkout > >> > >> it compiles fine with: > >> > >> mvn -Pdeveloper -Dsimulator -DskipTests clean install > >> > >> But when I run with: > >> > >> mvn -pl client jetty:run > >> > >> I get this: > >> > >> 2013-10-31 10:01:31.403:INFO::Started > SelectChannelConnector@0.0.0.0:8080 > >> [INFO] Started Jetty Server > >> Exception in thread "Timer-2" > >> org.springframework.beans.factory.BeanExpressionException: Expression > >> parsing failed; nested exception is > >> org.springframework.expression.spel.SpelEvaluationException: > EL1008E:(pos > >> 0): Field or property 'haPlanners' cannot be found on object of type > >> 'org.springframework.beans.factory.config.BeanExpressionContext' > >> at > >> > org.springframework.context.expression.StandardBeanExpressionResolver.eval > >> uate(StandardBeanExpressionResolver.java:142) > >> at > >> > org.springframework.beans.factory.support.AbstractBeanFactory.evaluateBean > >> DefinitionString(AbstractBeanFactory.java:1299) > >> at > >> > org.springframework.beans.factory.support.BeanDefinitionValueResolver.eval > >> uate(BeanDefinitionValueResolver.java:210) > >> at > >> > org.springframework.beans.factory.support.BeanDefinitionValueResolver.reso > >> lveValueIfNecessary(BeanDefinitionValueResolver.java:182) > >> at > >> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFacto > >> ry.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1360) > >> at > >> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFacto > >> ry.populateBean(AbstractAutowireCapableBeanFactory.java:1118) > >> at > >> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFacto > >> ry.configureBean(AbstractAutowireCapableBeanFactory.java:314) > >> at > >> > com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(Compone > >> ntContext.java:71) > >> at > >> com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:54) > >> at java.util.TimerThread.mainLoop(Timer.java:555) > >> at java.util.TimerThread.run(Timer.java:505) > >> Caused by: org.springframework.expression.spel.SpelEvaluationException: > >> EL1008E:(pos 0): Field or property 'haPlanners' cannot be found on > object > >> of type 'org.springframework.beans.factory.config.BeanExpressionContext' > >> at > >> > org.springframework.expression.spel.ast.PropertyOrFieldReference.readPrope > >> rty(PropertyOrFieldReference.java:208) > >> at > >> > org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueI > >> nternal(PropertyOrFieldReference.java:72) > >> at > >> > org.springframework.expression.spel.ast.CompoundExpression.getValueInterna > >> l(CompoundExpression.java:52) > >> at > >> > org.springframework.expression.spel.ast.SpelNodeImpl.getValue(SpelNodeImpl > >> .java:93) > >> at > >> > org.springframework.expression.spel.standard.SpelExpression.getValue(SpelE > >> xpression.java:88) > >> at > >> > org.springframework.context.expression.StandardBeanExpressionResolver.eval > >> uate(StandardBeanExpressionResolver.java:139) > >> ... 10 more > >> ERROR [cloud.api.ApiServlet] (770998563@qtp-534473248-6:) unknown > >> exception writing api response > >> java.lang.NullPointerException > >> at com.cloud.api.ApiServer.loginUser(ApiServer.java:7
Re: haPlanners error in fresh 4.2 checkout
I added the property and the server starts ok after being rebuilt. Thank Prachi for your help. Alex Ough On Mon, Nov 4, 2013 at 4:02 PM, Prachi Damle wrote: > I don't see this on 4.2 anymore, so it's hard to debug. But when I saw > this, the 'haPlanners' property was missing from my setup's > nonOssComponentContext.xml > > Are you guys using Simulator? > > I think there is one version for Simulator too - Do you see this property > defined in your client/tomcatconf/simulatorComponentContext.xml.in? > If it is not there, adding it in this file and recompiling should solve > this. > > Nitin - have you added it to simulatorComponentContext.xml? > > Prachi > > -Original Message- > From: Alex Ough [mailto:alex.o...@sungard.com] > Sent: Monday, November 04, 2013 11:29 AM > To: dev@cloudstack.apache.org > Subject: Re: haPlanners error in fresh 4.2 checkout > > I still have this issue > Any update on this? > > Thanks > Alex Ough > > > On Fri, Nov 1, 2013 at 4:41 AM, sebgoa wrote: > > > I deleted my local 4.2 branch and did a brand new checkout, same result. > > > > latest 4.2 does not work for me -same error as below - > > > > On Oct 31, 2013, at 8:34 PM, Srikanteswararao Talluri < > > srikanteswararao.tall...@citrix.com> wrote: > > > > > I too faced this issue today. Is there any resolution for this? > > > > > > Thanks, > > > ~Talluri > > > > > > On 31/10/13 2:35 pm, "sebgoa" wrote: > > > > > >> > > >> On Oct 29, 2013, at 7:56 PM, Nitin Mehta > > wrote: > > >> > > >>> Yeah, that should have cleaned the workspace. Did you see what > > >>> Prachi recommended below ? > > >> > > >> Yes and it's there. I do a fresh 4.2 checkout > > >> > > >> it compiles fine with: > > >> > > >> mvn -Pdeveloper -Dsimulator -DskipTests clean install > > >> > > >> But when I run with: > > >> > > >> mvn -pl client jetty:run > > >> > > >> I get this: > > >> > > >> 2013-10-31 10:01:31.403:INFO::Started > > SelectChannelConnector@0.0.0.0:8080 > > >> [INFO] Started Jetty Server > > >> Exception in thread "Timer-2" > > >> org.springframework.beans.factory.BeanExpressionException: > > >> Expression parsing failed; nested exception is > > >> org.springframework.expression.spel.SpelEvaluationException: > > EL1008E:(pos > > >> 0): Field or property 'haPlanners' cannot be found on object of > > >> type 'org.springframework.beans.factory.config.BeanExpressionContext' > > >> at > > >> > > org.springframework.context.expression.StandardBeanExpressionResolver. > > eval > > >> uate(StandardBeanExpressionResolver.java:142) > > >> at > > >> > > org.springframework.beans.factory.support.AbstractBeanFactory.evaluate > > Bean > > >> DefinitionString(AbstractBeanFactory.java:1299) > > >> at > > >> > > org.springframework.beans.factory.support.BeanDefinitionValueResolver. > > eval > > >> uate(BeanDefinitionValueResolver.java:210) > > >> at > > >> > > org.springframework.beans.factory.support.BeanDefinitionValueResolver. > > reso > > >> lveValueIfNecessary(BeanDefinitionValueResolver.java:182) > > >> at > > >> > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF > > acto > > >> ry.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1360) > > >> at > > >> > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF > > acto > > >> ry.populateBean(AbstractAutowireCapableBeanFactory.java:1118) > > >> at > > >> > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF > > acto > > >> ry.configureBean(AbstractAutowireCapableBeanFactory.java:314) > > >> at > > >> > > com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(Com > > pone > > >> ntContext.java:71) > > >> at > > >> > com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:54) > > >> at java.util.TimerThread.mainLoop(Timer.java:555) > > >> at java.util.TimerThread.run(Timer.java:505) > > >> Caused by
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
All, Among the 2 approaches, I uploaded the implemented codes of the first approach, master-slave architecture, here. https://github.com/alexoughsg/albatross And here is the design doc in the wiki. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions Please review them and let me know what you think if you're interested! Thanks Alex Ough On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough wrote: > Great! Thanks a lot, Daan. > > > On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland wrote: > >> you are added to jira, Alex >> >> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough wrote: >> > Thanks Chip, and can you also give a permission in Jira so that I can >> > assign myself in its jira? >> > >> > Alex Ough >> > >> > >> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers > >wrote: >> > >> >> Permission added. >> >> >> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote: >> >> > And I'd like to write the design document in the wiki page, but I >> don't >> >> > seem to have a permission to create pages. >> >> > So can anyone give me the permission? >> >> > >> >> > My account in the wiki is alex.o...@sungard.com >> >> > >> >> > Thanks in advance. >> >> > Alex Ough >> >> > >> >> > >> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough >> >> wrote: >> >> > >> >> > > I created a jira for this feature. >> >> > > >> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992 >> >> > > >> >> > > But it doesn't allow for me to assign it to myself, so any >> permission >> >> do I >> >> > > need for this? >> >> > > If so, can anyone give me this permission? >> >> > > >> >> > > If there is anything missing, let me know. >> >> > > Thanks >> >> > > Alex Ough >> >> > > >> >> > > >> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala < >> >> kishan.kav...@citrix.com>wrote: >> >> > > >> >> > >> > -Original Message- >> >> > >> > From: Alex Ough [mailto:alex.o...@sungard.com] >> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM >> >> > >> > To: dev@cloudstack.apache.org; u...@cloudstack.apache.org >> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among >> Multiple >> >> > >> > Regions >> >> > >> > >> >> > >> > All, >> >> > >> > >> >> > >> > Currently, under the environment of cloudstack with multiple >> >> regions, >> >> > >> each >> >> > >> > region has its own management server running with a separate >> >> database. >> >> > >> So if >> >> > >> > we want to support multiple regions and provide one point of >> entry >> >> for a >> >> > >> > customer, we need to duplicate domain/account/user information >> of >> >> that >> >> > >> > customer to all of the databases of regions the customer >> accesses, >> >> > >> which will >> >> > >> > cause data discrepancies when users update those data >> independently >> >> in >> >> > >> each >> >> > >> > management server. >> >> > >> > >> >> > >> > So I'd like to provide a way to sync up the data using the >> messaging >> >> > >> system >> >> > >> > introduced in 4.1.0. Using the events from each management >> server, >> >> > >> updates >> >> > >> > from each region can be propagated to the rest regions and they >> can >> >> be >> >> > >> > executed accordingly. >> >> > >> > >> >> > >> > I hope you guys have a chance to think about this and give some >> >> > >> feedbacks if >> >> > >> > interested. >> >> > >> > Thanks in advance. >> >> > >> > Alex Ough >> >> > >> >> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1]. >> Sync up >> >> > >> using messaging system is the right way to go. >> >> > >> >> >> > >> >> >> > >> [1] >> >> > >> >> >> >> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html >> >> > >> >> >> > >> >> >> > > >> >> >> >> >> >> >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
OK. 1) Do you mean the plug-in? If so, let me find out how to develop a plug-in and work on this. 2) Sure, let me add more information in the document. Thanks for your suggestions. Alex Ough On Tue, Nov 5, 2013 at 11:41 AM, Chip Childers wrote: > Alex, > > I've moved your page to the "Designs not committed to a release" > parent (instead of the 4.3 designs page), to align with both the Jira > record *and* the fact that feature freeze is about to happen for 4.3. > > As for the proposal itself, I have a couple of suggestions: > > 1) I'd like to see the implementation be part of the ACS runtime. > Having a separate python app for this sync feature seems like an admin > burden. > > 2) As far as the design document itself, I think that we need to see > more details on the proposed approach to sync, failure condition > handling, etc... > > -chip > > > On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough wrote: > > All, > > > > Among the 2 approaches, I uploaded the implemented codes of the first > > approach, master-slave architecture, here. > > https://github.com/alexoughsg/albatross > > > > And here is the design doc in the wiki. > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions > > > > Please review them and let me know what you think if you're interested! > > Thanks > > Alex Ough > > > > > > > > On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough > wrote: > > > >> Great! Thanks a lot, Daan. > >> > >> > >> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland >wrote: > >> > >>> you are added to jira, Alex > >>> > >>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough > wrote: > >>> > Thanks Chip, and can you also give a permission in Jira so that I can > >>> > assign myself in its jira? > >>> > > >>> > Alex Ough > >>> > > >>> > > >>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers < > chipchild...@apache.org > >>> >wrote: > >>> > > >>> >> Permission added. > >>> >> > >>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote: > >>> >> > And I'd like to write the design document in the wiki page, but I > >>> don't > >>> >> > seem to have a permission to create pages. > >>> >> > So can anyone give me the permission? > >>> >> > > >>> >> > My account in the wiki is alex.o...@sungard.com > >>> >> > > >>> >> > Thanks in advance. > >>> >> > Alex Ough > >>> >> > > >>> >> > > >>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough > > >>> >> wrote: > >>> >> > > >>> >> > > I created a jira for this feature. > >>> >> > > > >>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992 > >>> >> > > > >>> >> > > But it doesn't allow for me to assign it to myself, so any > >>> permission > >>> >> do I > >>> >> > > need for this? > >>> >> > > If so, can anyone give me this permission? > >>> >> > > > >>> >> > > If there is anything missing, let me know. > >>> >> > > Thanks > >>> >> > > Alex Ough > >>> >> > > > >>> >> > > > >>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala < > >>> >> kishan.kav...@citrix.com>wrote: > >>> >> > > > >>> >> > >> > -Original Message- > >>> >> > >> > From: Alex Ough [mailto:alex.o...@sungard.com] > >>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM > >>> >> > >> > To: dev@cloudstack.apache.org; u...@cloudstack.apache.org > >>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among > >>> Multiple > >>> >> > >> > Regions > >>> >> > >> > > >>> >> > >> > All, > >>> >> > >> > > >>> >> > >> > Currently, under the environment of cloudstack with mult
What is the cloudstack branch that has same codes with CloudPlatform-4.2.0-2-rhel6.4?
Is it master or any other branch? Thanks Alex Ough
Re: What is the cloudstack branch that has same codes with CloudPlatform-4.2.0-2-rhel6.4?
OK, thanks! On Wed, Nov 6, 2013 at 11:25 AM, Chip Childers wrote: > 4.2 but specifically the 4.2.0 tag > > > On Nov 6, 2013, at 9:18 AM, Alex Ough wrote: > > > > Is it master or any other branch? > > > > Thanks > > Alex Ough > >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
I'm having a difficulty finding documents about how to develop a plug-in. Anyone to help me find one? Thanks in advance. Alex Ough On Tue, Nov 5, 2013 at 2:18 PM, Alex Ough wrote: > OK. > > 1) Do you mean the plug-in? If so, let me find out how to develop a > plug-in and work on this. > 2) Sure, let me add more information in the document. > > Thanks for your suggestions. > Alex Ough > > > On Tue, Nov 5, 2013 at 11:41 AM, Chip Childers wrote: > >> Alex, >> >> I've moved your page to the "Designs not committed to a release" >> parent (instead of the 4.3 designs page), to align with both the Jira >> record *and* the fact that feature freeze is about to happen for 4.3. >> >> As for the proposal itself, I have a couple of suggestions: >> >> 1) I'd like to see the implementation be part of the ACS runtime. >> Having a separate python app for this sync feature seems like an admin >> burden. >> >> 2) As far as the design document itself, I think that we need to see >> more details on the proposed approach to sync, failure condition >> handling, etc... >> >> -chip >> >> >> On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough wrote: >> > All, >> > >> > Among the 2 approaches, I uploaded the implemented codes of the first >> > approach, master-slave architecture, here. >> > https://github.com/alexoughsg/albatross >> > >> > And here is the design doc in the wiki. >> > >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions >> > >> > Please review them and let me know what you think if you're interested! >> > Thanks >> > Alex Ough >> > >> > >> > >> > On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough >> wrote: >> > >> >> Great! Thanks a lot, Daan. >> >> >> >> >> >> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland < >> daan.hoogl...@gmail.com>wrote: >> >> >> >>> you are added to jira, Alex >> >>> >> >>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough >> wrote: >> >>> > Thanks Chip, and can you also give a permission in Jira so that I >> can >> >>> > assign myself in its jira? >> >>> > >> >>> > Alex Ough >> >>> > >> >>> > >> >>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers < >> chipchild...@apache.org >> >>> >wrote: >> >>> > >> >>> >> Permission added. >> >>> >> >> >>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote: >> >>> >> > And I'd like to write the design document in the wiki page, but I >> >>> don't >> >>> >> > seem to have a permission to create pages. >> >>> >> > So can anyone give me the permission? >> >>> >> > >> >>> >> > My account in the wiki is alex.o...@sungard.com >> >>> >> > >> >>> >> > Thanks in advance. >> >>> >> > Alex Ough >> >>> >> > >> >>> >> > >> >>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough < >> alex.o...@sungard.com> >> >>> >> wrote: >> >>> >> > >> >>> >> > > I created a jira for this feature. >> >>> >> > > >> >>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992 >> >>> >> > > >> >>> >> > > But it doesn't allow for me to assign it to myself, so any >> >>> permission >> >>> >> do I >> >>> >> > > need for this? >> >>> >> > > If so, can anyone give me this permission? >> >>> >> > > >> >>> >> > > If there is anything missing, let me know. >> >>> >> > > Thanks >> >>> >> > > Alex Ough >> >>> >> > > >> >>> >> > > >> >>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala < >> >>> >> kishan.kav...@citrix.com>wrote: >> >>> >> > > >> >>> >> > >> > -Original Message- >> >>
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
I don't mind implementing it in the core server unless there is anyone who thinks differently. Anyone with another thought? Thank Daan for your suggestion. Alex Ough On Thu, Nov 7, 2013 at 11:23 AM, Daan Hoogland wrote: > Alex, > > Why would you want to make this a plugin? It sounds like a function of > the core server. Don't you agree? > > regards, > Daan > > On Wed, Nov 6, 2013 at 10:59 PM, Alex Ough wrote: > > I'm having a difficulty finding documents about how to develop a plug-in. > > Anyone to help me find one? > > > > Thanks in advance. > > Alex Ough > > > > > > On Tue, Nov 5, 2013 at 2:18 PM, Alex Ough wrote: > > > >> OK. > >> > >> 1) Do you mean the plug-in? If so, let me find out how to develop a > >> plug-in and work on this. > >> 2) Sure, let me add more information in the document. > >> > >> Thanks for your suggestions. > >> Alex Ough > >> > >> > >> On Tue, Nov 5, 2013 at 11:41 AM, Chip Childers >wrote: > >> > >>> Alex, > >>> > >>> I've moved your page to the "Designs not committed to a release" > >>> parent (instead of the 4.3 designs page), to align with both the Jira > >>> record *and* the fact that feature freeze is about to happen for 4.3. > >>> > >>> As for the proposal itself, I have a couple of suggestions: > >>> > >>> 1) I'd like to see the implementation be part of the ACS runtime. > >>> Having a separate python app for this sync feature seems like an admin > >>> burden. > >>> > >>> 2) As far as the design document itself, I think that we need to see > >>> more details on the proposed approach to sync, failure condition > >>> handling, etc... > >>> > >>> -chip > >>> > >>> > >>> On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough > wrote: > >>> > All, > >>> > > >>> > Among the 2 approaches, I uploaded the implemented codes of the first > >>> > approach, master-slave architecture, here. > >>> > https://github.com/alexoughsg/albatross > >>> > > >>> > And here is the design doc in the wiki. > >>> > > >>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions > >>> > > >>> > Please review them and let me know what you think if you're > interested! > >>> > Thanks > >>> > Alex Ough > >>> > > >>> > > >>> > > >>> > On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough > >>> wrote: > >>> > > >>> >> Great! Thanks a lot, Daan. > >>> >> > >>> >> > >>> >> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland < > >>> daan.hoogl...@gmail.com>wrote: > >>> >> > >>> >>> you are added to jira, Alex > >>> >>> > >>> >>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough > >>> wrote: > >>> >>> > Thanks Chip, and can you also give a permission in Jira so that I > >>> can > >>> >>> > assign myself in its jira? > >>> >>> > > >>> >>> > Alex Ough > >>> >>> > > >>> >>> > > >>> >>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers < > >>> chipchild...@apache.org > >>> >>> >wrote: > >>> >>> > > >>> >>> >> Permission added. > >>> >>> >> > >>> >>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote: > >>> >>> >> > And I'd like to write the design document in the wiki page, > but I > >>> >>> don't > >>> >>> >> > seem to have a permission to create pages. > >>> >>> >> > So can anyone give me the permission? > >>> >>> >> > > >>> >>> >> > My account in the wiki is alex.o...@sungard.com > >>> >>> >> > > >>> >>> >> > Thanks in advance. > >>> >>> >> > Alex Ough > >>> >>> >> > > >>> >>> >> > > >>> >>
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
All, I'm new to the cloudstack project, so there can be something I missed for consideration, but the current region support seems to be inadequate (I hope this does not offend anyone...) because even if a customer can have multiple regions, the resources in each region are totally separated like resources of different customers. So my personal opinion is that it looks like this functionality is must-have not good-to-have to support the actual multiple regions. Once again, I can be wrong, and please correct me if so. Anyway, I've added more design specs in wiki including the consideration of the support as a plug-in in resolving the #1 restriction. Please review them and let me know what you think. Thanks Alex Ough On Sat, Nov 9, 2013 at 8:20 AM, Daan Hoogland wrote: > H Guys, > > Can you shoot at my claims below, please? > > syncing being optional does not conflict with the code being in the > core server. It seems that making a plugin for this is misuse of the > plugin mechanism. To me it is more of an option to switch on or of > with a global setting, having some extra configuration. > > Also, cloudstack being a slave to the real user db is a separate issue > from cloudstack syncing its copy. > > As for answerring the cap; this is a security answer as well as an > operational answer. > > thanks, > Daan > > On Sat, Nov 9, 2013 at 1:53 AM, Chip Childers > wrote: > > We are already (generally) AP for most infra changes really. I'd use > that model. Eventual consistency is better in this scenario. > > > >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal < > chiradeep.vit...@citrix.com> wrote: > >> > >> I'd also like to highlight that it isn't a trivial problem. > >> Let's say there's 3 regions: this means there are 3 copies of the user > >> database that are geographically separated by network links that fail > >> quite often (orders of magnitude more than intra-DC networks). > >> > >> Here we run into the consequences of the CAP theorem [1]. > >> We can either have a CP or AP system: either approach makes some > tradeoffs: > >> 1. If we run a AP system, then the challenge is to resolve conflicting > >> updates > >> 2. If we run a CP system, then the challenge is to detect partitions > >> reliably and disallow updates during partitions. > >> > >> [1] http://en.wikipedia.org/wiki/CAP_theorem > >> > >>> On 11/7/13 11:58 AM, "Chip Childers" wrote: > >>> > >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal > >>> wrote: > >>>> It may be an admin burden, but it has to be optional. There are other > >>>> ways > >>>> to achieve global sync (e.g., LDAP/AD/Oauth). > >>>> A lot of service providers who run cloudstack have their own user > >>>> database > >>>> / portal. In their implementations the CloudStack database is not the > >>>> master source of user records, but a slave. > >>> > >>> +1 to it being optional. > >> > >
Re: A question on vm migrations when hosts are set into a maintenance mode.
Thanks a lot for your confirmation, Marcus. I'll create a review request unless anyone has an objection. Thanks Alex Ough On Tue, Nov 12, 2013 at 3:37 PM, Marcus Sorensen wrote: > I have done parallel KVM migrations without issue, it's "supposed to > work". Really I think it's in the same boat as parallel start/stop. It > should work, but the config option is there just in case. I think we > should add it. > > On Thu, Oct 3, 2013 at 11:41 AM, Chip Childers > wrote: > > On Thu, Oct 03, 2013 at 11:44:46AM -0500, Alex Ough wrote: > >> I'm not sure what else commands 'MigrateCommand' actually execute in > >> addition to 'Start/Stop/CopyCommand', but can we include > 'MigrateCommand' > >> if it consists of only those 3 commands? > >> > >> Thanks > >> Alex Ough > > > > In the case of VMware, the migrate command is executed via the > > MigrateVMTask that's part of the VMware SDK (see > > vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java). > > > > For VMware, I know that vCenter will queue and process concurrent > > requests for migrations. Specifically, it will throttle the migrations > > happening, based on it's internal concurrency constraints, but the task > > queue will still accept more connections. Obviously the risk are the > > VMware layer tasks timing out if it takes too long for the task queue to > > complete. > > > > As for XenServer, it's happening in what appears to be a similar way > > (although the source host is the target for the migration API call). > > > > Check > > > plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java. > > > > I'm not familiar enough with XenServer's concurrency model for > > migrations. Any experts know the answer to if it can handle concurrency > > in a stable way? > > > > With KVM, it's obviously executing via the agent. Similarly to > > XenServer, I'm not familiar enough to know about concurrent operations. > > > > So do the HV experts on the list have any opinions about XenServer and > > KVM migration concurrency? > > > > -chip > > > > > >
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
Good point, Chiradeep, I'm not sure if you reviewed my design doc in the wiki, but my design is to just skip any actions for target resources that already took place by any means. But the issue is when conflict actions in the same resources (like create & delete the same users) are enqueued in reversed orders, which is hopefully rare. And to support consistency in the AP system, I'd like to provide a full sync up, which will sync up all data in all region servers by selecting a region as a master and force its data to other regions. Let me know what you think. Thanks Alex Ough On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal < chiradeep.vit...@citrix.com> wrote: > Missed this one. In a single region, the CloudStack DB is the master for > most operations. If the infra is not in the state the DB says it should > be, generally the approach is to whack it and make it conform. For some > exceptions (live migration/related use cases are exceptions) the DB is the > slave -- the point is that the inconsistency that inevitably arise in an > AP system need a conflict resolution system. In a single region, the > default is to assume that the MySQL DB is correct and handle other cases > carefully. > > In a multi-region case, there is no master: disable an account in one > region, and it may not propagate to the other regions for many hours/days. > You could add a user in one region and then add the same user in a > different region and conflict before the sync happens. > > This is of course not a problem unique to CloudStack -- people pay mucho > dinero for Global AD/LDAP sync. I don't think this is a problem for > CloudStack core to solve: I support the event-based mechanism for those > who want this facility. > > Distributed systems are hard and research continues to try and make > building these systems easier, but there are very few solutions for global > state synchronization (Google Spanner comes to mind) > > -- > Chiradeep > > > On 11/8/13 4:53 PM, "Chip Childers" wrote: > > >We are already (generally) AP for most infra changes really. I'd use that > >model. Eventual consistency is better in this scenario. > > > >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal > >> wrote: > >> > >> I'd also like to highlight that it isn't a trivial problem. > >> Let's say there's 3 regions: this means there are 3 copies of the user > >> database that are geographically separated by network links that fail > >> quite often (orders of magnitude more than intra-DC networks). > >> > >> Here we run into the consequences of the CAP theorem [1]. > >> We can either have a CP or AP system: either approach makes some > >>tradeoffs: > >> 1. If we run a AP system, then the challenge is to resolve conflicting > >> updates > >> 2. If we run a CP system, then the challenge is to detect partitions > >> reliably and disallow updates during partitions. > >> > >> [1] http://en.wikipedia.org/wiki/CAP_theorem > >> > >>> On 11/7/13 11:58 AM, "Chip Childers" wrote: > >>> > >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal > >>> wrote: > >>>> It may be an admin burden, but it has to be optional. There are other > >>>> ways > >>>> to achieve global sync (e.g., LDAP/AD/Oauth). > >>>> A lot of service providers who run cloudstack have their own user > >>>> database > >>>> / portal. In their implementations the CloudStack database is not the > >>>> master source of user records, but a slave. > >>> > >>> +1 to it being optional. > >> > > >
Re: A question on vm migrations when hosts are set into a maintenance mode.
All,
While merging my changes to 4.3 branch, I found that the option,
'execute.in.sequence.hypervisor.commands' is NOT used in Start/Stop/Copy
commands in 'VirtualMachineManagerImpl.java' any more as below.
*StopCommand stop = new StopCommand(vm, getExecuteInSequence());*
*protected boolean getExecuteInSequence() {*
* return false;*
*}*
As you see in the above, the function, 'getExecuteInSequence', just returns
false instead of getting the value from the global variable.
And one more change is that the file has been moved to
'engine/orchestration/src/com/cloud/vm' from 'server/src/com/cloud/vm'.
Am I missing something related with this or do we stop supporting this
option in 4.3?
I'm a little confused, so please help me resolve this.
Thanks
Alex Ough
On Tue, Nov 12, 2013 at 4:20 PM, Alex Ough wrote:
> Thanks a lot for your confirmation, Marcus.
> I'll create a review request unless anyone has an objection.
>
> Thanks
> Alex Ough
>
>
> On Tue, Nov 12, 2013 at 3:37 PM, Marcus Sorensen wrote:
>
>> I have done parallel KVM migrations without issue, it's "supposed to
>> work". Really I think it's in the same boat as parallel start/stop. It
>> should work, but the config option is there just in case. I think we
>> should add it.
>>
>> On Thu, Oct 3, 2013 at 11:41 AM, Chip Childers
>> wrote:
>> > On Thu, Oct 03, 2013 at 11:44:46AM -0500, Alex Ough wrote:
>> >> I'm not sure what else commands 'MigrateCommand' actually execute in
>> >> addition to 'Start/Stop/CopyCommand', but can we include
>> 'MigrateCommand'
>> >> if it consists of only those 3 commands?
>> >>
>> >> Thanks
>> >> Alex Ough
>> >
>> > In the case of VMware, the migrate command is executed via the
>> > MigrateVMTask that's part of the VMware SDK (see
>> > vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java).
>> >
>> > For VMware, I know that vCenter will queue and process concurrent
>> > requests for migrations. Specifically, it will throttle the migrations
>> > happening, based on it's internal concurrency constraints, but the task
>> > queue will still accept more connections. Obviously the risk are the
>> > VMware layer tasks timing out if it takes too long for the task queue to
>> > complete.
>> >
>> > As for XenServer, it's happening in what appears to be a similar way
>> > (although the source host is the target for the migration API call).
>> >
>> > Check
>> >
>> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java.
>> >
>> > I'm not familiar enough with XenServer's concurrency model for
>> > migrations. Any experts know the answer to if it can handle concurrency
>> > in a stable way?
>> >
>> > With KVM, it's obviously executing via the agent. Similarly to
>> > XenServer, I'm not familiar enough to know about concurrent operations.
>> >
>> > So do the HV experts on the list have any opinions about XenServer and
>> > KVM migration concurrency?
>> >
>> > -chip
>> >
>> >
>>
>>
>
Re: A question on vm migrations when hosts are set into a maintenance mode.
It sounds a little scary...
I looked at the history and found these.
8/9/ : file moved to engine by Alex Huang
9/16 : '_mgmtServer.getExecuteInSequence()' changed to
'getExecuteInSequence()' by Alex Huang
Hi Alex Huang,
I'm not sure if you're aware of this, but can you check this for me?
Thanks
Alex Ough
On Wed, Nov 13, 2013 at 11:18 AM, Marcus Sorensen wrote:
> I'm not sure. I know in the past when I've seen files change locations
> it has also clobbered updates to that file. Someone branched, did the
> reorganization work, and merged, while in-between the original file
> changed.
>
> On Wed, Nov 13, 2013 at 9:21 AM, Alex Ough wrote:
> > All,
> >
> > While merging my changes to 4.3 branch, I found that the option,
> > 'execute.in.sequence.hypervisor.commands' is NOT used in Start/Stop/Copy
> > commands in 'VirtualMachineManagerImpl.java' any more as below.
> >
> >
> > *StopCommand stop = new StopCommand(vm, getExecuteInSequence());*
> >
> > *protected boolean getExecuteInSequence() {*
> > * return false;*
> > *}*
> >
> > As you see in the above, the function, 'getExecuteInSequence', just
> returns
> > false instead of getting the value from the global variable.
> >
> > And one more change is that the file has been moved to
> > 'engine/orchestration/src/com/cloud/vm' from 'server/src/com/cloud/vm'.
> >
> > Am I missing something related with this or do we stop supporting this
> > option in 4.3?
> > I'm a little confused, so please help me resolve this.
> >
> > Thanks
> > Alex Ough
> >
> >
> >
> >
> > On Tue, Nov 12, 2013 at 4:20 PM, Alex Ough
> wrote:
> >
> >> Thanks a lot for your confirmation, Marcus.
> >> I'll create a review request unless anyone has an objection.
> >>
> >> Thanks
> >> Alex Ough
> >>
> >>
> >> On Tue, Nov 12, 2013 at 3:37 PM, Marcus Sorensen >wrote:
> >>
> >>> I have done parallel KVM migrations without issue, it's "supposed to
> >>> work". Really I think it's in the same boat as parallel start/stop. It
> >>> should work, but the config option is there just in case. I think we
> >>> should add it.
> >>>
> >>> On Thu, Oct 3, 2013 at 11:41 AM, Chip Childers
> >>> wrote:
> >>> > On Thu, Oct 03, 2013 at 11:44:46AM -0500, Alex Ough wrote:
> >>> >> I'm not sure what else commands 'MigrateCommand' actually execute in
> >>> >> addition to 'Start/Stop/CopyCommand', but can we include
> >>> 'MigrateCommand'
> >>> >> if it consists of only those 3 commands?
> >>> >>
> >>> >> Thanks
> >>> >> Alex Ough
> >>> >
> >>> > In the case of VMware, the migrate command is executed via the
> >>> > MigrateVMTask that's part of the VMware SDK (see
> >>> >
> vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java).
> >>> >
> >>> > For VMware, I know that vCenter will queue and process concurrent
> >>> > requests for migrations. Specifically, it will throttle the
> migrations
> >>> > happening, based on it's internal concurrency constraints, but the
> task
> >>> > queue will still accept more connections. Obviously the risk are the
> >>> > VMware layer tasks timing out if it takes too long for the task
> queue to
> >>> > complete.
> >>> >
> >>> > As for XenServer, it's happening in what appears to be a similar way
> >>> > (although the source host is the target for the migration API call).
> >>> >
> >>> > Check
> >>> >
> >>>
> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java.
> >>> >
> >>> > I'm not familiar enough with XenServer's concurrency model for
> >>> > migrations. Any experts know the answer to if it can handle
> concurrency
> >>> > in a stable way?
> >>> >
> >>> > With KVM, it's obviously executing via the agent. Similarly to
> >>> > XenServer, I'm not familiar enough to know about concurrent
> operations.
> >>> >
> >>> > So do the HV experts on the list have any opinions about XenServer
> and
> >>> > KVM migration concurrency?
> >>> >
> >>> > -chip
> >>> >
> >>> >
> >>>
> >>>
> >>
>
>
Re: A question on vm migrations when hosts are set into a maintenance mode.
Not sure if Alex Huang checked this, but can anyone help to resolve this?
Thanks
Alex Ough
On Wed, Nov 13, 2013 at 11:39 AM, Alex Ough wrote:
> It sounds a little scary...
>
> I looked at the history and found these.
>
> 8/9/ : file moved to engine by Alex Huang
> 9/16 : '_mgmtServer.getExecuteInSequence()' changed to
> 'getExecuteInSequence()' by Alex Huang
>
>
> Hi Alex Huang,
> I'm not sure if you're aware of this, but can you check this for me?
>
> Thanks
> Alex Ough
>
>
>
> On Wed, Nov 13, 2013 at 11:18 AM, Marcus Sorensen wrote:
>
>> I'm not sure. I know in the past when I've seen files change locations
>> it has also clobbered updates to that file. Someone branched, did the
>> reorganization work, and merged, while in-between the original file
>> changed.
>>
>> On Wed, Nov 13, 2013 at 9:21 AM, Alex Ough wrote:
>> > All,
>> >
>> > While merging my changes to 4.3 branch, I found that the option,
>> > 'execute.in.sequence.hypervisor.commands' is NOT used in Start/Stop/Copy
>> > commands in 'VirtualMachineManagerImpl.java' any more as below.
>> >
>> >
>> > *StopCommand stop = new StopCommand(vm, getExecuteInSequence());*
>> >
>> > *protected boolean getExecuteInSequence() {*
>> > * return false;*
>> > *}*
>> >
>> > As you see in the above, the function, 'getExecuteInSequence', just
>> returns
>> > false instead of getting the value from the global variable.
>> >
>> > And one more change is that the file has been moved to
>> > 'engine/orchestration/src/com/cloud/vm' from 'server/src/com/cloud/vm'.
>> >
>> > Am I missing something related with this or do we stop supporting this
>> > option in 4.3?
>> > I'm a little confused, so please help me resolve this.
>> >
>> > Thanks
>> > Alex Ough
>> >
>> >
>> >
>> >
>> > On Tue, Nov 12, 2013 at 4:20 PM, Alex Ough
>> wrote:
>> >
>> >> Thanks a lot for your confirmation, Marcus.
>> >> I'll create a review request unless anyone has an objection.
>> >>
>> >> Thanks
>> >> Alex Ough
>> >>
>> >>
>> >> On Tue, Nov 12, 2013 at 3:37 PM, Marcus Sorensen > >wrote:
>> >>
>> >>> I have done parallel KVM migrations without issue, it's "supposed to
>> >>> work". Really I think it's in the same boat as parallel start/stop. It
>> >>> should work, but the config option is there just in case. I think we
>> >>> should add it.
>> >>>
>> >>> On Thu, Oct 3, 2013 at 11:41 AM, Chip Childers
>> >>> wrote:
>> >>> > On Thu, Oct 03, 2013 at 11:44:46AM -0500, Alex Ough wrote:
>> >>> >> I'm not sure what else commands 'MigrateCommand' actually execute
>> in
>> >>> >> addition to 'Start/Stop/CopyCommand', but can we include
>> >>> 'MigrateCommand'
>> >>> >> if it consists of only those 3 commands?
>> >>> >>
>> >>> >> Thanks
>> >>> >> Alex Ough
>> >>> >
>> >>> > In the case of VMware, the migrate command is executed via the
>> >>> > MigrateVMTask that's part of the VMware SDK (see
>> >>> >
>> vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java).
>> >>> >
>> >>> > For VMware, I know that vCenter will queue and process concurrent
>> >>> > requests for migrations. Specifically, it will throttle the
>> migrations
>> >>> > happening, based on it's internal concurrency constraints, but the
>> task
>> >>> > queue will still accept more connections. Obviously the risk are
>> the
>> >>> > VMware layer tasks timing out if it takes too long for the task
>> queue to
>> >>> > complete.
>> >>> >
>> >>> > As for XenServer, it's happening in what appears to be a similar way
>> >>> > (although the source host is the target for the migration API call).
>> >>> >
>> >>> > Check
>> >>> >
>> >>>
>> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java.
>> >>> >
>> >>> > I'm not familiar enough with XenServer's concurrency model for
>> >>> > migrations. Any experts know the answer to if it can handle
>> concurrency
>> >>> > in a stable way?
>> >>> >
>> >>> > With KVM, it's obviously executing via the agent. Similarly to
>> >>> > XenServer, I'm not familiar enough to know about concurrent
>> operations.
>> >>> >
>> >>> > So do the HV experts on the list have any opinions about XenServer
>> and
>> >>> > KVM migration concurrency?
>> >>> >
>> >>> > -chip
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>
>>
>>
>
Re: A question on vm migrations when hosts are set into a maintenance mode.
I hate to sending the same emails over and over again, but I really need to
finalize this feature to be included in the next code freeze because this
feature is very critical in our inside project.
Anyone who can help, please?
Thanks
Alex Ough
On Thu, Nov 14, 2013 at 1:27 PM, Alex Ough wrote:
> Not sure if Alex Huang checked this, but can anyone help to resolve this?
>
> Thanks
> Alex Ough
>
>
> On Wed, Nov 13, 2013 at 11:39 AM, Alex Ough wrote:
>
>> It sounds a little scary...
>>
>> I looked at the history and found these.
>>
>> 8/9/ : file moved to engine by Alex Huang
>> 9/16 : '_mgmtServer.getExecuteInSequence()' changed to
>> 'getExecuteInSequence()' by Alex Huang
>>
>>
>> Hi Alex Huang,
>> I'm not sure if you're aware of this, but can you check this for me?
>>
>> Thanks
>> Alex Ough
>>
>>
>>
>> On Wed, Nov 13, 2013 at 11:18 AM, Marcus Sorensen wrote:
>>
>>> I'm not sure. I know in the past when I've seen files change locations
>>> it has also clobbered updates to that file. Someone branched, did the
>>> reorganization work, and merged, while in-between the original file
>>> changed.
>>>
>>> On Wed, Nov 13, 2013 at 9:21 AM, Alex Ough
>>> wrote:
>>> > All,
>>> >
>>> > While merging my changes to 4.3 branch, I found that the option,
>>> > 'execute.in.sequence.hypervisor.commands' is NOT used in
>>> Start/Stop/Copy
>>> > commands in 'VirtualMachineManagerImpl.java' any more as below.
>>> >
>>> >
>>> > *StopCommand stop = new StopCommand(vm, getExecuteInSequence());*
>>> >
>>> > *protected boolean getExecuteInSequence() {*
>>> > * return false;*
>>> > *}*
>>> >
>>> > As you see in the above, the function, 'getExecuteInSequence', just
>>> returns
>>> > false instead of getting the value from the global variable.
>>> >
>>> > And one more change is that the file has been moved to
>>> > 'engine/orchestration/src/com/cloud/vm' from 'server/src/com/cloud/vm'.
>>> >
>>> > Am I missing something related with this or do we stop supporting this
>>> > option in 4.3?
>>> > I'm a little confused, so please help me resolve this.
>>> >
>>> > Thanks
>>> > Alex Ough
>>> >
>>> >
>>> >
>>> >
>>> > On Tue, Nov 12, 2013 at 4:20 PM, Alex Ough
>>> wrote:
>>> >
>>> >> Thanks a lot for your confirmation, Marcus.
>>> >> I'll create a review request unless anyone has an objection.
>>> >>
>>> >> Thanks
>>> >> Alex Ough
>>> >>
>>> >>
>>> >> On Tue, Nov 12, 2013 at 3:37 PM, Marcus Sorensen >> >wrote:
>>> >>
>>> >>> I have done parallel KVM migrations without issue, it's "supposed to
>>> >>> work". Really I think it's in the same boat as parallel start/stop.
>>> It
>>> >>> should work, but the config option is there just in case. I think we
>>> >>> should add it.
>>> >>>
>>> >>> On Thu, Oct 3, 2013 at 11:41 AM, Chip Childers
>>> >>> wrote:
>>> >>> > On Thu, Oct 03, 2013 at 11:44:46AM -0500, Alex Ough wrote:
>>> >>> >> I'm not sure what else commands 'MigrateCommand' actually execute
>>> in
>>> >>> >> addition to 'Start/Stop/CopyCommand', but can we include
>>> >>> 'MigrateCommand'
>>> >>> >> if it consists of only those 3 commands?
>>> >>> >>
>>> >>> >> Thanks
>>> >>> >> Alex Ough
>>> >>> >
>>> >>> > In the case of VMware, the migrate command is executed via the
>>> >>> > MigrateVMTask that's part of the VMware SDK (see
>>> >>> >
>>> vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java).
>>> >>> >
>>> >>> > For VMware, I know that vCenter will queue and process concurrent
>>> >>> > requests for migrations. Specifically, it will throttle the
>>> migrations
>>> >>> > happening, based on it's internal concurrency constraints, but the
>>> task
>>> >>> > queue will still accept more connections. Obviously the risk are
>>> the
>>> >>> > VMware layer tasks timing out if it takes too long for the task
>>> queue to
>>> >>> > complete.
>>> >>> >
>>> >>> > As for XenServer, it's happening in what appears to be a similar
>>> way
>>> >>> > (although the source host is the target for the migration API
>>> call).
>>> >>> >
>>> >>> > Check
>>> >>> >
>>> >>>
>>> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java.
>>> >>> >
>>> >>> > I'm not familiar enough with XenServer's concurrency model for
>>> >>> > migrations. Any experts know the answer to if it can handle
>>> concurrency
>>> >>> > in a stable way?
>>> >>> >
>>> >>> > With KVM, it's obviously executing via the agent. Similarly to
>>> >>> > XenServer, I'm not familiar enough to know about concurrent
>>> operations.
>>> >>> >
>>> >>> > So do the HV experts on the list have any opinions about XenServer
>>> and
>>> >>> > KVM migration concurrency?
>>> >>> >
>>> >>> > -chip
>>> >>> >
>>> >>> >
>>> >>>
>>> >>>
>>> >>
>>>
>>>
>>
>
Re: A question on vm migrations when hosts are set into a maintenance mode.
Thank Parasanna & Sebastien,
I also got his email and sent an email.
Waiting for his reply...
Thanks
Alex Ough
On Sat, Nov 16, 2013 at 3:05 PM, Sebastien Goasguen wrote:
> cc Alex Huang to get his attention:
>
>
> On Nov 15, 2013, at 10:17 PM, Prasanna Santhanam wrote:
>
> > Alex, Could you just do a git blame on the file and copy the emails of
> > people who changed that bit of code? They may be able to help if Cc-ed
> > directly.
> >
> > Thanks,
> >
> > On Fri, Nov 15, 2013 at 01:49:07PM -0600, Alex Ough wrote:
> >> I hate to sending the same emails over and over again, but I really
> need to
> >> finalize this feature to be included in the next code freeze because
> this
> >> feature is very critical in our inside project.
> >>
> >> Anyone who can help, please?
> >> Thanks
> >> Alex Ough
> >>
> >>
> >> On Thu, Nov 14, 2013 at 1:27 PM, Alex Ough
> wrote:
> >>
> >>> Not sure if Alex Huang checked this, but can anyone help to resolve
> this?
> >>>
> >>> Thanks
> >>> Alex Ough
> >>>
> >>>
> >>> On Wed, Nov 13, 2013 at 11:39 AM, Alex Ough
> wrote:
> >>>
> >>>> It sounds a little scary...
> >>>>
> >>>> I looked at the history and found these.
> >>>>
> >>>> 8/9/ : file moved to engine by Alex Huang
> >>>> 9/16 : '_mgmtServer.getExecuteInSequence()' changed to
> >>>> 'getExecuteInSequence()' by Alex Huang
> >>>>
> >>>>
> >>>> Hi Alex Huang,
> >>>> I'm not sure if you're aware of this, but can you check this for me?
> >>>>
> >>>> Thanks
> >>>> Alex Ough
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Nov 13, 2013 at 11:18 AM, Marcus Sorensen <
> shadow...@gmail.com>wrote:
> >>>>
> >>>>> I'm not sure. I know in the past when I've seen files change
> locations
> >>>>> it has also clobbered updates to that file. Someone branched, did the
> >>>>> reorganization work, and merged, while in-between the original file
> >>>>> changed.
> >>>>>
> >>>>> On Wed, Nov 13, 2013 at 9:21 AM, Alex Ough
> >>>>> wrote:
> >>>>>> All,
> >>>>>>
> >>>>>> While merging my changes to 4.3 branch, I found that the option,
> >>>>>> 'execute.in.sequence.hypervisor.commands' is NOT used in
> >>>>> Start/Stop/Copy
> >>>>>> commands in 'VirtualMachineManagerImpl.java' any more as below.
> >>>>>>
> >>>>>>
> >>>>>> *StopCommand stop = new StopCommand(vm, getExecuteInSequence());*
> >>>>>>
> >>>>>> *protected boolean getExecuteInSequence() {*
> >>>>>> * return false;*
> >>>>>> *}*
> >>>>>>
> >>>>>> As you see in the above, the function, 'getExecuteInSequence', just
> >>>>> returns
> >>>>>> false instead of getting the value from the global variable.
> >>>>>>
> >>>>>> And one more change is that the file has been moved to
> >>>>>> 'engine/orchestration/src/com/cloud/vm' from
> 'server/src/com/cloud/vm'.
> >>>>>>
> >>>>>> Am I missing something related with this or do we stop supporting
> this
> >>>>>> option in 4.3?
> >>>>>> I'm a little confused, so please help me resolve this.
> >>>>>>
> >>>>>> Thanks
> >>>>>> Alex Ough
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Nov 12, 2013 at 4:20 PM, Alex Ough
> >>>>> wrote:
> >>>>>>
> >>>>>>> Thanks a lot for your confirmation, Marcus.
> >>>>>>> I'll create a review request unless anyone has an objection.
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>> Alex Ough
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Nov 12, 2013 at 3:37 PM, Marcus Sorensen <
Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
/com/cloud/region/service/RemoteDomainEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteUserEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserFullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserService.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/AutoGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/LocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/server/StatsCollector.java 45fc43e server/src/com/cloud/user/AccountManager.java 2e909c8 server/src/com/cloud/user/AccountManagerImpl.java d367653 server/src/com/cloud/user/DomainManager.java 7bc2abb server/src/com/cloud/user/DomainManagerImpl.java 036110b server/test/com/cloud/user/MockAccountManagerImpl.java 38cc1a84 server/test/com/cloud/user/MockDomainManagerImpl.java aab8001 setup/db/db/schema-421to430.sql 316cbe9 Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
All, I just sent a review request, so please take a look at it and let me know if you have any comments/suggests. https://reviews.apache.org/r/17790/ Thanks Alex Ough On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough wrote: > All, > > I'd like to have some suggestion about 2 things related with this. > > 1. The 'Full Scan' management > Now, I set it running every time a user logs in to the UI, but I think it > will be necessary to make it run with some interval also. > But I'm not familiar with the config file, so can anyone give some > directions how to manage the time interval in the config file and the best > way to run it with the time interval? > > 2. Repository of regions with their login information. > To send/receive requests to/from other regions using API interfaces, we > need the region information including login info of each region. > I was planning to use a table as a repository, but I think it is better to > store it in the config file to make the access a little lighter. > Any recommendation on this? > > Your reply with directions & comments will be very appreciated. > Thanks > Alex Ough > > > On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough wrote: > >> All, >> >> A little bit of updates after a long vacation, >> I'm currently creating automated test scripts that randomly >> create/delete/update domain/account/user objects in random regions to >> trigger the sync-up and full scans regularly. >> Once they are completed, I'll post it in the github also and submit the >> review requests for this implementation. >> >> Let me know if you have any comments. >> Thanks >> Alex Ough >> >> >> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough wrote: >> >>> All, >>> >>> I updated the wiki after some logic changes, so please review them, >>> especially "Full Scan", which is newly introduced. >>> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions >>> >>> And I implemented this functionality in Java and you can get the pull >>> request of it here. >>> (This does not include the 'full scan' yet and I'm currently working >>> on this to finalize.) >>> https://github.com/alexoughsg/Albatross/pull/1 >>> >>> Especially, I really want to have your review on the "Full Scan" logic >>> to confirm if it does not miss any cases. >>> Thanks for your interest and your feedback will be very helpful. >>> Alex Ough >>> >>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough >>> wrote: >>> > Good point, Chiradeep, >>> > >>> > I'm not sure if you reviewed my design doc in the wiki, but my design >>> is to >>> > just skip any actions for target resources that already took place by >>> any >>> > means. >>> > But the issue is when conflict actions in the same resources (like >>> create & >>> > delete the same users) are enqueued in reversed orders, which is >>> hopefully >>> > rare. >>> > >>> > And to support consistency in the AP system, I'd like to provide a >>> full sync >>> > up, which will sync up all data in all region servers by selecting a >>> region >>> > as a master and force its data to other regions. >>> > >>> > Let me know what you think. >>> > Thanks >>> > Alex Ough >>> > >>> > >>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal >>> > wrote: >>> >> >>> >> Missed this one. In a single region, the CloudStack DB is the master >>> for >>> >> most operations. If the infra is not in the state the DB says it >>> should >>> >> be, generally the approach is to whack it and make it conform. For >>> some >>> >> exceptions (live migration/related use cases are exceptions) the DB >>> is the >>> >> slave -- the point is that the inconsistency that inevitably arise in >>> an >>> >> AP system need a conflict resolution system. In a single region, the >>> >> default is to assume that the MySQL DB is correct and handle other >>> cases >>> >> carefully. >>> >> >>> >> In a multi-region case, there is no master: disable an account in one >>> >> region, and it may not propagate to the other regions for many >>> hours/days. >>
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
Hi Chiradeep, Thanks for your reply. The change is just to add timestamps when record has been changed to decide the time order when a same resource has been changed independently in different regions. The changes are minimum and additions, so I don't think they will cause any side effects. Thanks Alex Ough On Thu, Feb 6, 2014 at 1:35 PM, Chiradeep Vittal < chiradeep.vit...@citrix.com> wrote: > I am uncomfortable with changes to GenericDaoBase. Was this really > necessary? This feature was supposed to be "outside" CloudStack as much as > possible and optional. Yet it touches the most sensitive code in CloudStack. > > From: Alex Ough > Date: Thursday, February 6, 2014 6:29 AM > To: "dev@cloudstack.apache.org" > Cc: Chip Childers , Daan Hoogland < > daan.hoogl...@gmail.com>, Chiradeep Vittal , > Kishan Kavala > Subject: Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions > > All, > > I just sent a review request, so please take a look at it and let me > know if you have any comments/suggests. > > https://reviews.apache.org/r/17790/ > > Thanks > Alex Ough > > > On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough wrote: > >> All, >> >> I'd like to have some suggestion about 2 things related with this. >> >> 1. The 'Full Scan' management >> Now, I set it running every time a user logs in to the UI, but I think it >> will be necessary to make it run with some interval also. >> But I'm not familiar with the config file, so can anyone give some >> directions how to manage the time interval in the config file and the best >> way to run it with the time interval? >> >> 2. Repository of regions with their login information. >> To send/receive requests to/from other regions using API interfaces, we >> need the region information including login info of each region. >> I was planning to use a table as a repository, but I think it is better >> to store it in the config file to make the access a little lighter. >> Any recommendation on this? >> >> Your reply with directions & comments will be very appreciated. >> Thanks >> Alex Ough >> >> >> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough wrote: >> >>> All, >>> >>> A little bit of updates after a long vacation, >>> I'm currently creating automated test scripts that randomly >>> create/delete/update domain/account/user objects in random regions to >>> trigger the sync-up and full scans regularly. >>> Once they are completed, I'll post it in the github also and submit the >>> review requests for this implementation. >>> >>> Let me know if you have any comments. >>> Thanks >>> Alex Ough >>> >>> >>> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough wrote: >>> >>>> All, >>>> >>>> I updated the wiki after some logic changes, so please review them, >>>> especially "Full Scan", which is newly introduced. >>>> >>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions >>>> >>>> And I implemented this functionality in Java and you can get the pull >>>> request of it here. >>>> (This does not include the 'full scan' yet and I'm currently working >>>> on this to finalize.) >>>> https://github.com/alexoughsg/Albatross/pull/1 >>>> >>>> Especially, I really want to have your review on the "Full Scan" logic >>>> to confirm if it does not miss any cases. >>>> Thanks for your interest and your feedback will be very helpful. >>>> Alex Ough >>>> >>>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough >>>> wrote: >>>> > Good point, Chiradeep, >>>> > >>>> > I'm not sure if you reviewed my design doc in the wiki, but my design >>>> is to >>>> > just skip any actions for target resources that already took place by >>>> any >>>> > means. >>>> > But the issue is when conflict actions in the same resources (like >>>> create & >>>> > delete the same users) are enqueued in reversed orders, which is >>>> hopefully >>>> > rare. >>>> > >>>> > And to support consistency in the AP system, I'd like to provide a >>>> full sync >>>> > up, which will sync up all data in all region servers by selecting a >&
Re: [ACS 4.3] RC update
All, I'm not sure what the current status for the issue of "Could not find corresponding resource manager for simulator" during deploying infra in simulator mode. It looks like I'm still getting this error in 4.3. Your help will be very appreciated. Thanks Alex Ough On Thu, Jan 23, 2014 at 5:52 AM, Prasanna Santhanam wrote: > On Thu, Jan 23, 2014 at 11:39:30AM +, Santhosh Edukulla wrote: > > Simulator issue is fixed. It now works, please check. > > CLOUDSTACK-5875 probably needs to go to master as well. Didn't see a > commit there. Once it's fixed in master all jobs in [1] > will show success. These are run on a per-commit basis. > > [1] http://jenkins.buildacloud.org/view/simulator/ > > Thanks, > -- > Prasanna., > > > Powered by BigRock.com > > >
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
ATION server/src/com/cloud/region/service/RemoteAccountEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteDomainEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteUserEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserFullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserService.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/AutoGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/LocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/server/StatsCollector.java 45fc43e server/src/com/cloud/user/AccountManager.java 2e909c8 server/src/com/cloud/user/AccountManagerImpl.java d367653 server/src/com/cloud/user/DomainManager.java 7bc2abb server/src/com/cloud/user/DomainManagerImpl.java 036110b server/test/com/cloud/user/MockAccountManagerImpl.java 38cc1a84 server/test/com/cloud/user/MockDomainManagerImpl.java aab8001 setup/db/db/schema-421to430.sql 316cbe9 tools/devcloud/devcloud-advanced.cfg fb25d03 Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
Hi Daan, Does it look good to you? Thanks Alex Ough On Mon, Feb 10, 2014 at 1:07 PM, Alex Ough wrote: >This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/17790/ > Review request for cloudstack. > By Alex Ough. > > *Updated Feb. 10, 2014, 7:06 p.m.* > Changes > > Updates for Daan's request. > > *Repository: * cloudstack-git > Description > > Currently, under the environment of cloudstack with multiple regions, each > region has its own management server running with a separate database, which > will cause data discrepancies when users create/update/delete > domain/account/user data independently in each management server. So to > support multiple regions and provide one point of entry for each customer, > this implementation duplicates domain/account/user information of customers > in one region to all of the regions independently whenever there is any > change. > https://issues.apache.org/jira/browse/CLOUDSTACK-4992https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions > > Testing > > 1. Successfully tested real time synchronization as soon as resources are > created/deleted/modified in one region. > 2. Successfully tested full scans to synchronize resources that were missed > during real time synchronization because of any reasons like network > connection issues. > 3. The tests were done manually and also automatically by randomly generating > changes each region. > > Diffs (updated) > >- api/src/com/cloud/domain/Domain.java (befed07) >- api/src/com/cloud/event/EventTypes.java (77d2fab) >- api/src/com/cloud/user/Account.java (940a0eb) >- api/src/com/cloud/user/AccountService.java (8153a3f) >- api/src/com/cloud/user/User.java (dcf27a0) >- api/src/com/cloud/user/UserAccount.java (0cb0f69) >- api/src/org/apache/cloudstack/api/ApiConstants.java (68abf8d) >- api/src/org/apache/cloudstack/api/BaseCmd.java (acc1568) >- api/src/org/apache/cloudstack/api/response/AccountResponse.java >(957936b) >- api/src/org/apache/cloudstack/api/response/DomainResponse.java >(736a96c) >- api/src/org/apache/cloudstack/api/response/UserResponse.java >(e70a310) >- api/src/org/apache/cloudstack/region/Region.java (c696fb2) >- > engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml >(ea0bad9) >- engine/schema/src/com/cloud/domain/DomainVO.java (a87bedc) >- engine/schema/src/com/cloud/rmap/RmapVO.java (PRE-CREATION) >- engine/schema/src/com/cloud/rmap/dao/RmapDao.java (PRE-CREATION) >- engine/schema/src/com/cloud/rmap/dao/RmapDaoImpl.java (PRE-CREATION) >- engine/schema/src/com/cloud/user/AccountVO.java (4a7e73b) >- engine/schema/src/com/cloud/user/UserAccountVO.java (cfad617) >- engine/schema/src/com/cloud/user/UserVO.java (2a85758) >- engine/schema/src/org/apache/cloudstack/region/RegionVO.java >(36db8dd) >- framework/db/src/com/cloud/utils/db/Attribute.java (22fd969) >- framework/db/src/com/cloud/utils/db/GenericDao.java (ef25d7f) >- framework/db/src/com/cloud/utils/db/GenericDaoBase.java (177cd5b) >- framework/db/src/com/cloud/utils/db/SqlGenerator.java (473e839) >- > plugins/event-bus/rabbitmq/src/org/apache/cloudstack/mom/rabbitmq/AccountSubscriber.java >(PRE-CREATION) >- > plugins/event-bus/rabbitmq/src/org/apache/cloudstack/mom/rabbitmq/DomainSubscriber.java >(PRE-CREATION) >- > plugins/event-bus/rabbitmq/src/org/apache/cloudstack/mom/rabbitmq/MultiRegionEventBus.java >(PRE-CREATION) >- > plugins/event-bus/rabbitmq/src/org/apache/cloudstack/mom/rabbitmq/MultiRegionSubscriber.java >(PRE-CREATION) >- > plugins/event-bus/rabbitmq/src/org/apache/cloudstack/mom/rabbitmq/UserSubscriber.java >(PRE-CREATION) >- > plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java >(195e932) >- server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml >(PRE-CREATION) >- server/src/com/cloud/api/ApiDispatcher.java (9624c61) >- server/src/com/cloud/api/ApiResponseHelper.java (5bd8241) >- server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java (edbd3ae) >- server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java >(4ad8a8f) >- server/src/com/cloud/api/query/vo/AccountJoinVO.java (2ec45d3) >- server/src/com/cloud/api/query/vo/UserAccountJoinVO.java (c020c01) >- server/src/com/cloud/event/ActionEventUtils.java (00122de) >- server/src/com/cloud/projects/ProjectManagerImpl.java (22e2020) >- server/src/com/cl
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
We're just about to add unit tests, so I'll add them once completed. Thanks Alex Ough On Thu, Feb 13, 2014 at 3:12 PM, Daan Hoogland wrote: > h Alex, > > looks alright (i did mean with newlines but I don't know if this is > actually caught by our checkstyle run) I was wondering if you have any unit > tests for your code. I can see you did take testing seriously and I would > like to see this formalized for future reference and repetition on updates. > Can you add some stuff in this line? > > thanks, > Daan > > > On Thu, Feb 13, 2014 at 9:52 PM, Alex Ough wrote: > >> Hi Daan, >> Does it look good to you? >> >> Thanks >> Alex Ough >> >> >> On Mon, Feb 10, 2014 at 1:07 PM, Alex Ough wrote: >> >>>This is an automatically generated e-mail. To reply, visit: >>> https://reviews.apache.org/r/17790/ >>> Review request for cloudstack. >>> By Alex Ough. >>> >>> *Updated Feb. 10, 2014, 7:06 p.m.* >>> Changes >>> >>> Updates for Daan's request. >>> >>> *Repository: * cloudstack-git >>> Description >>> >>> Currently, under the environment of cloudstack with multiple regions, each >>> region has its own management server running with a separate database, >>> which will cause data discrepancies when users create/update/delete >>> domain/account/user data independently in each management server. So to >>> support multiple regions and provide one point of entry for each customer, >>> this implementation duplicates domain/account/user information of customers >>> in one region to all of the regions independently whenever there is any >>> change. >>> https://issues.apache.org/jira/browse/CLOUDSTACK-4992https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions >>> >>> Testing >>> >>> 1. Successfully tested real time synchronization as soon as resources are >>> created/deleted/modified in one region. >>> 2. Successfully tested full scans to synchronize resources that were missed >>> during real time synchronization because of any reasons like network >>> connection issues. >>> 3. The tests were done manually and also automatically by randomly >>> generating changes each region. >>> >>> Diffs (updated) >>> >>>- api/src/com/cloud/domain/Domain.java (befed07) >>>- api/src/com/cloud/event/EventTypes.java (77d2fab) >>>- api/src/com/cloud/user/Account.java (940a0eb) >>>- api/src/com/cloud/user/AccountService.java (8153a3f) >>>- api/src/com/cloud/user/User.java (dcf27a0) >>>- api/src/com/cloud/user/UserAccount.java (0cb0f69) >>>- api/src/org/apache/cloudstack/api/ApiConstants.java (68abf8d) >>>- api/src/org/apache/cloudstack/api/BaseCmd.java (acc1568) >>>- api/src/org/apache/cloudstack/api/response/AccountResponse.java >>>(957936b) >>>- api/src/org/apache/cloudstack/api/response/DomainResponse.java >>>(736a96c) >>>- api/src/org/apache/cloudstack/api/response/UserResponse.java >>>(e70a310) >>>- api/src/org/apache/cloudstack/region/Region.java (c696fb2) >>>- >>> engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml >>>(ea0bad9) >>>- engine/schema/src/com/cloud/domain/DomainVO.java (a87bedc) >>>- engine/schema/src/com/cloud/rmap/RmapVO.java (PRE-CREATION) >>>- engine/schema/src/com/cloud/rmap/dao/RmapDao.java (PRE-CREATION) >>>- engine/schema/src/com/cloud/rmap/dao/RmapDaoImpl.java >>>(PRE-CREATION) >>>- engine/schema/src/com/cloud/user/AccountVO.java (4a7e73b) >>>- engine/schema/src/com/cloud/user/UserAccountVO.java (cfad617) >>>- engine/schema/src/com/cloud/user/UserVO.java (2a85758) >>>- engine/schema/src/org/apache/cloudstack/region/RegionVO.java >>>(36db8dd) >>>- framework/db/src/com/cloud/utils/db/Attribute.java (22fd969) >>>- framework/db/src/com/cloud/utils/db/GenericDao.java (ef25d7f) >>>- framework/db/src/com/cloud/utils/db/GenericDaoBase.java (177cd5b) >>>- framework/db/src/com/cloud/utils/db/SqlGenerator.java (473e839) >>>- >>> plugins/event-bus/rabbitmq/src/org/apache/cloudstack/mom/rabbitmq/AccountSubscriber.java >>>(PRE-CREATION) >>>- >>> plugins/event-bus/rabbitmq/src/org/apache/cloudstack/mom/rabbitmq/DomainSu
Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
Hi Dave, "resources have different uuids in different regions even if they are identical" is not a restriction but a normal case because maintaining a same uuid can be difficult in some cases like when 2 different users create the same resource in different regions at the same time. But it will not be an issue if you still want to maintain the same UUIDs because the resource map table will have the same uuid for each resource across all regions. The only thing to be changed is to set the resource UUID in the API create calls to send the create requests to remote regions during the real time synchronization. Let me know if this is not clear. Thanks Alex Ough On Wed, Feb 19, 2014 at 1:47 PM, David Grizzanti < david.grizza...@sungard.com> wrote: > Hi Alex, > > One thing I wanted to ask about/mention on this was regarding the > restriction you have mentioned in the wiki on "resources have different > uuids in different regions even if they are identical". We discussed this > a bit offline, but I think it would be beneficial to allow for the the > UUIDs to be carried over to the other regions when you're replicating > resources. I'm not sure how others feel about this feature, but I know that > in our discussions we will need this feature if we are to rely on the sync > to create domains/accounts/user across all the regions as the UUID is in > the identifying factor of uniqueness. > > Let me know your thoughts on this and whether or not this can be added, at > least as an optional item if needed. > > Thanks > > -- > David Grizzanti > Software Engineer > Sungard Availability Services > e: david.grizza...@sungard.com > w: 215.446.1431 > c: 570.575.0315 > > On February 6, 2014 at 3:19:52 PM, Alex Ough > (alex.o...@sungard.com) > wrote: > > Hi Chiradeep, > Thanks for your reply. > > The change is just to add timestamps when record has been changed to > decide > the time order when a same resource has been changed independently in > different regions. > The changes are minimum and additions, so I don't think they will cause > any > side effects. > > Thanks > Alex Ough > > > On Thu, Feb 6, 2014 at 1:35 PM, Chiradeep Vittal < > chiradeep.vit...@citrix.com> wrote: > > > I am uncomfortable with changes to GenericDaoBase. Was this really > > necessary? This feature was supposed to be "outside" CloudStack as much > as > > possible and optional. Yet it touches the most sensitive code in > CloudStack. > > > > From: Alex Ough > > Date: Thursday, February 6, 2014 6:29 AM > > To: "dev@cloudstack.apache.org" > > Cc: Chip Childers , Daan Hoogland < > > daan.hoogl...@gmail.com>, Chiradeep Vittal , > > > Kishan Kavala > > Subject: Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple > Regions > > > > All, > > > > I just sent a review request, so please take a look at it and let me > > know if you have any comments/suggests. > > > > https://reviews.apache.org/r/17790/ > > > > Thanks > > Alex Ough > > > > > > On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough > wrote: > > > >> All, > >> > >> I'd like to have some suggestion about 2 things related with this. > >> > >> 1. The 'Full Scan' management > >> Now, I set it running every time a user logs in to the UI, but I think > it > >> will be necessary to make it run with some interval also. > >> But I'm not familiar with the config file, so can anyone give some > >> directions how to manage the time interval in the config file and the > best > >> way to run it with the time interval? > >> > >> 2. Repository of regions with their login information. > >> To send/receive requests to/from other regions using API interfaces, we > >> need the region information including login info of each region. > >> I was planning to use a table as a repository, but I think it is better > >> to store it in the config file to make the access a little lighter. > >> Any recommendation on this? > >> > >> Your reply with directions & comments will be very appreciated. > >> Thanks > >> Alex Ough > >> > >> > >> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough > wrote: > >> > >>> All, > >>> > >>> A little bit of updates after a long vacation, > >>> I'm currently creating automated test scripts that randomly > >>> create/delete/update domain/account/user objects in random regions to > >>> trigger the sync-up
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
Hi Daan, We started to work on implementing unit tests and checked in some of them, so please review them and let us know if there is any comments/suggestions. https://github.com/alexoughsg/Albatross/commit/fca94db36054ee35f5da85c69c2471430e0eebfe Thanks Alex Ough On Thu, Feb 13, 2014 at 3:15 PM, Alex Ough wrote: > We're just about to add unit tests, so I'll add them once completed. > > Thanks > Alex Ough > > > On Thu, Feb 13, 2014 at 3:12 PM, Daan Hoogland wrote: > >> h Alex, >> >> looks alright (i did mean with newlines but I don't know if this is >> actually caught by our checkstyle run) I was wondering if you have any unit >> tests for your code. I can see you did take testing seriously and I would >> like to see this formalized for future reference and repetition on updates. >> Can you add some stuff in this line? >> >> thanks, >> Daan >> >> >> On Thu, Feb 13, 2014 at 9:52 PM, Alex Ough wrote: >> >>> Hi Daan, >>> Does it look good to you? >>> >>> Thanks >>> Alex Ough >>> >>> >>> On Mon, Feb 10, 2014 at 1:07 PM, Alex Ough wrote: >>> >>>>This is an automatically generated e-mail. To reply, visit: >>>> https://reviews.apache.org/r/17790/ >>>> Review request for cloudstack. >>>> By Alex Ough. >>>> >>>> *Updated Feb. 10, 2014, 7:06 p.m.* >>>> Changes >>>> >>>> Updates for Daan's request. >>>> >>>> *Repository: * cloudstack-git >>>> Description >>>> >>>> Currently, under the environment of cloudstack with multiple regions, each >>>> region has its own management server running with a separate database, >>>> which will cause data discrepancies when users create/update/delete >>>> domain/account/user data independently in each management server. So to >>>> support multiple regions and provide one point of entry for each customer, >>>> this implementation duplicates domain/account/user information of >>>> customers in one region to all of the regions independently whenever there >>>> is any change. >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4992https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions >>>> >>>> Testing >>>> >>>> 1. Successfully tested real time synchronization as soon as resources are >>>> created/deleted/modified in one region. >>>> 2. Successfully tested full scans to synchronize resources that were >>>> missed during real time synchronization because of any reasons like >>>> network connection issues. >>>> 3. The tests were done manually and also automatically by randomly >>>> generating changes each region. >>>> >>>> Diffs (updated) >>>> >>>>- api/src/com/cloud/domain/Domain.java (befed07) >>>>- api/src/com/cloud/event/EventTypes.java (77d2fab) >>>>- api/src/com/cloud/user/Account.java (940a0eb) >>>>- api/src/com/cloud/user/AccountService.java (8153a3f) >>>>- api/src/com/cloud/user/User.java (dcf27a0) >>>>- api/src/com/cloud/user/UserAccount.java (0cb0f69) >>>>- api/src/org/apache/cloudstack/api/ApiConstants.java (68abf8d) >>>>- api/src/org/apache/cloudstack/api/BaseCmd.java (acc1568) >>>>- api/src/org/apache/cloudstack/api/response/AccountResponse.java >>>>(957936b) >>>>- api/src/org/apache/cloudstack/api/response/DomainResponse.java >>>>(736a96c) >>>>- api/src/org/apache/cloudstack/api/response/UserResponse.java >>>>(e70a310) >>>>- api/src/org/apache/cloudstack/region/Region.java (c696fb2) >>>>- >>>> engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml >>>>(ea0bad9) >>>>- engine/schema/src/com/cloud/domain/DomainVO.java (a87bedc) >>>>- engine/schema/src/com/cloud/rmap/RmapVO.java (PRE-CREATION) >>>>- engine/schema/src/com/cloud/rmap/dao/RmapDao.java (PRE-CREATION) >>>>- engine/schema/src/com/cloud/rmap/dao/RmapDaoImpl.java >>>>(PRE-CREATION) >>>>- engine/schema/src/com/cloud/user/AccountVO.java (4a7e73b) >>>>- engine/schema/src/com/cloud/user/UserAccountVO.java (cfad617) >>>>- engine/schema/src/com/cloud/user/UserVO.java (2a85758) >
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
Hi Daan, 1. Yes, we'll update the request once all the unit tests are completed. 2. And yes, it is a branch from 4.3 3. I'm using the global parameter, called 'region.auto.generation.interval' and the value is 0 (millisec) by default. If you set the value more than 0, it will automatically create/update/remove domains/accounts/users with that interval, which will trigger the scheduled full scan to sync the newly changed resources. Let me know if you need more info. Thanks Alex Ough On Thu, Feb 27, 2014 at 3:50 PM, Daan Hoogland wrote: > I see, great. > > I will find a moment to apply it and test. It looks good at first sight. > > Don't forget adding the license header to the new files. Are you > updating the review request with these tests? > > You are using a local branch to implement this on, > 'alexoughsg/Albatross' is a fork of cloudstack, is it? > Let's discuss merge back and testing. > Do you have a integration test script in Marvin and/or scenarios that > will ensure continued functionality for the sync feature? > > > thanks, > Daan > > On Thu, Feb 27, 2014 at 9:19 PM, Alex Ough wrote: > > Hi Daan, > > > > We started to work on implementing unit tests and checked in some of > them, > > so please review them and let us know if there is any > comments/suggestions. > > > > > https://github.com/alexoughsg/Albatross/commit/fca94db36054ee35f5da85c69c2471430e0eebfe > > > > Thanks > > Alex Ough > > > > > > On Thu, Feb 13, 2014 at 3:15 PM, Alex Ough > wrote: > >> > >> We're just about to add unit tests, so I'll add them once completed. > >> > >> Thanks > >> Alex Ough > >> > >> > >> On Thu, Feb 13, 2014 at 3:12 PM, Daan Hoogland > > >> wrote: > >>> > >>> h Alex, > >>> > >>> looks alright (i did mean with newlines but I don't know if this is > >>> actually caught by our checkstyle run) I was wondering if you have any > unit > >>> tests for your code. I can see you did take testing seriously and I > would > >>> like to see this formalized for future reference and repetition on > updates. > >>> Can you add some stuff in this line? > >>> > >>> thanks, > >>> Daan > >>> > >>> > >>> On Thu, Feb 13, 2014 at 9:52 PM, Alex Ough > wrote: > >>>> > >>>> Hi Daan, > >>>> Does it look good to you? > >>>> > >>>> Thanks > >>>> Alex Ough > >>>> > >>>> > >>>> On Mon, Feb 10, 2014 at 1:07 PM, Alex Ough > >>>> wrote: > >>>>> > >>>>> This is an automatically generated e-mail. To reply, visit: > >>>>> https://reviews.apache.org/r/17790/ > >>>>> > >>>>> Review request for cloudstack. > >>>>> By Alex Ough. > >>>>> > >>>>> Updated Feb. 10, 2014, 7:06 p.m. > >>>>> > >>>>> Changes > >>>>> > >>>>> Updates for Daan's request. > >>>>> > >>>>> Repository: cloudstack-git > >>>>> > >>>>> Description > >>>>> > >>>>> Currently, under the environment of cloudstack with multiple regions, > >>>>> each region has its own management server running with a separate > database, > >>>>> which will cause data discrepancies when users create/update/delete > >>>>> domain/account/user data independently in each management server. So > to > >>>>> support multiple regions and provide one point of entry for each > customer, > >>>>> this implementation duplicates domain/account/user information of > customers > >>>>> in one region to all of the regions independently whenever there is > any > >>>>> change. > >>>>> > >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4992 > >>>>> > >>>>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions > >>>>> > >>>>> Testing > >>>>> > >>>>> 1. Successfully tested real time synchronization as soon as resources > >>>>> are created/deleted/modified in one region. > >>>>> 2. Successfully tested full scans to sy
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
Can you tell me when the deadline for 4.4 is? Thanks Alex Ough On Sat, Mar 1, 2014 at 2:26 PM, Daan Hoogland wrote: > Ha Alex, > > branch 4.3 is going to be a problem. We will be able to release this > with 4.4 if we hurry. > > On Fri, Feb 28, 2014 at 4:01 PM, Alex Ough wrote: > > Hi Daan, > > > > 1. Yes, we'll update the request once all the unit tests are completed. > > 2. And yes, it is a branch from 4.3 > > 3. I'm using the global parameter, called > 'region.auto.generation.interval' > > and the value is 0 (millisec) by default. > > If you set the value more than 0, it will automatically > > create/update/remove domains/accounts/users with that interval, > > which will trigger the scheduled full scan to sync the newly changed > > resources. > > > > Let me know if you need more info. > > Thanks > > Alex Ough > > > > > > On Thu, Feb 27, 2014 at 3:50 PM, Daan Hoogland > > wrote: > >> > >> I see, great. > >> > >> I will find a moment to apply it and test. It looks good at first sight. > >> > >> Don't forget adding the license header to the new files. Are you > >> updating the review request with these tests? > >> > >> You are using a local branch to implement this on, > >> 'alexoughsg/Albatross' is a fork of cloudstack, is it? > >> Let's discuss merge back and testing. > >> Do you have a integration test script in Marvin and/or scenarios that > >> will ensure continued functionality for the sync feature? > >> > >> > >> thanks, > >> Daan > >> > >> On Thu, Feb 27, 2014 at 9:19 PM, Alex Ough > wrote: > >> > Hi Daan, > >> > > >> > We started to work on implementing unit tests and checked in some of > >> > them, > >> > so please review them and let us know if there is any > >> > comments/suggestions. > >> > > >> > > >> > > https://github.com/alexoughsg/Albatross/commit/fca94db36054ee35f5da85c69c2471430e0eebfe > >> > > >> > Thanks > >> > Alex Ough > >> > > >> > > >> > On Thu, Feb 13, 2014 at 3:15 PM, Alex Ough > >> > wrote: > >> >> > >> >> We're just about to add unit tests, so I'll add them once completed. > >> >> > >> >> Thanks > >> >> Alex Ough > >> >> > >> >> > >> >> On Thu, Feb 13, 2014 at 3:12 PM, Daan Hoogland > >> >> > >> >> wrote: > >> >>> > >> >>> h Alex, > >> >>> > >> >>> looks alright (i did mean with newlines but I don't know if this is > >> >>> actually caught by our checkstyle run) I was wondering if you have > any > >> >>> unit > >> >>> tests for your code. I can see you did take testing seriously and I > >> >>> would > >> >>> like to see this formalized for future reference and repetition on > >> >>> updates. > >> >>> Can you add some stuff in this line? > >> >>> > >> >>> thanks, > >> >>> Daan > >> >>> > >> >>> > >> >>> On Thu, Feb 13, 2014 at 9:52 PM, Alex Ough > >> >>> wrote: > >> >>>> > >> >>>> Hi Daan, > >> >>>> Does it look good to you? > >> >>>> > >> >>>> Thanks > >> >>>> Alex Ough > >> >>>> > >> >>>> > >> >>>> On Mon, Feb 10, 2014 at 1:07 PM, Alex Ough > >> >>>> wrote: > >> >>>>> > >> >>>>> This is an automatically generated e-mail. To reply, visit: > >> >>>>> https://reviews.apache.org/r/17790/ > >> >>>>> > >> >>>>> Review request for cloudstack. > >> >>>>> By Alex Ough. > >> >>>>> > >> >>>>> Updated Feb. 10, 2014, 7:06 p.m. > >> >>>>> > >> >>>>> Changes > >> >>>>> > >> >>>>> Updates for Daan's request. > >> >>>>> > >> >>>>> Repository: cloudstack-git > >> >>>>> > >> >>>>
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
Do you think it is ok to wrap up the implementation until the end of this week to make this included in 4.4 release? Thanks Alex Ough On Sun, Mar 2, 2014 at 2:10 PM, Daan Hoogland wrote: > two weeks from now, but under debate. Let's go with that, though. > > On Sun, Mar 2, 2014 at 1:17 AM, Alex Ough wrote: > > Can you tell me when the deadline for 4.4 is? > > > > Thanks > > Alex Ough > > > > > > On Sat, Mar 1, 2014 at 2:26 PM, Daan Hoogland > > wrote: > >> > >> Ha Alex, > >> > >> branch 4.3 is going to be a problem. We will be able to release this > >> with 4.4 if we hurry. > >> > >> On Fri, Feb 28, 2014 at 4:01 PM, Alex Ough > wrote: > >> > Hi Daan, > >> > > >> > 1. Yes, we'll update the request once all the unit tests are > completed. > >> > 2. And yes, it is a branch from 4.3 > >> > 3. I'm using the global parameter, called > >> > 'region.auto.generation.interval' > >> > and the value is 0 (millisec) by default. > >> > If you set the value more than 0, it will automatically > >> > create/update/remove domains/accounts/users with that interval, > >> > which will trigger the scheduled full scan to sync the newly > changed > >> > resources. > >> > > >> > Let me know if you need more info. > >> > Thanks > >> > Alex Ough > >> > > >> > > >> > On Thu, Feb 27, 2014 at 3:50 PM, Daan Hoogland < > daan.hoogl...@gmail.com> > >> > wrote: > >> >> > >> >> I see, great. > >> >> > >> >> I will find a moment to apply it and test. It looks good at first > >> >> sight. > >> >> > >> >> Don't forget adding the license header to the new files. Are you > >> >> updating the review request with these tests? > >> >> > >> >> You are using a local branch to implement this on, > >> >> 'alexoughsg/Albatross' is a fork of cloudstack, is it? > >> >> Let's discuss merge back and testing. > >> >> Do you have a integration test script in Marvin and/or scenarios that > >> >> will ensure continued functionality for the sync feature? > >> >> > >> >> > >> >> thanks, > >> >> Daan > >> >> > >> >> On Thu, Feb 27, 2014 at 9:19 PM, Alex Ough > >> >> wrote: > >> >> > Hi Daan, > >> >> > > >> >> > We started to work on implementing unit tests and checked in some > of > >> >> > them, > >> >> > so please review them and let us know if there is any > >> >> > comments/suggestions. > >> >> > > >> >> > > >> >> > > >> >> > > https://github.com/alexoughsg/Albatross/commit/fca94db36054ee35f5da85c69c2471430e0eebfe > >> >> > > >> >> > Thanks > >> >> > Alex Ough > >> >> > > >> >> > > >> >> > On Thu, Feb 13, 2014 at 3:15 PM, Alex Ough > >> >> > wrote: > >> >> >> > >> >> >> We're just about to add unit tests, so I'll add them once > completed. > >> >> >> > >> >> >> Thanks > >> >> >> Alex Ough > >> >> >> > >> >> >> > >> >> >> On Thu, Feb 13, 2014 at 3:12 PM, Daan Hoogland > >> >> >> > >> >> >> wrote: > >> >> >>> > >> >> >>> h Alex, > >> >> >>> > >> >> >>> looks alright (i did mean with newlines but I don't know if this > >> >> >>> is > >> >> >>> actually caught by our checkstyle run) I was wondering if you > have > >> >> >>> any > >> >> >>> unit > >> >> >>> tests for your code. I can see you did take testing seriously > and I > >> >> >>> would > >> >> >>> like to see this formalized for future reference and repetition > on > >> >> >>> updates. > >> >> >>> Can you add some stuff in this line? > >> >> >>> > >> >> >>> thanks, > >> &g
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
Yes, by 3/7. Thanks Alex Ough On Tue, Mar 4, 2014 at 12:30 PM, Daan Hoogland wrote: > This week? should be ok. end of next week is too late! > > On Tue, Mar 4, 2014 at 6:17 PM, Alex Ough wrote: > > Do you think it is ok to wrap up the implementation until the end of this > > week to make this included in 4.4 release? > > > > Thanks > > Alex Ough > > > > > > On Sun, Mar 2, 2014 at 2:10 PM, Daan Hoogland > > wrote: > >> > >> two weeks from now, but under debate. Let's go with that, though. > >> > >> On Sun, Mar 2, 2014 at 1:17 AM, Alex Ough > wrote: > >> > Can you tell me when the deadline for 4.4 is? > >> > > >> > Thanks > >> > Alex Ough > >> > > >> > > >> > On Sat, Mar 1, 2014 at 2:26 PM, Daan Hoogland < > daan.hoogl...@gmail.com> > >> > wrote: > >> >> > >> >> Ha Alex, > >> >> > >> >> branch 4.3 is going to be a problem. We will be able to release this > >> >> with 4.4 if we hurry. > >> >> > >> >> On Fri, Feb 28, 2014 at 4:01 PM, Alex Ough > >> >> wrote: > >> >> > Hi Daan, > >> >> > > >> >> > 1. Yes, we'll update the request once all the unit tests are > >> >> > completed. > >> >> > 2. And yes, it is a branch from 4.3 > >> >> > 3. I'm using the global parameter, called > >> >> > 'region.auto.generation.interval' > >> >> > and the value is 0 (millisec) by default. > >> >> > If you set the value more than 0, it will automatically > >> >> > create/update/remove domains/accounts/users with that interval, > >> >> > which will trigger the scheduled full scan to sync the newly > >> >> > changed > >> >> > resources. > >> >> > > >> >> > Let me know if you need more info. > >> >> > Thanks > >> >> > Alex Ough > >> >> > > >> >> > > >> >> > On Thu, Feb 27, 2014 at 3:50 PM, Daan Hoogland > >> >> > > >> >> > wrote: > >> >> >> > >> >> >> I see, great. > >> >> >> > >> >> >> I will find a moment to apply it and test. It looks good at first > >> >> >> sight. > >> >> >> > >> >> >> Don't forget adding the license header to the new files. Are you > >> >> >> updating the review request with these tests? > >> >> >> > >> >> >> You are using a local branch to implement this on, > >> >> >> 'alexoughsg/Albatross' is a fork of cloudstack, is it? > >> >> >> Let's discuss merge back and testing. > >> >> >> Do you have a integration test script in Marvin and/or scenarios > >> >> >> that > >> >> >> will ensure continued functionality for the sync feature? > >> >> >> > >> >> >> > >> >> >> thanks, > >> >> >> Daan > >> >> >> > >> >> >> On Thu, Feb 27, 2014 at 9:19 PM, Alex Ough > > >> >> >> wrote: > >> >> >> > Hi Daan, > >> >> >> > > >> >> >> > We started to work on implementing unit tests and checked in > some > >> >> >> > of > >> >> >> > them, > >> >> >> > so please review them and let us know if there is any > >> >> >> > comments/suggestions. > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > https://github.com/alexoughsg/Albatross/commit/fca94db36054ee35f5da85c69c2471430e0eebfe > >> >> >> > > >> >> >> > Thanks > >> >> >> > Alex Ough > >> >> >> > > >> >> >> > > >> >> >> > On Thu, Feb 13, 2014 at 3:15 PM, Alex Ough < > alex.o...@sungard.com> > >> >> >> > wrote: > >> >> >> >> > >> >> >> >> We're just about to add unit tests, so I'll add them once > >> >>
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
What branch needs to be based to create a review request of this implementation that will be included in 4.4 release? 4.3 or 4.3-forward? Thanks Alex Ough On Tue, Mar 4, 2014 at 1:25 PM, Alex Ough wrote: > Yes, by 3/7. > > Thanks > Alex Ough > > > On Tue, Mar 4, 2014 at 12:30 PM, Daan Hoogland wrote: > >> This week? should be ok. end of next week is too late! >> >> On Tue, Mar 4, 2014 at 6:17 PM, Alex Ough wrote: >> > Do you think it is ok to wrap up the implementation until the end of >> this >> > week to make this included in 4.4 release? >> > >> > Thanks >> > Alex Ough >> > >> > >> > On Sun, Mar 2, 2014 at 2:10 PM, Daan Hoogland >> > wrote: >> >> >> >> two weeks from now, but under debate. Let's go with that, though. >> >> >> >> On Sun, Mar 2, 2014 at 1:17 AM, Alex Ough >> wrote: >> >> > Can you tell me when the deadline for 4.4 is? >> >> > >> >> > Thanks >> >> > Alex Ough >> >> > >> >> > >> >> > On Sat, Mar 1, 2014 at 2:26 PM, Daan Hoogland < >> daan.hoogl...@gmail.com> >> >> > wrote: >> >> >> >> >> >> Ha Alex, >> >> >> >> >> >> branch 4.3 is going to be a problem. We will be able to release this >> >> >> with 4.4 if we hurry. >> >> >> >> >> >> On Fri, Feb 28, 2014 at 4:01 PM, Alex Ough >> >> >> wrote: >> >> >> > Hi Daan, >> >> >> > >> >> >> > 1. Yes, we'll update the request once all the unit tests are >> >> >> > completed. >> >> >> > 2. And yes, it is a branch from 4.3 >> >> >> > 3. I'm using the global parameter, called >> >> >> > 'region.auto.generation.interval' >> >> >> > and the value is 0 (millisec) by default. >> >> >> > If you set the value more than 0, it will automatically >> >> >> > create/update/remove domains/accounts/users with that interval, >> >> >> > which will trigger the scheduled full scan to sync the newly >> >> >> > changed >> >> >> > resources. >> >> >> > >> >> >> > Let me know if you need more info. >> >> >> > Thanks >> >> >> > Alex Ough >> >> >> > >> >> >> > >> >> >> > On Thu, Feb 27, 2014 at 3:50 PM, Daan Hoogland >> >> >> > >> >> >> > wrote: >> >> >> >> >> >> >> >> I see, great. >> >> >> >> >> >> >> >> I will find a moment to apply it and test. It looks good at first >> >> >> >> sight. >> >> >> >> >> >> >> >> Don't forget adding the license header to the new files. Are you >> >> >> >> updating the review request with these tests? >> >> >> >> >> >> >> >> You are using a local branch to implement this on, >> >> >> >> 'alexoughsg/Albatross' is a fork of cloudstack, is it? >> >> >> >> Let's discuss merge back and testing. >> >> >> >> Do you have a integration test script in Marvin and/or scenarios >> >> >> >> that >> >> >> >> will ensure continued functionality for the sync feature? >> >> >> >> >> >> >> >> >> >> >> >> thanks, >> >> >> >> Daan >> >> >> >> >> >> >> >> On Thu, Feb 27, 2014 at 9:19 PM, Alex Ough < >> alex.o...@sungard.com> >> >> >> >> wrote: >> >> >> >> > Hi Daan, >> >> >> >> > >> >> >> >> > We started to work on implementing unit tests and checked in >> some >> >> >> >> > of >> >> >> >> > them, >> >> >> >> > so please review them and let us know if there is any >> >> >> >> > comments/suggestions. >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> &g
Unit test failing in master
All, Anyone who can fix this failure in "cloudstack/plugins/alert-handlers/snmp-alerts"? Failed tests: appendTest(org.apache.cloudstack.alert.snmp.SnmpTrapAppenderTest): error snmpHelper list size not as expected expected:<0> but was:<2> Thanks Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17790/ --- (Updated March 8, 2014, 6:29 p.m.) Review request for cloudstack. Changes --- Unit tests are added. Repository: cloudstack-git Description --- Currently, under the environment of cloudstack with multiple regions, each region has its own management server running with a separate database, which will cause data discrepancies when users create/update/delete domain/account/user data independently in each management server. So to support multiple regions and provide one point of entry for each customer, this implementation duplicates domain/account/user information of customers in one region to all of the regions independently whenever there is any change. https://issues.apache.org/jira/browse/CLOUDSTACK-4992 https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions Diffs (updated) - api/src/com/cloud/domain/Domain.java c4755d7 api/src/com/cloud/event/EventTypes.java ec54ea1 api/src/com/cloud/user/Account.java 99ef954 api/src/com/cloud/user/AccountService.java a9be292 api/src/com/cloud/user/User.java 36e9028 api/src/com/cloud/user/UserAccount.java c5a0637 api/src/org/apache/cloudstack/api/ApiConstants.java 14df653 api/src/org/apache/cloudstack/api/BaseCmd.java e869ddf api/src/org/apache/cloudstack/api/response/AccountResponse.java b7d30ca api/src/org/apache/cloudstack/api/response/DomainResponse.java 0c0281e api/src/org/apache/cloudstack/api/response/UserResponse.java 40e1561 api/src/org/apache/cloudstack/region/Region.java df64e44 engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml 08efb83 engine/schema/src/com/cloud/domain/DomainVO.java f6494b3 engine/schema/src/com/cloud/user/AccountVO.java fb1b58a engine/schema/src/com/cloud/user/UserAccountVO.java cef9239 engine/schema/src/com/cloud/user/UserVO.java 68879f6 engine/schema/src/org/apache/cloudstack/region/RegionVO.java 608bd2b framework/db/src/com/cloud/utils/db/Attribute.java 82c2bdb framework/db/src/com/cloud/utils/db/GenericDao.java cb401cd framework/db/src/com/cloud/utils/db/GenericDaoBase.java cecea84 framework/db/src/com/cloud/utils/db/SqlGenerator.java befe34b plugins/alert-handlers/snmp-alerts/test/org/apache/cloudstack/alert/snmp/SnmpTrapAppenderTest.java 7db3073 plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java fa7be58 server/src/com/cloud/api/ApiDispatcher.java 5bdefe7 server/src/com/cloud/api/ApiResponseHelper.java 81bfe21 server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java 5e087fd server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 9724d99 server/src/com/cloud/projects/ProjectManagerImpl.java 5a0ed1c server/src/com/cloud/server/StatsCollector.java 548587c server/src/com/cloud/user/AccountManager.java 983caf1 server/src/com/cloud/user/AccountManagerImpl.java 186cfb2 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java b2a478e server/test/com/cloud/user/MockAccountManagerImpl.java 62e7fc8 server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb setup/db/db/schema-430to440.sql acc7e21 Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
/service/AccountService.java PRE-CREATION server/src/com/cloud/region/service/BaseService.java PRE-CREATION server/src/com/cloud/region/service/DomainFullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/DomainService.java PRE-CREATION server/src/com/cloud/region/service/FullScanner.java PRE-CREATION server/src/com/cloud/region/service/FullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/LocalAccountManager.java PRE-CREATION server/src/com/cloud/region/service/LocalDomainManager.java PRE-CREATION server/src/com/cloud/region/service/LocalUserManager.java PRE-CREATION server/src/com/cloud/region/service/RemoteAccountEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteDomainEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteUserEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserFullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserService.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/AutoGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/LocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/server/StatsCollector.java 548587c server/src/com/cloud/user/AccountManager.java 983caf1 server/src/com/cloud/user/AccountManagerImpl.java 186cfb2 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java b2a478e server/test/com/cloud/region/api_interface/AccountInterfaceTest.java PRE-CREATION server/test/com/cloud/region/api_interface/BaseInterfaceTest.java PRE-CREATION server/test/com/cloud/region/api_interface/DomainInterfaceTest.java PRE-CREATION server/test/com/cloud/region/api_interface/UserInterfaceTest.java PRE-CREATION server/test/com/cloud/region/service/AccountFullSyncProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/BaseServiceTest.java PRE-CREATION server/test/com/cloud/region/service/DomainFullSyncProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/RemoteAccountEventProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/RemoteDomainEventProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/RemoteUserEventProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/UserFullSyncProcessorTest.java PRE-CREATION server/test/com/cloud/region/simulator/AccountLocalGeneratorEventTest.java PRE-CREATION server/test/com/cloud/region/simulator/AccountLocalGeneratorTest.java PRE-CREATION server/test/com/cloud/region/simulator/DomainLocalGeneratorEventTest.java PRE-CREATION server/test/com/cloud/region/simulator/DomainLocalGeneratorTest.java PRE-CREATION server/test/com/cloud/region/simulator/LocalGeneratorTest.java PRE-CREATION server/test/com/cloud/region/simulator/UserLocalGeneratorEventTest.java PRE-CREATION server/test/com/cloud/region/simulator/UserLocalGeneratorTest.java PRE-CREATION server/test/com/cloud/user/MockAccountManagerImpl.java 62e7fc8 server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb setup/db/db/schema-430to440.sql acc7e21 Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
server/src/com/cloud/region/api_interface/DomainInterface.java PRE-CREATION server/src/com/cloud/region/api_interface/UserInterface.java PRE-CREATION server/src/com/cloud/region/service/AccountFullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/AccountService.java PRE-CREATION server/src/com/cloud/region/service/BaseService.java PRE-CREATION server/src/com/cloud/region/service/DomainFullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/DomainService.java PRE-CREATION server/src/com/cloud/region/service/FullScanner.java PRE-CREATION server/src/com/cloud/region/service/FullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/LocalAccountManager.java PRE-CREATION server/src/com/cloud/region/service/LocalDomainManager.java PRE-CREATION server/src/com/cloud/region/service/LocalUserManager.java PRE-CREATION server/src/com/cloud/region/service/RemoteAccountEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteDomainEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/RemoteUserEventProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserFullSyncProcessor.java PRE-CREATION server/src/com/cloud/region/service/UserService.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/AccountLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/AutoGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/DomainLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/region/simulator/LocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGenerator.java PRE-CREATION server/src/com/cloud/region/simulator/UserLocalGeneratorEvent.java PRE-CREATION server/src/com/cloud/server/StatsCollector.java 548587c server/src/com/cloud/user/AccountManager.java 983caf1 server/src/com/cloud/user/AccountManagerImpl.java 186cfb2 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java b2a478e server/test/com/cloud/region/api_interface/AccountInterfaceTest.java PRE-CREATION server/test/com/cloud/region/api_interface/BaseInterfaceTest.java PRE-CREATION server/test/com/cloud/region/api_interface/DomainInterfaceTest.java PRE-CREATION server/test/com/cloud/region/api_interface/UserInterfaceTest.java PRE-CREATION server/test/com/cloud/region/service/AccountFullSyncProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/BaseServiceTest.java PRE-CREATION server/test/com/cloud/region/service/DomainFullSyncProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/RemoteAccountEventProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/RemoteDomainEventProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/RemoteUserEventProcessorTest.java PRE-CREATION server/test/com/cloud/region/service/UserFullSyncProcessorTest.java PRE-CREATION server/test/com/cloud/region/simulator/AccountLocalGeneratorEventTest.java PRE-CREATION server/test/com/cloud/region/simulator/AccountLocalGeneratorTest.java PRE-CREATION server/test/com/cloud/region/simulator/DomainLocalGeneratorEventTest.java PRE-CREATION server/test/com/cloud/region/simulator/DomainLocalGeneratorTest.java PRE-CREATION server/test/com/cloud/region/simulator/LocalGeneratorTest.java PRE-CREATION server/test/com/cloud/region/simulator/UserLocalGeneratorEventTest.java PRE-CREATION server/test/com/cloud/region/simulator/UserLocalGeneratorTest.java PRE-CREATION server/test/com/cloud/user/MockAccountManagerImpl.java 62e7fc8 server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb setup/db/db/schema-430to440.sql acc7e21 Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Please see my reply/question in blue.
Thanks
Alex Ough
On Mon, Mar 10, 2014 at 1:25 PM, Chiradeep Vittal <
chiradeep.vit...@citrix.com> wrote:
> I haven¹t looked at it because it is huge. But preliminary scan:
>
> - there are unit tests missing for changes to core CS code
>
Unit tests for only new classes were added because I couldn't find
unit tests of the ones I modified
> - uses com.amazonaws.util.json (why?)
>
They are used to handle the json objects. Let me know if you want
me to replace it with another.
- code format does not follow coding convention ( placement of {}, camel
> case api_interface )
>
Let me work on this.
> - package namespace is com.cloud instead of org.apache for new files
>
I didn't know that. So do I need to use 'org.apache' package for
all new classes?
- no file-level comments. What does LocalAccountManager do? Why does it
> exist? Etc.
>
Let me work on this
> - No interfaces, only abstract classes. No use of @Inject. While this
> might be OK, it does make it harder to test and does not follow the rest
> of ACS conventions.
>
I don't think there is any interface, but let me know if you find
any.
Any recommendation to replace @inject?
>
> I would urge the submitter to break up the submission.
> A) the changes to CS core, with explanations / comments and tests
> B) the sync service should be an interface with concrete implementations
> in its own package
> C) more tests
>
can you give me a little specific what kind of tests you need more?
> On 3/10/14, 3:37 AM, "Daan Hoogland" wrote:
>
> >Hi everyody,
> >
> >The people at sungard have been making this change for 4.4 and I want
> >to merge/apply it this week. It is more then a screenfull and might
> >cause issue is a setup or two.
> >
> >have a look at https://reviews.apache.org/r/17790/
> >
> >a ref to ticket and fs page are in the review request.
> >
> >kind regards,
> >--
> >Daan
>
>
>
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
I worked on a couple of items, so can you give me the confirmation for the
rest items so that I can wrap this up?
I really want to include this into 4.4.
Thanks
Alex Ough
On Mon, Mar 10, 2014 at 3:41 PM, Alex Ough wrote:
> Please see my reply/question in blue.
>
> Thanks
> Alex Ough
>
>
> On Mon, Mar 10, 2014 at 1:25 PM, Chiradeep Vittal <
> chiradeep.vit...@citrix.com> wrote:
>
>> I haven¹t looked at it because it is huge. But preliminary scan:
>>
>> - there are unit tests missing for changes to core CS code
>>
> Unit tests for only new classes were added because I couldn't
> find unit tests of the ones I modified
>
>
>> - uses com.amazonaws.util.json (why?)
>>
> They are used to handle the json objects. Let me know if you
> want me to replace it with another.
>
> - code format does not follow coding convention ( placement of {}, camel
>> case api_interface )
>>
> Done
>
>
>> - package namespace is com.cloud instead of org.apache for new files
>>
> I didn't know that. So do I need to use 'org.apache' package for
> all new classes?
>
> - no file-level comments. What does LocalAccountManager do? Why does it
>> exist? Etc.
>>
> Done.
>
>
>> - No interfaces, only abstract classes. No use of @Inject. While this
>> might be OK, it does make it harder to test and does not follow the rest
>> of ACS conventions.
>>
> I don't think there is any interface, but let me know if you find
> any.
> Any recommendation to replace @inject?
>
>>
>> I would urge the submitter to break up the submission.
>> A) the changes to CS core, with explanations / comments and tests
>> B) the sync service should be an interface with concrete implementations
>> in its own package
>> C) more tests
>>
> can you give me a little specific what kind of tests you need
> more?
>
>
>
>> On 3/10/14, 3:37 AM, "Daan Hoogland" wrote:
>>
>> >Hi everyody,
>> >
>> >The people at sungard have been making this change for 4.4 and I want
>> >to merge/apply it this week. It is more then a screenfull and might
>> >cause issue is a setup or two.
>> >
>> >have a look at https://reviews.apache.org/r/17790/
>> >
>> >a ref to ticket and fs page are in the review request.
>> >
>> >kind regards,
>> >--
>> >Daan
>>
>>
>>
>
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Hi Daan,
I didn't update the patch after the couple of works because I wanted to do
it with others Chiradeep asked if any.
Let me know when you want me to.
Thanks
Alex Ough
On Tue, Mar 11, 2014 at 4:37 PM, Daan Hoogland wrote:
> I will call the merge in a moment, but won't do it immediately.
>
> Be there for me when any issues come up. If not I will revert it.
>
> @Chiradeep: did Alex answer your concerns adequately?
>
>
> kind regards,
>
> On Tue, Mar 11, 2014 at 6:43 PM, Alex Ough wrote:
> > I worked on a couple of items, so can you give me the confirmation for
> the
> > rest items so that I can wrap this up?
> > I really want to include this into 4.4.
> >
> > Thanks
> > Alex Ough
> >
> >
> > On Mon, Mar 10, 2014 at 3:41 PM, Alex Ough
> wrote:
> >
> >> Please see my reply/question in blue.
> >>
> >> Thanks
> >> Alex Ough
> >>
> >>
> >> On Mon, Mar 10, 2014 at 1:25 PM, Chiradeep Vittal <
> >> chiradeep.vit...@citrix.com> wrote:
> >>
> >>> I haven¹t looked at it because it is huge. But preliminary scan:
> >>>
> >>> - there are unit tests missing for changes to core CS code
> >>>
> >> Unit tests for only new classes were added because I couldn't
> >> find unit tests of the ones I modified
> >>
> >>
> >>> - uses com.amazonaws.util.json (why?)
> >>>
> >> They are used to handle the json objects. Let me know if you
> >> want me to replace it with another.
> >>
> >> - code format does not follow coding convention ( placement of {}, camel
> >>> case api_interface )
> >>>
> >> Done
> >>
> >>
> >>> - package namespace is com.cloud instead of org.apache for new files
> >>>
> >> I didn't know that. So do I need to use 'org.apache' package
> for
> >> all new classes?
> >>
> >> - no file-level comments. What does LocalAccountManager do? Why does it
> >>> exist? Etc.
> >>>
> >> Done.
> >>
> >>
> >>> - No interfaces, only abstract classes. No use of @Inject. While this
> >>> might be OK, it does make it harder to test and does not follow the
> rest
> >>> of ACS conventions.
> >>>
> >> I don't think there is any interface, but let me know if you
> find
> >> any.
> >> Any recommendation to replace @inject?
> >>
> >>>
> >>> I would urge the submitter to break up the submission.
> >>> A) the changes to CS core, with explanations / comments and tests
> >>> B) the sync service should be an interface with concrete
> implementations
> >>> in its own package
> >>> C) more tests
> >>>
> >> can you give me a little specific what kind of tests you need
> >> more?
> >>
> >>
> >>
> >>> On 3/10/14, 3:37 AM, "Daan Hoogland" wrote:
> >>>
> >>> >Hi everyody,
> >>> >
> >>> >The people at sungard have been making this change for 4.4 and I want
> >>> >to merge/apply it this week. It is more then a screenfull and might
> >>> >cause issue is a setup or two.
> >>> >
> >>> >have a look at https://reviews.apache.org/r/17790/
> >>> >
> >>> >a ref to ticket and fs page are in the review request.
> >>> >
> >>> >kind regards,
> >>> >--
> >>> >Daan
> >>>
> >>>
> >>>
> >>
>
>
>
> --
> Daan
>
>
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Hi Chiradeep,
Can you give me the example of the Singleton service class you mentioned?
I'm not sure if you are asking the name changes or else because those
classes are abstract classes and do not need to be singleton class.
And let me try the refactoring and ask confirmations to you, so I hope I
can get the reply soon.
Thanks
Alex Ough
On Wed, Mar 12, 2014 at 4:13 AM, Daan Hoogland wrote:
> H Alex, I considder Chiradeeps comments quite valid and serious enough
> to anticipate not making friday 14:00 CET. That would mean no merge
> before 4.4. Can you live with that?
>
> On Wed, Mar 12, 2014 at 6:40 AM, Chiradeep Vittal
> wrote:
> > Hi Alex,
> >
> > If you look at the general design of CloudStack, there are Singleton
> > service interfaces which are then implemented with real classes. This
> > facilitates easy testing by mocking the interface. In your new files
> > (BaseInterface, which by the way is NOT an interface, AccountService,
> > which is NOT a service like other CloudStack services, and so on), this
> > design paradigm is not followed. Therefore it is incongruous to the rest
> > of the code base.
> >
> > Furthermore this has been plopped right in the middle of other core
> > CloudStack code (server/src/com/cloud/region/). If you look at the
> > EventBus logic, it has been written as a plugin, thereby enabling users
> > who do not need it to disable it. This level of configuration is needed
> > and I can't find it.
> >
> > So, to summarize:
> > 1. Split it into patches. Patch A is the change to the core DAOs, db
> > logic, schema upgrade code, etc.
> > 2. Patch B is the actual sync service written as an optional plugin with
> > the package name space of org.apache.
> >
> > On 3/11/14, 3:09 PM, "Alex Ough" wrote:
> >
> >>Hi Daan,
> >>
> >>I didn't update the patch after the couple of works because I wanted to
> do
> >>it with others Chiradeep asked if any.
> >>Let me know when you want me to.
> >>
> >>Thanks
> >>Alex Ough
> >>
> >>
> >>On Tue, Mar 11, 2014 at 4:37 PM, Daan Hoogland
> >>wrote:
> >>
> >>> I will call the merge in a moment, but won't do it immediately.
> >>>
> >>> Be there for me when any issues come up. If not I will revert it.
> >>>
> >>> @Chiradeep: did Alex answer your concerns adequately?
> >>>
> >>>
> >>> kind regards,
> >>>
> >>> On Tue, Mar 11, 2014 at 6:43 PM, Alex Ough
> >>>wrote:
> >>> > I worked on a couple of items, so can you give me the confirmation
> for
> >>> the
> >>> > rest items so that I can wrap this up?
> >>> > I really want to include this into 4.4.
> >>> >
> >>> > Thanks
> >>> > Alex Ough
> >>> >
> >>> >
> >>> > On Mon, Mar 10, 2014 at 3:41 PM, Alex Ough
> >>> wrote:
> >>> >
> >>> >> Please see my reply/question in blue.
> >>> >>
> >>> >> Thanks
> >>> >> Alex Ough
> >>> >>
> >>> >>
> >>> >> On Mon, Mar 10, 2014 at 1:25 PM, Chiradeep Vittal <
> >>> >> chiradeep.vit...@citrix.com> wrote:
> >>> >>
> >>> >>> I haven¹t looked at it because it is huge. But preliminary scan:
> >>> >>>
> >>> >>> - there are unit tests missing for changes to core CS code
> >>> >>>
> >>> >> Unit tests for only new classes were added because I
> >>>couldn't
> >>> >> find unit tests of the ones I modified
> >>> >>
> >>> >>
> >>> >>> - uses com.amazonaws.util.json (why?)
> >>> >>>
> >>> >> They are used to handle the json objects. Let me know if
> you
> >>> >> want me to replace it with another.
> >>> >>
> >>> >> - code format does not follow coding convention ( placement of {},
> >>>camel
> >>> >>> case api_interface )
> >>> >>>
> >>> >> Done
> >>> >>
> >>> >>
> >>> >>> - package namespace is com.cloud instead of org.apache for new
> files
> >>> >>>
> >>> >> I didn't know that. So do I need to use 'org.ap
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17790/ --- (Updated March 12, 2014, 3:14 p.m.) Review request for cloudstack. Changes --- This is a patch A that includes changes in core classes. Repository: cloudstack-git Description --- Currently, under the environment of cloudstack with multiple regions, each region has its own management server running with a separate database, which will cause data discrepancies when users create/update/delete domain/account/user data independently in each management server. So to support multiple regions and provide one point of entry for each customer, this implementation duplicates domain/account/user information of customers in one region to all of the regions independently whenever there is any change. https://issues.apache.org/jira/browse/CLOUDSTACK-4992 https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions Diffs (updated) - api/src/com/cloud/domain/Domain.java c4755d7 api/src/com/cloud/event/EventTypes.java ec54ea1 api/src/com/cloud/user/Account.java 99ef954 api/src/com/cloud/user/AccountService.java a9be292 api/src/com/cloud/user/User.java 36e9028 api/src/com/cloud/user/UserAccount.java c5a0637 api/src/org/apache/cloudstack/api/ApiConstants.java 14df653 api/src/org/apache/cloudstack/api/BaseCmd.java e869ddf api/src/org/apache/cloudstack/api/response/AccountResponse.java b7d30ca api/src/org/apache/cloudstack/api/response/DomainResponse.java 0c0281e api/src/org/apache/cloudstack/api/response/UserResponse.java 40e1561 api/src/org/apache/cloudstack/region/Region.java df64e44 engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml 08efb83 engine/schema/src/com/cloud/domain/DomainVO.java f6494b3 engine/schema/src/com/cloud/user/AccountVO.java fb1b58a engine/schema/src/com/cloud/user/UserAccountVO.java cef9239 engine/schema/src/com/cloud/user/UserVO.java 68879f6 engine/schema/src/org/apache/cloudstack/region/RegionVO.java 608bd2b framework/db/src/com/cloud/utils/db/Attribute.java 82c2bdb framework/db/src/com/cloud/utils/db/GenericDao.java cb401cd framework/db/src/com/cloud/utils/db/GenericDaoBase.java cecea84 framework/db/src/com/cloud/utils/db/SqlGenerator.java befe34b plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java fa7be58 server/src/com/cloud/api/ApiDispatcher.java 5bdefe7 server/src/com/cloud/api/ApiResponseHelper.java 81bfe21 server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java 5e087fd server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 9724d99 server/src/com/cloud/projects/ProjectManagerImpl.java 5a0ed1c server/src/com/cloud/server/StatsCollector.java 548587c server/src/com/cloud/user/AccountManager.java 983caf1 server/src/com/cloud/user/AccountManagerImpl.java 186cfb2 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java b2a478e server/test/com/cloud/user/MockAccountManagerImpl.java 62e7fc8 server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb setup/db/db/schema-430to440.sql acc7e21 Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
I uploaded the patch A that includes only core changes, so please review it and let me know if you have any comments. Thanks Alex Ough On Wed, Mar 12, 2014 at 8:24 AM, Alex Ough wrote: > Hi Chiradeep, > > Can you give me the example of the Singleton service class you mentioned? > I'm not sure if you are asking the name changes or else because those > classes are abstract classes and do not need to be singleton class. > > And let me try the refactoring and ask confirmations to you, so I hope I > can get the reply soon. > > Thanks > Alex Ough > > > On Wed, Mar 12, 2014 at 4:13 AM, Daan Hoogland wrote: > >> H Alex, I considder Chiradeeps comments quite valid and serious enough >> to anticipate not making friday 14:00 CET. That would mean no merge >> before 4.4. Can you live with that? >> >> On Wed, Mar 12, 2014 at 6:40 AM, Chiradeep Vittal >> wrote: >> > Hi Alex, >> > >> > If you look at the general design of CloudStack, there are Singleton >> > service interfaces which are then implemented with real classes. This >> > facilitates easy testing by mocking the interface. In your new files >> > (BaseInterface, which by the way is NOT an interface, AccountService, >> > which is NOT a service like other CloudStack services, and so on), this >> > design paradigm is not followed. Therefore it is incongruous to the rest >> > of the code base. >> > >> > Furthermore this has been plopped right in the middle of other core >> > CloudStack code (server/src/com/cloud/region/). If you look at the >> > EventBus logic, it has been written as a plugin, thereby enabling users >> > who do not need it to disable it. This level of configuration is needed >> > and I can't find it. >> > >> > So, to summarize: >> > 1. Split it into patches. Patch A is the change to the core DAOs, db >> > logic, schema upgrade code, etc. >> > 2. Patch B is the actual sync service written as an optional plugin with >> > the package name space of org.apache. >> > >> > On 3/11/14, 3:09 PM, "Alex Ough" wrote: >> > >> >>Hi Daan, >> >> >> >>I didn't update the patch after the couple of works because I wanted to >> do >> >>it with others Chiradeep asked if any. >> >>Let me know when you want me to. >> >> >> >>Thanks >> >>Alex Ough >> >> >> >> >> >>On Tue, Mar 11, 2014 at 4:37 PM, Daan Hoogland >> >>wrote: >> >> >> >>> I will call the merge in a moment, but won't do it immediately. >> >>> >> >>> Be there for me when any issues come up. If not I will revert it. >> >>> >> >>> @Chiradeep: did Alex answer your concerns adequately? >> >>> >> >>> >> >>> kind regards, >> >>> >> >>> On Tue, Mar 11, 2014 at 6:43 PM, Alex Ough >> >>>wrote: >> >>> > I worked on a couple of items, so can you give me the confirmation >> for >> >>> the >> >>> > rest items so that I can wrap this up? >> >>> > I really want to include this into 4.4. >> >>> > >> >>> > Thanks >> >>> > Alex Ough >> >>> > >> >>> > >> >>> > On Mon, Mar 10, 2014 at 3:41 PM, Alex Ough >> >>> wrote: >> >>> > >> >>> >> Please see my reply/question in blue. >> >>> >> >> >>> >> Thanks >> >>> >> Alex Ough >> >>> >> >> >>> >> >> >>> >> On Mon, Mar 10, 2014 at 1:25 PM, Chiradeep Vittal < >> >>> >> chiradeep.vit...@citrix.com> wrote: >> >>> >> >> >>> >>> I haven¹t looked at it because it is huge. But preliminary scan: >> >>> >>> >> >>> >>> - there are unit tests missing for changes to core CS code >> >>> >>> >> >>> >> Unit tests for only new classes were added because I >> >>>couldn't >> >>> >> find unit tests of the ones I modified >> >>> >> >> >>> >> >> >>> >>> - uses com.amazonaws.util.json (why?) >> >>> >>> >> >>> >> They are used to handle the json objects. Let me know i
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
server/test/org/apache/cloudstack/region/multi/api/DomainInterfaceTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/api/UserInterfaceTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/service/AccountFullSyncProcessorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/service/BaseServiceTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/service/DomainFullSyncProcessorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/service/RemoteAccountEventProcessorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/service/RemoteDomainEventProcessorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/service/RemoteUserEventProcessorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/service/UserFullSyncProcessorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/AccountLocalGeneratorEventTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/AccountLocalGeneratorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/DomainLocalGeneratorEventTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/DomainLocalGeneratorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/LocalGeneratorTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/UserLocalGeneratorEventTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/UserLocalGeneratorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
And I also uploaded the patch B that includes new implementation to support multi regions. Thanks Alex Ough On Wed, Mar 12, 2014 at 10:17 AM, Alex Ough wrote: > I uploaded the patch A that includes only core changes, so please review > it and let me know if you have any comments. > > Thanks > Alex Ough > > > On Wed, Mar 12, 2014 at 8:24 AM, Alex Ough wrote: > >> Hi Chiradeep, >> >> Can you give me the example of the Singleton service class you mentioned? >> I'm not sure if you are asking the name changes or else because those >> classes are abstract classes and do not need to be singleton class. >> >> And let me try the refactoring and ask confirmations to you, so I hope I >> can get the reply soon. >> >> Thanks >> Alex Ough >> >> >> On Wed, Mar 12, 2014 at 4:13 AM, Daan Hoogland >> wrote: >> >>> H Alex, I considder Chiradeeps comments quite valid and serious enough >>> to anticipate not making friday 14:00 CET. That would mean no merge >>> before 4.4. Can you live with that? >>> >>> On Wed, Mar 12, 2014 at 6:40 AM, Chiradeep Vittal >>> wrote: >>> > Hi Alex, >>> > >>> > If you look at the general design of CloudStack, there are Singleton >>> > service interfaces which are then implemented with real classes. This >>> > facilitates easy testing by mocking the interface. In your new files >>> > (BaseInterface, which by the way is NOT an interface, AccountService, >>> > which is NOT a service like other CloudStack services, and so on), this >>> > design paradigm is not followed. Therefore it is incongruous to the >>> rest >>> > of the code base. >>> > >>> > Furthermore this has been plopped right in the middle of other core >>> > CloudStack code (server/src/com/cloud/region/). If you look at the >>> > EventBus logic, it has been written as a plugin, thereby enabling users >>> > who do not need it to disable it. This level of configuration is needed >>> > and I can't find it. >>> > >>> > So, to summarize: >>> > 1. Split it into patches. Patch A is the change to the core DAOs, db >>> > logic, schema upgrade code, etc. >>> > 2. Patch B is the actual sync service written as an optional plugin >>> with >>> > the package name space of org.apache. >>> > >>> > On 3/11/14, 3:09 PM, "Alex Ough" wrote: >>> > >>> >>Hi Daan, >>> >> >>> >>I didn't update the patch after the couple of works because I wanted >>> to do >>> >>it with others Chiradeep asked if any. >>> >>Let me know when you want me to. >>> >> >>> >>Thanks >>> >>Alex Ough >>> >> >>> >> >>> >>On Tue, Mar 11, 2014 at 4:37 PM, Daan Hoogland >>> >>wrote: >>> >> >>> >>> I will call the merge in a moment, but won't do it immediately. >>> >>> >>> >>> Be there for me when any issues come up. If not I will revert it. >>> >>> >>> >>> @Chiradeep: did Alex answer your concerns adequately? >>> >>> >>> >>> >>> >>> kind regards, >>> >>> >>> >>> On Tue, Mar 11, 2014 at 6:43 PM, Alex Ough >>> >>>wrote: >>> >>> > I worked on a couple of items, so can you give me the confirmation >>> for >>> >>> the >>> >>> > rest items so that I can wrap this up? >>> >>> > I really want to include this into 4.4. >>> >>> > >>> >>> > Thanks >>> >>> > Alex Ough >>> >>> > >>> >>> > >>> >>> > On Mon, Mar 10, 2014 at 3:41 PM, Alex Ough >>> >>> wrote: >>> >>> > >>> >>> >> Please see my reply/question in blue. >>> >>> >> >>> >>> >> Thanks >>> >>> >> Alex Ough >>> >>> >> >>> >>> >> >>> >>> >> On Mon, Mar 10, 2014 at 1:25 PM, Chiradeep Vittal < >>> >>> >> chiradeep.vit...@citrix.com> wrote: >>> >>> >> >>> >>> >>> I haven¹t looked at it because it is huge. But preliminary scan: >>> >&g
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Chiradeep, Any comments on them? Thanks Alex Ough On Wed, Mar 12, 2014 at 10:58 AM, Alex Ough wrote: > And I also uploaded the patch B that includes new implementation to > support multi regions. > > Thanks > Alex Ough > > > On Wed, Mar 12, 2014 at 10:17 AM, Alex Ough wrote: > >> I uploaded the patch A that includes only core changes, so please review >> it and let me know if you have any comments. >> >> Thanks >> Alex Ough >> >> >> On Wed, Mar 12, 2014 at 8:24 AM, Alex Ough wrote: >> >>> Hi Chiradeep, >>> >>> Can you give me the example of the Singleton service class you mentioned? >>> I'm not sure if you are asking the name changes or else because those >>> classes are abstract classes and do not need to be singleton class. >>> >>> And let me try the refactoring and ask confirmations to you, so I hope I >>> can get the reply soon. >>> >>> Thanks >>> Alex Ough >>> >>> >>> On Wed, Mar 12, 2014 at 4:13 AM, Daan Hoogland >>> wrote: >>> >>>> H Alex, I considder Chiradeeps comments quite valid and serious enough >>>> to anticipate not making friday 14:00 CET. That would mean no merge >>>> before 4.4. Can you live with that? >>>> >>>> On Wed, Mar 12, 2014 at 6:40 AM, Chiradeep Vittal >>>> wrote: >>>> > Hi Alex, >>>> > >>>> > If you look at the general design of CloudStack, there are Singleton >>>> > service interfaces which are then implemented with real classes. This >>>> > facilitates easy testing by mocking the interface. In your new files >>>> > (BaseInterface, which by the way is NOT an interface, AccountService, >>>> > which is NOT a service like other CloudStack services, and so on), >>>> this >>>> > design paradigm is not followed. Therefore it is incongruous to the >>>> rest >>>> > of the code base. >>>> > >>>> > Furthermore this has been plopped right in the middle of other core >>>> > CloudStack code (server/src/com/cloud/region/). If you look at the >>>> > EventBus logic, it has been written as a plugin, thereby enabling >>>> users >>>> > who do not need it to disable it. This level of configuration is >>>> needed >>>> > and I can't find it. >>>> > >>>> > So, to summarize: >>>> > 1. Split it into patches. Patch A is the change to the core DAOs, db >>>> > logic, schema upgrade code, etc. >>>> > 2. Patch B is the actual sync service written as an optional plugin >>>> with >>>> > the package name space of org.apache. >>>> > >>>> > On 3/11/14, 3:09 PM, "Alex Ough" wrote: >>>> > >>>> >>Hi Daan, >>>> >> >>>> >>I didn't update the patch after the couple of works because I wanted >>>> to do >>>> >>it with others Chiradeep asked if any. >>>> >>Let me know when you want me to. >>>> >> >>>> >>Thanks >>>> >>Alex Ough >>>> >> >>>> >> >>>> >>On Tue, Mar 11, 2014 at 4:37 PM, Daan Hoogland >>>> >>wrote: >>>> >> >>>> >>> I will call the merge in a moment, but won't do it immediately. >>>> >>> >>>> >>> Be there for me when any issues come up. If not I will revert it. >>>> >>> >>>> >>> @Chiradeep: did Alex answer your concerns adequately? >>>> >>> >>>> >>> >>>> >>> kind regards, >>>> >>> >>>> >>> On Tue, Mar 11, 2014 at 6:43 PM, Alex Ough >>>> >>>wrote: >>>> >>> > I worked on a couple of items, so can you give me the >>>> confirmation for >>>> >>> the >>>> >>> > rest items so that I can wrap this up? >>>> >>> > I really want to include this into 4.4. >>>> >>> > >>>> >>> > Thanks >>>> >>> > Alex Ough >>>> >>> > >>>> >>> > >>>> >>> > On Mon, Mar 10, 2014 at 3:41 PM, Alex Ough >>> > >>>> >>> wrote: >>>> >
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Hi Alena, Patch B, I'm not quite familiar with java, so I have a little difficulty in following your recommendation. Can you send me an example using 'BaseInterface' and/or 'AccountInterface'? It will be very helpful and appreciated. Patch A, To reduce the number of requests to the remote regions because the syncing is always using the api requests a lot to get information of domains/accounts/users from remote regions. And let me change the concatenation into using StringBuilder. Thanks a lot for your reply. Alex Ough On Thu, Mar 13, 2014 at 2:41 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > Alex, I have some comments. > > Patch B. > > * You shouldn't make your service a part of cloud-mom-rabbitmq plugin. > Your subscribers/implementation are specific to your feature, and you need > to introduce a special plugin just for your service. > * AccountInterface and BaseInterface are still regular classes. You should > split them into Service interface /ManagerImpl or Manager interface > /ManagerImpl as Chiradeep suggested. > * Once you extract services interfaces, make sure you don't use VO objects > in methods signatures. > * You should really get a use of @Manager interface and @Inject > annotations for autowiring your managers instead of setting them up using > ComponentContext.getComponent() in each of your manager classes. > > > > Patch A. > > * AccountResponse. Why did you add domain path to the account response? > You can always retrieve it by a) get domain info from account response b) > execute listDomains&domainId to get the domain path. Try not to overload > the response with attributes that don't belong to response's first class > object. > > > Generic comments. > > * I can see that you do a lot of string concatenation this way: paramStr > += "&email=" + email + "&firstname=" + firstName + "&lastname=" + lastName > + "&accounttype=" + accountType; > I would suggest to use StringBuilder instead. > > > > > On 3/13/14, 9:33 AM, "Alex Ough" wrote: > > >Chiradeep, > > > >Any comments on them? > > > >Thanks > >Alex Ough > > > > > >On Wed, Mar 12, 2014 at 10:58 AM, Alex Ough > wrote: > > > >> And I also uploaded the patch B that includes new implementation to > >> support multi regions. > >> > >> Thanks > >> Alex Ough > >> > >> > >> On Wed, Mar 12, 2014 at 10:17 AM, Alex Ough > >>wrote: > >> > >>> I uploaded the patch A that includes only core changes, so please > >>>review > >>> it and let me know if you have any comments. > >>> > >>> Thanks > >>> Alex Ough > >>> > >>> > >>> On Wed, Mar 12, 2014 at 8:24 AM, Alex Ough > >>>wrote: > >>> > >>>> Hi Chiradeep, > >>>> > >>>> Can you give me the example of the Singleton service class you > >>>>mentioned? > >>>> I'm not sure if you are asking the name changes or else because those > >>>> classes are abstract classes and do not need to be singleton class. > >>>> > >>>> And let me try the refactoring and ask confirmations to you, so I > >>>>hope I > >>>> can get the reply soon. > >>>> > >>>> Thanks > >>>> Alex Ough > >>>> > >>>> > >>>> On Wed, Mar 12, 2014 at 4:13 AM, Daan Hoogland > >>>>wrote: > >>>> > >>>>> H Alex, I considder Chiradeeps comments quite valid and serious > >>>>>enough > >>>>> to anticipate not making friday 14:00 CET. That would mean no merge > >>>>> before 4.4. Can you live with that? > >>>>> > >>>>> On Wed, Mar 12, 2014 at 6:40 AM, Chiradeep Vittal > >>>>> wrote: > >>>>> > Hi Alex, > >>>>> > > >>>>> > If you look at the general design of CloudStack, there are > >>>>>Singleton > >>>>> > service interfaces which are then implemented with real classes. > >>>>>This > >>>>> > facilitates easy testing by mocking the interface. In your new > >>>>>files > >>>>> > (BaseInterface, which by the way is NOT an interface, > >>>>>AccountService, > >>>>> > which is NOT a service like ot
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Patch B,
1. The reason why I use abstract classes instead of interfaces is because
there are some basic methods that are used among the inherited classes, so
I'm not sure why it has to be an interface.
2. These are the abstract base classes along with their inherited classes
and they are grouped by their behavior.
- BaseInterface - UserInterface, AccountInterface, DomainInterface
- BaseService - UserService, AccountService, DomainService
- FullSyncProcessor - UserFullSyncProcessor, AccountFullSyncProcessor,
DomainFullSyncProcessor
- RemoteEventProcessor - RemoteUserEventProcessor,
RemoteAccountEventProcessor, RemoteDomainEventProcessor
=> Do you recommend to refactor them into defining interfaces and
creating one class implementing all methods related with each user, account
and domain?
3. Let me work on changing names.
Thanks
Alex Ough
On Thu, Mar 13, 2014 at 3:14 PM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:
> Alex, see inline.
>
> -Alena.
>
> From: Alex Ough
> Date: Thursday, March 13, 2014 at 1:00 PM
> To: Alena Prokharchyk
> Cc: "dev@cloudstack.apache.org" , Chiradeep
> Vittal
>
> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
> Among Multiple Regions
>
> Hi Alena,
>
> Patch B,
> I'm not quite familiar with java, so I have a little difficulty in
> following your recommendation.
> Can you send me an example using 'BaseInterface' and/or 'AccountInterface'?
>
>
>- What is called an interface in java:
>http://docs.oracle.com/javase/tutorial/java/concepts/interface.html.
>Its a place where all your methods are defined w/o actual implementation.
>- Look at any of cloudStack Managers implementation. Take for example:
>
>
>1. AcccountManagerImpl.java - class where all the methods are
>implemented. Part of the server package
>2. AccountManagerImpl implements 2 interfaces - AccountManager and
>AccountService. If you want any of your methods to be used by
>plugins/services, define them in Service interface. If all of them are
>meant to be used just inside your plugin/or cloudstack core/server - define
>them in Manager interface.
>3. I would suggest you rename your classes/interfaces by adding your
>feature specific keyword to the name. CloudStack already has AccountService
>interface. And BaseInterface name is way too generic. Plus you shouldn't
>really put an "Interface" to the name.
>
>
>
>It will be very helpful and appreciated.
>
> Patch A,
> To reduce the number of requests to the remote regions
> because the syncing is always using the api requests a lot to get
> information of domains/accounts/users from remote regions.
>
>
>
>- you can't ,modify cloudStack core/server code only to fix problem
>that is specific to your plugin/service. The solution for your case will be
>- create in memory data structure where you keep account/domain
>information. Account->domain relationship don't change along account
> lifecycle, as well as the domain path doesn't change for the domain once
>its created. And get the domain path from there.
>
>
>
> And let me change the concatenation into using StringBuilder.
>
> Thanks a lot for your reply.
> Alex Ough
>
>
> On Thu, Mar 13, 2014 at 2:41 PM, Alena Prokharchyk <
> alena.prokharc...@citrix.com> wrote:
>
>> Alex, I have some comments.
>>
>> Patch B.
>>
>> * You shouldn't make your service a part of cloud-mom-rabbitmq plugin.
>> Your subscribers/implementation are specific to your feature, and you need
>> to introduce a special plugin just for your service.
>> * AccountInterface and BaseInterface are still regular classes. You should
>> split them into Service interface /ManagerImpl or Manager interface
>> /ManagerImpl as Chiradeep suggested.
>> * Once you extract services interfaces, make sure you don't use VO objects
>> in methods signatures.
>> * You should really get a use of @Manager interface and @Inject
>> annotations for autowiring your managers instead of setting them up using
>> ComponentContext.getComponent() in each of your manager classes.
>>
>>
>>
>> Patch A.
>>
>> * AccountResponse. Why did you add domain path to the account response?
>> You can always retrieve it by a) get domain info from account response b)
>> execute listDomains&domainId to get the domain path. Try not to overload
>> the response with attributes that don't belong to response's first class
>> object.
>>
>>
>> Generic comments.
>>
>> * I
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
They are not called outside and only called from 'subscriber' classes and
FullScanner class.
Do you think these name changes are ok?
- BaseInterface - UserInterface, AccountInterface, DomainInterface
=> APICaller - APIUserCaller, APIAccountCaller, APIDomainCaller
- BaseService - UserService, AccountService, DomainService
=> RemoteResourceProcessor - RemoteUserProcessor,
RemoteAccountProcessor, RemoteDomainProcessor
- FullSyncProcessor - UserFullSyncProcessor, AccountFullSyncProcessor,
DomainFullSyncProcessor
=> no change
- RemoteEventProcessor - RemoteUserEventProcessor,
RemoteAccountEventProcessor, RemoteDomainEventProcessor
=> no change
Thanks
Alex Ough
On Thu, Mar 13, 2014 at 3:58 PM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:
> You extract stuff into interfaces when the methods are meant to be called
> from different classes/Managers. Do you implement to add APIs for your
> plugins? Can your plugin be used by any other CS manager - RegionManager
> for example? If the answer is yes, then you would need an interface. If
> not, abstract class is fine, just remove Interface/Service from the name.
> Also rename your classes to reflect the purpose they are developed for;
> account/user/domain is way too generic and already used in other CS
> packages.
>
>
>
>
>
> On 3/13/14, 1:50 PM, "Alex Ough" wrote:
>
> >Patch B,
> >
> >1. The reason why I use abstract classes instead of interfaces is because
> >there are some basic methods that are used among the inherited classes, so
> >I'm not sure why it has to be an interface.
> >
> >2. These are the abstract base classes along with their inherited classes
> >and they are grouped by their behavior.
> >- BaseInterface - UserInterface, AccountInterface, DomainInterface
> >- BaseService - UserService, AccountService, DomainService
> >- FullSyncProcessor - UserFullSyncProcessor, AccountFullSyncProcessor,
> >DomainFullSyncProcessor
> >- RemoteEventProcessor - RemoteUserEventProcessor,
> >RemoteAccountEventProcessor, RemoteDomainEventProcessor
> >
> > => Do you recommend to refactor them into defining interfaces and
> >creating one class implementing all methods related with each user,
> >account
> >and domain?
> >
> >3. Let me work on changing names.
> >
> >Thanks
> >Alex Ough
> >
> >
> >
> >
> >On Thu, Mar 13, 2014 at 3:14 PM, Alena Prokharchyk <
> >alena.prokharc...@citrix.com> wrote:
> >
> >> Alex, see inline.
> >>
> >> -Alena.
> >>
> >> From: Alex Ough
> >> Date: Thursday, March 13, 2014 at 1:00 PM
> >> To: Alena Prokharchyk
> >> Cc: "dev@cloudstack.apache.org" , Chiradeep
> >> Vittal
> >>
> >> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
> >> Among Multiple Regions
> >>
> >> Hi Alena,
> >>
> >> Patch B,
> >> I'm not quite familiar with java, so I have a little difficulty in
> >> following your recommendation.
> >> Can you send me an example using 'BaseInterface' and/or
> >>'AccountInterface'?
> >>
> >>
> >>- What is called an interface in java:
> >>http://docs.oracle.com/javase/tutorial/java/concepts/interface.html.
> >>Its a place where all your methods are defined w/o actual
> >>implementation.
> >>- Look at any of cloudStack Managers implementation. Take for
> >>example:
> >>
> >>
> >>1. AcccountManagerImpl.java - class where all the methods are
> >>implemented. Part of the server package
> >>2. AccountManagerImpl implements 2 interfaces - AccountManager and
> >>AccountService. If you want any of your methods to be used by
> >>plugins/services, define them in Service interface. If all of them
> >>are
> >>meant to be used just inside your plugin/or cloudstack core/server -
> >>define
> >>them in Manager interface.
> >>3. I would suggest you rename your classes/interfaces by adding your
> >>feature specific keyword to the name. CloudStack already has
> >>AccountService
> >>interface. And BaseInterface name is way too generic. Plus you
> >>shouldn't
> >>really put an "Interface" to the name.
> >>
> >>
> >>
> >>It will be very helpful and appreciated.
> >>
> >> Patch A,
> >> To reduce the number of re
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
What I think based on your comments are
1. Well, I have a different thought. I'd rather have separated classes
instead of having a class with lots of methods, which makes the maintenance
easier. And as you show as an example, it is possible to create a method
and merge them, but I think it is better to provide separate methods that
are exposed outside so that the callers can know what to call with ease.
2. Let me look at that.
3. I'm not sure why you think they are adapters, but let me look at that
class.
4. OK, let me work on this.
5. The urls are stored in the region table along with username and
password. Please see 'RegionVO' under
'engine/schema/src/org/apache/cloudstack/region'.
Thanks
Alex Ough
On Thu, Mar 13, 2014 at 5:49 PM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:
> Alex,
>
> There are so many classes, and it makes it hard to see/review the feature.
> Can you come up with some sort of visual diagram, so its easier to see
> which component is responsible for what task, and how they interact with
> each other.
>
> My suggestions:
>
> 1) I think it would make sense to merge all the classes doing utils tasks
> (forming and sending Apis to CS) - UserInterface, AccountInterface,
> DomainInterface - to a single util class. They do return generic types
> anyway - JsonArray/JsonObject, and they do perform a generic util task -
> forming and sending the request to the CS. I would merge them all with the
> BaseInterface and name it with the name indicating that this class is
> responsible for sending API calls against CS. And this would be your util
> class.
>
>
> You should also come up with some generic method that forms the requests
> to CS APIs to make the code cleaner.
>
> For example, look at these 2:
>
>
> public JSONObject lockUser(String userId) throws Exception {
> String paramStr = "command=lockUser&id=" + userId +
> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(), "UTF-8");
> return sendApacheGet(paramStr);
> }
>
>
> public JSONObject disableUser(String userId) throws Exception {
>
> String paramStr = "command=disableUser&id=" + userId +
> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(), "UTF-8");
> return sendApacheGet(paramStr);
> }
>
>
> You repeat appending json and session key in all of them. Why not create
> some generic method where you pass a) commandName 2) map of parameters,
> and make that method return JsonObject/JsonArray?
>
>
> 2) I would suggest you utilize Spring framework in your code and auto wire
> all the dependencies by using @Inject rather than locating them with the
> components lifecycle. Please refer to Apache Wiki:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Using+Spring+in+Clou
> dStack
>
>
>
>
> 3) AccountFullSyncProcessor/DomainFullSyncProcessor/.
> These looks like adaptors to me. Have you looked at how AdapterBase is
> utilized in CS? You should take a look at it.
>
>
> 4) I see that you have a folder called "simulator". Does this folder
> contain Test classes used by some kind of simulator? Or would they be used
> in production? If its just for testing, please rename them by putting the
> word "Simulator" in the name. Look at how other simulator classes are
> named in CS (SimulatorImageStoreLifeCycleImpl.java for example)
>
> 5) In the code, I haven't noticed where you store/read the end point URL
> to the CS Management server - the server address you are gonna send your
> requests to. If for example, you have MS1 and MS2, will your plugin from
> MS1 ever sends a request to MS2? And if it sends the request only to MS1,
> what ip address it uses?
>
>
>
> -Alena.
>
> On 3/13/14, 2:58 PM, "Alex Ough" wrote:
>
> >They are not called outside and only called from 'subscriber' classes and
> >FullScanner class.
> >
> >Do you think these name changes are ok?
> >
> >- BaseInterface - UserInterface, AccountInterface, DomainInterface
> > => APICaller - APIUserCaller, APIAccountCaller, APIDomainCaller
> >
> >- BaseService - UserService, AccountService, DomainService
> > => RemoteResourceProcessor - RemoteUserProcessor,
> >RemoteAccountProcessor, RemoteDomainProcessor
> >
> >- FullSyncProcessor - UserFullSyncProcessor, AccountFullSyncProcessor,
> >DomainFullSyncProcessor
> > => no change
> >
> >- RemoteEventProcessor - RemoteUserEventProcessor,
> >RemoteAccountEventProcessor, RemoteDomainEventProcessor
> > => no change
> >
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Hi Alena,
It looks like this was not included in version 4.4.
Anyway, do you have any comments on my reply?
Thanks
Alex Ough
On Thu, Mar 13, 2014 at 9:48 PM, Alex Ough wrote:
> What I think based on your comments are
>
> 1. Well, I have a different thought. I'd rather have separated classes
> instead of having a class with lots of methods, which makes the maintenance
> easier. And as you show as an example, it is possible to create a method
> and merge them, but I think it is better to provide separate methods that
> are exposed outside so that the callers can know what to call with ease.
>
> 2. Let me look at that.
>
> 3. I'm not sure why you think they are adapters, but let me look at that
> class.
>
> 4. OK, let me work on this.
>
> 5. The urls are stored in the region table along with username and
> password. Please see 'RegionVO' under
> 'engine/schema/src/org/apache/cloudstack/region'.
>
> Thanks
> Alex Ough
>
>
> On Thu, Mar 13, 2014 at 5:49 PM, Alena Prokharchyk <
> alena.prokharc...@citrix.com> wrote:
>
>> Alex,
>>
>> There are so many classes, and it makes it hard to see/review the feature.
>> Can you come up with some sort of visual diagram, so its easier to see
>> which component is responsible for what task, and how they interact with
>> each other.
>>
>> My suggestions:
>>
>> 1) I think it would make sense to merge all the classes doing utils tasks
>> (forming and sending Apis to CS) - UserInterface, AccountInterface,
>> DomainInterface - to a single util class. They do return generic types
>> anyway - JsonArray/JsonObject, and they do perform a generic util task -
>> forming and sending the request to the CS. I would merge them all with the
>> BaseInterface and name it with the name indicating that this class is
>> responsible for sending API calls against CS. And this would be your util
>> class.
>>
>>
>> You should also come up with some generic method that forms the requests
>> to CS APIs to make the code cleaner.
>>
>> For example, look at these 2:
>>
>>
>> public JSONObject lockUser(String userId) throws Exception {
>> String paramStr = "command=lockUser&id=" + userId +
>> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(),
>> "UTF-8");
>> return sendApacheGet(paramStr);
>> }
>>
>>
>> public JSONObject disableUser(String userId) throws Exception {
>>
>> String paramStr = "command=disableUser&id=" + userId +
>> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(),
>> "UTF-8");
>> return sendApacheGet(paramStr);
>> }
>>
>>
>> You repeat appending json and session key in all of them. Why not create
>> some generic method where you pass a) commandName 2) map of parameters,
>> and make that method return JsonObject/JsonArray?
>>
>>
>> 2) I would suggest you utilize Spring framework in your code and auto wire
>> all the dependencies by using @Inject rather than locating them with the
>> components lifecycle. Please refer to Apache Wiki:
>>
>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Using+Spring+in+Clou
>> dStack
>>
>>
>>
>>
>> 3) AccountFullSyncProcessor/DomainFullSyncProcessor/.
>> These looks like adaptors to me. Have you looked at how AdapterBase is
>> utilized in CS? You should take a look at it.
>>
>>
>> 4) I see that you have a folder called "simulator". Does this folder
>> contain Test classes used by some kind of simulator? Or would they be used
>> in production? If its just for testing, please rename them by putting the
>> word "Simulator" in the name. Look at how other simulator classes are
>> named in CS (SimulatorImageStoreLifeCycleImpl.java for example)
>>
>> 5) In the code, I haven't noticed where you store/read the end point URL
>> to the CS Management server - the server address you are gonna send your
>> requests to. If for example, you have MS1 and MS2, will your plugin from
>> MS1 ever sends a request to MS2? And if it sends the request only to MS1,
>> what ip address it uses?
>>
>>
>>
>> -Alena.
>>
>> On 3/13/14, 2:58 PM, "Alex Ough" wrote:
>>
>> >They are not called outside and only called from 'subscriber' classes and
>> >FullScanner class.
>> >
>> >Do you think these name changes are ok?
>> >
>> >- BaseInterface - UserInterface, A
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
/cloudstack/region/multi/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION server/test/org/apache/cloudstack/region/multi/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION setup/db/db/schema-430to440.sql acc7e21 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Alena,
It took a little time to update the patch because I had a vacation last
week.
Now I uploaded the updated patch for review with below status, so please
check them and let me know what you think.
I hope it to be merged soon to wrap this up asap.
1. no change with waiting for comments on my recommendation.
2. The two things to turn on the sync-up among multiple regions is to
change the class name of "eventNotificationBus" into "MultiRegionEventBus"
from "RabbitMQEventBus" as below and change the value of
'region.full.scan.interval' in Global Settings. And the new classes are
never used unless the feature is turned on, so I'm not sure if auto wiring
is necessary here and Chiradeep asked not to use @inject in his initial
review, so I removed them.
3. They are not adaptors, but inherited classes to process
domain/account/user objects separately.
4. Done
5. Same
6. I removed 'domain path' from AccountResponse & UserResponse.
Thanks
Alex Ough
On Thu, Mar 13, 2014 at 9:48 PM, Alex Ough wrote:
> What I think based on your comments are
>
> 1. Well, I have a different thought. I'd rather have separated classes
> instead of having a class with lots of methods, which makes the maintenance
> easier. And as you show as an example, it is possible to create a method
> and merge them, but I think it is better to provide separate methods that
> are exposed outside so that the callers can know what to call with ease.
>
> 2. Let me look at that.
>
> 3. I'm not sure why you think they are adapters, but let me look at that
> class.
>
> 4. OK, let me work on this.
>
> 5. The urls are stored in the region table along with username and
> password. Please see 'RegionVO' under
> 'engine/schema/src/org/apache/cloudstack/region'.
>
> Thanks
> Alex Ough
>
>
> On Thu, Mar 13, 2014 at 5:49 PM, Alena Prokharchyk <
> alena.prokharc...@citrix.com> wrote:
>
>> Alex,
>>
>> There are so many classes, and it makes it hard to see/review the feature.
>> Can you come up with some sort of visual diagram, so its easier to see
>> which component is responsible for what task, and how they interact with
>> each other.
>>
>> My suggestions:
>>
>> 1) I think it would make sense to merge all the classes doing utils tasks
>> (forming and sending Apis to CS) - UserInterface, AccountInterface,
>> DomainInterface - to a single util class. They do return generic types
>> anyway - JsonArray/JsonObject, and they do perform a generic util task -
>> forming and sending the request to the CS. I would merge them all with the
>> BaseInterface and name it with the name indicating that this class is
>> responsible for sending API calls against CS. And this would be your util
>> class.
>>
>>
>> You should also come up with some generic method that forms the requests
>> to CS APIs to make the code cleaner.
>>
>> For example, look at these 2:
>>
>>
>> public JSONObject lockUser(String userId) throws Exception {
>> String paramStr = "command=lockUser&id=" + userId +
>> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(),
>> "UTF-8");
>> return sendApacheGet(paramStr);
>> }
>>
>>
>> public JSONObject disableUser(String userId) throws Exception {
>>
>> String paramStr = "command=disableUser&id=" + userId +
>> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(),
>> "UTF-8");
>> return sendApacheGet(paramStr);
>> }
>>
>>
>> You repeat appending json and session key in all of them. Why not create
>> some generic method where you pass a) commandName 2) map of parameters,
>> and make that method return JsonObject/JsonArray?
>>
>>
>> 2) I would suggest you utilize Spring framework in your code and auto wire
>> all the dependencies by using @Inject rather than locating them with the
>> components lifecycle. Please refer to Apache Wiki:
>>
>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Using+Spring+in+Clou
>> dStack
>>
>>
>>
>>
>> 3) AccountFullSyncProcessor/DomainFullSyncProcessor/.
>> These looks like adaptors to me. Have you looked at how AdapterBase is
>> utilized in CS? You should take a look at it.
>>
>>
>> 4) I see that you have a folder called "simulator". Does this folder
>> contain Test classes used by some kind of simulator? Or would they be used
>> in production? If its just for testing, please rename them by putting the
>> word &q
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Sure, I'm looking forward to it.
Thanks
Alex Ough
On Wed, Mar 26, 2014 at 9:39 AM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:
> Alex, I'm attending a conference today, will review the patch tomorrow.
>
> -Alena
>
> From: Alex Ough
> Date: Wednesday, March 26, 2014 at 6:35 AM
> To: Alena Prokharchyk
> Cc: "dev@cloudstack.apache.org" , Chiradeep
> Vittal
>
> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
> Among Multiple Regions
>
> Alena,
>
> It took a little time to update the patch because I had a vacation last
> week.
> Now I uploaded the updated patch for review with below status, so please
> check them and let me know what you think.
> I hope it to be merged soon to wrap this up asap.
>
> 1. no change with waiting for comments on my recommendation.
>
> 2. The two things to turn on the sync-up among multiple regions is to
> change the class name of "eventNotificationBus" into "MultiRegionEventBus"
> from "RabbitMQEventBus" as below and change the value of
> 'region.full.scan.interval' in Global Settings. And the new classes are
> never used unless the feature is turned on, so I'm not sure if auto wiring
> is necessary here and Chiradeep asked not to use @inject in his initial
> review, so I removed them.
>class="org.apache.cloudstack.mom.rabbitmq.MultiRegionEventBus">
>
> 3. They are not adaptors, but inherited classes to process
> domain/account/user objects separately.
>
> 4. Done
>
> 5. Same
>
> 6. I removed 'domain path' from AccountResponse & UserResponse.
>
> Thanks
> Alex Ough
>
>
> On Thu, Mar 13, 2014 at 9:48 PM, Alex Ough wrote:
>
>> What I think based on your comments are
>>
>> 1. Well, I have a different thought. I'd rather have separated classes
>> instead of having a class with lots of methods, which makes the maintenance
>> easier. And as you show as an example, it is possible to create a method
>> and merge them, but I think it is better to provide separate methods that
>> are exposed outside so that the callers can know what to call with ease.
>>
>> 2. Let me look at that.
>>
>> 3. I'm not sure why you think they are adapters, but let me look at
>> that class.
>>
>> 4. OK, let me work on this.
>>
>> 5. The urls are stored in the region table along with username and
>> password. Please see 'RegionVO' under
>> 'engine/schema/src/org/apache/cloudstack/region'.
>>
>> Thanks
>> Alex Ough
>>
>>
>> On Thu, Mar 13, 2014 at 5:49 PM, Alena Prokharchyk <
>> alena.prokharc...@citrix.com> wrote:
>>
>>> Alex,
>>>
>>> There are so many classes, and it makes it hard to see/review the
>>> feature.
>>> Can you come up with some sort of visual diagram, so its easier to see
>>> which component is responsible for what task, and how they interact with
>>> each other.
>>>
>>> My suggestions:
>>>
>>> 1) I think it would make sense to merge all the classes doing utils tasks
>>> (forming and sending Apis to CS) - UserInterface, AccountInterface,
>>> DomainInterface - to a single util class. They do return generic types
>>> anyway - JsonArray/JsonObject, and they do perform a generic util task -
>>> forming and sending the request to the CS. I would merge them all with
>>> the
>>> BaseInterface and name it with the name indicating that this class is
>>> responsible for sending API calls against CS. And this would be your util
>>> class.
>>>
>>>
>>> You should also come up with some generic method that forms the requests
>>> to CS APIs to make the code cleaner.
>>>
>>> For example, look at these 2:
>>>
>>>
>>> public JSONObject lockUser(String userId) throws Exception {
>>> String paramStr = "command=lockUser&id=" + userId +
>>> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(),
>>> "UTF-8");
>>> return sendApacheGet(paramStr);
>>> }
>>>
>>>
>>> public JSONObject disableUser(String userId) throws Exception {
>>>
>>> String paramStr = "command=disableUser&id=" + userId +
>>> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(),
>>> "UTF-8");
>>> return sendApacheGet(paramStr);
>>> }
>>>
>>>
>>> Yo
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Sorry, my bad. I mis-read your comment.
Thanks for pointing it out.
Alex Ough
On Wed, Mar 26, 2014 at 12:25 PM, Chiradeep Vittal <
chiradeep.vit...@citrix.com> wrote:
> I didn't say "do not use auto wiring". I said the convention is to use
> @Inject which you didn't have.
>
> From: Alena Prokharchyk
> Date: Wednesday, March 26, 2014 at 7:39 AM
> To: Alex Ough , "dev@cloudstack.apache.org" <
> dev@cloudstack.apache.org>
> Cc: Chiradeep Vittal
>
> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
> Among Multiple Regions
>
> Alex, I'm attending a conference today, will review the patch tomorrow.
>
> -Alena
>
> From: Alex Ough
> Date: Wednesday, March 26, 2014 at 6:35 AM
> To: Alena Prokharchyk
> Cc: "dev@cloudstack.apache.org" , Chiradeep
> Vittal
> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
> Among Multiple Regions
>
> Alena,
>
> It took a little time to update the patch because I had a vacation last
> week.
> Now I uploaded the updated patch for review with below status, so please
> check them and let me know what you think.
> I hope it to be merged soon to wrap this up asap.
>
> 1. no change with waiting for comments on my recommendation.
>
> 2. The two things to turn on the sync-up among multiple regions is to
> change the class name of "eventNotificationBus" into "MultiRegionEventBus"
> from "RabbitMQEventBus" as below and change the value of
> 'region.full.scan.interval' in Global Settings. And the new classes are
> never used unless the feature is turned on, so I'm not sure if auto wiring
> is necessary here and Chiradeep asked not to use @inject in his initial
> review, so I removed them.
>class="org.apache.cloudstack.mom.rabbitmq.MultiRegionEventBus">
>
> 3. They are not adaptors, but inherited classes to process
> domain/account/user objects separately.
>
> 4. Done
>
> 5. Same
>
> 6. I removed 'domain path' from AccountResponse & UserResponse.
>
> Thanks
> Alex Ough
>
>
> On Thu, Mar 13, 2014 at 9:48 PM, Alex Ough wrote:
>
>> What I think based on your comments are
>>
>> 1. Well, I have a different thought. I'd rather have separated classes
>> instead of having a class with lots of methods, which makes the maintenance
>> easier. And as you show as an example, it is possible to create a method
>> and merge them, but I think it is better to provide separate methods that
>> are exposed outside so that the callers can know what to call with ease.
>>
>> 2. Let me look at that.
>>
>> 3. I'm not sure why you think they are adapters, but let me look at
>> that class.
>>
>> 4. OK, let me work on this.
>>
>> 5. The urls are stored in the region table along with username and
>> password. Please see 'RegionVO' under
>> 'engine/schema/src/org/apache/cloudstack/region'.
>>
>> Thanks
>> Alex Ough
>>
>>
>> On Thu, Mar 13, 2014 at 5:49 PM, Alena Prokharchyk <
>> alena.prokharc...@citrix.com> wrote:
>>
>>> Alex,
>>>
>>> There are so many classes, and it makes it hard to see/review the
>>> feature.
>>> Can you come up with some sort of visual diagram, so its easier to see
>>> which component is responsible for what task, and how they interact with
>>> each other.
>>>
>>> My suggestions:
>>>
>>> 1) I think it would make sense to merge all the classes doing utils tasks
>>> (forming and sending Apis to CS) - UserInterface, AccountInterface,
>>> DomainInterface - to a single util class. They do return generic types
>>> anyway - JsonArray/JsonObject, and they do perform a generic util task -
>>> forming and sending the request to the CS. I would merge them all with
>>> the
>>> BaseInterface and name it with the name indicating that this class is
>>> responsible for sending API calls against CS. And this would be your util
>>> class.
>>>
>>>
>>> You should also come up with some generic method that forms the requests
>>> to CS APIs to make the code cleaner.
>>>
>>> For example, look at these 2:
>>>
>>>
>>> public JSONObject lockUser(String userId) throws Exception {
>>> String paramStr = "command=lockUser&id=" + userId +
>>> "&response=json&sessionkey=" + URLEncoder.encode(getSessionKey(),
>>> "UTF-8");
>>> return
How to use @Inject
All, I'm trying to change "ComponentContext.getComponent(RmapDao.class)" to using @Inject, but I can't get the Dao object even after adding a configuration file with below information. http://www.springframework.org/schema/beans"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xmlns:context="http://www.springframework.org/schema/context"; xmlns:aop="http://www.springframework.org/schema/aop"; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd"; > Can anyone show me what needs to be fixed? Thanks Alex Ough
Re: How to use @Inject
It seems like I wrote a wrong type in ComponentContext. getComponent(RmapDao.class). It should be ComponentContext.getComponent(DomainDao.class) Thanks Alex Ough On Mon, Mar 31, 2014 at 5:53 PM, Alex Ough wrote: > All, > > I'm trying to change "ComponentContext.getComponent(RmapDao.class)" to > using @Inject, but I can't get the Dao object even after adding a > configuration file with below information. > > http://www.springframework.org/schema/beans"; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >xmlns:context="http://www.springframework.org/schema/context"; >xmlns:aop="http://www.springframework.org/schema/aop"; >xsi:schemaLocation="http://www.springframework.org/schema/beans > > http://www.springframework.org/schema/beans/spring-beans-3.0.xsd > http://www.springframework.org/schema/aop > http://www.springframework.org/schema/aop/spring-aop-3.0.xsd > http://www.springframework.org/schema/context > > http://www.springframework.org/schema/context/spring-context-3.0.xsd"; > > > > > > > > > > Can anyone show me what needs to be fixed? > Thanks > Alex Ough > >
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGenerator.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEvent.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriber.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/AccountInterfaceTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/BaseInterfaceTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/DomainInterfaceTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/UserInterfaceTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/AccountFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/BaseServiceTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/DomainFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteAccountEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteDomainEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteUserEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/UserFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriberTest.java PRE-CREATION Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
All,
I updated the patches as per Alena's request.
Let me know if there is anything missing/incorrect.
Thanks
Alex Ough
On Wed, Mar 26, 2014 at 1:40 PM, Alex Ough wrote:
> Sorry, my bad. I mis-read your comment.
>
> Thanks for pointing it out.
> Alex Ough
>
>
> On Wed, Mar 26, 2014 at 12:25 PM, Chiradeep Vittal <
> chiradeep.vit...@citrix.com> wrote:
>
>> I didn't say "do not use auto wiring". I said the convention is to use
>> @Inject which you didn't have.
>>
>> From: Alena Prokharchyk
>> Date: Wednesday, March 26, 2014 at 7:39 AM
>> To: Alex Ough , "dev@cloudstack.apache.org" <
>> dev@cloudstack.apache.org>
>> Cc: Chiradeep Vittal
>>
>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>> Among Multiple Regions
>>
>> Alex, I'm attending a conference today, will review the patch tomorrow.
>>
>> -Alena
>>
>> From: Alex Ough
>> Date: Wednesday, March 26, 2014 at 6:35 AM
>> To: Alena Prokharchyk
>> Cc: "dev@cloudstack.apache.org" , Chiradeep
>> Vittal
>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>> Among Multiple Regions
>>
>> Alena,
>>
>> It took a little time to update the patch because I had a vacation last
>> week.
>> Now I uploaded the updated patch for review with below status, so please
>> check them and let me know what you think.
>> I hope it to be merged soon to wrap this up asap.
>>
>> 1. no change with waiting for comments on my recommendation.
>>
>> 2. The two things to turn on the sync-up among multiple regions is to
>> change the class name of "eventNotificationBus" into "MultiRegionEventBus"
>> from "RabbitMQEventBus" as below and change the value of
>> 'region.full.scan.interval' in Global Settings. And the new classes are
>> never used unless the feature is turned on, so I'm not sure if auto wiring
>> is necessary here and Chiradeep asked not to use @inject in his initial
>> review, so I removed them.
>> > class="org.apache.cloudstack.mom.rabbitmq.MultiRegionEventBus">
>>
>> 3. They are not adaptors, but inherited classes to process
>> domain/account/user objects separately.
>>
>> 4. Done
>>
>> 5. Same
>>
>> 6. I removed 'domain path' from AccountResponse & UserResponse.
>>
>> Thanks
>> Alex Ough
>>
>>
>> On Thu, Mar 13, 2014 at 9:48 PM, Alex Ough wrote:
>>
>>> What I think based on your comments are
>>>
>>> 1. Well, I have a different thought. I'd rather have separated classes
>>> instead of having a class with lots of methods, which makes the maintenance
>>> easier. And as you show as an example, it is possible to create a method
>>> and merge them, but I think it is better to provide separate methods that
>>> are exposed outside so that the callers can know what to call with ease.
>>>
>>> 2. Let me look at that.
>>>
>>> 3. I'm not sure why you think they are adapters, but let me look at
>>> that class.
>>>
>>> 4. OK, let me work on this.
>>>
>>> 5. The urls are stored in the region table along with username and
>>> password. Please see 'RegionVO' under
>>> 'engine/schema/src/org/apache/cloudstack/region'.
>>>
>>> Thanks
>>> Alex Ough
>>>
>>>
>>> On Thu, Mar 13, 2014 at 5:49 PM, Alena Prokharchyk <
>>> alena.prokharc...@citrix.com> wrote:
>>>
>>>> Alex,
>>>>
>>>> There are so many classes, and it makes it hard to see/review the
>>>> feature.
>>>> Can you come up with some sort of visual diagram, so its easier to see
>>>> which component is responsible for what task, and how they interact with
>>>> each other.
>>>>
>>>> My suggestions:
>>>>
>>>> 1) I think it would make sense to merge all the classes doing utils
>>>> tasks
>>>> (forming and sending Apis to CS) - UserInterface, AccountInterface,
>>>> DomainInterface - to a single util class. They do return generic types
>>>> anyway - JsonArray/JsonObject, and they do perform a generic util task -
>>>> forming and sending the request to the CS. I would merge them all with
>>>> the
>>>> BaseInterface and name it with the name indicating that this class is
>>>> re
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Forgot to mention this, but it was a great help from Darren.
Thanks again, Darren!
Alex Ough
On Thu, Apr 3, 2014 at 11:56 AM, Alex Ough wrote:
> All,
>
> I updated the patches as per Alena's request.
>
> Let me know if there is anything missing/incorrect.
> Thanks
> Alex Ough
>
>
> On Wed, Mar 26, 2014 at 1:40 PM, Alex Ough wrote:
>
>> Sorry, my bad. I mis-read your comment.
>>
>> Thanks for pointing it out.
>> Alex Ough
>>
>>
>> On Wed, Mar 26, 2014 at 12:25 PM, Chiradeep Vittal <
>> chiradeep.vit...@citrix.com> wrote:
>>
>>> I didn't say "do not use auto wiring". I said the convention is to use
>>> @Inject which you didn't have.
>>>
>>> From: Alena Prokharchyk
>>> Date: Wednesday, March 26, 2014 at 7:39 AM
>>> To: Alex Ough , "dev@cloudstack.apache.org" <
>>> dev@cloudstack.apache.org>
>>> Cc: Chiradeep Vittal
>>>
>>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>>> Among Multiple Regions
>>>
>>> Alex, I'm attending a conference today, will review the patch
>>> tomorrow.
>>>
>>> -Alena
>>>
>>> From: Alex Ough
>>> Date: Wednesday, March 26, 2014 at 6:35 AM
>>> To: Alena Prokharchyk
>>> Cc: "dev@cloudstack.apache.org" , Chiradeep
>>> Vittal
>>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>>> Among Multiple Regions
>>>
>>> Alena,
>>>
>>> It took a little time to update the patch because I had a vacation
>>> last week.
>>> Now I uploaded the updated patch for review with below status, so please
>>> check them and let me know what you think.
>>> I hope it to be merged soon to wrap this up asap.
>>>
>>> 1. no change with waiting for comments on my recommendation.
>>>
>>> 2. The two things to turn on the sync-up among multiple regions is to
>>> change the class name of "eventNotificationBus" into "MultiRegionEventBus"
>>> from "RabbitMQEventBus" as below and change the value of
>>> 'region.full.scan.interval' in Global Settings. And the new classes are
>>> never used unless the feature is turned on, so I'm not sure if auto wiring
>>> is necessary here and Chiradeep asked not to use @inject in his initial
>>> review, so I removed them.
>>> >> class="org.apache.cloudstack.mom.rabbitmq.MultiRegionEventBus">
>>>
>>> 3. They are not adaptors, but inherited classes to process
>>> domain/account/user objects separately.
>>>
>>> 4. Done
>>>
>>> 5. Same
>>>
>>> 6. I removed 'domain path' from AccountResponse & UserResponse.
>>>
>>> Thanks
>>> Alex Ough
>>>
>>>
>>> On Thu, Mar 13, 2014 at 9:48 PM, Alex Ough wrote:
>>>
>>>> What I think based on your comments are
>>>>
>>>> 1. Well, I have a different thought. I'd rather have separated
>>>> classes instead of having a class with lots of methods, which makes the
>>>> maintenance easier. And as you show as an example, it is possible to create
>>>> a method and merge them, but I think it is better to provide separate
>>>> methods that are exposed outside so that the callers can know what to call
>>>> with ease.
>>>>
>>>> 2. Let me look at that.
>>>>
>>>> 3. I'm not sure why you think they are adapters, but let me look at
>>>> that class.
>>>>
>>>> 4. OK, let me work on this.
>>>>
>>>> 5. The urls are stored in the region table along with username and
>>>> password. Please see 'RegionVO' under
>>>> 'engine/schema/src/org/apache/cloudstack/region'.
>>>>
>>>> Thanks
>>>> Alex Ough
>>>>
>>>>
>>>> On Thu, Mar 13, 2014 at 5:49 PM, Alena Prokharchyk <
>>>> alena.prokharc...@citrix.com> wrote:
>>>>
>>>>> Alex,
>>>>>
>>>>> There are so many classes, and it makes it hard to see/review the
>>>>> feature.
>>>>> Can you come up with some sort of visual diagram, so its easier to see
>>>>> which component is responsible for what task, and how they interact
>>>>> with
>>>>> each
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Well, I'm not sure about that because the help is about how to use @Inject
in the Spring framework.
On Thu, Apr 3, 2014 at 12:49 PM, Alena Prokharchyk <
alena.prokharc...@citrix.com> wrote:
> Alex, please feel free to update Wiki docs related to the questions you
> got Darren's help from. I think this info would be useful for all CS
> committers.
>
> Thank you,
> Alena.
>
> From: Alex Ough
> Date: Thursday, April 3, 2014 at 9:22 AM
> To: Chiradeep Vittal , Alena Prokharchyk <
> alena.prokharc...@citrix.com>, Darren Shepherd >
> Cc: "dev@cloudstack.apache.org"
>
> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
> Among Multiple Regions
>
> Forgot to mention this, but it was a great help from Darren.
> Thanks again, Darren!
> Alex Ough
>
>
> On Thu, Apr 3, 2014 at 11:56 AM, Alex Ough wrote:
>
>> All,
>>
>> I updated the patches as per Alena's request.
>>
>> Let me know if there is anything missing/incorrect.
>> Thanks
>> Alex Ough
>>
>>
>> On Wed, Mar 26, 2014 at 1:40 PM, Alex Ough wrote:
>>
>>> Sorry, my bad. I mis-read your comment.
>>>
>>> Thanks for pointing it out.
>>> Alex Ough
>>>
>>>
>>> On Wed, Mar 26, 2014 at 12:25 PM, Chiradeep Vittal <
>>> chiradeep.vit...@citrix.com> wrote:
>>>
>>>> I didn't say "do not use auto wiring". I said the convention is to
>>>> use @Inject which you didn't have.
>>>>
>>>> From: Alena Prokharchyk
>>>> Date: Wednesday, March 26, 2014 at 7:39 AM
>>>> To: Alex Ough , "dev@cloudstack.apache.org" <
>>>> dev@cloudstack.apache.org>
>>>> Cc: Chiradeep Vittal
>>>>
>>>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>>>> Among Multiple Regions
>>>>
>>>> Alex, I'm attending a conference today, will review the patch
>>>> tomorrow.
>>>>
>>>> -Alena
>>>>
>>>> From: Alex Ough
>>>> Date: Wednesday, March 26, 2014 at 6:35 AM
>>>> To: Alena Prokharchyk
>>>> Cc: "dev@cloudstack.apache.org" , Chiradeep
>>>> Vittal
>>>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>>>> Among Multiple Regions
>>>>
>>>> Alena,
>>>>
>>>> It took a little time to update the patch because I had a vacation
>>>> last week.
>>>> Now I uploaded the updated patch for review with below status, so
>>>> please check them and let me know what you think.
>>>> I hope it to be merged soon to wrap this up asap.
>>>>
>>>> 1. no change with waiting for comments on my recommendation.
>>>>
>>>> 2. The two things to turn on the sync-up among multiple regions is to
>>>> change the class name of "eventNotificationBus" into "MultiRegionEventBus"
>>>> from "RabbitMQEventBus" as below and change the value of
>>>> 'region.full.scan.interval' in Global Settings. And the new classes are
>>>> never used unless the feature is turned on, so I'm not sure if auto wiring
>>>> is necessary here and Chiradeep asked not to use @inject in his initial
>>>> review, so I removed them.
>>>> >>> class="org.apache.cloudstack.mom.rabbitmq.MultiRegionEventBus">
>>>>
>>>> 3. They are not adaptors, but inherited classes to process
>>>> domain/account/user objects separately.
>>>>
>>>> 4. Done
>>>>
>>>> 5. Same
>>>>
>>>> 6. I removed 'domain path' from AccountResponse & UserResponse.
>>>>
>>>> Thanks
>>>> Alex Ough
>>>>
>>>>
>>>> On Thu, Mar 13, 2014 at 9:48 PM, Alex Ough wrote:
>>>>
>>>>> What I think based on your comments are
>>>>>
>>>>> 1. Well, I have a different thought. I'd rather have separated
>>>>> classes instead of having a class with lots of methods, which makes the
>>>>> maintenance easier. And as you show as an example, it is possible to
>>>>> create
>>>>> a method and merge them, but I think it is better to provide separate
>>>>> methods that are exposed outside so that the callers can know what to call
>>>>>
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriberTest.java PRE-CREATION plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDispatcher.java 95074e2 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions
/SimulatorAccountLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriberTest.java PRE-CREATION plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDispatcher.java 95074e2 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20099/ --- Review request for cloudstack. Repository: cloudstack-git Description --- This is the review request for the core changes related with #17790 that has only the new plugin codes. Diffs - api/src/com/cloud/domain/Domain.java 365a705 api/src/com/cloud/event/EventTypes.java 39ef710 api/src/com/cloud/user/Account.java b912e51 api/src/com/cloud/user/AccountService.java 7e37b38 api/src/com/cloud/user/User.java 36e9028 api/src/com/cloud/user/UserAccount.java c5a0637 api/src/org/apache/cloudstack/api/ApiConstants.java fdb4558 api/src/org/apache/cloudstack/api/BaseCmd.java f6f21ae api/src/org/apache/cloudstack/api/command/admin/region/AddRegionCmd.java f6743ba api/src/org/apache/cloudstack/api/command/admin/region/UpdateRegionCmd.java b08cbbb api/src/org/apache/cloudstack/api/response/AccountResponse.java 2e50c51 api/src/org/apache/cloudstack/api/response/DomainResponse.java 0c0281e api/src/org/apache/cloudstack/api/response/RegionResponse.java 6c74fa6 api/src/org/apache/cloudstack/api/response/UserResponse.java 40e1561 api/src/org/apache/cloudstack/region/Region.java df64e44 api/src/org/apache/cloudstack/region/RegionService.java afefcc7 api/test/org/apache/cloudstack/api/command/test/RegionCmdTest.java 10c3d85 client/pom.xml d8dbde7 engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml 489b37d engine/schema/src/com/cloud/domain/DomainVO.java f6494b3 engine/schema/src/com/cloud/user/AccountVO.java 0f5a044 engine/schema/src/com/cloud/user/UserAccountVO.java cef9239 engine/schema/src/com/cloud/user/UserVO.java 68879f6 engine/schema/src/org/apache/cloudstack/region/RegionVO.java 608bd2b framework/db/src/com/cloud/utils/db/Attribute.java 82c2bdb framework/db/src/com/cloud/utils/db/GenericDao.java cb401cd framework/db/src/com/cloud/utils/db/GenericDaoBase.java 2052aad framework/db/src/com/cloud/utils/db/SqlGenerator.java befe34b framework/db/test/com/cloud/utils/db/GenericDaoBaseTest.java aef0c69 framework/db/test/com/cloud/utils/db/SqlGeneratorTest.java PRE-CREATION plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDispatcher.java 95074e2 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/20099/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20099/ --- (Updated April 7, 2014, 7:13 p.m.) Review request for cloudstack. Changes --- The is the patch for the core changes. Repository: cloudstack-git Description --- This is the review request for the core changes related with #17790 that has only the new plugin codes. Diffs (updated) - api/src/com/cloud/domain/Domain.java 365a705 api/src/com/cloud/event/EventTypes.java 39ef710 api/src/com/cloud/user/Account.java b912e51 api/src/com/cloud/user/AccountService.java 7e37b38 api/src/com/cloud/user/User.java 36e9028 api/src/com/cloud/user/UserAccount.java c5a0637 api/src/org/apache/cloudstack/api/ApiConstants.java fdb4558 api/src/org/apache/cloudstack/api/BaseCmd.java f6f21ae api/src/org/apache/cloudstack/api/command/admin/region/AddRegionCmd.java f6743ba api/src/org/apache/cloudstack/api/command/admin/region/UpdateRegionCmd.java b08cbbb api/src/org/apache/cloudstack/api/response/AccountResponse.java 2e50c51 api/src/org/apache/cloudstack/api/response/DomainResponse.java 0c0281e api/src/org/apache/cloudstack/api/response/RegionResponse.java 6c74fa6 api/src/org/apache/cloudstack/api/response/UserResponse.java 40e1561 api/src/org/apache/cloudstack/region/Region.java df64e44 api/src/org/apache/cloudstack/region/RegionService.java afefcc7 api/test/org/apache/cloudstack/api/command/test/RegionCmdTest.java 10c3d85 client/pom.xml d8dbde7 engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml 489b37d engine/schema/src/com/cloud/domain/DomainVO.java f6494b3 engine/schema/src/com/cloud/user/AccountVO.java 0f5a044 engine/schema/src/com/cloud/user/UserAccountVO.java cef9239 engine/schema/src/com/cloud/user/UserVO.java 68879f6 engine/schema/src/org/apache/cloudstack/region/RegionVO.java 608bd2b framework/db/src/com/cloud/utils/db/Attribute.java 82c2bdb framework/db/src/com/cloud/utils/db/GenericDao.java cb401cd framework/db/src/com/cloud/utils/db/GenericDaoBase.java 2052aad framework/db/src/com/cloud/utils/db/SqlGenerator.java befe34b framework/db/test/com/cloud/utils/db/GenericDaoBaseTest.java aef0c69 framework/db/test/com/cloud/utils/db/SqlGeneratorTest.java PRE-CREATION plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDispatcher.java 95074e2 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/20099/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions (New Plugin)
-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriberTest.java PRE-CREATION plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDispatcher.java 95074e2 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions (New Plugin)
/SimulatorAccountLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriberTest.java PRE-CREATION plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDispatcher.java 95074e2 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions (New Plugin)
/SimulatorAccountLocalGeneratorEvent.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAutoGenerator.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGenerator.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEvent.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGenerator.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGenerator.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEvent.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriber.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/AccountCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/BaseCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/DomainCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/UserCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/AccountFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/BaseServiceTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/DomainFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteAccountEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteDomainEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteUserEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/UserFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriberTest.java PRE-CREATION Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
yes, that is correct. On Mon, Apr 7, 2014 at 3:27 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > Alex, one more question. Is this patch meant to be a part of master branch > only (4.5)? I guess so as your changes affect 4.4-4.5 db upgrade path. > > Please confirm, > Alena. > > On 4/7/14, 12:13 PM, "Alex Ough" wrote: > > > > >--- > >This is an automatically generated e-mail. To reply, visit: > >https://reviews.apache.org/r/20099/ > >--- > > > >(Updated April 7, 2014, 7:13 p.m.) > > > > > >Review request for cloudstack. > > > > > >Changes > >--- > > > >The is the patch for the core changes. > > > > > >Repository: cloudstack-git > > > > > >Description > >--- > > > >This is the review request for the core changes related with #17790 that > >has only the new plugin codes. > > > > > >Diffs (updated) > >- > > > > api/src/com/cloud/domain/Domain.java 365a705 > > api/src/com/cloud/event/EventTypes.java 39ef710 > > api/src/com/cloud/user/Account.java b912e51 > > api/src/com/cloud/user/AccountService.java 7e37b38 > > api/src/com/cloud/user/User.java 36e9028 > > api/src/com/cloud/user/UserAccount.java c5a0637 > > api/src/org/apache/cloudstack/api/ApiConstants.java fdb4558 > > api/src/org/apache/cloudstack/api/BaseCmd.java f6f21ae > > > >api/src/org/apache/cloudstack/api/command/admin/region/AddRegionCmd.java > >f6743ba > > > >api/src/org/apache/cloudstack/api/command/admin/region/UpdateRegionCmd.jav > >a b08cbbb > > api/src/org/apache/cloudstack/api/response/AccountResponse.java 2e50c51 > > api/src/org/apache/cloudstack/api/response/DomainResponse.java 0c0281e > > api/src/org/apache/cloudstack/api/response/RegionResponse.java 6c74fa6 > > api/src/org/apache/cloudstack/api/response/UserResponse.java 40e1561 > > api/src/org/apache/cloudstack/region/Region.java df64e44 > > api/src/org/apache/cloudstack/region/RegionService.java afefcc7 > > api/test/org/apache/cloudstack/api/command/test/RegionCmdTest.java > >10c3d85 > > client/pom.xml d8dbde7 > > > >engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core > >-daos-context.xml 489b37d > > engine/schema/src/com/cloud/domain/DomainVO.java f6494b3 > > engine/schema/src/com/cloud/user/AccountVO.java 0f5a044 > > engine/schema/src/com/cloud/user/UserAccountVO.java cef9239 > > engine/schema/src/com/cloud/user/UserVO.java 68879f6 > > engine/schema/src/org/apache/cloudstack/region/RegionVO.java 608bd2b > > framework/db/src/com/cloud/utils/db/Attribute.java 82c2bdb > > framework/db/src/com/cloud/utils/db/GenericDao.java cb401cd > > framework/db/src/com/cloud/utils/db/GenericDaoBase.java 2052aad > > framework/db/src/com/cloud/utils/db/SqlGenerator.java befe34b > > framework/db/test/com/cloud/utils/db/GenericDaoBaseTest.java aef0c69 > > framework/db/test/com/cloud/utils/db/SqlGeneratorTest.java PRE-CREATION > > > >plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/netwo > >rk/contrail/management/MockAccountManager.java 957f708 > > plugins/pom.xml 9b391b8 > > > >server/resources/META-INF/cloudstack/core/spring-server-core-managers-cont > >ext.xml fc1c7e2 > > server/src/com/cloud/api/ApiDispatcher.java 95074e2 > > server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b > > server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b > > server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 > > server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 > > server/src/com/cloud/api/query/vo/AccountJoinVO.java 8d642ed > > server/src/com/cloud/api/query/vo/UserAccountJoinVO.java ed29284 > > server/src/com/cloud/event/ActionEventUtils.java 28e5680 > > server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 > > server/src/com/cloud/user/AccountManager.java 03bf842 > > server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 > > server/src/com/cloud/user/DomainManager.java f72b18a > > server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 > > server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 > > server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 > > server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 > > server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 > > server/test
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
Alena/Alex, I think I need to give some explanation how this works. There are 2 ways of sync. 1. The real time sync : Whenever a resource is create/updated/removed, each region gets the event of that and requests the same job to all remote regions using API interfaces, which will create/update/remove the same resource in each remote region in the real time. 2. Full scan : There can be some interruption (whatever the reason is) that makes fail the real time sync, so each region scans its own resources with an interval and compares them with those of remote regions. This is a little tricky because we need to find out whose change is later when there is any discrepancy between resources in the local & remote regions and this is why we need maintain the created/updated/removed times. 1) When a resource exists in both local (L) and remote regions (R1) - compare the 'updated' times of both - Update the local's resource using remote resource attributes only if the updated time of the remote region is later - Store the "updated" time of the remote resource (not the current time) as the local's updated time - If we store the current time as the updated time, it will cause an mis-sync because if the resource was updated in another remote region (R2) a little after it was updated in the remote region(R1), update in R2 region will be overwritten by the update in R1 because the current time is later than the updated time of R2 2) When a resource exists only in the local region - Find if there was a removal event of this resource in the remote region - and if so, compare the time when the event occurred with the created time in the local region - Remove the resource from the local only if the event time is later - Like #1, store the time the event was occurred as the 'removed' time of the removed resource in the local region 3) When a resource exists only in the remote region - Find if the resource in the local has been removed - If so, compare the removal time in the local and the created time in the remote - Create the resource in the local only if either the local was not removed or the created time of the remote is later than the removal in local - LIke #1 & #2, store the time of the created of the remote region as the 'created' time of the newly created in the local region. I hope this will help you understand how the create/updated/removed times are managed. If there is no failure in the real time sync (#1), we'll not need the full scan (#2) and we don't need to worry about the times, but there is no way to guarantee #1 not to be failed, so #2 is very important to be supported. Please let me know what you think and we can have a conference call if you want. Thanks Alex Ough On Mon, Apr 7, 2014 at 8:39 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20099/#review39753 > --- > > > > Alex, just discussed your changes to existing CS code > (Account/Domain/UserManager, VOs, Daos, GenericDaoBase), with Alex Huang. > Here are the concerns: > > > 1) there is no need to update the Removed field. Once the entry is > removed, you should purely rely on the removed not to be null, in order to > update it in the second region database. > > 2) The big issue - you can't trigger generic dao fields modification - > Modified/Removed - through the Management/Services layer. Once the field is > defined in Generic Dao Attributes, it should be populated by generic dao > only. It should never be populated by the business logic component what > your service is. > > You should remove the field updates from all the methods I've mentioned in > my previous review (for example, > > AccountManager > > public boolean disableAccount(long accountId, Date modified) throws > ConcurrentOperationException, ResourceUnavailableException;) > > and find another way of keeping track of modified attribute update for the > CS objects. > > Alex suggested a following solution: > = > * Every time user/domain/account is created/updated/removed in CS, certain > Action event is generated. There is a way to pubish this event to the event > bus, either rabbitMQ or in-memory bus. > * Instead of scanning user/domain/account tables, your plugin should > listen to the create/update/remove user/domain/account events published to > the event bus, and update the user/domain/account in all regions in the > system accordingly > * Of course, you have to ski
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
And all 4 of the recommendations Alex Hwang gave were already implemented to support the real time synchronization (#1), but like I said in the previous email, we need to support the full scan (#2) to cover any failures during the synchronization. Thanks Alex Ough On Tue, Apr 8, 2014 at 12:10 AM, Alex Ough wrote: > Alena/Alex, > > I think I need to give some explanation how this works. > > There are 2 ways of sync. > > 1. The real time sync : Whenever a resource is create/updated/removed, > each region gets the event of that and requests the same job to all remote > regions using API interfaces, which will create/update/remove the same > resource in each remote region in the real time. > > 2. Full scan : There can be some interruption (whatever the reason is) > that makes fail the real time sync, so each region scans its own resources > with an interval and compares them with those of remote regions. This is a > little tricky because we need to find out whose change is later when there > is any discrepancy between resources in the local & remote regions and this > is why we need maintain the created/updated/removed times. > > 1) When a resource exists in both local (L) and remote regions (R1) > - compare the 'updated' times of both > - Update the local's resource using remote resource attributes > only if the updated time of the remote region is later > - Store the "updated" time of the remote resource (not the current > time) as the local's updated time > - If we store the current time as the updated time, it will cause > an mis-sync > because if the resource was updated in another remote region > (R2) a little after it was updated in the remote region(R1), > update in R2 region will be overwritten by the update in R1 > because the current time is later than the updated time of R2 > > 2) When a resource exists only in the local region >- Find if there was a removal event of this resource in the remote > region >- and if so, compare the time when the event occurred with the > created time in the local region >- Remove the resource from the local only if the event time is later >- Like #1, store the time the event was occurred as the 'removed' > time of the removed resource in the local region > > 3) When a resource exists only in the remote region > - Find if the resource in the local has been removed > - If so, compare the removal time in the local and the created time > in the remote > - Create the resource in the local only if either the local was not > removed or the created time of the remote is later than the removal in local > - LIke #1 & #2, store the time of the created of the remote region > as the 'created' time of the newly created in the local region. > > I hope this will help you understand how the create/updated/removed times > are managed. > If there is no failure in the real time sync (#1), we'll not need the full > scan (#2) and we don't need to worry about the times, > but there is no way to guarantee #1 not to be failed, so #2 is very > important to be supported. > > Please let me know what you think and we can have a conference call if you > want. > Thanks > Alex Ough > > > On Mon, Apr 7, 2014 at 8:39 PM, Alena Prokharchyk < > alena.prokharc...@citrix.com> wrote: > >> >> --- >> This is an automatically generated e-mail. To reply, visit: >> https://reviews.apache.org/r/20099/#review39753 >> --- >> >> >> >> Alex, just discussed your changes to existing CS code >> (Account/Domain/UserManager, VOs, Daos, GenericDaoBase), with Alex Huang. >> Here are the concerns: >> >> >> 1) there is no need to update the Removed field. Once the entry is >> removed, you should purely rely on the removed not to be null, in order to >> update it in the second region database. >> >> 2) The big issue - you can't trigger generic dao fields modification - >> Modified/Removed - through the Management/Services layer. Once the field is >> defined in Generic Dao Attributes, it should be populated by generic dao >> only. It should never be populated by the business logic component what >> your service is. >> >> You should remove the field updates from all the methods I've mentioned >> in my previous review (for example, >> >> AccountManager >> >> public boolean disableAccount(long accountId, Date modified) throws >> ConcurrentOpe
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
Alena, It would be really nice if I had this feedback when I presented this at the beginning. Anyway, so are only these 2 things necessary to be merged into the master? 1. find another way to store the created/updated/removed datetimes of the resources 2. provide a way to disable this feature. Let me try to resolve those 2 and let you know once done, so please let me know now if there is anything else needed. Thanks Alex Ough On Tue, Apr 8, 2014 at 1:02 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > Alex, thank you for the explanation. But I think there should be > another way of saving all the "updated"/"created" info rather than updating > the UserVO/AccountVO/DomainVO fields directly. > > The one way I can think of - you can create a helper table where you > store the ref UUID->Region->UpdatedTime stamps for all Local/Remote > regions, and you update/compare those fields. Your component is a business > logic component, and it shouldn't modify the fields originally set by > GenericDao (Removed/Created/Modified). You should operate only with > parameters that are exposed to changes coming through the APIs. > > -Alena. > > From: Alex Ough > Date: Monday, April 7, 2014 at 9:16 PM > To: "dev@cloudstack.apache.org" , Alena > Prokharchyk , Alex Huang < > alex.hu...@citrix.com> > Subject: Re: Review Request 20099: Domain-Account-User Sync Up Among > Multiple Regions (Core Changes) > > And all 4 of the recommendations Alex Hwang gave were already > implemented to support the real time synchronization (#1), > but like I said in the previous email, we need to support the full scan > (#2) to cover any failures during the synchronization. > > Thanks > Alex Ough > > > > On Tue, Apr 8, 2014 at 12:10 AM, Alex Ough wrote: > >> Alena/Alex, >> >> I think I need to give some explanation how this works. >> >> There are 2 ways of sync. >> >> 1. The real time sync : Whenever a resource is create/updated/removed, >> each region gets the event of that and requests the same job to all remote >> regions using API interfaces, which will create/update/remove the same >> resource in each remote region in the real time. >> >> 2. Full scan : There can be some interruption (whatever the reason is) >> that makes fail the real time sync, so each region scans its own resources >> with an interval and compares them with those of remote regions. This is a >> little tricky because we need to find out whose change is later when there >> is any discrepancy between resources in the local & remote regions and this >> is why we need maintain the created/updated/removed times. >> >> 1) When a resource exists in both local (L) and remote regions (R1) >> - compare the 'updated' times of both >> - Update the local's resource using remote resource attributes >> only if the updated time of the remote region is later >> - Store the "updated" time of the remote resource (not the >> current time) as the local's updated time >> - If we store the current time as the updated time, it will cause >> an mis-sync >> because if the resource was updated in another remote region >> (R2) a little after it was updated in the remote region(R1), >> update in R2 region will be overwritten by the update in R1 >> because the current time is later than the updated time of R2 >> >> 2) When a resource exists only in the local region >>- Find if there was a removal event of this resource in the remote >> region >>- and if so, compare the time when the event occurred with the >> created time in the local region >>- Remove the resource from the local only if the event time is >> later >>- Like #1, store the time the event was occurred as the 'removed' >> time of the removed resource in the local region >> >> 3) When a resource exists only in the remote region >> - Find if the resource in the local has been removed >> - If so, compare the removal time in the local and the created time >> in the remote >> - Create the resource in the local only if either the local was not >> removed or the created time of the remote is later than the removal in local >> - LIke #1 & #2, store the time of the created of the remote region >> as the 'created' time of the newly created in the local region. >> >> I hope this will help you understand how the create/updated/removed >> times are managed. >> If there is no failu
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
Yes, I really hope so from now on :-) For #2, the current setting is NOT syncing the resources and these are the steps to enable this feature - Set the "mode" value of the remote regions to sync into 'sync on' - Set the value of global parameter, "region.full.scan.interval" (the default value is '0"), into value of milliseconds to enable the Full Scan Let me know if you want a different way. For #1, after another thought, I'm not still clear why those datetime values can't be added to their attributes. As you recommended, we can use another table to track them, but there can be an issue of partial commits when only the resource are successfully created/updated/removed and their datetime values fail to be stored because they are NOT in the same transaction. And we also need to track the 'removed' datetime because when a resource exists in only either the local or the remote region, we don't know whether it was created but it failed to be created in another region or it was removed but it failed to be removed in another region. That's why I track the "removed" datetime and preserve the original datetime. Let me think about the #1 more closely and let you know what I find. So please let me know if you have any other suggestion related with this. Thanks Alex Ough On Tue, Apr 8, 2014 at 2:09 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > Sorry, Alex, for not providing the input earlier, I understand that > requests like this shouldn't come the last minute. But unfortunately your > FS didn't have enough information to caught these design problems - the > fact that the GenericDaoFields modified/removed get modified directly - > from the very start, and only by reviewing the review board ticket you > could see it. Hopefully the next time the experience will be more smooth > for both parties. Meanwhile I'm ready to help with whatever questions you > have. > > Yes, only 2 things need to be fixed. #2 should be pretty > straightforward. Make sure your component injects the global config > variable if you decide to make a switch via global config. For that, your > class has to implement Configurable interface. You can find examples of how > this interface is utilized by existing CS components. Here is the FS: > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuration > > For #1 - I don't think you need to keep track of Removed date times. > Once the removed field is set in Region1, its safe to just go ahead and > remove the entry in all the rest of the regions. You can't (and don't need > to) perform any other operation on the resource if its removed in one of > the regions. > You just have to fix it in created/updated(modified) fields. > > -Alena. > > From: Alex Ough > Date: Tuesday, April 8, 2014 at 10:39 AM > To: Alena Prokharchyk > Cc: "dev@cloudstack.apache.org" , Alex Huang < > alex.hu...@citrix.com> > > Subject: Re: Review Request 20099: Domain-Account-User Sync Up Among > Multiple Regions (Core Changes) > > Alena, > > It would be really nice if I had this feedback when I presented this at > the beginning. > Anyway, so are only these 2 things necessary to be merged into the master? > > 1. find another way to store the created/updated/removed datetimes of > the resources > 2. provide a way to disable this feature. > > Let me try to resolve those 2 and let you know once done, > so please let me know now if there is anything else needed. > > Thanks > Alex Ough > > > On Tue, Apr 8, 2014 at 1:02 PM, Alena Prokharchyk < > alena.prokharc...@citrix.com> wrote: > >> Alex, thank you for the explanation. But I think there should be >> another way of saving all the "updated"/"created" info rather than updating >> the UserVO/AccountVO/DomainVO fields directly. >> >> The one way I can think of - you can create a helper table where you >> store the ref UUID->Region->UpdatedTime stamps for all Local/Remote >> regions, and you update/compare those fields. Your component is a business >> logic component, and it shouldn't modify the fields originally set by >> GenericDao (Removed/Created/Modified). You should operate only with >> parameters that are exposed to changes coming through the APIs. >> >> -Alena. >> >> From: Alex Ough >> Date: Monday, April 7, 2014 at 9:16 PM >> To: "dev@cloudstack.apache.org" , Alena >> Prokharchyk , Alex Huang < >> alex.hu...@citrix.com> >> Subject: Re: Review Request 20099: Domain-Account-User Sync Up Among >> Multiple Regions (Core Changes) >> >> And
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
#2 : I will, and YES, you need to change the modes of all regions, which also means that you can sync the resources only in the regions you select. #1: Well... it looks like I still need more information why they can't be added to their attributes. They are also managed through the their dao interfaces, and the values are assigned only during the full scan to preserve the original value when they are missed. Thanks Alex Ough On Tue, Apr 8, 2014 at 2:29 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > > > On 4/8/14, 11:26 AM, "Alex Ough" wrote: > > >Yes, I really hope so from now on :-) > > > > > >For #2, the current setting is NOT syncing the resources and these are > >the steps to enable this feature > >- Set the "mode" value of the remote regions to sync into 'sync on' > >- Set the value of global parameter, "region.full.scan.interval" (the > >default value is '0"), into value of milliseconds to enable the Full Scan > >Let me know if you want a different way. > > > Sounds good to me. We have to document it though very clearly under ³how > to disable the feature² section. When you set the mode, you have to set it > for all the regions to completely disable the feature, right? > > > > > > >For #1, after another thought, I'm not still clear why those datetime > >values can't be added to their attributes. As you recommended, we can use > >another table to track them, but there can be an issue of partial commits > >when only the resource are successfully > > created/updated/removed and their datetime values fail to be stored > >because they are NOT in the same transaction. > > > > > >And we also need to track the 'removed' datetime because when a resource > >exists in only either the local or the remote region, we don't know > >whether it was created but it failed to be created in another region or > >it was removed but it failed to be removed > > in another region. That's why I track the "removed" datetime and > >preserve the original datetime. > > I¹m fine with keeping the removed field, as long as you don¹t update this > field in CS DB. > > > > > > > >Let me think about the #1 more closely and let you know what I find. > >So please let me know if you have any other suggestion related with this. > >Thanks > >Alex Ough > > > > > > > >On Tue, Apr 8, 2014 at 2:09 PM, Alena Prokharchyk > > wrote: > > > >Sorry, Alex, for not providing the input earlier, I understand that > >requests like this shouldn¹t come the last minute. But unfortunately your > >FS didn¹t have enough information to caught these design problems the > >fact that the GenericDaoFields modified/removed > > get modified directly - from the very start, and only by reviewing the > >review board ticket you could see it. Hopefully the next time the > >experience will be more smooth for both parties. Meanwhile I¹m ready to > >help with whatever questions you have. > > > > > >Yes, only 2 things need to be fixed. #2 should be pretty straightforward. > >Make sure your component injects the global config variable if you decide > >to make a switch via global config. For that, your class has to implement > >Configurable interface. You can > > find examples of how this interface is utilized by existing CS > >components. Here is the FS: > > > > > >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuration > > > > > >For #1 I don¹t think you need to keep track of Removed date times. Once > >the removed field is set in Region1, its safe to just go ahead and remove > >the entry in all the rest of the regions. You can¹t (and don¹t need to) > >perform any other operation on the > > resource if its removed in one of the regions. > >You just have to fix it in created/updated(modified) fields. > > > > > >-Alena. > > > > > >From: Alex Ough > >Date: Tuesday, April 8, 2014 at 10:39 AM > >To: Alena Prokharchyk > >Cc: "dev@cloudstack.apache.org" , Alex Huang > > > > > >Subject: Re: Review Request 20099: Domain-Account-User Sync Up Among > >Multiple Regions (Core Changes) > > > > > > > > > > > >Alena, > > > > > >It would be really nice if I had this feedback when I presented this at > >the beginning. > >Anyway, so are only these 2 things necessary to be merged into the master? > > > > > >1. find another way to store the created/u
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
I understand that, but I don't understand why their values can't be assigned through the dao methods when necessary. But if you still feel uncomfortable, do you think it is ok to include the job to store the datetime values in another table into the same transaction of creating/removing/updating the resources in their manager methods to guarantee the atomicity? Thanks Alex Ough On Tue, Apr 8, 2014 at 2:45 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > Alex, > > Take for example the Removed field. Its not being managed through none of > the CS interfaces, and you can't set this value neither from the API, nor > from the Orchestration layer. Its being set automatically in GenericDao > once the object is set for removal by the very same Dao. > > The same applies to the Created/Updated fields. > > > -Alena. > > On 4/8/14, 11:41 AM, "Alex Ough" wrote: > > >#2 : I will, and YES, you need to change the modes of all regions, which > >also means that you can sync the resources only in the regions you select. > > > > > >#1: Well... it looks like I still need more information why they can't be > >added to their attributes. They are also managed through the their dao > >interfaces, and the values are assigned only during the full scan to > >preserve the original value when they > > are missed. > > > > > >Thanks > >Alex Ough > > > > > >On Tue, Apr 8, 2014 at 2:29 PM, Alena Prokharchyk > > wrote: > > > > > > > >On 4/8/14, 11:26 AM, "Alex Ough" wrote: > > > >>Yes, I really hope so from now on :-) > >> > >> > >>For #2, the current setting is NOT syncing the resources and these are > >>the steps to enable this feature > >>- Set the "mode" value of the remote regions to sync into 'sync on' > >>- Set the value of global parameter, "region.full.scan.interval" (the > >>default value is '0"), into value of milliseconds to enable the Full Scan > >>Let me know if you want a different way. > > > > > > > >Sounds good to me. We have to document it though very clearly under ³how > >to disable the feature² section. When you set the mode, you have to set it > >for all the regions to completely disable the feature, right? > > > >> > >> > >>For #1, after another thought, I'm not still clear why those datetime > >>values can't be added to their attributes. As you recommended, we can use > >>another table to track them, but there can be an issue of partial commits > >>when only the resource are successfully > >> created/updated/removed and their datetime values fail to be stored > >>because they are NOT in the same transaction. > >> > >> > >>And we also need to track the 'removed' datetime because when a resource > >>exists in only either the local or the remote region, we don't know > >>whether it was created but it failed to be created in another region or > >>it was removed but it failed to be removed > >> in another region. That's why I track the "removed" datetime and > >>preserve the original datetime. > > > > > >I¹m fine with keeping the removed field, as long as you don¹t update this > >field in CS DB. > > > > > >> > >> > >>Let me think about the #1 more closely and let you know what I find. > >>So please let me know if you have any other suggestion related with this. > >>Thanks > >>Alex Ough > >> > >> > >> > >>On Tue, Apr 8, 2014 at 2:09 PM, Alena Prokharchyk > >> wrote: > >> > >>Sorry, Alex, for not providing the input earlier, I understand that > >>requests like this shouldn¹t come the last minute. But unfortunately your > >>FS didn¹t have enough information to caught these design problems - the > >>fact that the GenericDaoFields modified/removed > >> get modified directly - from the very start, and only by reviewing the > >>review board ticket you could see it. Hopefully the next time the > >>experience will be more smooth for both parties. Meanwhile I¹m ready to > >>help with whatever questions you have. > >> > >> > >>Yes, only 2 things need to be fixed. #2 should be pretty straightforward. > >>Make sure your component injects the global config variable if you decide > >>to make a switch via global config. For that, your class has to implement > >>Configurable interf
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20099/ --- (Updated April 16, 2014, 7:06 p.m.) Review request for cloudstack. Changes --- This is another update for core. Repository: cloudstack-git Description --- This is the review request for the core changes related with #17790 that has only the new plugin codes. Diffs (updated) - api/src/com/cloud/event/EventTypes.java 39ef710 api/src/com/cloud/user/AccountService.java 7e37b38 api/src/org/apache/cloudstack/api/ApiConstants.java fdb4558 api/src/org/apache/cloudstack/api/BaseCmd.java f6f21ae api/src/org/apache/cloudstack/api/command/admin/region/AddRegionCmd.java f6743ba api/src/org/apache/cloudstack/api/command/admin/region/UpdateRegionCmd.java b08cbbb api/src/org/apache/cloudstack/api/response/AccountResponse.java 2e50c51 api/src/org/apache/cloudstack/api/response/DomainResponse.java 0c0281e api/src/org/apache/cloudstack/api/response/RegionResponse.java 6c74fa6 api/src/org/apache/cloudstack/api/response/UserResponse.java 40e1561 api/src/org/apache/cloudstack/region/Region.java df64e44 api/src/org/apache/cloudstack/region/RegionService.java afefcc7 api/test/org/apache/cloudstack/api/command/test/RegionCmdTest.java 10c3d85 client/pom.xml d8dbde7 engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml 489b37d engine/schema/src/org/apache/cloudstack/multiregion/RmapVO.java PRE-CREATION engine/schema/src/org/apache/cloudstack/multiregion/RsyncVO.java PRE-CREATION engine/schema/src/org/apache/cloudstack/multiregion/dao/RmapDao.java PRE-CREATION engine/schema/src/org/apache/cloudstack/multiregion/dao/RmapDaoImpl.java PRE-CREATION engine/schema/src/org/apache/cloudstack/multiregion/dao/RsyncDao.java PRE-CREATION engine/schema/src/org/apache/cloudstack/multiregion/dao/RsyncDaoImpl.java PRE-CREATION engine/schema/src/org/apache/cloudstack/region/RegionVO.java 608bd2b plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDispatcher.java 95074e2 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/dao/AccountJoinDaoImpl.java ecd97c7 server/src/com/cloud/api/query/dao/UserAccountJoinDaoImpl.java 923a238 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/multiregion/RsyncManager.java PRE-CREATION server/src/com/cloud/multiregion/RsyncManagerImpl.java PRE-CREATION server/src/com/cloud/projects/ProjectManagerImpl.java d10c059 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/networkoffering/ChildTestConfiguration.java 22516c0 server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/20099/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions (New Plugin)
/simulator/SimulatorDomainLocalGeneratorEvent.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGenerator.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGenerator.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEvent.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriber.java PRE-CREATION plugins/event-bus/multiregion/src/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriber.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/AccountCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/BaseCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/DomainCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/api/UserCallerTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/AccountFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/BaseServiceTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/DomainFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteAccountEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteDomainEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/RemoteUserEventProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/service/UserFullSyncProcessorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorAccountLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorDomainLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorEventTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/simulator/SimulatorUserLocalGeneratorTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/AccountSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/DomainSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/MultiRegionSubscriberTest.java PRE-CREATION plugins/event-bus/multiregion/test/org/apache/cloudstack/mom/multiregion/subscriber/UserSubscriberTest.java PRE-CREATION Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up Among Multiple Regions
Hi Alena,
It was a nice to actually meet you during the conference and I updated the
review requests, so please review them and let me know if there is anything
not expected.
Thanks
Alex Ough
On Thu, Apr 3, 2014 at 1:01 PM, Alex Ough wrote:
> Well, I'm not sure about that because the help is about how to use @Inject
> in the Spring framework.
>
>
> On Thu, Apr 3, 2014 at 12:49 PM, Alena Prokharchyk <
> alena.prokharc...@citrix.com> wrote:
>
>> Alex, please feel free to update Wiki docs related to the questions you
>> got Darren's help from. I think this info would be useful for all CS
>> committers.
>>
>> Thank you,
>> Alena.
>>
>> From: Alex Ough
>> Date: Thursday, April 3, 2014 at 9:22 AM
>> To: Chiradeep Vittal , Alena Prokharchyk <
>> alena.prokharc...@citrix.com>, Darren Shepherd <
>> darren.sheph...@citrix.com>
>> Cc: "dev@cloudstack.apache.org"
>>
>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>> Among Multiple Regions
>>
>> Forgot to mention this, but it was a great help from Darren.
>> Thanks again, Darren!
>> Alex Ough
>>
>>
>> On Thu, Apr 3, 2014 at 11:56 AM, Alex Ough wrote:
>>
>>> All,
>>>
>>> I updated the patches as per Alena's request.
>>>
>>> Let me know if there is anything missing/incorrect.
>>> Thanks
>>> Alex Ough
>>>
>>>
>>> On Wed, Mar 26, 2014 at 1:40 PM, Alex Ough wrote:
>>>
>>>> Sorry, my bad. I mis-read your comment.
>>>>
>>>> Thanks for pointing it out.
>>>> Alex Ough
>>>>
>>>>
>>>> On Wed, Mar 26, 2014 at 12:25 PM, Chiradeep Vittal <
>>>> chiradeep.vit...@citrix.com> wrote:
>>>>
>>>>> I didn't say "do not use auto wiring". I said the convention is to
>>>>> use @Inject which you didn't have.
>>>>>
>>>>> From: Alena Prokharchyk
>>>>> Date: Wednesday, March 26, 2014 at 7:39 AM
>>>>> To: Alex Ough , "dev@cloudstack.apache.org" <
>>>>> dev@cloudstack.apache.org>
>>>>> Cc: Chiradeep Vittal
>>>>>
>>>>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>>>>> Among Multiple Regions
>>>>>
>>>>> Alex, I'm attending a conference today, will review the patch
>>>>> tomorrow.
>>>>>
>>>>> -Alena
>>>>>
>>>>> From: Alex Ough
>>>>> Date: Wednesday, March 26, 2014 at 6:35 AM
>>>>> To: Alena Prokharchyk
>>>>> Cc: "dev@cloudstack.apache.org" ,
>>>>> Chiradeep Vittal
>>>>> Subject: Re: [DISCUSS]{BEHAVIORAL-CHANGE]Domain-Account-User Sync Up
>>>>> Among Multiple Regions
>>>>>
>>>>> Alena,
>>>>>
>>>>> It took a little time to update the patch because I had a vacation
>>>>> last week.
>>>>> Now I uploaded the updated patch for review with below status, so
>>>>> please check them and let me know what you think.
>>>>> I hope it to be merged soon to wrap this up asap.
>>>>>
>>>>> 1. no change with waiting for comments on my recommendation.
>>>>>
>>>>> 2. The two things to turn on the sync-up among multiple regions is
>>>>> to change the class name of "eventNotificationBus" into
>>>>> "MultiRegionEventBus" from "RabbitMQEventBus" as below and change the
>>>>> value
>>>>> of 'region.full.scan.interval' in Global Settings. And the new classes are
>>>>> never used unless the feature is turned on, so I'm not sure if auto wiring
>>>>> is necessary here and Chiradeep asked not to use @inject in his initial
>>>>> review, so I removed them.
>>>>> >>>> class="org.apache.cloudstack.mom.rabbitmq.MultiRegionEventBus">
>>>>>
>>>>> 3. They are not adaptors, but inherited classes to process
>>>>> domain/account/user objects separately.
>>>>>
>>>>> 4. Done
>>>>>
>>>>> 5. Same
>>>>>
>>>>> 6. I removed 'domain path' from AccountResponse & UserResponse.
>>>>>
>>>>> Than
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
> On April 17, 2014, 8:10 p.m., Alena Prokharchyk wrote: > > 1) Alex, I don't quite approve the fact that the responses were modified > > just to support your feature. User/account of region1 has absolutely no > > idea of syncing that your plugin is doing as well as he has 0 idea that it > > exists in multiple databases. I would suggest to keep these parameters in > > the DB of your component, and reflect all the sync updates there. > > And in the API responses account should only see actual > > created/removed/modified parameters reflecting the time when the account > > was created/modified/removed from the very first DB of your region. > > > > Just think about your component as of a plugin running on top of CS (and > > that plugin can be enabled/disabled at any time), and do the syncing w/o CS > > code knowing about it. CS just has to return you all the information that > > originally was set through the CS (w/o the help of your component). All > > extra work your component does, should be stored in your component DB. You > > can add a helper table/API where you reflect the sync time between the > > accounts/domains/users, I'm fine with that. > > > > 2) #2 is lined with #1. Please remove all the occurances of > > _rsyncMgr.update from AccountManagerImpl. As I said, your component should > > act as a plugin, and you shouldn't inject it to the CS core managers. Make > > all updates directly from RSyncManager after account/domain/user is > > created/modified/removed in the AccountManager; code without interfering > > with AccountManager. > > > > 3)RegionResponse.java > > > > * Please add more details to the new parameter description, its not clear > > what it returns now: > > > > "the active of the region" > > > > * add "since" annotation > > 1) & 2) I'm a little frustrated because I think you confirmed that the updates can be inside the Account/Domain managers to guarantee the atomicity of the transaction when we spoke during the conference. And the reason why the API responses need the synced datetime is because the synchronization needs to compare which changes are later. - Alex ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20099/#review40690 --- On April 16, 2014, 7:06 p.m., Alex Ough wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20099/ > --- > > (Updated April 16, 2014, 7:06 p.m.) > > > Review request for cloudstack. > > > Repository: cloudstack-git > > > Description > --- > > This is the review request for the core changes related with #17790 that has > only the new plugin codes. > > > Diffs > - > > api/src/com/cloud/event/EventTypes.java 39ef710 > api/src/com/cloud/user/AccountService.java 7e37b38 > api/src/org/apache/cloudstack/api/ApiConstants.java fdb4558 > api/src/org/apache/cloudstack/api/BaseCmd.java f6f21ae > api/src/org/apache/cloudstack/api/command/admin/region/AddRegionCmd.java > f6743ba > api/src/org/apache/cloudstack/api/command/admin/region/UpdateRegionCmd.java > b08cbbb > api/src/org/apache/cloudstack/api/response/AccountResponse.java 2e50c51 > api/src/org/apache/cloudstack/api/response/DomainResponse.java 0c0281e > api/src/org/apache/cloudstack/api/response/RegionResponse.java 6c74fa6 > api/src/org/apache/cloudstack/api/response/UserResponse.java 40e1561 > api/src/org/apache/cloudstack/region/Region.java df64e44 > api/src/org/apache/cloudstack/region/RegionService.java afefcc7 > api/test/org/apache/cloudstack/api/command/test/RegionCmdTest.java 10c3d85 > client/pom.xml d8dbde7 > > engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml > 489b37d > engine/schema/src/org/apache/cloudstack/multiregion/RmapVO.java > PRE-CREATION > engine/schema/src/org/apache/cloudstack/multiregion/RsyncVO.java > PRE-CREATION > engine/schema/src/org/apache/cloudstack/multiregion/dao/RmapDao.java > PRE-CREATION > engine/schema/src/org/apache/cloudstack/multiregion/dao/RmapDaoImpl.java > PRE-CREATION > engine/schema/src/org/apache/cloudstack/multiregion/dao/RsyncDao.java > PRE-CREATION > engine/schema/src/org/apache/cloudstack/multiregion/dao/RsyncDaoImpl.java > PRE-CREATION > engine/sc
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
when & how do you want to have it? On Mon, Apr 28, 2014 at 1:18 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > Alex, please see the answers inline. May be it would be a good idea if we > discuss the high level architecture - what DB changes you are going to > make, how CS apis would or wouldn¹t get affected - before you do the > coding? It would definitely save us time, and won¹t lead to further > frustration. I¹m more than willing to help. > > On 4/28/14, 6:59 AM, "Alex Ough" wrote: > > > > > > >> On April 17, 2014, 8:10 p.m., Alena Prokharchyk wrote: > >> > 1) Alex, I don't quite approve the fact that the responses were > >>modified just to support your feature. User/account of region1 has > >>absolutely no idea of syncing that your plugin is doing as well as he > >>has 0 idea that it exists in multiple databases. I would suggest to keep > >>these parameters in the DB of your component, and reflect all the sync > >>updates there. > >> > And in the API responses account should only see actual > >>created/removed/modified parameters reflecting the time when the account > >>was created/modified/removed from the very first DB of your region. > >> > > >> > Just think about your component as of a plugin running on top of CS > >>(and that plugin can be enabled/disabled at any time), and do the > >>syncing w/o CS code knowing about it. CS just has to return you all the > >>information that originally was set through the CS (w/o the help of your > >>component). All extra work your component does, should be stored in your > >>component DB. You can add a helper table/API where you reflect the sync > >>time between the accounts/domains/users, I'm fine with that. > >> > > >> > 2) #2 is lined with #1. Please remove all the occurances of > >>_rsyncMgr.update from AccountManagerImpl. As I said, your component > >>should act as a plugin, and you shouldn't inject it to the CS core > >>managers. Make all updates directly from RSyncManager after > >>account/domain/user is created/modified/removed in the AccountManager; > >>code without interfering with AccountManager. > >> > > >> > 3)RegionResponse.java > >> > > >> > * Please add more details to the new parameter description, its not > >>clear what it returns now: > >> > > >> > "the active of the region" > >> > > >> > * add "since" annotation > >> > > > > >1) & 2) I'm a little frustrated because I think you confirmed that the > >updates can be inside the Account/Domain managers to guarantee the > >atomicity of the transaction when we spoke during the conference. > > > Perhaps we misunderstood each other. When you/me/Alex Huang talked about > it at the collab, we concluded that all the changes related to synced > entries, should be done in Account/domain managers of YOUR component that > does account/domain sync, not the CS itself. While the current fix just > does the trick of not updating the initial created/updated/modified > fields, but by introducing the fields that do duplicated functionality - > sync_created/sync_updated/synced_removed - but across the regions. > > > > > > >And the reason why the API responses need the synced datetime is because > >the synchronization needs to compare which changes are later. > > As I said multiple times before, your component should act as any third > party component doing CS DB sync, and run all the comparisons on the layer > above Account/Domain layer of the CS. No modifications should be done to > AccountVO/DomainVO as well as the CS account/domain managers. The synced > data should be stored in the helper VO objects of your components, and the > manager of your components should perform the comparison w/o impacting the > CS code. When user looks at the API response, he should see no difference > whether the CS runs with multi region setup, or not. > > > You can also utilize *DetailsVO tables of the CS (user_details for user, > account_details for account, domain_details for domain) to store the > Synced time, instead of creating your helper VO object. These tables are > meant for storing CS object details, that can be used by third party > components/services. Refer to the following FS: > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+bett > er+control+over+first+class+objects+in+CS > > > Account/Domain API responses shouldn¹t get affected either. If you utilize > resourceDetails tables,
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
what do have any web conferencing app? If not, we can use the webex. Let me know what time is convenient for you. On Mon, Apr 28, 2014 at 1:46 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > Alex, that is totally up to you. I'm ready to review/talk any time. > > -alena. > > On 4/28/14, 10:35 AM, "Alex Ough" wrote: > > >when & how do you want to have it? > > > > > >On Mon, Apr 28, 2014 at 1:18 PM, Alena Prokharchyk < > >alena.prokharc...@citrix.com> wrote: > > > >> Alex, please see the answers inline. May be it would be a good idea if > >>we > >> discuss the high level architecture - what DB changes you are going to > >> make, how CS apis would or wouldn¹t get affected - before you do the > >> coding? It would definitely save us time, and won¹t lead to further > >> frustration. I¹m more than willing to help. > >> > >> On 4/28/14, 6:59 AM, "Alex Ough" wrote: > >> > >> > > >> > > >> >> On April 17, 2014, 8:10 p.m., Alena Prokharchyk wrote: > >> >> > 1) Alex, I don't quite approve the fact that the responses were > >> >>modified just to support your feature. User/account of region1 has > >> >>absolutely no idea of syncing that your plugin is doing as well as he > >> >>has 0 idea that it exists in multiple databases. I would suggest to > >>keep > >> >>these parameters in the DB of your component, and reflect all the sync > >> >>updates there. > >> >> > And in the API responses account should only see actual > >> >>created/removed/modified parameters reflecting the time when the > >>account > >> >>was created/modified/removed from the very first DB of your region. > >> >> > > >> >> > Just think about your component as of a plugin running on top of CS > >> >>(and that plugin can be enabled/disabled at any time), and do the > >> >>syncing w/o CS code knowing about it. CS just has to return you all > >>the > >> >>information that originally was set through the CS (w/o the help of > >>your > >> >>component). All extra work your component does, should be stored in > >>your > >> >>component DB. You can add a helper table/API where you reflect the > >>sync > >> >>time between the accounts/domains/users, I'm fine with that. > >> >> > > >> >> > 2) #2 is lined with #1. Please remove all the occurances of > >> >>_rsyncMgr.update from AccountManagerImpl. As I said, your component > >> >>should act as a plugin, and you shouldn't inject it to the CS core > >> >>managers. Make all updates directly from RSyncManager after > >> >>account/domain/user is created/modified/removed in the AccountManager; > >> >>code without interfering with AccountManager. > >> >> > > >> >> > 3)RegionResponse.java > >> >> > > >> >> > * Please add more details to the new parameter description, its not > >> >>clear what it returns now: > >> >> > > >> >> > "the active of the region" > >> >> > > >> >> > * add "since" annotation > >> >> > > >> > > >> >1) & 2) I'm a little frustrated because I think you confirmed that the > >> >updates can be inside the Account/Domain managers to guarantee the > >> >atomicity of the transaction when we spoke during the conference. > >> > >> > >> Perhaps we misunderstood each other. When you/me/Alex Huang talked about > >> it at the collab, we concluded that all the changes related to synced > >> entries, should be done in Account/domain managers of YOUR component > >>that > >> does account/domain sync, not the CS itself. While the current fix just > >> does the trick of not updating the initial created/updated/modified > >> fields, but by introducing the fields that do duplicated functionality - > >> sync_created/sync_updated/synced_removed - but across the regions. > >> > >> > >> > >> > > >> >And the reason why the API responses need the synced datetime is > >>because > >> >the synchronization needs to compare which changes are later. > >> > >> As I said multiple times before, your component should act as any third > >
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
What is your timezone? Can we have it tomorrow morning? On Mon, Apr 28, 2014 at 3:19 PM, Alena Prokharchyk < alena.prokharc...@citrix.com> wrote: > We can discuss it today at 3pm, Alex, if the time works for you. WebEx is > fine, but we should make sure to record the session + share follow up > notes with the community. The best approach would be - to update the FS > with the design changes for this particular part of the code. > > Will be waiting for the webex invitation from you. > > -Alena. > > On 4/28/14, 11:23 AM, "Alex Ough" wrote: > > >what do have any web conferencing app? > >If not, we can use the webex. > >Let me know what time is convenient for you. > > > > > >On Mon, Apr 28, 2014 at 1:46 PM, Alena Prokharchyk < > >alena.prokharc...@citrix.com> wrote: > > > >> Alex, that is totally up to you. I'm ready to review/talk any time. > >> > >> -alena. > >> > >> On 4/28/14, 10:35 AM, "Alex Ough" wrote: > >> > >> >when & how do you want to have it? > >> > > >> > > >> >On Mon, Apr 28, 2014 at 1:18 PM, Alena Prokharchyk < > >> >alena.prokharc...@citrix.com> wrote: > >> > > >> >> Alex, please see the answers inline. May be it would be a good idea > >>if > >> >>we > >> >> discuss the high level architecture - what DB changes you are going > >>to > >> >> make, how CS apis would or wouldn¹t get affected - before you do the > >> >> coding? It would definitely save us time, and won¹t lead to further > >> >> frustration. I¹m more than willing to help. > >> >> > >> >> On 4/28/14, 6:59 AM, "Alex Ough" wrote: > >> >> > >> >> > > >> >> > > >> >> >> On April 17, 2014, 8:10 p.m., Alena Prokharchyk wrote: > >> >> >> > 1) Alex, I don't quite approve the fact that the responses were > >> >> >>modified just to support your feature. User/account of region1 has > >> >> >>absolutely no idea of syncing that your plugin is doing as well as > >>he > >> >> >>has 0 idea that it exists in multiple databases. I would suggest to > >> >>keep > >> >> >>these parameters in the DB of your component, and reflect all the > >>sync > >> >> >>updates there. > >> >> >> > And in the API responses account should only see actual > >> >> >>created/removed/modified parameters reflecting the time when the > >> >>account > >> >> >>was created/modified/removed from the very first DB of your region. > >> >> >> > > >> >> >> > Just think about your component as of a plugin running on top > >>of CS > >> >> >>(and that plugin can be enabled/disabled at any time), and do the > >> >> >>syncing w/o CS code knowing about it. CS just has to return you all > >> >>the > >> >> >>information that originally was set through the CS (w/o the help of > >> >>your > >> >> >>component). All extra work your component does, should be stored in > >> >>your > >> >> >>component DB. You can add a helper table/API where you reflect the > >> >>sync > >> >> >>time between the accounts/domains/users, I'm fine with that. > >> >> >> > > >> >> >> > 2) #2 is lined with #1. Please remove all the occurances of > >> >> >>_rsyncMgr.update from AccountManagerImpl. As I said, your component > >> >> >>should act as a plugin, and you shouldn't inject it to the CS core > >> >> >>managers. Make all updates directly from RSyncManager after > >> >> >>account/domain/user is created/modified/removed in the > >>AccountManager; > >> >> >>code without interfering with AccountManager. > >> >> >> > > >> >> >> > 3)RegionResponse.java > >> >> >> > > >> >> >> > * Please add more details to the new parameter description, its > >>not > >> >> >>clear what it returns now: > >> >> >> > > >> >> >> > "the active of the region" > >> >> >> > > >> >> >> > * add &
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20099/ --- (Updated May 4, 2014, 9:12 p.m.) Review request for cloudstack. Changes --- core changes Repository: cloudstack-git Description --- This is the review request for the core changes related with #17790 that has only the new plugin codes. Diffs (updated) - api/src/com/cloud/event/EventTypes.java 39ef710 api/src/com/cloud/user/AccountService.java 7e37b38 api/src/org/apache/cloudstack/api/ApiConstants.java fdb4558 api/src/org/apache/cloudstack/api/BaseCmd.java f6f21ae api/src/org/apache/cloudstack/api/ResponseGenerator.java 10fb6df api/src/org/apache/cloudstack/api/command/admin/region/AddRegionCmd.java f6743ba api/src/org/apache/cloudstack/api/command/admin/region/UpdateRegionCmd.java b08cbbb api/src/org/apache/cloudstack/api/response/RegionResponse.java 6c74fa6 api/src/org/apache/cloudstack/query/QueryService.java 01af631 api/src/org/apache/cloudstack/region/Region.java df64e44 api/src/org/apache/cloudstack/region/RegionService.java afefcc7 api/test/org/apache/cloudstack/api/command/test/RegionCmdTest.java 10c3d85 client/pom.xml d8dbde7 client/tomcatconf/commands.properties.in 45debe4 engine/schema/resources/META-INF/cloudstack/core/spring-engine-schema-core-daos-context.xml 489b37d engine/schema/src/org/apache/cloudstack/region/RegionVO.java 608bd2b plugins/network-elements/juniper-contrail/test/org/apache/cloudstack/network/contrail/management/MockAccountManager.java 957f708 plugins/pom.xml 9b391b8 server/resources/META-INF/cloudstack/core/spring-server-core-managers-context.xml fc1c7e2 server/src/com/cloud/api/ApiDBUtils.java 67e47f7 server/src/com/cloud/api/ApiResponseHelper.java 38f2f0b server/src/com/cloud/api/dispatch/ParamProcessWorker.java e9bdd8b server/src/com/cloud/api/query/QueryManagerImpl.java 3abb944 server/src/com/cloud/api/query/ViewResponseHelper.java d06e1d1 server/src/com/cloud/event/ActionEventUtils.java 28e5680 server/src/com/cloud/server/ManagementServerImpl.java bce2930 server/src/com/cloud/user/AccountManager.java 03bf842 server/src/com/cloud/user/AccountManagerImpl.java 2070ee6 server/src/com/cloud/user/DomainManager.java f72b18a server/src/com/cloud/user/DomainManagerImpl.java fbbe0c2 server/src/org/apache/cloudstack/region/RegionManager.java 6f25481 server/src/org/apache/cloudstack/region/RegionManagerImpl.java 8910714 server/src/org/apache/cloudstack/region/RegionServiceImpl.java 98cf500 server/test/com/cloud/api/dispatch/ParamProcessWorkerTest.java 12051a6 server/test/com/cloud/user/MockAccountManagerImpl.java f373cba server/test/com/cloud/user/MockDomainManagerImpl.java 7dddefb server/test/org/apache/cloudstack/region/RegionManagerTest.java d7bc537 setup/db/db/schema-440to450.sql 2bd5386 ui/scripts/regions.js 66dae8c Diff: https://reviews.apache.org/r/20099/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 17790: Domain-Account-User Sync Up Among Multiple Regions (New Plugin)
/subscriber/UserSubscriberTest.java PRE-CREATION server/src/com/cloud/api/query/dao/SyncAccountJoinDao.java PRE-CREATION server/src/com/cloud/api/query/dao/SyncAccountJoinDaoImpl.java PRE-CREATION server/src/com/cloud/api/query/dao/SyncUserAccountJoinDao.java PRE-CREATION server/src/com/cloud/api/query/dao/SyncUserAccountJoinDaoImpl.java PRE-CREATION server/src/com/cloud/multiregion/RsyncManager.java PRE-CREATION server/src/com/cloud/multiregion/RsyncManagerImpl.java PRE-CREATION Diff: https://reviews.apache.org/r/17790/diff/ Testing --- 1. Successfully tested real time synchronization as soon as resources are created/deleted/modified in one region. 2. Successfully tested full scans to synchronize resources that were missed during real time synchronization because of any reasons like network connection issues. 3. The tests were done manually and also automatically by randomly generating changes each region. Thanks, Alex Ough
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
> On May 5, 2014, 6:27 p.m., Alena Prokharchyk wrote: > > Alex, generic logic looks good to me. But still some things need to be > > fixed. > > > > 1) RegionVO > > > > @Column(name = "active") > > private boolean active; > > > > Explicitly set it to be active by default. Just setting it in the DB might > > not be enough because DAO would always default boolean to false if not set > > in the constructor/intance variable declaration. It should look like: > > > > @Column(name = "active") > > > > private boolean active = true; > > > > 2) Please make the description for the new parameters in the addRegion API > > , more descriptive > > > > @Parameter(name = ApiConstants.MODE, type = CommandType.STRING, required = > > true, description = "Region service active") > > 55 > > private String mode; > > > > > > 3) AccountManager > > > > * boolean updateUser(final Long id, String userName, String firstName, > > String lastName, String password, String email, String apiKey, String > > secretKey, String timeZone, Account.State state); > > > > Why do we pass account state to the call? there should be only user info > > passed in. If the account needs to be updated, do it in the separate > > command; if you need to retrive the account state - get the account object > > from the user object. Please remove this parameter > > > > * Why did you change the method signature for the > > > > UserAccount disableUser(long userId) to disableUser(User user)? The only > > one info that you have to pass to the manager - is the userId. Please > > change it back. > > > > The same for boolean enableUser(User user), boolean lockAccount(Account > > account); > > > > These changes don't seem to be critical or used for/by your plugin, so > > please change them back to the original way of doing things. Its not a good > > practice to change methods defined in the interfaces - > > AccountManager/AccountService - unless there is a real need for it. > > > > > > * boolean updateAccount(AccountVO account, String newAccountName, String > > newNetworkDomain, final Map details, Account.State state); > > > > As updateAccount doesn't allow account state update, please remove > > Account.State from the method signature > > > > > > 4) ApiResponseHelper.java > > > > Why did you change createDomainResponse method? > > > > public DomainResponse createDomainResponse(Domain domain)? > > > > > > To summarize it all, I would suggest to eliminate (revert) all the changes > > done to account/domain managers that your code doesn't actually need. It > > would make the changes list much shorter and easier for review. for 3) AccountManager * updateUser in 'state' : it is necessary during the Full Sync when an object's information is not same with one in other regions. * new signatures : they are necessary during the Full Sync not to generate events while changing them. They are just another methods to manage objects, so can you show me why they are not permitted? - Alex --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20099/#review42175 --- On May 4, 2014, 9:12 p.m., Alex Ough wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20099/ > --- > > (Updated May 4, 2014, 9:12 p.m.) > > > Review request for cloudstack. > > > Repository: cloudstack-git > > > Description > --- > > This is the review request for the core changes related with #17790 that has > only the new plugin codes. > > > Diffs > - > > api/src/com/cloud/event/EventTypes.java 39ef710 > api/src/com/cloud/user/AccountService.java 7e37b38 > api/src/org/apache/cloudstack/api/ApiConstants.java fdb4558 > api/src/org/apache/cloudstack/api/BaseCmd.java f6f21ae > api/src/org/apache/cloudstack/api/ResponseGenerator.java 10fb6df > api/src/org/apache/cloudstack/api/command/admin/region/AddRegionCmd.java > f6743ba > api/src/org/apache/cloudstack/api/command/admin/region/UpdateRegionCmd.java > b08cbbb > api/src/org/apache/cloudstack/api/response/RegionResponse.java 6c74fa6 > api/src/org/apache/cloudstack/query/QueryServ
Re: Review Request 20099: Domain-Account-User Sync Up Among Multiple Regions (Core Changes)
> On May 5, 2014, 6:27 p.m., Alena Prokharchyk wrote:
> > Alex, generic logic looks good to me. But still some things need to be
> > fixed.
> >
> > 1) RegionVO
> >
> > @Column(name = "active")
> > private boolean active;
> >
> > Explicitly set it to be active by default. Just setting it in the DB might
> > not be enough because DAO would always default boolean to false if not set
> > in the constructor/intance variable declaration. It should look like:
> >
> > @Column(name = "active")
> >
> > private boolean active = true;
> >
> > 2) Please make the description for the new parameters in the addRegion API
> > , more descriptive
> >
> > @Parameter(name = ApiConstants.MODE, type = CommandType.STRING, required =
> > true, description = "Region service active")
> > 55
> > private String mode;
> >
> >
> > 3) AccountManager
> >
> > * boolean updateUser(final Long id, String userName, String firstName,
> > String lastName, String password, String email, String apiKey, String
> > secretKey, String timeZone, Account.State state);
> >
> > Why do we pass account state to the call? there should be only user info
> > passed in. If the account needs to be updated, do it in the separate
> > command; if you need to retrive the account state - get the account object
> > from the user object. Please remove this parameter
> >
> > * Why did you change the method signature for the
> >
> > UserAccount disableUser(long userId) to disableUser(User user)? The only
> > one info that you have to pass to the manager - is the userId. Please
> > change it back.
> >
> > The same for boolean enableUser(User user), boolean lockAccount(Account
> > account);
> >
> > These changes don't seem to be critical or used for/by your plugin, so
> > please change them back to the original way of doing things. Its not a good
> > practice to change methods defined in the interfaces -
> > AccountManager/AccountService - unless there is a real need for it.
> >
> >
> > * boolean updateAccount(AccountVO account, String newAccountName, String
> > newNetworkDomain, final Map details, Account.State state);
> >
> > As updateAccount doesn't allow account state update, please remove
> > Account.State from the method signature
> >
> >
> > 4) ApiResponseHelper.java
> >
> > Why did you change createDomainResponse method?
> >
> > public DomainResponse createDomainResponse(Domain domain)?
> >
> >
> > To summarize it all, I would suggest to eliminate (revert) all the changes
> > done to account/domain managers that your code doesn't actually need. It
> > would make the changes list much shorter and easier for review.
>
> Alex Ough wrote:
> for 3) AccountManager
>
> * updateUser in 'state' : it is necessary during the Full Sync when an
> object's information is not same with one in other regions.
> * new signatures : they are necessary during the Full Sync not to
> generate events while changing them. They are just another methods to manage
> objects, so can you show me why they are not permitted?
>
>
> Alena Prokharchyk wrote:
> Alex, correct me if I'm wrong. In most of your code you use CS APIs to
> retrieve/update/delete/disable(disable changes the state) the CS objects
> (user/domain/account - updateAccount/updateDomain/deleteAccount etc). And you
> do it in each reason, and that generates the events for all the regions when
> these APIs are processed. Like for example, I see these calls:
>
> public JSONObject updateAccount(String accountId, String currentName,
> String newName, String details, String domainId, String networkDomain) throws
> APIFailureException {
> 139
> 140
> StringBuilder param = buildCmd("updateAccount");
> 141
> param.append(append(ApiConstants.ID, accountId));
> 142
> if (currentName != null)
> param.append(append(ApiConstants.ACCOUNT, currentName));
> 143
> if (newName != null) param.append(append(ApiConstants.NEW_NAME,
> newName));
> 144
> if (details != null)
> param.append(append(ApiConstants.ACCOUNT_DETAILS, details));
> 145
> if (domainId != null) param.append(append(ApiConstants.DOMAIN_ID,
> domainId));
> 146
> if (networkDomain != null)
> param.append(append(ApiCons
