It's not valid if you've got code that says does string 'vlan://untagged'
equal 'untagged'.
On Fri, May 30, 2014 at 10:17 AM, Daan Hoogland <daan.hoogl...@gmail.com>
wrote:
> CLOUDSTACK-5505 looks alright. As for the solution; Isn't 'untagged' a
> valid uri in itself? I would expect it would have always the value
> without the 'vlan://'. That said a solution is better then no
> solution, maybe the db upgrade path is best.
>
> Andrija, can you try as Marcus suggests, editing the db to change
> 'untagged' to 'vlan://untagged'?
>
> thanks,
> Daan
>
> On Fri, May 30, 2014 at 6:03 PM, Marcus <shadow...@gmail.com> wrote:
> > If that works, then I think the fix is better to update the database
> > upgrade script to look for this and change the vlan_id. That way new
> > installs and upgrades have consistent data. We could fix it in the code
> by
> > filtering it through BroadcastDomainType.Vlan.toUri(ipAddr.getVlanTag()),
> > but then later when someone expects the vlan id to be in URI format
> > somewhere else it will randomly break on people who did an upgrade. The
> fix
> > to CLOUDSTACK-5505 did this, avoiding the mismatch in format in the DB by
> > fetching the data from elsewhere that filtered it into URI format, and
> then
> > we just ended up hitting it again here.
> >
> >
> > On Fri, May 30, 2014 at 9:57 AM, Marcus <shadow...@gmail.com> wrote:
> >
> >> Actually, if you're in the position to play a bit... New deployments
> seem
> >> to work, and I believe it's because that broadcastUri is stored in the
> db
> >> in the new format:
> >>
> >> mysql> select id,vlan_id from vlan where network_id = (select id from
> >> networks where traffic_type="Public");
> >> +----+-----------------+
> >> | id | vlan_id |
> >> +----+-----------------+
> >> | 1 | vlan://untagged |
> >> +----+-----------------+
> >> 1 row in set (0.00 sec)
> >>
> >>
> >> I believe 4.2 and earlier would say just 'untagged' there. If you want
> to
> >> attempt changing that value to include vlan:// (if it is in fact
> missing),
> >> then restarting everything, that may fix the issue.
> >>
> >>
> >>
> >> On Fri, May 30, 2014 at 9:38 AM, Marcus <shadow...@gmail.com> wrote:
> >>
> >>> I thinnk the commit that caused the change was this or related to it.
> Is
> >>> there any way you could test a fix? Do you need me to build 4.3 RPMs
> or can
> >>> I just provide a patch? What works for you?
> >>>
> >>> commit 53d09c6f1843f04c5f1ab76be9419f5584302d1e
> >>> Date: Mon Aug 5 11:52:40 2013 +0200
> >>>
> >>> uri code per broadcast/isolation type , default is to accept
> anything
> >>> as uri , vlan and lswitch need some extra tlc
> >>>
> >>>
> >>>
> >>> On Fri, May 30, 2014 at 9:36 AM, Marcus <shadow...@gmail.com> wrote:
> >>>
> >>>> Note the differences in broadcastUri, here is your plug command:
> >>>>
> >>>> {
> >>>> "com.cloud.agent.api.PlugNicCommand": {
> >>>> "nic": {
> >>>> "deviceId": 1,
> >>>> "networkRateMbps": 99999,
> >>>> "defaultNic": true,
> >>>> "uuid": "6c782af3-2071-4543-acdc-cb30096e89ff",
> >>>> "ip": "46.232.xxx.246",
> >>>> "netmask": "255.255.255.0",
> >>>> "gateway": "46.232.xxx.1",
> >>>> "mac": "06:53:82:00:00:25",
> >>>> "broadcastType": "Vlan",
> >>>> "type": "Public",
> >>>> "broadcastUri": "vlan://untagged",
> >>>> "isolationUri": "vlan://untagged",
> >>>> "isSecurityGroupEnabled": false,
> >>>> "name": "breth1-500"
> >>>> },
> >>>> "instanceName": "r-801-VM",
> >>>> "vmType": "DomainRouter",
> >>>> "wait": 0
> >>>> }
> >>>> }
> >>>>
> >>>> and here is your ip associate command:
> >>>>
> >>>> {
> >>>> "com.cloud.agent.api.routing.IpAssocVpcCommand": {
> >>>> "ipAddresses": [
> >>>> {
> >>>> "accountId": 11,
> >>>> "publicIp": "46.232.xxx.246",
> >>>> "sourceNat": true,
> >>>> "add": true,
> >>>> "oneToOneNat": false,
> >>>> "firstIP": false,
> >>>> "broadcastUri": "untagged",
> >>>> "vlanGateway": "46.232.xxx.1",
> >>>> "vlanNetmask": "255.255.255.0",
> >>>> "vifMacAddress": "06:53:82:00:00:25",
> >>>> "networkRate": 99999,
> >>>> "trafficType": "Public",
> >>>> "networkName": "breth1-500"
> >>>> }
> >>>> ],
> >>>> "accessDetails": {
> >>>> "router.guest.ip": "46.232.xxx.246",
> >>>> "zone.network.type": "Advanced",
> >>>> "router.name": "r-801-VM",
> >>>> "router.ip": "169.254.0.52"
> >>>> },
> >>>> "wait": 0
> >>>> }
> >>>> }
> >>>>
> >>>>
> >>>> On Fri, May 30, 2014 at 9:27 AM, Andrija Panic <
> andrija.pa...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> yes, correct, eth1 is present, and can be started by static IP
> >>>>> configuration...
> >>>>>
> >>>>>
> >>>>> On 30 May 2014 17:25, Marcus <shadow...@gmail.com> wrote:
> >>>>>
> >>>>> > Let me make sure I understand... the 'Plug' of the nic works fine,
> as
> >>>>> it
> >>>>> > seems you do have an eth1 and can manually assign the IP to get it
> to
> >>>>> work?
> >>>>> > If that's the case then there's probably not an issue in
> >>>>> BridgeVifDriver or
> >>>>> > the XML. It is definitely in fetching/matching the eth device here
> >>>>> >
> >>>>> > vpc_ipassoc.sh 169.254.0.52 -A -l 46.232.xxx.246 -c ethnull -g
> >>>>> > 46.232.xxx.1 -m 24 -n 46.232.xxx.0
> >>>>> >
> >>>>> >
> >>>>> > On Fri, May 30, 2014 at 8:33 AM, Andrija Panic <
> >>>>> andrija.pa...@gmail.com>
> >>>>> > wrote:
> >>>>> >
> >>>>> > > Nope, started, did check, it is reported now as Debian 5 VM, but
> >>>>> still
> >>>>> > > doesn't work...
> >>>>> > > Rebooted VPC (destroyed VR, new one created...)
> >>>>> > >
> >>>>> > > $ virsh dumpxml r-812-VM
> >>>>> > > ...
> >>>>> > > <description>Debian GNU/Linux 5(64-bit)</description>
> >>>>> > > ...
> >>>>> > > <os>
> >>>>> > > <type arch='x86_64' machine='rhel6.5.0'>hvm</type>
> >>>>> > > <boot dev='cdrom'/>
> >>>>> > > <boot dev='hd'/>
> >>>>> > > </os>
> >>>>> > >
> >>>>> > > :(
> >>>>> > >
> >>>>> > >
> >>>>> > >
> >>>>> > > On 30 May 2014 16:24, Andrija Panic <andrija.pa...@gmail.com>
> >>>>> wrote:
> >>>>> > >
> >>>>> > > > I confirm, the highest is Debian 5 64bit
> >>>>> > > >
> >>>>> > > > Per docs
> >>>>> > > >
> >>>>> > >
> >>>>> >
> >>>>>
> http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.3/rnotes.html#upgrade-from-4-2-x-to-4-3
> >>>>> > > > , you should use Debian 7.0 64bit as OS type for system-kvm-4.3
> >>>>> > > template...
> >>>>> > > > (same for xen and vmware templatest)
> >>>>> > > >
> >>>>> > > > Will change now DB to point to debian 5, and let you know.
> >>>>> > > >
> >>>>> > > > This is guest_os_id: 184 | 2 | NULL |
> >>>>> > > > 986a0e98-39d6-11e3-8f93-0025904e4412 | Debian GNU/Linux
> 7(64-bit)
> >>>>> > > >
> >>>>> > > >
> >>>>> > > > On 30 May 2014 15:45, Joris van Lieshout <
> >>>>> > > jvanliesh...@schubergphilis.com>
> >>>>> > > > wrote:
> >>>>> > > >
> >>>>> > > >> Andrija,
> >>>>> > > >>
> >>>>> > > >> The thing is I don't know who the os matching on KVM works.
> >>>>> There must
> >>>>> > > be
> >>>>> > > >> a way to list supported os types.
> >>>>> > > >>
> >>>>> > > >> I also did some queuing on the guest_os_hypervisor table (ACS
> >>>>> 4.3)
> >>>>> > and I
> >>>>> > > >> don't see Debian 7 for KVM listed.
> >>>>> > > >>
> >>>>> > > >> select * from guest_os join guest_os_hypervisor on
> >>>>> > > >> guest_os.id=guest_os_hypervisor.guest_os_id where
> >>>>> > > >> guest_os_hypervisor.hypervisor_type='KVM' and
> >>>>> > > >> guest_os_hypervisor.guest_os_name like '%Debian%';
> >>>>> > > >>
> >>>>> > > >> What was the guest_os_id you where using? Could you try id 72
> >>>>> (Debian
> >>>>> > 5
> >>>>> > > >> 64-bit)? Adjust both os_type_id in vm_instance and vm_template
> >>>>> (where
> >>>>> > > >> type='SYSTEM' and hypervisor_type='KVM').
> >>>>> > > >>
> >>>>> > > >> Kind regards,
> >>>>> > > >> Joris van Lieshout
> >>>>> > > >>
> >>>>> > > >> Schuberg Philis
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > > >> On 30/05/14 15:33, "Andrija Panic" <andrija.pa...@gmail.com>
> >>>>> wrote:
> >>>>> > > >>
> >>>>> > > >> >Joris,
> >>>>> > > >> >
> >>>>> > > >> >do you have recommendation on how in particular to try ? I'm
> >>>>> not sure
> >>>>> > > how
> >>>>> > > >> >to fix that, except playing with editing systemvm-4.3
> template
> >>>>> to
> >>>>> > > define
> >>>>> > > >> >it
> >>>>> > > >> >as another OS type... ?
> >>>>> > > >> >
> >>>>> > > >> >Thanks again,
> >>>>> > > >> >Andrija
> >>>>> > > >> >
> >>>>> > > >> >
> >>>>> > > >> >On 30 May 2014 15:30, Joris van Lieshout <
> >>>>> > > >> jvanliesh...@schubergphilis.com>
> >>>>> > > >> >wrote:
> >>>>> > > >> >
> >>>>> > > >> >> I've read back a bit in the code and if you look at
> >>>>> > > >> BridgeVifDriver.java
> >>>>> > > >> >> (this is where the log message with the nic profile is
> >>>>> generated)
> >>>>> > you
> >>>>> > > >> >>can
> >>>>> > > >> >> see that the nic information might be off already once ACS
> >>>>> hits the
> >>>>> > > >> >> LibvirtVMDef.InterfaceDef plug function. This leads be to
> >>>>> believer
> >>>>> > > that
> >>>>> > > >> >> the HVM/PV OS mismatch issue might still be related. Try
> >>>>> fixing
> >>>>> > that
> >>>>> > > >> >> first. At least it will allow us to exclude this from the
> >>>>> list.
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >> >> Kind regards,
> >>>>> > > >> >> Joris van Lieshout
> >>>>> > > >> >>
> >>>>> > > >> >> Schuberg Philis
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >> >> On 30/05/14 15:26, "Andrija Panic" <
> andrija.pa...@gmail.com>
> >>>>> > wrote:
> >>>>> > > >> >>
> >>>>> > > >> >> >OK, thanks Joris.
> >>>>> > > >> >> >
> >>>>> > > >> >> >I will try playing with OS version option, on the
> >>>>> systemvm-kvm-4.3
> >>>>> > > >> >> >template...
> >>>>> > > >> >> >
> >>>>> > > >> >> >Let me know if I can help with anything more.
> >>>>> > > >> >> >
> >>>>> > > >> >> >Thanks.
> >>>>> > > >> >> >Andrija
> >>>>> > > >> >> >
> >>>>> > > >> >> >
> >>>>> > > >> >> >On 30 May 2014 15:19, Joris van Lieshout
> >>>>> > > >> >><jvanliesh...@schubergphilis.com
> >>>>> > > >> >> >
> >>>>> > > >> >> >wrote:
> >>>>> > > >> >> >
> >>>>> > > >> >> >> Hi Andrija,
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> That does sound familiar and in the start xml of KVM you
> >>>>> can see
> >>>>> > > >> >>"<type
> >>>>> > > >> >> >> arch='x86_64' machine='pc'>hvm</type>". I don't know
> >>>>> KVM+ACS
> >>>>> > well
> >>>>> > > >> >>enough
> >>>>> > > >> >> >> to judge if this is the cause but I thing focusing on
> >>>>> getting
> >>>>> > the
> >>>>> > > VR
> >>>>> > > >> >> >> started as PV guest might be worth trying. On the other
> >>>>> hand I
> >>>>> > do
> >>>>> > > >> see
> >>>>> > > >> >> >> patchviasocket.pl being executed successfully...
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> The other thing I see is, and now we're getting into
> java
> >>>>> code,
> >>>>> > is
> >>>>> > > >> >>this:
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> 2014-05-30 14:41:01,386{GMT} DEBUG
> >>>>> > [kvm.resource.BridgeVifDriver]
> >>>>> > > >> >> >> (agentRequest-Handler-3:)
> >>>>> > > >> >> >>nic=[Nic:Public-46.232.xxx.246-vlan://untagged]
> >>>>> > > >> >> >> 2014-05-30 14:41:01,502{GMT} DEBUG [cloud.agent.Agent]
> >>>>> > > >> >> >> (agentRequest-Handler-3:) Processing command:
> >>>>> > > >> >> >> com.cloud.agent.api.routing.IpAssocVpcCommand
> >>>>> > > >> >> >> 2014-05-30 14:41:01,506{GMT} DEBUG
> >>>>> > > >> >> >> [resource.virtualnetwork.VirtualRoutingResource]
> >>>>> > > >> >> >>(agentRequest-Handler-3:)
> >>>>> > > >> >> >> Executing:
> >>>>> > > >> >> >>
> >>>>> > /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh
> >>>>> > > >> >> >> vpc_ipassoc.sh 169.254.0.52 -A -l 46.232.xxx.246 -c
> >>>>> ethnull -g
> >>>>> > > >> >> >> 46.232.xxx.1 -m 24 -n 46.232.xxx.0
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> My suspicion is that somewhere in the translation from
> the
> >>>>> nic
> >>>>> > > >> >>profile
> >>>>> > > >> >> >>to
> >>>>> > > >> >> >> the actual route_proxy.sh command ACS failes to find the
> >>>>> nic id
> >>>>> > > and
> >>>>> > > >> >> >> returns null.
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> Let me dig a bit deeper and see what I can find but
> this is
> >>>>> > where
> >>>>> > > we
> >>>>> > > >> >> >>might
> >>>>> > > >> >> >> need some help from someone with knowledge of this pice
> of
> >>>>> the
> >>>>> > > code.
> >>>>> > > >> >>:)
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> Kind regards,
> >>>>> > > >> >> >> Joris van Lieshout
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> Schuberg Philis
> >>>>> > > >> >> >> Boeingavenue 271
> >>>>> > > >> >> >> 1119 PD Schiphol-Rijk
> >>>>> > > >> >> >> schubergphilis.com
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> +31 20-7506672
> >>>>> > > >> >> >> +31 6-51428188
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> On 30/05/14 14:49, "Andrija Panic" <
> >>>>> andrija.pa...@gmail.com>
> >>>>> > > wrote:
> >>>>> > > >> >> >>
> >>>>> > > >> >> >> >Hi Joris,
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >I have turned on DEBUG loging in agent.log on
> cs1.xxx/net
> >>>>> host:
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >So, management logs again:
> http://pastebin.com/F6BRf7Y9
> >>>>> > > >> >> >> >Agent logs on cs1.xxx: http://pastebin.com/BJauKbaC
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >Not playing smart, but there is some error:
> >>>>> > > >> >> >> >[kvm.resource.KVMGuestOsMapper]
> >>>>> > > >> >> >> >(agentRequest-Handler-3:) Can't find the mapping of
> guest
> >>>>> os:
> >>>>> > > >> Debian
> >>>>> > > >> >> >> >GNU/Linux 7(64-bit)
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >Best,
> >>>>> > > >> >> >> >Andrija
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >On 30 May 2014 14:26, Joris van Lieshout
> >>>>> > > >> >> >><jvanliesh...@schubergphilis.com
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >wrote:
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >> Hi Andrija,
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> Bold formatting does not come trough on the dev
> list. :)
> >>>>> > > >> >> >> >> But u might need a bit more info.
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> At a certain point I see this line
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> 2014-05-30 13:56:23,935 DEBUG [c.c.a.t.Request]
> >>>>> > > >> >> >> >> (Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Seq
> >>>>> 1-609104082:
> >>>>> > > >> >>Sending
> >>>>> > > >> >> >> {
> >>>>> > > >> >> >> >> Cmd , MgmtId: 161344838950, via: 1(cs1.xxxxx.net),
> >>>>> Ver: v1,
> >>>>> > > >> Flags:
> >>>>> > > >> >> >> >>100111,
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>[{"com.cloud.agent.api.StartCommand":{"vm":{"id":801,"name":"r-801-VM
> >>>>> > > >> >>>>>>".
> >>>>> > > >> >> >>>>..
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> This is where the information is passed on to the
> agent
> >>>>> > > handles.
> >>>>> > > >> >>For
> >>>>> > > >> >> >>XS
> >>>>> > > >> >> >> >> this would initiate an agent handler on the
> management
> >>>>> server
> >>>>> > > but
> >>>>> > > >> >>for
> >>>>> > > >> >> >> >>KVM,
> >>>>> > > >> >> >> >> if I remember correctly, it passed the command on to
> the
> >>>>> > > >> >>cloudstack
> >>>>> > > >> >> >> >>agent
> >>>>> > > >> >> >> >> service on the hypervisor.
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> Can you check the cloud service log on the KVM
> >>>>> hypervisor
> >>>>> > > >> >>executing
> >>>>> > > >> >> >>the
> >>>>> > > >> >> >> >> request? it's this server cs1.xxxxx.net and then
> >>>>> search top
> >>>>> > > down
> >>>>> > > >> >>for
> >>>>> > > >> >> >> >> 609104082 in the log. See if you can provide the log
> >>>>> from the
> >>>>> > > >> >>agent
> >>>>> > > >> >> >> >> handler thread started by that sequence.
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> Kind regards,
> >>>>> > > >> >> >> >> Joris van Lieshout
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> Schuberg Philis
> >>>>> > > >> >> >> >> Boeingavenue 271
> >>>>> > > >> >> >> >> 1119 PD Schiphol-Rijk
> >>>>> > > >> >> >> >> schubergphilis.com
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> +31 20-7506672
> >>>>> > > >> >> >> >> +31 6-51428188
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> On 30/05/14 14:08, "Andrija Panic" <
> >>>>> andrija.pa...@gmail.com>
> >>>>> > > >> >>wrote:
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >> >Hi Joris,
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >here is the management log:
> >>>>> http://pastebin.com/zxnKxFhk
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >Interesting parts (to me): in bold
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >2014-05-30 13:56:21,899 DEBUG
> >>>>> > > >> >>[o.a.c.s.m.AncientDataMotionStrategy]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> copyAsync
> >>>>> > > inspecting
> >>>>> > > >> >>src
> >>>>> > > >> >> >> >>type
> >>>>> > > >> >> >> >> >TEMPLATE copyAsync inspecting dest type VOLUME
> >>>>> > > >> >> >> >> >2014-05-30 13:56:21,905 DEBUG [c.c.a.t.Request]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Seq
> >>>>> > 4-1248669612:
> >>>>> > > >> >> >>Sending
> >>>>> > > >> >> >> >>{
> >>>>> > > >> >> >> >> >Cmd , MgmtId: 161344838950, via: 4(cs2.xxxxx.net),
> >>>>> Ver: v1,
> >>>>> > > >> >>Flags:
> >>>>> > > >> >> >> >> 100011,
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org
> >>>>> > > >> >>>>>>>.a
> >>>>> > > >> >> >>>>>pa
> >>>>> > > >> >> >> >>>ch
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>e.cloudstack.storage.to.TemplateObjectTO":{"path":"1adc1d2e-56ae-4a0
> >>>>> > > >> >>>>>>>f-
> >>>>> > > >> >> >>>>>b0
> >>>>> > > >> >> >> >>>b4
> >>>>> > > >> >> >> >> >-5e351e7cae55","origUrl":"
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-mas
> >>>>> > > >> >> >> >>t
> >>>>> > > >> >> >> >> >er-kvm.qcow2.bz2
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>","uuid":"1adc1d2e-56ae-4a0f-b0b4-5e351e7cae55","id":414,"format":"Q
> >>>>> > > >> >>>>>>>CO
> >>>>> > > >> >> >>>>>W2
> >>>>> > > >> >> >> >>>",
> >>>>> > > >> >> >> >>
> >>>>> > >"accountId":2,"checksum":"85a1bed07bf43cbf022451cb2ecae4ff","
> >>>>> > > >> >> >> >> >*hvm":true*
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>,"displayText":"systemvm-kvm-4.3","imageDataStore":{"org.apache.clou
> >>>>> > > >> >>>>>>>ds
> >>>>> > > >> >> >>>>>ta
> >>>>> > > >> >> >> >>>ck
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>.storage.to.PrimaryDataStoreTO":{"uuid":"5b93422e-1a66-353d-88a8-220
> >>>>> > > >> >>>>>>>3f
> >>>>> > > >> >> >>>>>79
> >>>>> > > >> >> >> >>>b1
> >>>>> > > >> >> >> >> >dc6","id":209,"poolType":"RBD","host":"
> >>>>> > > >> >> >> >> >cephmon.xxxxx.net
> >>>>> > > >> ","path":"cloudstack","port":6789,"url":"RBD://
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> cephmon.xxxxx.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-8
> >>>>> > > >> >> >> >>8
> >>>>> > > >> >> >> >> >a8-2203f79b1dc6
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>"}},"name":"414-2-ec331e74-5858-3153-91a9-1d706d9c533e","hypervisorT
> >>>>> > > >> >>>>>>>yp
> >>>>> > > >> >> >>>>>e"
> >>>>> > > >> >> >> >>>:"
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{
> >>>>> > > >> >>>>>>>"u
> >>>>> > > >> >> >>>>>ui
> >>>>> > > >> >> >> >>>d"
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>:"9c440d3b-cba5-4960-b8bf-dca90291cd2b","volumeType":"ROOT","dataSto
> >>>>> > > >> >>>>>>>re
> >>>>> > > >> >> >>>>>":
> >>>>> > > >> >> >> >>>{"
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5b9342
> >>>>> > > >> >>>>>>>2e
> >>>>> > > >> >> >>>>>-1
> >>>>> > > >> >> >> >>>a6
> >>>>> > > >> >> >> >>
> >>>>> >6-353d-88a8-2203f79b1dc6","id":209,"poolType":"RBD","host":"
> >>>>> > > >> >> >> >> >cephmon.xxxxx.net
> >>>>> > > >> ","path":"cloudstack","port":6789,"url":"RBD://
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> cephmon.xxxxx.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-8
> >>>>> > > >> >> >> >>8
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>a8-2203f79b1dc6"}},"name":"ROOT-801","size":2621440000,"volumeId":10
> >>>>> > > >> >>>>>>>64
> >>>>> > > >> >> >>>>>,"
> >>>>> > > >> >> >> >>>vm
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>Name":"r-801-VM","accountId":11,"format":"RAW","id":1064,"deviceId":
> >>>>> > > >> >>>>>>>0,
> >>>>> > > >> >> >>>>>"h
> >>>>> > > >> >> >> >>>yp
> >>>>> > > >> >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>ervisorType":"KVM"}},"executeInSequence":false,"options":{},"wait":0}}
> >>>>> > > >> >>>>>]
> >>>>> > > >> >> >> >> >}
> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,742 DEBUG [c.c.a.t.Request]
> >>>>> > > >> >> >> >> >(AgentManager-Handler-12:null) Seq 4-1248669612:
> >>>>> Processing:
> >>>>> > > {
> >>>>> > > >> >> >>Ans: ,
> >>>>> > > >> >> >> >> >MgmtId: 161344838950, via: 4, Ver: v1, Flags: 10,
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{
> >>>>> > > >> >>>>>>>"o
> >>>>> > > >> >> >>>>>rg
> >>>>> > > >> >> >> >>>.a
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>pache.cloudstack.storage.to.VolumeObjectTO":{"size":2621440000,"path
> >>>>> > > >> >>>>>>>":
> >>>>> > > >> >> >>>>>"9
> >>>>> > > >> >> >> >>>c4
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>40d3b-cba5-4960-b8bf-dca90291cd2b","accountId":0,"format":"RAW","id"
> >>>>> > > >> >>>>>>>:0
> >>>>> > > >> >> >>>>>}}
> >>>>> > > >> >> >> >>>,"
> >>>>> > > >> >> >> >> >result":true,"wait":0}}]
> >>>>> > > >> >> >> >> >}
> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,742 DEBUG [c.c.a.t.Request]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Seq
> >>>>> > 4-1248669612:
> >>>>> > > >> >> >> >>Received: {
> >>>>> > > >> >> >> >> >Ans: , MgmtId: 161344838950, via: 4, Ver: v1, Flags:
> >>>>> 10, {
> >>>>> > > >> >> >> >>CopyCmdAnswer
> >>>>> > > >> >> >> >> >} }
> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> > > >> >> >> >> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Removing
> >>>>> nic
> >>>>> > > >> >>NicProfile[
> >>>>> > > >> >> >> >> >*1092-801-null*-46.232.xxx.246-vlan://untagged of
> type
> >>>>> > Public
> >>>>> > > >> >>from
> >>>>> > > >> >> >>the
> >>>>> > > >> >> >> >> >nics
> >>>>> > > >> >> >> >> >passed on vm start. The nic will be plugged later
> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> > > >> >> >> >> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Removing
> >>>>> nic
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>NicProfile[1093-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.1.1-vl
> >>>>> > > >> >>>>>>>an
> >>>>> > > >> >> >>>>>:/
> >>>>> > > >> >> >> >>>/4
> >>>>> > > >> >> >> >> >4
> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm start. The
> >>>>> nic will
> >>>>> > > be
> >>>>> > > >> >> >>plugged
> >>>>> > > >> >> >> >> >later
> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> > > >> >> >> >> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Removing
> >>>>> nic
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>NicProfile[1094-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.3.1-vl
> >>>>> > > >> >>>>>>>an
> >>>>> > > >> >> >>>>>:/
> >>>>> > > >> >> >> >>>/4
> >>>>> > > >> >> >> >> >3
> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm start. The
> >>>>> nic will
> >>>>> > > be
> >>>>> > > >> >> >>plugged
> >>>>> > > >> >> >> >> >later
> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> > > >> >> >> >> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Removing
> >>>>> nic
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>NicProfile[1095-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.4.1-vl
> >>>>> > > >> >>>>>>>an
> >>>>> > > >> >> >>>>>:/
> >>>>> > > >> >> >> >>>/3
> >>>>> > > >> >> >> >> >004
> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm start. The
> >>>>> nic will
> >>>>> > > be
> >>>>> > > >> >> >>plugged
> >>>>> > > >> >> >> >> >later
> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> > > >> >> >> >> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Removing
> >>>>> nic
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >>
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>>>NicProfile[1095-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.4.1-vl
> >>>>> > > >> >>>>>>>an
> >>>>> > > >> >> >>>>>:/
> >>>>> > > >> >> >> >>>/3
> >>>>> > > >> >> >> >> >004
> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm start
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >Thanks,
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >On 30 May 2014 13:54, Joris van Lieshout
> >>>>> > > >> >> >> >><jvanliesh...@schubergphilis.com
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >wrote:
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >> Hi Andrija,
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >> Just the start of the VR should be sufficient.
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >> Kind regards,
> >>>>> > > >> >> >> >> >> Joris van Lieshout
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >> Schuberg Philis
> >>>>> > > >> >> >> >> >> Boeingavenue 271
> >>>>> > > >> >> >> >> >> 1119 PD Schiphol-Rijk
> >>>>> > > >> >> >> >> >> schubergphilis.com
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >> +31 20-7506672
> >>>>> > > >> >> >> >> >> +31 6-51428188
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >> On 30/05/14 13:48, "Andrija Panic" <
> >>>>> > andrija.pa...@gmail.com
> >>>>> > > >
> >>>>> > > >> >> >>wrote:
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >Hi Joris,
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >just to be sure - you want me to capture the log
> >>>>> from the
> >>>>> > > >> >>moment
> >>>>> > > >> >> >>I
> >>>>> > > >> >> >> >> >>reboot
> >>>>> > > >> >> >> >> >> >router - or you want me to stop it, then start
> >>>>> capturing
> >>>>> > > log,
> >>>>> > > >> >>and
> >>>>> > > >> >> >> >> >>start it
> >>>>> > > >> >> >> >> >> >(and continue capture untill ethnull errors
> inside
> >>>>> VR) ?
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >Thanks,
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >On 30 May 2014 13:39, Joris van Lieshout
> >>>>> > > >> >> >> >> >><jvanliesh...@schubergphilis.com
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >wrote:
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> Hi Andrija,
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> Thanks for the answers. In deed your situation
> is
> >>>>> > > different
> >>>>> > > >> >>so
> >>>>> > > >> >> >> >> >>PV/HVM is
> >>>>> > > >> >> >> >> >> >> not the issue.
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> When reading back the log output you have
> >>>>> provided I
> >>>>> > > noted
> >>>>> > > >> >>that
> >>>>> > > >> >> >> >>the
> >>>>> > > >> >> >> >> >>VR
> >>>>> > > >> >> >> >> >> >> messages log indicates that it's waiting for
> >>>>> ethnull to
> >>>>> > > be
> >>>>> > > >> >>up.
> >>>>> > > >> >> >> >>This
> >>>>> > > >> >> >> >> >> >>raises
> >>>>> > > >> >> >> >> >> >> the question where null was introduced instead
> of
> >>>>> 1.
> >>>>> > The
> >>>>> > > >> ACS
> >>>>> > > >> >> >> >> >>management
> >>>>> > > >> >> >> >> >> >> log output you send was, what I think, later
> down
> >>>>> the
> >>>>> > > road
> >>>>> > > >> >> >>where
> >>>>> > > >> >> >> >>ACS
> >>>>> > > >> >> >> >> >> >>gives
> >>>>> > > >> >> >> >> >> >> up trying to wait for the VR to come up. If you
> >>>>> would
> >>>>> > > >> >>capture
> >>>>> > > >> >> >>the
> >>>>> > > >> >> >> >> >> >> job-executor in the management log from
> >>>>> startCommand
> >>>>> > till
> >>>>> > > >> >>the
> >>>>> > > >> >> >> >> >>exception,
> >>>>> > > >> >> >> >> >> >> do you see anywhere a mention of ethnull? You
> >>>>> might
> >>>>> > need
> >>>>> > > to
> >>>>> > > >> >> >>reed
> >>>>> > > >> >> >> >>into
> >>>>> > > >> >> >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> DirectAgent executing the startCommand to find
> a
> >>>>> clue.
> >>>>> > > The
> >>>>> > > >> >> >>thing
> >>>>> > > >> >> >> >>is
> >>>>> > > >> >> >> >> >> >>that I
> >>>>> > > >> >> >> >> >> >> only have experience with XS based environment
> so
> >>>>> I
> >>>>> > > cannot
> >>>>> > > >> >> >>point
> >>>>> > > >> >> >> >>you
> >>>>> > > >> >> >> >> >>to
> >>>>> > > >> >> >> >> >> >> the exact output to look for. On XS, at least,
> it
> >>>>> is
> >>>>> > > >> >> >> >> >> >> "[c.c.h.x.r.CitrixResourceBase]
> >>>>> > > >> >>(DirectAgent-351:ctx-4a51bb9e)
> >>>>> > > >> >> >> >> >>Created a
> >>>>> > > >> >> >> >> >> >> vif e4c362bd-764b-f651-dc9a-1abd5cb33c43 on 1"
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> Kind regards,
> >>>>> > > >> >> >> >> >> >> Joris van Lieshout
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> Schuberg Philis
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> On 30/05/14 10:48, "Andrija Panic" <
> >>>>> > > >> andrija.pa...@gmail.com>
> >>>>> > > >> >> >> >>wrote:
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >Hi Deen,
> >>>>> > > >> >> >> >> >> >> >no, in DB there is field "vlan_id" with value
> >>>>> > > "untagged" -
> >>>>> > > >> >> >>that
> >>>>> > > >> >> >> >> >> >> >"vlan://untagged" is shown from ACS gui, and
> is
> >>>>> used
> >>>>> > in
> >>>>> > > >> API
> >>>>> > > >> >> >>call
> >>>>> > > >> >> >> >>(or
> >>>>> > > >> >> >> >> >> >> >better
> >>>>> > > >> >> >> >> >> >> >said commands that are seen in management
> server
> >>>>> > logs).
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >Best,
> >>>>> > > >> >> >> >> >> >> >Andrija
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >On 30 May 2014 10:37, Daan Hoogland
> >>>>> > > >> >><daan.hoogl...@gmail.com>
> >>>>> > > >> >> >> >> wrote:
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> Andrija,
> >>>>> > > >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> Do not just assign a second net vlan://500
> You
> >>>>> have
> >>>>> > > one
> >>>>> > > >> >>like
> >>>>> > > >> >> >> >>that
> >>>>> > > >> >> >> >> >>and
> >>>>> > > >> >> >> >> >> >> >> you don't want conflicting nets using the
> same
> >>>>> > vlan. I
> >>>>> > > >> am
> >>>>> > > >> >> >> >> >>wondering
> >>>>> > > >> >> >> >> >> >> >> why 'untagged' comes out as
> 'vlan://untagged'.
> >>>>> I
> >>>>> > think
> >>>>> > > >> >>that
> >>>>> > > >> >> >>is
> >>>>> > > >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> >> bug. Did you find the string
> 'vlan://untagged'
> >>>>> in
> >>>>> > your
> >>>>> > > >> >>db?
> >>>>> > > >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> On Fri, May 30, 2014 at 10:20 AM, Andrija
> Panic
> >>>>> > > >> >> >> >> >> >> >><andrija.pa...@gmail.com>
> >>>>> > > >> >> >> >> >> >> >> wrote:
> >>>>> > > >> >> >> >> >> >> >> > Hi Joris,
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > thank you for taking time to address this
> >>>>> issue :)
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > So...:
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > - I'm on KVM (stock CentOS 6.2 patched by
> >>>>> Inktank
> >>>>> > > for
> >>>>> > > >> >>CEPH
> >>>>> > > >> >> >> >> >> >>support),
> >>>>> > > >> >> >> >> >> >> >>OS
> >>>>> > > >> >> >> >> >> >> >> is
> >>>>> > > >> >> >> >> >> >> >> > Centos 6.5, libvirt 1.2.3 compiled.
> >>>>> > > >> >> >> >> >> >> >> > - ACS 4.3 having problems, ACS 4.2.1 was
> fine
> >>>>> > > >> >> >> >> >> >> >> > - not XS, so I guess no answers for this
> >>>>> part :)
> >>>>> > > >> >> >> >> >> >> >> > - guest_os_id is 184 = Debian 7 x64
> >>>>> > > >> >> >> >> >> >> >> > - SVM = systemvm-kvm-4.3 = os type 184 =
> >>>>> Debian 7
> >>>>> > > x64
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > This worked previously on 4.2.1 = template
> >>>>> was
> >>>>> > > >> ofcourse
> >>>>> > > >> >> >> >> >> >> >>systemvm-kvm-4.2
> >>>>> > > >> >> >> >> >> >> >> -
> >>>>> > > >> >> >> >> >> >> >> > but that was also Debian 7 x64 type... so
> >>>>> this
> >>>>> > > should
> >>>>> > > >> >>not
> >>>>> > > >> >> >>be
> >>>>> > > >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> >>issues
> >>>>> > > >> >> >> >> >> >> >> > (guest not supported by host...)
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > The only thing that might be out of
> >>>>> "standard" =
> >>>>> > all
> >>>>> > > >> >>SVMs
> >>>>> > > >> >> >> >>are on
> >>>>> > > >> >> >> >> >> >>CEPH
> >>>>> > > >> >> >> >> >> >> >>-
> >>>>> > > >> >> >> >> >> >> >> > there are official docs on altering
> database
> >>>>> to
> >>>>> > make
> >>>>> > > >> >>some
> >>>>> > > >> >> >>new
> >>>>> > > >> >> >> >> >> >>System
> >>>>> > > >> >> >> >> >> >> >> > Offering as default for SSVM and CPVM -
> what
> >>>>> I
> >>>>> > did,
> >>>>> > > I
> >>>>> > > >> >>also
> >>>>> > > >> >> >> >>have
> >>>>> > > >> >> >> >> >> >>done
> >>>>> > > >> >> >> >> >> >> >>same
> >>>>> > > >> >> >> >> >> >> >> > config in DB, to make VR use another
> System
> >>>>> > Offering
> >>>>> > > >> as
> >>>>> > > >> >> >> >>default
> >>>>> > > >> >> >> >> >>-
> >>>>> > > >> >> >> >> >> >> >>which
> >>>>> > > >> >> >> >> >> >> >> is
> >>>>> > > >> >> >> >> >> >> >> > NOT explained in the docs - you could use
> >>>>> "Change
> >>>>> > > >> >> >> >>Offering..."
> >>>>> > > >> >> >> >> >> >>button
> >>>>> > > >> >> >> >> >> >> >>on
> >>>>> > > >> >> >> >> >> >> >> > exiting, shutdown VR to change it per
> docs...
> >>>>> > > >> >> >> >> >> >> >> > But still this worked all fine on 4.2.1...
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > - regarding /var/cache/cloud/cmdline the
> >>>>> content
> >>>>> > is
> >>>>> > > >> >> >>folowing
> >>>>> > > >> >> >> >>at
> >>>>> > > >> >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> >> moment
> >>>>> > > >> >> >> >> >> >> >> > root@r-801-VM:~# cat
> >>>>> /var/cache/cloud/cmdline
> >>>>> > > >> >> >> >> >> >> >> > vpccidr=10.0.0.0/8
> domain=cscloud.internal
> >>>>> > > >> dns1=8.8.8.8
> >>>>> > > >> >> >> dns2=
> >>>>> > > >> >> >> >> >> >> >> template=domP
> >>>>> > > >> >> >> >> >> >> >> > name=r-801-VM eth0ip=169.254.0.75
> >>>>> > > eth0mask=255.255.0.0
> >>>>> > > >> >> >> >> >> >>type=vpcrouter
> >>>>> > > >> >> >> >> >> >> >> > disable_rp_filter=true
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > Also please note that only eth1 does not
> >>>>> have IP
> >>>>> > > info,
> >>>>> > > >> >> >>eth0
> >>>>> > > >> >> >> >> >> >>(control
> >>>>> > > >> >> >> >> >> >> >> > 169.xxx) and all other eh2 and up that are
> >>>>> used
> >>>>> > for
> >>>>> > > >> >>Tiers
> >>>>> > > >> >> >> >>get IP
> >>>>> > > >> >> >> >> >> >>info
> >>>>> > > >> >> >> >> >> >> >> fine.
> >>>>> > > >> >> >> >> >> >> >> > I could also manually add IP for eth1
> >>>>> (public NIC)
> >>>>> > > and
> >>>>> > > >> >> >>start
> >>>>> > > >> >> >> >> >>ifup
> >>>>> > > >> >> >> >> >> >> >>eth1 -
> >>>>> > > >> >> >> >> >> >> >> > and it works fine, but adding new IP Port
> >>>>> > Forwarding
> >>>>> > > >> >>etc
> >>>>> > > >> >> >>does
> >>>>> > > >> >> >> >> >>not
> >>>>> > > >> >> >> >> >> >> >>work...
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > Daan or somebody said it could be realted
> to
> >>>>> my
> >>>>> > > >> >>"Public"
> >>>>> > > >> >> >> >>network
> >>>>> > > >> >> >> >> >> >>(in
> >>>>> > > >> >> >> >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> >> > Zones, Physical Network, eth1 listing) is
> NOT
> >>>>> > tagged
> >>>>> > > >> >> >> >> >> >> >>(vlan://untagged)...
> >>>>> > > >> >> >> >> >> >> >> > Interestingly the only VR that does work
> >>>>> fine is
> >>>>> > the
> >>>>> > > >> VR
> >>>>> > > >> >> >>used
> >>>>> > > >> >> >> >>in
> >>>>> > > >> >> >> >> >> >>Shared
> >>>>> > > >> >> >> >> >> >> >> > network, but that VR is using IP from
> Guest
> >>>>> IP
> >>>>> > range
> >>>>> > > >> >>(also
> >>>>> > > >> >> >> >> >> >>efectively
> >>>>> > > >> >> >> >> >> >> >> > public IPs but on vlan 500)
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > I was instructed to try to change Public
> IP
> >>>>> range
> >>>>> > > from
> >>>>> > > >> >> >> >>untagged
> >>>>> > > >> >> >> >> >>to
> >>>>> > > >> >> >> >> >> >> >>vlan
> >>>>> > > >> >> >> >> >> >> >> > 500, but I'm not sure how to do this, if
> >>>>> there is
> >>>>> > > any
> >>>>> > > >> >>way
> >>>>> > > >> >> >>at
> >>>>> > > >> >> >> >>all
> >>>>> > > >> >> >> >> >> >> >>(editing
> >>>>> > > >> >> >> >> >> >> >> > "vlan" table and changing to vlan 500 does
> >>>>> not
> >>>>> > work,
> >>>>> > > >> >>after
> >>>>> > > >> >> >> >> >> >>rebooting
> >>>>> > > >> >> >> >> >> >> >>VR
> >>>>> > > >> >> >> >> >> >> >> > from ACS gui).
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > :)
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > So, not sure what is roughly expected date
> >>>>> for
> >>>>> > 4.4,
> >>>>> > > >> but
> >>>>> > > >> >> >>right
> >>>>> > > >> >> >> >> >>now,
> >>>>> > > >> >> >> >> >> >>I'm
> >>>>> > > >> >> >> >> >> >> >> > pretty stuck with a big problem of all VPC
> >>>>> not
> >>>>> > > >> >> >>operational at
> >>>>> > > >> >> >> >> >> >>all...
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > Thanks,
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > On 30 May 2014 08:27, Joris van Lieshout <
> >>>>> > > >> >> >> >> >> >> >> jvanliesh...@schubergphilis.com>
> >>>>> > > >> >> >> >> >> >> >> > wrote:
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> Hi Andrija,
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> Daan asked me to have a look at this as
> >>>>> well.
> >>>>> > > Looking
> >>>>> > > >> >>at
> >>>>> > > >> >> >>you
> >>>>> > > >> >> >> >> >> >>issue I
> >>>>> > > >> >> >> >> >> >> >> >> recall having seen something similar.
> Back
> >>>>> then
> >>>>> > > when
> >>>>> > > >> >> >> >>upgrading
> >>>>> > > >> >> >> >> >> >>4.2.1
> >>>>> > > >> >> >> >> >> >> >>to
> >>>>> > > >> >> >> >> >> >> >> >> 4.3 I though it had to do with out own
> >>>>> custom
> >>>>> > build
> >>>>> > > >> >>svm
> >>>>> > > >> >> >> >> >>template.
> >>>>> > > >> >> >> >> >> >> >> >> Let me fire off some questions before
> >>>>> explaining
> >>>>> > > what
> >>>>> > > >> >>the
> >>>>> > > >> >> >> >>cause
> >>>>> > > >> >> >> >> >> >>was
> >>>>> > > >> >> >> >> >> >> >>in
> >>>>> > > >> >> >> >> >> >> >> our
> >>>>> > > >> >> >> >> >> >> >> >> case. :)
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> - what hypervisor (and version) are you
> >>>>> using?
> >>>>> > > >> >> >> >> >> >> >> >> - if XS, is the new VR a para-virtualised
> >>>>> > instance
> >>>>> > > >> >>(PV)
> >>>>> > > >> >> >>or
> >>>>> > > >> >> >> >> >> >>hardware
> >>>>> > > >> >> >> >> >> >> >> >> assisted (HVM)? Do a "xe vm-param-list"
> on
> >>>>> the VR
> >>>>> > > >> uuid
> >>>>> > > >> >> >>and
> >>>>> > > >> >> >> >> >>check
> >>>>> > > >> >> >> >> >> >>that
> >>>>> > > >> >> >> >> >> >> >> >> param PV-args is set and HVM-boot-policy
> is
> >>>>> > unset.
> >>>>> > > >> >> >> >> >> >> >> >> - what is the OS type of the VR in ACS
> >>>>> > (guest_os_id
> >>>>> > > >> in
> >>>>> > > >> >> >> >> >>vm_instance
> >>>>> > > >> >> >> >> >> >> >>table
> >>>>> > > >> >> >> >> >> >> >> >> and match with table guest_os)
> >>>>> > > >> >> >> >> >> >> >> >> - what is the OS type of the SVM
> template?
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> Now for the explaining. :)
> >>>>> > > >> >> >> >> >> >> >> >> In our case the OS type of the new
> template
> >>>>> was
> >>>>> > not
> >>>>> > > >> >> >> >>supported
> >>>>> > > >> >> >> >> >>on
> >>>>> > > >> >> >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> >> >> XenServer version we are running.
> Therefore
> >>>>> the
> >>>>> > VR
> >>>>> > > >> was
> >>>>> > > >> >> >> >>started
> >>>>> > > >> >> >> >> >>by
> >>>>> > > >> >> >> >> >> >>XS
> >>>>> > > >> >> >> >> >> >> >>as
> >>>>> > > >> >> >> >> >> >> >> a
> >>>>> > > >> >> >> >> >> >> >> >> HVM guest. System vms on XS rely on the
> >>>>> arguments
> >>>>> > > >> >>passed
> >>>>> > > >> >> >>to
> >>>>> > > >> >> >> >> >>them
> >>>>> > > >> >> >> >> >> >>in
> >>>>> > > >> >> >> >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> >> >> PV-args param (ends up on the guest in
> >>>>> > > >> >> >> >>/var/cache/cloud/cmdline
> >>>>> > > >> >> >> >> >> >> >>which in
> >>>>> > > >> >> >> >> >> >> >> >> turn is used by cloud-early-config) in
> >>>>> order to
> >>>>> > > work.
> >>>>> > > >> >> >> >>cmdline
> >>>>> > > >> >> >> >> >> >> >>contains
> >>>>> > > >> >> >> >> >> >> >> the
> >>>>> > > >> >> >> >> >> >> >> >> NIC configuration information.
> >>>>> > > >> >> >> >> >> >> >> >> So, long story short, if a VR gets
> started
> >>>>> as a
> >>>>> > HVM
> >>>>> > > >> it
> >>>>> > > >> >> >>will
> >>>>> > > >> >> >> >>not
> >>>>> > > >> >> >> >> >> >>get
> >>>>> > > >> >> >> >> >> >> >>the
> >>>>> > > >> >> >> >> >> >> >> >> information needed to configure it's
> NICs.
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> Workaround
> >>>>> > > >> >> >> >> >> >> >> >> We corrected the os_type_id in the DB
> (yes
> >>>>> I know
> >>>>> > > >> >>editing
> >>>>> > > >> >> >> >>the
> >>>>> > > >> >> >> >> >>DB
> >>>>> > > >> >> >> >> >> >>is
> >>>>> > > >> >> >> >> >> >> >> >> something you usually don't want but
> there
> >>>>> is no
> >>>>> > > >> other
> >>>>> > > >> >> >>way
> >>>>> > > >> >> >> >>in
> >>>>> > > >> >> >> >> >>this
> >>>>> > > >> >> >> >> >> >> >>case)
> >>>>> > > >> >> >> >> >> >> >> >> of the existing VR's and of the
> >>>>> systemvmtemplate
> >>>>> > to
> >>>>> > > >> >> >> >>something
> >>>>> > > >> >> >> >> >> >> >>supported
> >>>>> > > >> >> >> >> >> >> >> by
> >>>>> > > >> >> >> >> >> >> >> >> XenServer.
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> Kind regards,
> >>>>> > > >> >> >> >> >> >> >> >> Joris van Lieshout
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> Schuberg Philis
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> On 29/05/14 12:18, "Andrija Panic"
> >>>>> > > >> >> >><andrija.pa...@gmail.com
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >> >> >>wrote:
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >They are 2 traffic types on 1 physical
> net
> >>>>> (that
> >>>>> > > is
> >>>>> > > >> >>both
> >>>>> > > >> >> >> >> >>tagged
> >>>>> > > >> >> >> >> >> >>vlan
> >>>>> > > >> >> >> >> >> >> >> 500,
> >>>>> > > >> >> >> >> >> >> >> >> >and untagged packets travel over same
> KVM
> >>>>> > bridge,
> >>>>> > > >> and
> >>>>> > > >> >> >>over
> >>>>> > > >> >> >> >> >>eth1
> >>>>> > > >> >> >> >> >> >>to
> >>>>> > > >> >> >> >> >> >> >> outside
> >>>>> > > >> >> >> >> >> >> >> >> >world)...
> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >On 29 May 2014 12:04, Daan Hoogland
> >>>>> > > >> >> >> >><daan.hoogl...@gmail.com>
> >>>>> > > >> >> >> >> >> >> wrote:
> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >> Are these two traffic types in one
> >>>>> physical
> >>>>> > net?
> >>>>> > > >> or
> >>>>> > > >> >> >>two
> >>>>> > > >> >> >> >> >> >>physical
> >>>>> > > >> >> >> >> >> >> >>nets
> >>>>> > > >> >> >> >> >> >> >> >> >> on the same interface (seems wrong).
> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> On Thu, May 29, 2014 at 11:35 AM,
> Jayapal
> >>>>> > Reddy
> >>>>> > > >> >>Uradi
> >>>>> > > >> >> >> >> >> >> >> >> >> <jayapalreddy.ur...@citrix.com>
> wrote:
> >>>>> > > >> >> >> >> >> >> >> >> >> > I don't think editing DB table will
> >>>>> work.
> >>>>> > > >> >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >> > -Jayapal
> >>>>> > > >> >> >> >> >> >> >> >> >> > On 29-May-2014, at 2:52 PM, Andrija
> >>>>> Panic
> >>>>> > > >> >> >> >> >> >> >><andrija.pa...@gmail.com
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >> wrote:
> >>>>> > > >> >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >> >> It's like this:
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >> I have public subnet /24.
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >> half is dedicated for Guest traffic
> >>>>> (vlan
> >>>>> > > 500)
> >>>>> > > >> >>and
> >>>>> > > >> >> >>the
> >>>>> > > >> >> >> >> >> >>second
> >>>>> > > >> >> >> >> >> >> >> half is
> >>>>> > > >> >> >> >> >> >> >> >> >> >> dedicated to Public traffic/network
> >>>>> (no
> >>>>> > vlan
> >>>>> > > >> >>tags,
> >>>>> > > >> >> >> >>that
> >>>>> > > >> >> >> >> >>is
> >>>>> > > >> >> >> >> >> >> >> untagged
> >>>>> > > >> >> >> >> >> >> >> >> >> packets)
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >> Both vlan500 and untagged packets
> >>>>> travel
> >>>>> > over
> >>>>> > > >> >> >>physical
> >>>>> > > >> >> >> >> >>eth1
> >>>>> > > >> >> >> >> >> >> >> >> >>interface on
> >>>>> > > >> >> >> >> >> >> >> >> >> >> hypervisors and can reach Internet.
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >> Thanks,
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >> On 29 May 2014 11:06, Daan Hoogland
> >>>>> > > >> >> >> >> >> >><daan.hoogl...@gmail.com>
> >>>>> > > >> >> >> >> >> >> >> wrote:
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>> On Thu, May 29, 2014 at 10:57 AM,
> >>>>> Andrija
> >>>>> > > >> >>Panic <
> >>>>> > > >> >> >> >> >> >> >> >> >> andrija.pa...@gmail.com>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>> wrote:
> >>>>> > > >> >> >> >> >> >> >> >> >> >>>> 500
> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>> is 500 the vlan of your
> guestnetwork
> >>>>> or
> >>>>> > your
> >>>>> > > >> >> >>physical
> >>>>> > > >> >> >> >> >> >>network?
> >>>>> > > >> >> >> >> >> >> >> You
> >>>>> > > >> >> >> >> >> >> >> >> >> >>> wouldn't want to have two nets
> with
> >>>>> vlan
> >>>>> > > 500!
> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>> --
> >>>>> > > >> >> >> >> >> >> >> >> >> >>> Daan
> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >> --
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> >> Andrija Panić
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> --------------------------------------
> >>>>> > > >> >> >> >> >> >> >> >> >> >> http://admintweets.com
> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> --------------------------------------
> >>>>> > > >> >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >> --
> >>>>> > > >> >> >> >> >> >> >> >> >> Daan
> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >--
> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >> >Andrija Panić
> >>>>> > > >> >> >> >> >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >> >> >> >> >> > http://admintweets.com
> >>>>> > > >> >> >> >> >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > --
> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >> > Andrija Panić
> >>>>> > > >> >> >> >> >> >> >> > --------------------------------------
> >>>>> > > >> >> >> >> >> >> >> > http://admintweets.com
> >>>>> > > >> >> >> >> >> >> >> > --------------------------------------
> >>>>> > > >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >> --
> >>>>> > > >> >> >> >> >> >> >> Daan
> >>>>> > > >> >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >--
> >>>>> > > >> >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >> >Andrija Panić
> >>>>> > > >> >> >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >> >> >> > http://admintweets.com
> >>>>> > > >> >> >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >>
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >--
> >>>>> > > >> >> >> >> >> >
> >>>>> > > >> >> >> >> >> >Andrija Panić
> >>>>> > > >> >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >> >> > http://admintweets.com
> >>>>> > > >> >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >>
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >--
> >>>>> > > >> >> >> >> >
> >>>>> > > >> >> >> >> >Andrija Panić
> >>>>> > > >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >> > http://admintweets.com
> >>>>> > > >> >> >> >> >--------------------------------------
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >>
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >--
> >>>>> > > >> >> >> >
> >>>>> > > >> >> >> >Andrija Panić
> >>>>> > > >> >> >> >--------------------------------------
> >>>>> > > >> >> >> > http://admintweets.com
> >>>>> > > >> >> >> >--------------------------------------
> >>>>> > > >> >> >>
> >>>>> > > >> >> >>
> >>>>> > > >> >> >
> >>>>> > > >> >> >
> >>>>> > > >> >> >--
> >>>>> > > >> >> >
> >>>>> > > >> >> >Andrija Panić
> >>>>> > > >> >> >--------------------------------------
> >>>>> > > >> >> > http://admintweets.com
> >>>>> > > >> >> >--------------------------------------
> >>>>> > > >> >>
> >>>>> > > >> >>
> >>>>> > > >> >
> >>>>> > > >> >
> >>>>> > > >> >--
> >>>>> > > >> >
> >>>>> > > >> >Andrija Panić
> >>>>> > > >> >--------------------------------------
> >>>>> > > >> > http://admintweets.com
> >>>>> > > >> >--------------------------------------
> >>>>> > > >>
> >>>>> > > >>
> >>>>> > > >
> >>>>> > > >
> >>>>> > > > --
> >>>>> > > >
> >>>>> > > > Andrija Panić
> >>>>> > > > --------------------------------------
> >>>>> > > > http://admintweets.com
> >>>>> > > > --------------------------------------
> >>>>> > > >
> >>>>> > >
> >>>>> > >
> >>>>> > >
> >>>>> > > --
> >>>>> > >
> >>>>> > > Andrija Panić
> >>>>> > > --------------------------------------
> >>>>> > > http://admintweets.com
> >>>>> > > --------------------------------------
> >>>>> > >
> >>>>> >
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Andrija Panić
> >>>>> --------------------------------------
> >>>>> http://admintweets.com
> >>>>> --------------------------------------
> >>>>>
> >>>>
> >>>>
> >>>
> >>
>
>
>
> --
> Daan
>
