Re: Circumventing VXLAN MTU issues

[Andrija Panic]

Hi Nux,

there is one rare contributions from my side :D to the ACS (documentation,
2.5 years ago) - check it here:
http://docs.cloudstack.apache.org/en/latest/networking/vxlan.html#important-note-on-mtu-size

We are using it extensively, feel free to ask anything if needed.

Cheers

On 20 November 2017 at 19:38, Nux!  wrote:

> Yeah, I changed the MTU to 9000 on the Guest network interface and
> connectivity is now fine out of the box.
> Hopefully I won't need to set jumbo frames in the VMs any time soon. :-)
>
> Thanks for the tips.
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Simon Weller" 
> > To: "users" 
> > Cc: "dev" 
> > Sent: Monday, 20 November, 2017 17:36:31
> > Subject: Re: Circumventing VXLAN MTU issues
>
> > Change your host interface MTU to something a lot higher.
> >
> >
> > 
> > From: Nux! 
> > Sent: Monday, November 20, 2017 10:51 AM
> > To: users
> > Cc: dev
> > Subject: Re: Circumventing VXLAN MTU issues
> >
> > Hi,
> >
> > I probably can use jumbo frames, but for now my lab is restricted to a
> single
> > machine.
> > Anything I can do in this situation?
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > - Original Message -
> >> From: "Simon Weller" 
> >> To: "dev" , "users" <
> us...@cloudstack.apache.org>
> >> Sent: Monday, 20 November, 2017 16:38:33
> >> Subject: Re: Circumventing VXLAN MTU issues
> >
> >> Lucian,
> >>
> >> Can you run jumbos on your switches?
> >>
> >> - Si
> >>
> >>
> >> 
> >> From: Nux! 
> >> Sent: Monday, November 20, 2017 10:08 AM
> >> To: dev
> >> Cc: users
> >> Subject: Circumventing VXLAN MTU issues
> >>
> >> Hello,
> >>
> >> I am playing around with the native VXLAN implementation and I have of
> course
> >> hit the situation where MTU on the host gets chomped by 50 bytes and
> traffic
> >> crawls to a stop in the VMs.
> >> How are you people circumventing this? The easiest, but kind of
> inconvenient for
> >> customers would be to set MTU 1450 inside the VM, though it could be
> baked into
> >> the templates or via cloud-init.
> >>
> >> Is the OpenVSwitch implementation (of VXLAN) suffer from the same
> problem, I
> >> understand ovs can dynamically adjust MTUs to compensate (?), but at
> the same
> >> time this VXLAN implementation suffers from some limitations (multicast
> etc).
> >>
> >> Lucian
> >>
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> > > www.nux.ro
>



-- 

Andrija Panić

Re: Circumventing VXLAN MTU issues

[Nux!]

Thanks Andrija, well done, I have indeed read your document.
Using MTU 9000 solved my problem. :)

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Andrija Panic" 
> To: "dev" 
> Cc: "users" 
> Sent: Thursday, 21 December, 2017 14:09:43
> Subject: Re: Circumventing VXLAN MTU issues

> Hi Nux,
> 
> there is one rare contributions from my side :D to the ACS (documentation,
> 2.5 years ago) - check it here:
> http://docs.cloudstack.apache.org/en/latest/networking/vxlan.html#important-note-on-mtu-size
> 
> We are using it extensively, feel free to ask anything if needed.
> 
> Cheers
> 
> On 20 November 2017 at 19:38, Nux!  wrote:
> 
>> Yeah, I changed the MTU to 9000 on the Guest network interface and
>> connectivity is now fine out of the box.
>> Hopefully I won't need to set jumbo frames in the VMs any time soon. :-)
>>
>> Thanks for the tips.
>>
>> Lucian
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> - Original Message -
>> > From: "Simon Weller" 
>> > To: "users" 
>> > Cc: "dev" 
>> > Sent: Monday, 20 November, 2017 17:36:31
>> > Subject: Re: Circumventing VXLAN MTU issues
>>
>> > Change your host interface MTU to something a lot higher.
>> >
>> >
>> > 
>> > From: Nux! 
>> > Sent: Monday, November 20, 2017 10:51 AM
>> > To: users
>> > Cc: dev
>> > Subject: Re: Circumventing VXLAN MTU issues
>> >
>> > Hi,
>> >
>> > I probably can use jumbo frames, but for now my lab is restricted to a
>> single
>> > machine.
>> > Anything I can do in this situation?
>> >
>> > --
>> > Sent from the Delta quadrant using Borg technology!
>> >
>> > Nux!
>> > www.nux.ro
>> >
>> > - Original Message -
>> >> From: "Simon Weller" 
>> >> To: "dev" , "users" <
>> us...@cloudstack.apache.org>
>> >> Sent: Monday, 20 November, 2017 16:38:33
>> >> Subject: Re: Circumventing VXLAN MTU issues
>> >
>> >> Lucian,
>> >>
>> >> Can you run jumbos on your switches?
>> >>
>> >> - Si
>> >>
>> >>
>> >> 
>> >> From: Nux! 
>> >> Sent: Monday, November 20, 2017 10:08 AM
>> >> To: dev
>> >> Cc: users
>> >> Subject: Circumventing VXLAN MTU issues
>> >>
>> >> Hello,
>> >>
>> >> I am playing around with the native VXLAN implementation and I have of
>> course
>> >> hit the situation where MTU on the host gets chomped by 50 bytes and
>> traffic
>> >> crawls to a stop in the VMs.
>> >> How are you people circumventing this? The easiest, but kind of
>> inconvenient for
>> >> customers would be to set MTU 1450 inside the VM, though it could be
>> baked into
>> >> the templates or via cloud-init.
>> >>
>> >> Is the OpenVSwitch implementation (of VXLAN) suffer from the same
>> problem, I
>> >> understand ovs can dynamically adjust MTUs to compensate (?), but at
>> the same
>> >> time this VXLAN implementation suffers from some limitations (multicast
>> etc).
>> >>
>> >> Lucian
>> >>
>> >>
>> >> --
>> >> Sent from the Delta quadrant using Borg technology!
>> >>
>> >> Nux!
>> > > www.nux.ro
>>
> 
> 
> 
> --
> 
> Andrija Panić

Re: Circumventing VXLAN MTU issues

[Andrija Panic]

while there, here is another one (just below that one) :) which I learned
the very hard way, after 2 years in production (when clients started
complaining on random traffic drop) :
http://docs.cloudstack.apache.org/en/latest/networking/vxlan.html#important-note-on-max-number-of-multicast-groups-and-thus-vxlan-intefaces

Very sneaky and ugly: Ubuntu kernel 3.16 = random traffic drops, but all
vxlan interfaces are UP etc..., Ubuntu kernel 4.x vxlan interface would NOT
come  up AT ALL, so a hard drop of packets, and there is where we saw the
issue...

Cheers


On 21 December 2017 at 15:13, Nux!  wrote:

> Thanks Andrija, well done, I have indeed read your document.
> Using MTU 9000 solved my problem. :)
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Andrija Panic" 
> > To: "dev" 
> > Cc: "users" 
> > Sent: Thursday, 21 December, 2017 14:09:43
> > Subject: Re: Circumventing VXLAN MTU issues
>
> > Hi Nux,
> >
> > there is one rare contributions from my side :D to the ACS
> (documentation,
> > 2.5 years ago) - check it here:
> > http://docs.cloudstack.apache.org/en/latest/networking/
> vxlan.html#important-note-on-mtu-size
> >
> > We are using it extensively, feel free to ask anything if needed.
> >
> > Cheers
> >
> > On 20 November 2017 at 19:38, Nux!  wrote:
> >
> >> Yeah, I changed the MTU to 9000 on the Guest network interface and
> >> connectivity is now fine out of the box.
> >> Hopefully I won't need to set jumbo frames in the VMs any time soon. :-)
> >>
> >> Thanks for the tips.
> >>
> >> Lucian
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >> - Original Message -
> >> > From: "Simon Weller" 
> >> > To: "users" 
> >> > Cc: "dev" 
> >> > Sent: Monday, 20 November, 2017 17:36:31
> >> > Subject: Re: Circumventing VXLAN MTU issues
> >>
> >> > Change your host interface MTU to something a lot higher.
> >> >
> >> >
> >> > 
> >> > From: Nux! 
> >> > Sent: Monday, November 20, 2017 10:51 AM
> >> > To: users
> >> > Cc: dev
> >> > Subject: Re: Circumventing VXLAN MTU issues
> >> >
> >> > Hi,
> >> >
> >> > I probably can use jumbo frames, but for now my lab is restricted to a
> >> single
> >> > machine.
> >> > Anything I can do in this situation?
> >> >
> >> > --
> >> > Sent from the Delta quadrant using Borg technology!
> >> >
> >> > Nux!
> >> > www.nux.ro
> >> >
> >> > - Original Message -
> >> >> From: "Simon Weller" 
> >> >> To: "dev" , "users" <
> >> us...@cloudstack.apache.org>
> >> >> Sent: Monday, 20 November, 2017 16:38:33
> >> >> Subject: Re: Circumventing VXLAN MTU issues
> >> >
> >> >> Lucian,
> >> >>
> >> >> Can you run jumbos on your switches?
> >> >>
> >> >> - Si
> >> >>
> >> >>
> >> >> 
> >> >> From: Nux! 
> >> >> Sent: Monday, November 20, 2017 10:08 AM
> >> >> To: dev
> >> >> Cc: users
> >> >> Subject: Circumventing VXLAN MTU issues
> >> >>
> >> >> Hello,
> >> >>
> >> >> I am playing around with the native VXLAN implementation and I have
> of
> >> course
> >> >> hit the situation where MTU on the host gets chomped by 50 bytes and
> >> traffic
> >> >> crawls to a stop in the VMs.
> >> >> How are you people circumventing this? The easiest, but kind of
> >> inconvenient for
> >> >> customers would be to set MTU 1450 inside the VM, though it could be
> >> baked into
> >> >> the templates or via cloud-init.
> >> >>
> >> >> Is the OpenVSwitch implementation (of VXLAN) suffer from the same
> >> problem, I
> >> >> understand ovs can dynamically adjust MTUs to compensate (?), but at
> >> the same
> >> >> time this VXLAN implementation suffers from some limitations
> (multicast
> >> etc).
> >> >>
> >> >> Lucian
> >> >>
> >> >>
> >> >> --
> >> >> Sent from the Delta quadrant using Borg technology!
> >> >>
> >> >> Nux!
> >> > > www.nux.ro
> >>
> >
> >
> >
> > --
> >
> > Andrija Panić
>



-- 

Andrija Panić

Re: Circumventing VXLAN MTU issues

[Andrija Panic]

@devs any plan to implement VXLAN as isolation for also Private Gateway
functionality ?

On 21 December 2017 at 15:28, Andrija Panic  wrote:

> while there, here is another one (just below that one) :) which I learned
> the very hard way, after 2 years in production (when clients started
> complaining on random traffic drop) : http://docs.cloudstack.
> apache.org/en/latest/networking/vxlan.html#important-note-on-max-number-
> of-multicast-groups-and-thus-vxlan-intefaces
>
> Very sneaky and ugly: Ubuntu kernel 3.16 = random traffic drops, but all
> vxlan interfaces are UP etc..., Ubuntu kernel 4.x vxlan interface would NOT
> come  up AT ALL, so a hard drop of packets, and there is where we saw the
> issue...
>
> Cheers
>
>
> On 21 December 2017 at 15:13, Nux!  wrote:
>
>> Thanks Andrija, well done, I have indeed read your document.
>> Using MTU 9000 solved my problem. :)
>>
>> Lucian
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> - Original Message -
>> > From: "Andrija Panic" 
>> > To: "dev" 
>> > Cc: "users" 
>> > Sent: Thursday, 21 December, 2017 14:09:43
>> > Subject: Re: Circumventing VXLAN MTU issues
>>
>> > Hi Nux,
>> >
>> > there is one rare contributions from my side :D to the ACS
>> (documentation,
>> > 2.5 years ago) - check it here:
>> > http://docs.cloudstack.apache.org/en/latest/networking/vxlan
>> .html#important-note-on-mtu-size
>> >
>> > We are using it extensively, feel free to ask anything if needed.
>> >
>> > Cheers
>> >
>> > On 20 November 2017 at 19:38, Nux!  wrote:
>> >
>> >> Yeah, I changed the MTU to 9000 on the Guest network interface and
>> >> connectivity is now fine out of the box.
>> >> Hopefully I won't need to set jumbo frames in the VMs any time soon.
>> :-)
>> >>
>> >> Thanks for the tips.
>> >>
>> >> Lucian
>> >>
>> >> --
>> >> Sent from the Delta quadrant using Borg technology!
>> >>
>> >> Nux!
>> >> www.nux.ro
>> >>
>> >> - Original Message -
>> >> > From: "Simon Weller" 
>> >> > To: "users" 
>> >> > Cc: "dev" 
>> >> > Sent: Monday, 20 November, 2017 17:36:31
>> >> > Subject: Re: Circumventing VXLAN MTU issues
>> >>
>> >> > Change your host interface MTU to something a lot higher.
>> >> >
>> >> >
>> >> > 
>> >> > From: Nux! 
>> >> > Sent: Monday, November 20, 2017 10:51 AM
>> >> > To: users
>> >> > Cc: dev
>> >> > Subject: Re: Circumventing VXLAN MTU issues
>> >> >
>> >> > Hi,
>> >> >
>> >> > I probably can use jumbo frames, but for now my lab is restricted to
>> a
>> >> single
>> >> > machine.
>> >> > Anything I can do in this situation?
>> >> >
>> >> > --
>> >> > Sent from the Delta quadrant using Borg technology!
>> >> >
>> >> > Nux!
>> >> > www.nux.ro
>> >> >
>> >> > - Original Message -
>> >> >> From: "Simon Weller" 
>> >> >> To: "dev" , "users" <
>> >> us...@cloudstack.apache.org>
>> >> >> Sent: Monday, 20 November, 2017 16:38:33
>> >> >> Subject: Re: Circumventing VXLAN MTU issues
>> >> >
>> >> >> Lucian,
>> >> >>
>> >> >> Can you run jumbos on your switches?
>> >> >>
>> >> >> - Si
>> >> >>
>> >> >>
>> >> >> 
>> >> >> From: Nux! 
>> >> >> Sent: Monday, November 20, 2017 10:08 AM
>> >> >> To: dev
>> >> >> Cc: users
>> >> >> Subject: Circumventing VXLAN MTU issues
>> >> >>
>> >> >> Hello,
>> >> >>
>> >> >> I am playing around with the native VXLAN implementation and I have
>> of
>> >> course
>> >> >> hit the situation where MTU on the host gets chomped by 50 bytes and
>> >> traffic
>> >> >> crawls to a stop in the VMs.
>> >> >> How are you people circumventing this? The easiest, but kind of
>> >> inconvenient for
>> >> >> customers would be to set MTU 1450 inside the VM, though it could be
>> >> baked into
>> >> >> the templates or via cloud-init.
>> >> >>
>> >> >> Is the OpenVSwitch implementation (of VXLAN) suffer from the same
>> >> problem, I
>> >> >> understand ovs can dynamically adjust MTUs to compensate (?), but at
>> >> the same
>> >> >> time this VXLAN implementation suffers from some limitations
>> (multicast
>> >> etc).
>> >> >>
>> >> >> Lucian
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Sent from the Delta quadrant using Borg technology!
>> >> >>
>> >> >> Nux!
>> >> > > www.nux.ro
>> >>
>> >
>> >
>> >
>> > --
>> >
>> > Andrija Panić
>>
>
>
>
> --
>
> Andrija Panić
>



-- 

Andrija Panić

Re: [UPDATE] Debian 9 "stretch" systemvmtemplate for master

[Rohit Yadav]

All,


In the last smoke test run yesterday, all of them passed on VMware 5.5u3, 
VMware 6.5+dvswitch, CentOS7 and XenServer 6.5sp1. The PR has now two code 
reviews (approved/lgtms) to qualify for merging this, however, I'll hold 
merging the PR until another smoketest round since master has changed and at 
that same time request for more reviews. If anyone has any concerns or 
objections to merging this, please do share.


I'll also kick component tests tomorrow or over the weekends for VMware, KVM, 
and XenServer and make adjustments to tests etc if necessary.


Regards,

Rohit Yadav

Software Architect, ShapeBlue

http://rohityadav.cloud | @rhtyd


  __?.o/  Apache CloudStack
 ()# The best IaaS cloud platform
(___(_)   https://cloudstack.apache.org



From: Rohit Yadav 
Sent: Tuesday, December 19, 2017 12:38:49 AM
To: dev@cloudstack.apache.org; Wido den Hollander
Cc: us...@cloudstack.apache.org
Subject: Re: [UPDATE] Debian 9 "stretch" systemvmtemplate for master

Hi Wido,

Thanks. I've verified, virtio-scsi seems to work for me. Qemu guest agent also 
works, I was able to write poc code to get rid of patchviasocket.py as well. 
Can you help review and test the PR?

Regards.

From: Wido den Hollander 
Sent: Saturday, December 9, 2017 12:32:24 AM
To: dev@cloudstack.apache.org; Rohit Yadav
Cc: us...@cloudstack.apache.org
Subject: Re: [UPDATE] Debian 9 "stretch" systemvmtemplate for master

Awesome work!

More replies below

On 12/08/2017 03:58 PM, Rohit Yadav wrote:
> All,
>
>
> Our effort to move to Debian9 systemvmtemplate seems to be soon coming to 
> conclusion, the following high-level goals have been achieved so far:
>
>
> - Several infra improvements such as faster patching (no reboots on 
> patching), smaller setup/patch scripts and even smaller cloud-early-config, 
> old file cleanups and directory/filesystem refactorings
>
> - Tested and boots/runs on KVM, VMware, XenServer and HyperV (thanks to Paul 
> for hyperv)
>
> - Boots, patches, runs systemvm/VR in about 10s (tested with KVM/XenServer 
> and NFS+SSDs) with faster console-proxy (cloud) service launch
>
> - Disk size reduced to 2GB from the previous 3+GB with still bigger /var/log 
> partition
>
> - Migration to systemd based cloud-early-config, cloud services etc (thanks 
> Wido!)
>
> - Strongswan provided vpn/ipsec improvements (ports based on work from 
> Will/Syed)
>
> - Several fixes to redundant virtual routers and scripts for VPC (ports from 
> Remi json/gzip PR and additional fixes/improvements to execute update_config 
> faster)
>
> - Packages installation improvements (thanks to Rene for review)
>
> - Several integration test fixes -- all smoke tests passing on KVM and most 
> on XenServer, work on fixing VMware test failures is on-going
>
> - Several UI/UX improvements and systemvm python codebase linting/unit tests 
> added to Travis
>
>
> Here's the pull request:
>
> https://github.com/apache/cloudstack/pull/2211
>
>
> I've temporarily hosted the templates here:
>
> http://hydra.yadav.xyz/debian9/
>
>
> Outstanding tasks/issues:
>
> - Should we skip rVR related tests for VMware noting a reference to a jira 
> ticket to renable them once the feature is support for VMware?
>
> - Fix intermittent failures for XenServer and test failures on VMware
>
> - Misc issues and items (full checklist available on the PR)
>
> - Review and additional test effort from community
>
>
> After your due review, if we're able to show that the test results are on par 
> with the previous 4.9.2.0/4.9.3.0 smoke test results (i.e. most are passing) 
> on XenServer, KVM, and VMware I would proceed with merging the PR by end of 
> this month. Thoughts, comments?
>

We might want to look/verify that this works:

- Running with VirtIO-SCSI under KVM (allows disk trimming)
- Make sure the Qemu Guest Agent works

If those two things work we can keep the footprint of the SSVM rather small.

Wido

>
> Regards.
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

rohit.ya...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

RE: Master Blockers and Criticals

[Paul Angus]

Good news - the list is shrinking.

But can someone reach out to or tap Jayapal on the shoulder, we really need 
some response on his blocker ticket.

Key PrioritySummary 
AssigneeReporter
CLOUDSTACK-9885 Blocker VPC RVR: On deleting first tier and configuring Private 
GW both VRs becoming MASTER Jayapal Reddy   Jayapal Reddy
CLOUDSTACK-10127Critical4.9 / 4.10 KVM + openvswitch + vpc + 
static nat / secondary ip on eth2? Frank Maximus   Sven 
Vogel
CLOUDSTACK-9892 CriticalPrimary storage resource check is broken when 
using root disk size override to deploy VMKoushik Das Koushik Das
CLOUDSTACK-9862 Criticallist template with id= no longer work as domain 
admin   Khosrow Moossavi
Pierre-Luc Dion
CLOUDSTACK-10128CriticalTemplate from snapshot not merging vhd 
filesRafael Weingärtner 
 Marcelo Lima
CLOUDSTACK-9837 CriticalUpon stoping and starting an InternalLbVM 
device from CloudStack, HAProxy service on the device is not being started 
properly


   Unassigned  Mani Prashanth Varma Manthena



Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com] 
Sent: 18 December 2017 17:05
To: dev@cloudstack.apache.org
Cc: Boris Stoyanov ; Rohit Yadav 

Subject: RE: Master Blockers and Criticals

Hi All, here is an updated summary of the open Critical and Blocker Issues in 
Jira.
If you are working on any of these issues, please whether you believe that you 
will have this issue closed by 8th Jan.

@Jayapal Reddy please respond to the pings on the subject of the blocker that 
you have raised and are working on.

Key PrioritySummary 
AssigneeReporter
CLOUDSTACK-9885 Blocker VPC RVR: On deleting first tier and configuring Private 
GW both VRs becoming MASTER Jayapal Reddy   Jayapal Reddy
CLOUDSTACK-10127Critical4.9 / 4.10 KVM + openvswitch + vpc + 
static nat / secondary ip on eth2? Frank Maximus   Sven 
Vogel
CLOUDSTACK-9892 CriticalPrimary storage resource check is broken when 
using root disk size override to deploy VMKoushik Das Koushik Das
CLOUDSTACK-9862 Criticallist template with id= no longer work as domain 
admin   Unassigned  Pierre-Luc Dion
CLOUDSTACK-10128CriticalTemplate from snapshot not merging vhd 
filesRafael Weingärtner 
 Marcelo Lima
CLOUDSTACK-9964 CriticalSnapahots are getting deleted if VM is assigned 
to another user Pavan Kumar Aravapalli  Pavan 
Kumar Aravapalli
CLOUDSTACK-9855 CriticalVPC RVR when master guest interface is down 
backup VR is not switched to Master Jayapal Reddy   Jayapal Reddy
CLOUDSTACK-9837 CriticalUpon stoping and starting an InternalLbVM 
device from CloudStack, HAProxy service on the device is not being started 
properly

Unassigned  Mani Prashanth 
Varma Manthena

project = CLOUDSTACK AND issuetype = Bug AND status in (Open, "In Progress", 
Reopened) AND priority in (Blocker, Critical) AND affectedVersion in (4.10.0.0, 
4.10.1.0, 4.11.0.0, Future) ORDER BY priority DESC, updated DESC



Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 


-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com]
Sent: 11 December 2017 11:00
To: dev@cloudstack.apache.org
Cc: Boris Stoyanov ; Rohit Yadav 

Subject: Master Blockers and Criticals

Hi All,
Please find a summary - of open critical and blocker bugs in 4.11 If you know 
of one which is missing please update Jira accordingly.  I will chase Assignees 
individually (via ML) to get status updates...


KeySummary  

   Priority 
Assignee  Reporter
CLOUDSTACK-9885  VPC RVR: On deletin

Release Notes

[Paul Angus]

Hi guys I'm hoping to get a head start on release notes over Xmas. I believe 
some guys have come up with a tool which gets human readable what's new and 
known issues...

Could someone point me in the right direction please.

(And merry Christmas and a happy new year to everyone)


paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue