Hi Koushik, If you see the code, we use ipset only for SGs. It makes sense to use nethash for storing cidrs for a rule using nethash as iphash will try to fill in individual IPs that break if /8 or /4 cidr is added that results in iphash entry overflow (by default it allows 65535 ips to a set). You can reproduce this issue by adding a /4 or /8 cidr and tail the xs host’s cloud.log or do a ipset -L to verify.
The changes I’ve made will remove and add a nethash type of ipset entry if the an old entry exists of type iphash, this is safe as rule application is idempotent (they get applied again every time we add/remove a ingress/egress rule). > On 22-Apr-2015, at 7:19 am, Koushik Das <[email protected]> wrote: > > Would it make sense to use nethash or iphash based on the ipset version? In > Rohit's change I see that for both XS 6.2 and 6.5 nethash is used. > > -----Original Message----- > From: Tim Mackey [mailto:[email protected]] > Sent: Wednesday, 22 April 2015 0:11 > To: [email protected] > Subject: Re: XenServer 6.5 - Security Groups > > Geoff, > > I just went through the changelog for ipset, and its a pretty safe bet that's > where the problems are. In XenServer 6.2, ipset is version 4.5 while > XenServer 6.5 uses ipset version 6.11. With version 5, upstream did a major > rewrite of ipset. iptables also changed, but it's not as dramatic. > I'm going to try and build with Rohit's change and test tonight or tomorrow. > > -tim > > On Tue, Apr 21, 2015 at 2:41 AM, Abhinandan Prateek < > [email protected]> wrote: > >> The security groups are working but with errors. All the recommended >> settings are in place. >> >> In the log files there are many delete failures like: >> >> Failed to delete rule log file /var/run/cloud/i-2-4-VM.ip >> 10.51.212.24-cloud.log:2015-04-17 11:45:08 DEBUG [root] Ignoring >> failure to delete rules for vm i-2-4-VM >> 10.51.212.24-cloud.log:2015-04-17 11:45:08 DEBUG [root] Ignoring >> failure to delete rules for vm i-2-4-VM >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] Ignoring >> failure to delete ebtables chain for vm i-2-4-VM >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] Creating ipset >> chain .... i-2-4-VM >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] Writing log to >> /var/run/cloud/i-2-4-VM.log >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] Programmed >> default rules for vm i-2-4-VM >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] No change in >> default info set of vm i-2-4-VM >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] Seqno increased >> from -1 to 17: reprogamming ingress rules for vm i-2-4-VM >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] Programming >> network rules for vm i-2-4-VM seqno=17 numrules=3 >> signature=fd07c7713376d8906fb71eb8328224ca guestIp=10.170.20.61, >> update iptables, reason=seqno_change_or_sig_change >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] ['iptables', >> '-I', 'i-2-4-VM', '-p', 'icmp', '--icmp-type', 'any', '-j', 'ACCEPT'] >> 10.51.212.24-cloud.log:2015-04-17 11:45:09 DEBUG [root] ipset chain >> already existsi-2-4-VM_tcp_22_22 >> >> >> Looks like due to delete failure the subsequent rule change fails to >> apply ? >> >> -abhi >> >>> On 21-Apr-2015, at 9:56 am, Koushik Das <[email protected]> wrote: >>> >>> What is the output of "cat /etc/sysctl.conf"? Update it as per >> http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/xenserver.html. >> If there was an upgrade from XS 6.2 to XS 6.5, then changes to >> sysctl.conf is not persisted. >>> >>> >>> On 21-Apr-2015, at 6:24 AM, Tim Mackey <[email protected]> wrote: >>> >>>> Geoff, was this a fresh install of both XenServer and CloudStack, >>>> or was there any post install steps or upgrades? I'm thinking of >>>> setting this >> up >>>> and tracing things tomorrow. Might be good to get your logs to >>>> compare against what I see. >>>> On Apr 20, 2015 3:24 PM, "Geoff Higginbottom" < >>>> [email protected]> wrote: >>>> >>>>> FYI we have just re-deployed the environment using ACS 4.4.3 and >> XenServer >>>>> 6.2 and now it is all working as expected. >>>>> >>>>> We now need to work out if the problem is with ACS 4.5.1 or >>>>> XenServer >> 6.5 >>>>> >>>>> Regards >>>>> >>>>> Geoff Higginbottom >>>>> >>>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >>>>> >>>>> [email protected] >>>>> >>>>> -----Original Message----- >>>>> From: Geoff Higginbottom [mailto:[email protected]] >>>>> Sent: 20 April 2015 17:52 >>>>> To: [email protected] >>>>> Subject: RE: XenServer 6.5 - Security Groups >>>>> >>>>> Many thank Somesh >>>>> >>>>> Regards >>>>> >>>>> Geoff Higginbottom >>>>> >>>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >>>>> >>>>> [email protected] >>>>> >>>>> -----Original Message----- >>>>> From: Somesh Naidu [mailto:[email protected]] >>>>> Sent: 20 April 2015 15:55 >>>>> To: [email protected] >>>>> Subject: RE: XenServer 6.5 - Security Groups >>>>> >>>>> I am running 4.5.0. I probably don't use my lab setup that often >>>>> to encounter these inconsistencies. I will look for any XS >>>>> hotfixes that >> have >>>>> fixes around this area and let you know if I find one. >>>>> >>>>> Somesh >>>>> CloudPlatform Escalations >>>>> Citrix Systems, Inc. >>>>> >>>>> -----Original Message----- >>>>> From: Geoff Higginbottom [mailto:[email protected]] >>>>> Sent: Monday, April 20, 2015 10:42 AM >>>>> To: [email protected] >>>>> Subject: RE: XenServer 6.5 - Security Groups >>>>> >>>>> We are currently testing a 4.5.1 CloudStack deployment with >>>>> XenServer >> 6.5 >>>>> and the Security Groups are very temperamental. >>>>> >>>>> Sometimes they allow traffic, sometimes they block it. Lots of >>>>> errors >> in >>>>> the cloud.log file on the XenServer saying failed to update >>>>> iptables >> etc >>>>> >>>>> Regards >>>>> >>>>> Geoff Higginbottom >>>>> >>>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >>>>> >>>>> [email protected] >>>>> >>>>> -----Original Message----- >>>>> From: Somesh Naidu [mailto:[email protected]] >>>>> Sent: 20 April 2015 15:38 >>>>> To: [email protected] >>>>> Subject: RE: XenServer 6.5 - Security Groups >>>>> >>>>> I am running that in my lab and haven't encountered any issues. >>>>> But >> note I >>>>> may not be covering a whole lot of use cases. >>>>> >>>>> Any specific issues that you are facing? >>>>> >>>>> Somesh >>>>> CloudPlatform Escalations >>>>> Citrix Systems, Inc. >>>>> >>>>> From: Geoff Higginbottom [mailto:[email protected]] >>>>> Sent: Monday, April 20, 2015 10:12 AM >>>>> To: [email protected] >>>>> Subject: XenServer 6.5 - Security Groups >>>>> >>>>> Hi, >>>>> >>>>> Is anyone using XenServer 6.5 with a Basic Zone with Security >>>>> Groups, >> and >>>>> if so, are you experiencing any problems? >>>>> >>>>> Regards >>>>> >>>>> Geoff Higginbottom >>>>> CTO / Cloud Architect >>>>> >>>>> [Description: Mail Logo Bottom Align] >>>>> >>>>> D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel: >>>>> +442036030540> | M: +447968161581<tel:+447968161581> >>>>> >>>>> [email protected]<mailto: >> [email protected]> >>>>> | >>>>> >> http://secure-web.cisco.com/1_m6FCDbSUBvddH6wuttV8bB9VyxT7_ZrQdTyy8rP6 >> zdZKcxYpIktkHNIjZklPdlONoacLb2wS34ty66n-dfvn7WNlvJGXbkgF-rfzXJIBVgkyV1 >> bWAuhg1ncLBIz0-ryRfTsoRfwGBsEIt25u879S1c92XJ8aaph3rzYIY6Uiu8jaw0-PbMR0 >> 566HUKmboL-/http%3A%2F%2Fwww.shapeblue.com >>>>> <htp://http:/ >>>>> >> secure-web.cisco.com/1wlSAn7D75G6NWQjnyhpieSqjDnAwYRwKZrJ72dxbEczCABPj >> XADRKyn6yB7bVLg9qX8BOLiQhHMSkV1rTaJ3hvk6hQ4ZzHbyevQP48d5wh6D6YO3kcvr3r >> On_PWYrsvxVV9sr6C6aGf_HdUQ3FT_1PNOgdKuZ9jugNdc1URHALWNa7pbbFucbfGdvls3 >> -lSD/http%3A%2F%2Fwww.shapeblue.com%2F >>> >>>>> | Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru> >>>>> >>>>> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS >>>>> >>>>> Find out more about ShapeBlue and our range of CloudStack related >> services >>>>> >>>>> IaaS Cloud Design & Build< >>>>> >> http://secure-web.cisco.com/1ELd3cItcRBs-I31ofdP3jOF-t5vDpGEXoTGIusThW >> jIMyrHdvguDFWxEKII1DpOPXKDDDImob8MuTMkIOh0DfliYEz52VWpOigayNR-6cF6Dzf3 >> dkzh0mUtqxQCELej15kwPVIay8Hnyq2qDPsk3GX6lbS2hLeHonsKEDnq9dfG4l6UNA09F9 >> JEETtHIJ3cr/http%3A%2F%2Fshapeblue.com%2Fiaas-cloud-design-and-build%2 >> F%2F >>>>>> >>>>> CSForge - rapid IaaS deployment framework< >>>>> >> http://secure-web.cisco.com/1ODcR5OFg2RiGzgEwRUsJ_RHtEXWdR_wS6Gwj38jVh >> LMjWOi6nRP7iqtV0pbpZbKCWDhQ2YQjAVHPQ-GeIUUTEYSbL-dWm_tWuRNFl2m7wZQERQG >> Y8kKwe5zxpG62bne2dQ_I9ezxt1376h_8tUsX-_cJH00m2tY_m1MBRt0jYviBewtgYbRYA >> 3_YSuGJyf0b/http%3A%2F%2Fshapeblue.com%2Fcsforge%2F >>>>>> >>>>> CloudStack Consulting< >>>>> >> http://secure-web.cisco.com/1cHy5w5vKDJaJg7SS2Vl9lgoBUd5-Ctdx-nSQIOSgA >> YOtsNFyI-p1B9xnYhDSzCkQrlPCu_1rcLbxKXcOmmKgV5lQrnuILkwNUCH1-HNmOqOxlRr >> PN0BK0-SBrw_fPqMQcqv6qDCSmVfFLIQ0SKY7o7FxndebfXimroPuUgzPxG644IFgFtJ1l >> 4VyPra9xahV/http%3A%2F%2Fshapeblue.com%2Fcloudstack-consultancy%2F >>>>>> >>>>> CloudStack Software Engineering< >>>>> >> http://secure-web.cisco.com/16oTFNuacBSBoCrEA-Y3fa-IkXuZ_6jOtLjLL5uvID >> YECRH4C2TkWaCdSGEDG7Yosw_kIt0VuO1RF_X8xuKyAOAlLo7EzlT137BUjUZDBCCVDpn2 >> ciRQsV76hRJLSpY20jWP1r_JsQW26ODBMbML47upXGOSHgi2BrAH7LMN81TXQE6rtnRdtR >> VtWkvcVOiL_/http%3A%2F%2Fshapeblue.com%2Fcloudstack-software-engineeri >> ng%2F >>>>>> >>>>> CloudStack Infrastructure Support< >>>>> >> http://secure-web.cisco.com/1XcMscsGdPZOa4Spu1DD753IpqjztZn6hg--j4Kk64 >> JZjKIx-X0l3LHiIz5ImUtbnqwLEL1fWPdS-67j4kZ_ZJbJM0bXSgl_Llk-0FZ9QD6UGLJH >> eYpAi_vNUqDNDKAUuaNq3WbvmfHpT5DC00NLc_6COAn6VpP-bav7xRmbrHhPz5ZGxdZdJ8 >> CuPdBnDdFAL/http%3A%2F%2Fshapeblue.com%2Fcloudstack-infrastructure-sup >> port%2F >>>>>> >>>>> CloudStack Bootcamp Training Courses< >>>>> >> http://secure-web.cisco.com/1IP7LA-sNa8Vs_Z95rp5Zj8LFSfGl_DyYmspoHab4c >> tH0YCsceU9s_Y4MJ3QiHte2TtwQQOtjPc8QKbwsqFOt40DABpdZY_gKDB3-LZ6AmpKityj >> PbFocbUhLHtfkQPk0OA1TLP0d945z-jHQvjDIDwAuV3FDBSCQcoWRgJrUAK5bS4rCs_awT >> HVwXPviuzNt/http%3A%2F%2Fshapeblue.com%2Fcloudstack-training%2F >>>>>> >>>>> >>>>> This email and any attachments to it may be confidential and are >> intended >>>>> solely for the use of the individual to whom it is addressed. Any >> views or >>>>> opinions expressed are solely those of the author and do not >> necessarily >>>>> represent those of Shape Blue Ltd or related companies. If you are >>>>> not >> the >>>>> intended recipient of this email, you must neither take any action >> based >>>>> upon its contents, nor copy or show it to anyone. Please contact >>>>> the >> sender >>>>> if you believe you have received this email in error. Shape Blue >>>>> Ltd >> is a >>>>> company incorporated in England & Wales. ShapeBlue Services India >>>>> LLP >> is a >>>>> company incorporated in India and is operated under license from >>>>> Shape >> Blue >>>>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated >>>>> in >> Brasil >>>>> and is operated under license from Shape Blue Ltd. ShapeBlue SA >>>>> Pty >> Ltd is >>>>> a company registered by The Republic of South Africa and is traded >> under >>>>> license from Shape Blue Ltd. ShapeBlue is a registered trademark. >>>>> Find out more about ShapeBlue and our range of CloudStack related >> services >>>>> >>>>> IaaS Cloud Design & Build< >>>>> >> http://secure-web.cisco.com/1qxoX785ZmMjnboIy1y1h-SUu_Jl3hTKt7iFo-u0eD >> B_68cG34d1HlEG4zx9qHSGnDCXt3rdD6PdBGhBMkKFr6TWSsXQTpFVL9GAImd-sRsd2kGJ >> 38Fc1bypBet4Ma8g1O7JC-iJNcCosukzONTTQRKIY2HIFS-S2pDUKxtdqIgVFcujjU1IYi >> 9A9PHQdjs5M/http%3A%2F%2Fshapeblue.com%2Fiaas-cloud-design-and-build%2 >> F%2F >>>>>> >>>>> CSForge – rapid IaaS deployment framework< >>>>> >> http://secure-web.cisco.com/1j6QBweSqTOqdCPbKiQTwEDmH1Z8zZ3Aww9Cp4cQCQ >> 8FiNw5J9x8n9ye4woAUe0Tvfo_Y3EKzsqnuNxXkDUqhJyQzP_VE7gdxLfiEZlVnKDIYEBB >> Sq2F7Noe_lJK3XgAHiX8pDP2HHLFdlrN5N4VPnBdKY52J7EwM_VJsKKLjRa6Nf8K8jQGmW >> SOh97Py1IPa/http%3A%2F%2Fshapeblue.com%2Fcsforge%2F >>>>>> >>>>> CloudStack Consulting< >>>>> >> http://secure-web.cisco.com/1uKIqKaPysisjAYs5hynrA56Q4iyjJ6ZW0ldQ9t_Fk >> qGa-aTFvPqNPv7d5bQy7_raKeeYVR2X5SlmduWvBzURm1fqAPHAx-LTUkbCOz7jQDPeH4R >> YXdHtvJNKFNn5ZV1MZOPl9ENOF88ufx3hzqgEQTXLAm8IYqOn9lhY1ltPGp_VruC6FOXjm >> Uftm-RdZ0tb/http%3A%2F%2Fshapeblue.com%2Fcloudstack-consultancy%2F >>>>>> >>>>> CloudStack Software Engineering< >>>>> >> http://secure-web.cisco.com/1xgQxAp3AgT66n1JVvu_H7rGJmsaOavPG5JI3ZBPdT >> GXqb8nGqYOTlXiWAEwzEK1-IBoM_lPfT0wLuG-g2GGrsZAF7cK-AxI-QwI9Y0oFD6QhC4m >> 7v2I9t_43xKbgmquECFL241xvibLrnA8a1L2SMIYRHaYuLS-oDboFVrPq0P9XJvjZATKer >> UOGbzhHUks2/http%3A%2F%2Fshapeblue.com%2Fcloudstack-software-engineeri >> ng%2F >>>>>> >>>>> CloudStack Infrastructure Support< >>>>> >> http://secure-web.cisco.com/1ItjOvHT5ychUQBDrFa3tOBXnRhrFWZWg2bFqJ55Kx >> Ee43wZcla2yGUlE6-3TY4MZ0dzDL2ZP773WHjUgWR9dMMajOU4llAV5JLGmFaouZUQrthY >> mScYm0RcIKnev_oPl_wYKDzwxSJVr_2UQ-0zvl2glUqZ8nch8LSeZXniAYL0f6X0_wIqny >> Ig3nI76jbBi/http%3A%2F%2Fshapeblue.com%2Fcloudstack-infrastructure-sup >> port%2F >>>>>> >>>>> CloudStack Bootcamp Training Courses< >>>>> >> http://secure-web.cisco.com/12fS8JP2fzHcOQK6CpY9sqXQ2-FtuG8WTF2NS01TTj >> kJBS703san5ODj19uns__Xhrq4vx6pyMAm78clkVLtb-_-olAxf3nPnRW3FPOdfSV9l7JX >> 1PMr2vui_FrlXIZGvvmFQCBu0Yx29UUYJ0cL0pszFiMCCHHSbxpjWm_1FFOSQ4Y11Vlz3n >> TwSxKZCwL1h/http%3A%2F%2Fshapeblue.com%2Fcloudstack-training%2F >>>>>> >>>>> >>>>> This email and any attachments to it may be confidential and are >> intended >>>>> solely for the use of the individual to whom it is addressed. Any >> views or >>>>> opinions expressed are solely those of the author and do not >> necessarily >>>>> represent those of Shape Blue Ltd or related companies. If you are >>>>> not >> the >>>>> intended recipient of this email, you must neither take any action >> based >>>>> upon its contents, nor copy or show it to anyone. Please contact >>>>> the >> sender >>>>> if you believe you have received this email in error. Shape Blue >>>>> Ltd >> is a >>>>> company incorporated in England & Wales. ShapeBlue Services India >>>>> LLP >> is a >>>>> company incorporated in India and is operated under license from >>>>> Shape >> Blue >>>>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated >>>>> in >> Brasil >>>>> and is operated under license from Shape Blue Ltd. ShapeBlue SA >>>>> Pty >> Ltd is >>>>> a company registered by The Republic of South Africa and is traded >> under >>>>> license from Shape Blue Ltd. ShapeBlue is a registered trademark. >>>>> Find out more about ShapeBlue and our range of CloudStack related >> services >>>>> >>>>> IaaS Cloud Design & Build< >>>>> >> http://secure-web.cisco.com/1HCcKu4sOWwt8wKuvvBzvkgT_jpnVnvJ0pPOMSuJQW >> bIVaLn8RcXvJMv7V7g10ENr4Z1YRfWoqPH3uZGVy1RDN2N7CXaIh1KjxNSQ4r3P6GPdLTg >> 5F7o_h2MyTz9RscDOXkRzeltObQ-4v5G40iALCLjpVduhmte8lxO4j6ZXA3XE4DwRlb6Uv >> mh596Vq4GFh/http%3A%2F%2Fshapeblue.com%2Fiaas-cloud-design-and-build%2 >> F%2F >>> >>>>> CSForge – rapid IaaS deployment framework< >> http://secure-web.cisco.com/1gRiwM4pWI-2iYTQ4OyMvMz-g0D9-fodjXf8vvxQEw >> BNQXngmPAghHp8DeE_ll-480fWIKrPECcE7DV3Oy2spltvkm9fnho8FPF5fp9kZqFd4unH >> tQ3LR4HIwGzlOLzT9LbqsM5GmXhl8qFd3T1lrNe-nsV-l6QpI3lGPOHF5dq5piUpIcyyiN >> rAIZyxdE5Iz/http%3A%2F%2Fshapeblue.com%2Fcsforge%2F >>> >>>>> CloudStack Consulting< >> http://secure-web.cisco.com/1qx3j5uTQZieJNjr6-gqoAfg_Ot1-O6JuPK5Z7mjQj >> 0m_G_aXCdUrYfjxeDGqemCMFH6sSstX6wvMcvj0Xhi-j6CWEHOyC52t0sEjnTZ4_tGWDod >> kENa76yJbiFUbeWLddtBSfc2cnZSfXvzTc2FCJ80Q3rN8adsKOvtgke3L7x6AUOFcKbGbr >> mtnZkFOK9Y3/http%3A%2F%2Fshapeblue.com%2Fcloudstack-consultancy%2F >>> >>>>> CloudStack Software Engineering< >>>>> >> http://secure-web.cisco.com/1fI-iSVt0IffxPGEoPM7WxSztmXHE3_Ps8Ah6gU8Lb >> lj6Nry22hu2SBMfyNZVvFsiKPGKuPTtK4BhLYAT1-p_3uLlEbQmh5sYIf3cjUb3v7q_Cm9 >> l_kjX-_ZA18Ly8fxUcp4Y4wZmJQaJ6V1cmb7FGwcAieqlmw0XK5bTkZpILjVF600m9Uqtx >> 67qw3cozd0-/http%3A%2F%2Fshapeblue.com%2Fcloudstack-software-engineeri >> ng%2F >>> >>>>> CloudStack Infrastructure Support< >>>>> >> http://secure-web.cisco.com/1h2KOum5k0kRATlEW9S-kvBJhhygbGvXAKBdPDt-em >> pix9d9UU2Qk_SONB3UERJu4Eknea2FignD76PFiJiXEbQWyKW5pz9eRG1DJBFYYsTDXefn >> _-QJ-y27fjSLgzNOybA1pxWUFpWmiXCZ_tv9c66bi08Tx3i4-OwQ0Fo0s6BBGbxuOXqT11 >> hgKDKP14uYG/http%3A%2F%2Fshapeblue.com%2Fcloudstack-infrastructure-sup >> port%2F >>> >>>>> CloudStack Bootcamp Training Courses< >>>>> >> http://secure-web.cisco.com/1Zszm7zKQ89XqBbDviBqfAIEcz5-nSVQJePJCBo_sK >> fr6unk95_QAkR_lbawqEcw6PSISsnYGBpY7D-oNiYrDxZ0y8AXh3Nu7NhpSvyEsQ6GK_1H >> qI9iz7W1D_2B8Hh3zk1v-IkJC33H0sO087k8Rl_hRXewfvGG2PnNR_waCKlSpB3xxD-AM7 >> m2RWhvnAr5_/http%3A%2F%2Fshapeblue.com%2Fcloudstack-training%2F >>> >>>>> >>>>> This email and any attachments to it may be confidential and are >> intended >>>>> solely for the use of the individual to whom it is addressed. Any >> views or >>>>> opinions expressed are solely those of the author and do not >> necessarily >>>>> represent those of Shape Blue Ltd or related companies. If you are >>>>> not >> the >>>>> intended recipient of this email, you must neither take any action >> based >>>>> upon its contents, nor copy or show it to anyone. Please contact >>>>> the >> sender >>>>> if you believe you have received this email in error. Shape Blue >>>>> Ltd >> is a >>>>> company incorporated in England & Wales. ShapeBlue Services India >>>>> LLP >> is a >>>>> company incorporated in India and is operated under license from >>>>> Shape >> Blue >>>>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated >>>>> in >> Brasil >>>>> and is operated under license from Shape Blue Ltd. ShapeBlue SA >>>>> Pty >> Ltd is >>>>> a company registered by The Republic of South Africa and is traded >> under >>>>> license from Shape Blue Ltd. ShapeBlue is a registered trademark. >>>>> Find out more about ShapeBlue and our range of CloudStack related >> services >>>>> >>>>> IaaS Cloud Design & Build< >>>>> >> http://secure-web.cisco.com/1HCcKu4sOWwt8wKuvvBzvkgT_jpnVnvJ0pPOMSuJQW >> bIVaLn8RcXvJMv7V7g10ENr4Z1YRfWoqPH3uZGVy1RDN2N7CXaIh1KjxNSQ4r3P6GPdLTg >> 5F7o_h2MyTz9RscDOXkRzeltObQ-4v5G40iALCLjpVduhmte8lxO4j6ZXA3XE4DwRlb6Uv >> mh596Vq4GFh/http%3A%2F%2Fshapeblue.com%2Fiaas-cloud-design-and-build%2 >> F%2F >>> >>>>> CSForge – rapid IaaS deployment framework< >> http://secure-web.cisco.com/1gRiwM4pWI-2iYTQ4OyMvMz-g0D9-fodjXf8vvxQEw >> BNQXngmPAghHp8DeE_ll-480fWIKrPECcE7DV3Oy2spltvkm9fnho8FPF5fp9kZqFd4unH >> tQ3LR4HIwGzlOLzT9LbqsM5GmXhl8qFd3T1lrNe-nsV-l6QpI3lGPOHF5dq5piUpIcyyiN >> rAIZyxdE5Iz/http%3A%2F%2Fshapeblue.com%2Fcsforge%2F >>> >>>>> CloudStack Consulting< >> http://secure-web.cisco.com/1qx3j5uTQZieJNjr6-gqoAfg_Ot1-O6JuPK5Z7mjQj >> 0m_G_aXCdUrYfjxeDGqemCMFH6sSstX6wvMcvj0Xhi-j6CWEHOyC52t0sEjnTZ4_tGWDod >> kENa76yJbiFUbeWLddtBSfc2cnZSfXvzTc2FCJ80Q3rN8adsKOvtgke3L7x6AUOFcKbGbr >> mtnZkFOK9Y3/http%3A%2F%2Fshapeblue.com%2Fcloudstack-consultancy%2F >>> >>>>> CloudStack Software Engineering< >>>>> >> http://secure-web.cisco.com/1fI-iSVt0IffxPGEoPM7WxSztmXHE3_Ps8Ah6gU8Lb >> lj6Nry22hu2SBMfyNZVvFsiKPGKuPTtK4BhLYAT1-p_3uLlEbQmh5sYIf3cjUb3v7q_Cm9 >> l_kjX-_ZA18Ly8fxUcp4Y4wZmJQaJ6V1cmb7FGwcAieqlmw0XK5bTkZpILjVF600m9Uqtx >> 67qw3cozd0-/http%3A%2F%2Fshapeblue.com%2Fcloudstack-software-engineeri >> ng%2F >>> >>>>> CloudStack Infrastructure Support< >>>>> >> http://secure-web.cisco.com/1h2KOum5k0kRATlEW9S-kvBJhhygbGvXAKBdPDt-em >> pix9d9UU2Qk_SONB3UERJu4Eknea2FignD76PFiJiXEbQWyKW5pz9eRG1DJBFYYsTDXefn >> _-QJ-y27fjSLgzNOybA1pxWUFpWmiXCZ_tv9c66bi08Tx3i4-OwQ0Fo0s6BBGbxuOXqT11 >> hgKDKP14uYG/http%3A%2F%2Fshapeblue.com%2Fcloudstack-infrastructure-sup >> port%2F >>> >>>>> CloudStack Bootcamp Training Courses< >>>>> >> http://secure-web.cisco.com/1Zszm7zKQ89XqBbDviBqfAIEcz5-nSVQJePJCBo_sK >> fr6unk95_QAkR_lbawqEcw6PSISsnYGBpY7D-oNiYrDxZ0y8AXh3Nu7NhpSvyEsQ6GK_1H >> qI9iz7W1D_2B8Hh3zk1v-IkJC33H0sO087k8Rl_hRXewfvGG2PnNR_waCKlSpB3xxD-AM7 >> m2RWhvnAr5_/http%3A%2F%2Fshapeblue.com%2Fcloudstack-training%2F >>> >>>>> >>>>> This email and any attachments to it may be confidential and are >> intended >>>>> solely for the use of the individual to whom it is addressed. Any >> views or >>>>> opinions expressed are solely those of the author and do not >> necessarily >>>>> represent those of Shape Blue Ltd or related companies. If you are >>>>> not >> the >>>>> intended recipient of this email, you must neither take any action >> based >>>>> upon its contents, nor copy or show it to anyone. Please contact >>>>> the >> sender >>>>> if you believe you have received this email in error. Shape Blue >>>>> Ltd >> is a >>>>> company incorporated in England & Wales. ShapeBlue Services India >>>>> LLP >> is a >>>>> company incorporated in India and is operated under license from >>>>> Shape >> Blue >>>>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated >>>>> in >> Brasil >>>>> and is operated under license from Shape Blue Ltd. ShapeBlue SA >>>>> Pty >> Ltd is >>>>> a company registered by The Republic of South Africa and is traded >> under >>>>> license from Shape Blue Ltd. ShapeBlue is a registered trademark. >> >> Find out more about ShapeBlue and our range of CloudStack related >> services >> >> IaaS Cloud Design & Build< >> http://secure-web.cisco.com/16ZhKVCxpDiLOMfG0KoobAxJdPCGylgMGC4ff40FkI >> Lnwv3sfWkrF11ZiaWiWp_MaixFzm3UJ7_SxeJ4VdO-w-EuNvf36BXA_cdOaWagqj-K2d6E >> WTucQIrzrRpggZMW3VEMQsZmOjFpbjmrFohbbLCkUnOuI5ZpaLOuLarq8mEb5Yair5MFio >> 5JriHjVmjpH/http%3A%2F%2Fshapeblue.com%2Fiaas-cloud-design-and-build%2 >> F%2F> CSForge – rapid IaaS deployment >> framework<http://secure-web.cisco.com/1Hu49aYH-ljiGLyM8Y6AYg3byu29WC3n >> slE2rIbv4hEEwE53kdAP8MM5MOOs7DaIToFjs349S0T3hFZVf6Z8aQFHm5K9FVbtGAHxXw >> hWhUNyx1UGMM99lfkE9egbxtFVMLB9gzuIdVPT192OmjHAooVLkPuEYZlmJucT8TyD5Z6G >> VVs4a8XxHR30FK2fZMBuk/http%3A%2F%2Fshapeblue.com%2Fcsforge%2F> >> CloudStack >> Consulting<http://secure-web.cisco.com/1DOLRZAEX2KboQIYliDlYWBFRcrZ29r >> _mliz6jcayp6qPMSEHgSqo6eObcernRBGBiZUJoFkEJJb8cX57Egcul9f3MS4EDcAhLRJx >> t3LUZBMKM2EzGUs3myKQ76RchS0MpYMify2KcD6ZPKsOkY_C8e09fVA7-PfFGi2gqPdonW >> AtBa8IiJDXMlxqtJOaJ-DZ/http%3A%2F%2Fshapeblue.com%2Fcloudstack-consult >> ancy%2F> >> CloudStack Software Engineering< >> http://secure-web.cisco.com/1_Zc-KEGwvB3JL7scCU4XmxUOl3ihNxO-m01laIS3G >> sDTgI8DLVmDPiYXbQ2m970UXWV4dGPuxhbU0orjFic2gZvao4nf4R_-Tb_4IjKb1Tw7ZRi >> 4yLeA2n6zgOmFgh4GjsQJs5qr0ebl_xZgECk-wLU-jrBgDuatR9wJDLZMqO9Nbe9EiuVAJ >> JDaN5yp-flo/http%3A%2F%2Fshapeblue.com%2Fcloudstack-software-engineeri >> ng%2F> >> CloudStack Infrastructure Support< >> http://secure-web.cisco.com/1sphQWGSTolEyeiP7McFSVziezTmepytz4xYiudeqW >> ZaSdVE9vOWXr8delOcmuZ9CtnoGgKtBVjQoVY_l3PMshHDm6Aujv_tTIkHYVb_vyRKZOyQ >> Ks_k6jMqj8xy0Ild8LzdHWsYAV1Mtf9dZOxwHKkkDPQj7PIayTdh4f8Yl0qlapZk9xeScH >> GYjtenAxYTE/http%3A%2F%2Fshapeblue.com%2Fcloudstack-infrastructure-sup >> port%2F> >> CloudStack Bootcamp Training Courses< >> http://secure-web.cisco.com/1WpzBMnCsSEbEItVmMIeuNJGXhd5NXbqall1Fv4scE >> lQtR1WBhPQSYCyublcSWp6QdpkEdcl3U0_dgZ1ofn14y2sGeru1DxivNGbyxLxnd-4-N5s >> RG6SnYO4YH48pOPDEtjgFeGzo1cSL_0EXs2gwGoabzMS5i8qeHNN1l5Pc_WODb850wHxCT >> -YDPK59FgZE/http%3A%2F%2Fshapeblue.com%2Fcloudstack-training%2F> >> >> This email and any attachments to it may be confidential and are >> intended solely for the use of the individual to whom it is addressed. >> Any views or opinions expressed are solely those of the author and do >> not necessarily represent those of Shape Blue Ltd or related >> companies. If you are not the intended recipient of this email, you >> must neither take any action based upon its contents, nor copy or show >> it to anyone. Please contact the sender if you believe you have >> received this email in error. Shape Blue Ltd is a company incorporated >> in England & Wales. ShapeBlue Services India LLP is a company >> incorporated in India and is operated under license from Shape Blue >> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in >> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA >> Pty Ltd is a company registered by The Republic of South Africa and is >> traded under license from Shape Blue Ltd. ShapeBlue is a registered >> trademark. >> Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | [email protected] Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
