+1. Would like an update on the possibility of using Mono to *build* it so it can be on jenkins.
On 5/23/13 7:22 PM, "Donal Lafferty" <donal.laffe...@citrix.com> wrote: >Reproduced from >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress > >Apologies for the length - skip to the heading that interests you. > >h1. Update on Hyper-V plugin work (2013-05-23): > >h2. New plugin Architecture: > >https://cwiki.apache.org/confluence/download/attachments/31822487/Phase2_H >yperV_Remote_Agent.png > >h3. How does is the new architecture an improvement? > >The plugin's architecture uses JSON over HTTP. This allows the plugin >agent to be written in C# rather than Java. .NET has tools to create WMI >wrappers that allows the Hyper-V API calls to be quickly rewritten in C#. > The result is easier to maintain and does not suffer the IP problems of >the Python used in phase 1. An all C# agent can be contained in a single >C#/.NET Windows Service. The result is a more robust design, because >there are far fewer points of failure. > >Switching from CloudStack Message bus to JSON over HTTP communications >allows the plugin's remote agent to be implemented in C#. Previously, >this agent was written in Java to take advantage to the existing >CloudStack message bus agent. The CloudStack message bus is difficult to >port due it's reliance on the Java NIO API. The NIO API is peculiar to >Java. As a consequence, porting to even similar languages such as C# is >difficult. Design work is required that would consume significant time. >In contrast, JSON over HTTP passes kernel commands as JSON in the body of >an HTTP POST. This solution is RPC-like and not RESTful, because the >destination URI that corresponds to the command and not the resource >being manipulated. The HTTP stack removes the dependency on NIO. HTTP >stacks themselves are easy to come by regardless of the language or >platform. Using an RPC approach avoids undertaking design work to >translate incoming kernel commands to a suitable RESTful equivalent. >Therefore, JSON over HTTP minimises the effort required to adopt the best >language for a platform. > >C# code to call the Hyper-V's API directly removes IP restrictions and >complexity of the existing Python code. Hyper-V is controlled through >Windows Management Interface (WMI). WMI provides a uniform framework >through which services and operating subsystems can provide information >and register control APIs. All Hyper-V control packages ultimately call >this API. Phase 1 of the plugin controlled Hyper-V using a port of >cloud.com's OpenStack driver for Hyper-V. The driver could be easily >repurposed, because the OpenStack information model is similar to that of >CloudStack. However, this Python code has IP, support, and coding >drawbacks. The code is a derivative of an existing work, which prevented >immediate donation to ASF. The code used a third party WMI wrapper >package that limited the version of Python that could be used. Finally, >the python wrapper did not address discover of WMI objects, properties >and methods. In contrast, the C# code can be transferred to ASF >immediately. The strongly typed wrappers rely on C# and .NET, which is >well supported. These wrappers are simple to update as they are machine >generated code. More importantly, they enable property and method >discovery through autocomplete. Therefore, the C# version is easier to >maintain and addresses Apache's copyright requirements. > >Access to native error handling, APIs and services make the C#/.NET agent >more robust. Previously, the remote agent created a new process every >time Python was invoked. Communications with this new process were over >stdin and stdout. The serialisation requirements meant exceptions could >not be used for error handling. Also, the design was vulnerable to >deadlock. Should the new process not return but not die either, the call >would be stuck. Now that the agent is written entirely in C# agent, it >can pass errors in a single exception rather than a series of method >returns. Furthermore, using Python along side Java in phase 1 provides >access to platform-specific APIs not exposed by the Java Runtime >Environment (JRE). In contrast, .NET allows direct access to Windows >APIs, which means the C# agent has no need for external processes. >Finally, additional scripts were required to start/stop/restart the phase >1 agent. With phase 2, the agent is implemented with hooks that allow it >to be launched as a Windows Service. Using these hooks, the Windows >Service Control Manager (SCM) takes care of executing the agent as if it >were a daemon. Therefore, switching to a single language and process >means the new remote agent has fewer points of failure. > >An additional strength of this design is that pushes the >hypervisor-specific code out of the server component. Previously, a >hypervisor's remote agent would have to register itself with the >management server. This meant that the agent had to somehow know its >zone, pod, and cluster location before it joined itself to the cloud. >For example, the KVM server component uses SSH to launch the remote agent >with the necessary configuration data. In our new design, the remote >agent can be told its cloud configuration after it has been started. >This removes the need for a hypervisor-specific control to launch remote >agent. Therefore, the resulting server component is much more reusable. > >To summarise, the plugin has an updated communications stack called JSON >over HTTP. This has simplified the implementation of the remote agent >and the corresponding server component. The result is a more robust and >reusable design, and a new implementation that meets Apache CloudStack's >IP requirements. > > >h2. Implementation Status: > >A VisualStudio 2012 SP1 solution on GitHub >([https://github.com/lafferty/cshv3/tree/master/plugins/hypervisors/hyperv >/DotNet/ServerResource]) holds the plugin's new remote agent. VM and >volume creation / deletion commands are mostly complete. Additional >storage commands are required to allow transfer of template and volume >images. Attach / detach commands should be added to allow System VM >patching. > >The solution includes unit tests. The tests are a port of the functional >tests from phase 1. At the time of writing, the port was still being >completed. > >The implementation avoids using features that would require the >hypervisor to run Windows Server 2012 in the parent partition. The image >store uses S3 for template storage, because the Windows NFS client is an >advanced feature that is not available with the basic and free Hyper-V >Server 2012. Likewise, the remote agent hosts its own HTTP stack to >avoid any dependency of Microsoft's Internet Information Services (IIS) >web server. > >A Server component is available that uses JSON over HTTP to communicate >the remote agent. The existing unit tests are being updated to call this >server component. Additional scripting is required to launch the remote >agent so that HTTP requests can be properly received and acted on. The >server component's Discoverer algorithm needs to be updated. > >The build and test system has dependencies on Microsoft tools and >platforms. To build the system with free tools, the remote agent should >be compiled against VisualStudio express. To build with Linux, the >solution should be compiled with Mono. However, neither of these options >have been tried. > > >h2. IP Dependencies > >The C# source code and configuration files that are input for code >generate are in the process of being marked with the Apache header >([http://www.apache.org/legal/src-headers.html#headers]) to indicate that >Apache CloudStack will hold the copyright. > >Third party binaries used by the solution are the Microsoft .NET >Framework 4.5, NuGet packages, and AWS .NET SDK. The AWS SDK is used to >access S3 storage. The NuGet packages provide logging, JSON >serialisation, and a light weight HTTP stack. The background to NuGet is >that it is akin to a Maven repo, but each packages has a corresponding >web page that includes details of the license for use of the package. > >Licenses for the NuGet packages are enumerated below. I do not have the >.NET Framework 4.5 EULA to hand. > > >Apache License, Version 2.0 >([http://logging.apache.org/log4net/license.html)]: > >AWS .NET SDK ([http://aws.amazon.com/sdkfornet/faqs/#13]) >Log4net 2.0.0 ([http://nuget.org/packages/log4net/]) > > >The MIT License (MIT): > >Newtonsoft.Json 4.5.11 [http://json.codeplex.com/license] > > >MICROSOFT ASP.NET MODEL VIEW CONTROLLER 4 EULA >([http://www.microsoft.com/web/webpi/eula/mvc_4_eula_enu.htm]) > >Microsoft ASP.NET Web API Core Libraries 4.0.20710.0 >[http://nuget.org/packages/Microsoft.AspNet.WebApi.Core/4.0.20710.0] >NuGet id="Microsoft.AspNet.WebApi.Core" version="4.0.20710.0" >Microsoft ASP.NET Web API Client Libraries 4.0.20710.0 >[http://nuget.org/packages/Microsoft.AspNet.WebApi.Client/4.0.20710.0] >NuGet id="Microsoft.AspNet.WebApi.Client" version="4.0.20710.0" >Microsoft .NET Framework 4 HTTP Client Libraries 2.0.20710.0 >[http://nuget.org/packages/Microsoft.Net.Http/2.0.20710.0] NuGet >id="Microsoft.Net.Http" version="2.0.20710.0" >Microsoft ASP.NET Web API Self Host 4.0.20918.0 >[http://nuget.org/packages/Microsoft.AspNet.WebApi.SelfHost/4.0.20918.0] >NuGet id="Microsoft.AspNet.WebApi.SelfHost" version="4.0.20918.0" > > > >h2. System VM services: > >The plugin depends on system VM services that are coded separately. >System VM services refer to CloudStack functionality that is offloaded >from the management server to VMs that sit in the cloud. The services in >question are template/snapshot storage, networking services, and console >access. Traditionally, CloudStack has implemented these services using >system VMs: secondary storage VM for image persistence, virtual router >VM for networking services, and console VM for access to guest VM >consoles. For the Hyper-V plugin to work properly, CloudStack has to be >able to offer these same services on Hyper-V hypervisors. > >The image that the CloudStack database points to for Hyper-V system VM >needs to be updated with a version that includes the console proxy. The >latest system VM includes Hyper-V kernel modules. This provides it with >the high performance network driver to act as a virtual router VM. The >secondary storage VM capabilities are not required. The secondary >storage VM allows CloudStack to upload images to an NFS store. However, >a NFS client is not available in the "Hyper-V Server" that we are >targeting. Therefore, we will not be using NFS for image storage. >Finally, the console services must be updated. Hyper-V's guest consoles >are accessed through RDP, which means an RDP proxy must be added to the >system VM. Therefore, changes to the system VM appear to be limited to >the addition of a Hyper-V compatible console proxy.
