Reproduced from https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress
Apologies for the length - skip to the heading that interests you. h1. Update on Hyper-V plugin work (2013-05-23): h2. New plugin Architecture: https://cwiki.apache.org/confluence/download/attachments/31822487/Phase2_HyperV_Remote_Agent.png h3. How does is the new architecture an improvement? The plugin's architecture uses JSON over HTTP. This allows the plugin agent to be written in C# rather than Java. .NET has tools to create WMI wrappers that allows the Hyper-V API calls to be quickly rewritten in C#. The result is easier to maintain and does not suffer the IP problems of the Python used in phase 1. An all C# agent can be contained in a single C#/.NET Windows Service. The result is a more robust design, because there are far fewer points of failure. Switching from CloudStack Message bus to JSON over HTTP communications allows the plugin's remote agent to be implemented in C#. Previously, this agent was written in Java to take advantage to the existing CloudStack message bus agent. The CloudStack message bus is difficult to port due it's reliance on the Java NIO API. The NIO API is peculiar to Java. As a consequence, porting to even similar languages such as C# is difficult. Design work is required that would consume significant time. In contrast, JSON over HTTP passes kernel commands as JSON in the body of an HTTP POST. This solution is RPC-like and not RESTful, because the destination URI that corresponds to the command and not the resource being manipulated. The HTTP stack removes the dependency on NIO. HTTP stacks themselves are easy to come by regardless of the language or platform. Using an RPC approach avoids undertaking design work to translate incoming kernel commands to a suitable RESTful equivalent. Therefore, JSON over HTTP minimises the effort required to adopt the best language for a platform. C# code to call the Hyper-V's API directly removes IP restrictions and complexity of the existing Python code. Hyper-V is controlled through Windows Management Interface (WMI). WMI provides a uniform framework through which services and operating subsystems can provide information and register control APIs. All Hyper-V control packages ultimately call this API. Phase 1 of the plugin controlled Hyper-V using a port of cloud.com's OpenStack driver for Hyper-V. The driver could be easily repurposed, because the OpenStack information model is similar to that of CloudStack. However, this Python code has IP, support, and coding drawbacks. The code is a derivative of an existing work, which prevented immediate donation to ASF. The code used a third party WMI wrapper package that limited the version of Python that could be used. Finally, the python wrapper did not address discover of WMI objects, properties and methods. In contrast, the C# code can be transferred to ASF immediately. The strongly typed wrappers rely on C# and .NET, which is well supported. These wrappers are simple to update as they are machine generated code. More importantly, they enable property and method discovery through autocomplete. Therefore, the C# version is easier to maintain and addresses Apache's copyright requirements. Access to native error handling, APIs and services make the C#/.NET agent more robust. Previously, the remote agent created a new process every time Python was invoked. Communications with this new process were over stdin and stdout. The serialisation requirements meant exceptions could not be used for error handling. Also, the design was vulnerable to deadlock. Should the new process not return but not die either, the call would be stuck. Now that the agent is written entirely in C# agent, it can pass errors in a single exception rather than a series of method returns. Furthermore, using Python along side Java in phase 1 provides access to platform-specific APIs not exposed by the Java Runtime Environment (JRE). In contrast, .NET allows direct access to Windows APIs, which means the C# agent has no need for external processes. Finally, additional scripts were required to start/stop/restart the phase 1 agent. With phase 2, the agent is implemented with hooks that allow it to be launched as a Windows Service. Using these hooks, the Windows Service Control Manager (SCM) takes care of executing the agent as if it were a daemon. Therefore, switching to a single language and process means the new remote agent has fewer points of failure. An additional strength of this design is that pushes the hypervisor-specific code out of the server component. Previously, a hypervisor's remote agent would have to register itself with the management server. This meant that the agent had to somehow know its zone, pod, and cluster location before it joined itself to the cloud. For example, the KVM server component uses SSH to launch the remote agent with the necessary configuration data. In our new design, the remote agent can be told its cloud configuration after it has been started. This removes the need for a hypervisor-specific control to launch remote agent. Therefore, the resulting server component is much more reusable. To summarise, the plugin has an updated communications stack called JSON over HTTP. This has simplified the implementation of the remote agent and the corresponding server component. The result is a more robust and reusable design, and a new implementation that meets Apache CloudStack's IP requirements. h2. Implementation Status: A VisualStudio 2012 SP1 solution on GitHub ([https://github.com/lafferty/cshv3/tree/master/plugins/hypervisors/hyperv/DotNet/ServerResource]) holds the plugin's new remote agent. VM and volume creation / deletion commands are mostly complete. Additional storage commands are required to allow transfer of template and volume images. Attach / detach commands should be added to allow System VM patching. The solution includes unit tests. The tests are a port of the functional tests from phase 1. At the time of writing, the port was still being completed. The implementation avoids using features that would require the hypervisor to run Windows Server 2012 in the parent partition. The image store uses S3 for template storage, because the Windows NFS client is an advanced feature that is not available with the basic and free Hyper-V Server 2012. Likewise, the remote agent hosts its own HTTP stack to avoid any dependency of Microsoft's Internet Information Services (IIS) web server. A Server component is available that uses JSON over HTTP to communicate the remote agent. The existing unit tests are being updated to call this server component. Additional scripting is required to launch the remote agent so that HTTP requests can be properly received and acted on. The server component's Discoverer algorithm needs to be updated. The build and test system has dependencies on Microsoft tools and platforms. To build the system with free tools, the remote agent should be compiled against VisualStudio express. To build with Linux, the solution should be compiled with Mono. However, neither of these options have been tried. h2. IP Dependencies The C# source code and configuration files that are input for code generate are in the process of being marked with the Apache header ([http://www.apache.org/legal/src-headers.html#headers]) to indicate that Apache CloudStack will hold the copyright. Third party binaries used by the solution are the Microsoft .NET Framework 4.5, NuGet packages, and AWS .NET SDK. The AWS SDK is used to access S3 storage. The NuGet packages provide logging, JSON serialisation, and a light weight HTTP stack. The background to NuGet is that it is akin to a Maven repo, but each packages has a corresponding web page that includes details of the license for use of the package. Licenses for the NuGet packages are enumerated below. I do not have the .NET Framework 4.5 EULA to hand. Apache License, Version 2.0 ([http://logging.apache.org/log4net/license.html)]: AWS .NET SDK ([http://aws.amazon.com/sdkfornet/faqs/#13]) Log4net 2.0.0 ([http://nuget.org/packages/log4net/]) The MIT License (MIT): Newtonsoft.Json 4.5.11 [http://json.codeplex.com/license] MICROSOFT ASP.NET MODEL VIEW CONTROLLER 4 EULA ([http://www.microsoft.com/web/webpi/eula/mvc_4_eula_enu.htm]) Microsoft ASP.NET Web API Core Libraries 4.0.20710.0 [http://nuget.org/packages/Microsoft.AspNet.WebApi.Core/4.0.20710.0] NuGet id="Microsoft.AspNet.WebApi.Core" version="4.0.20710.0" Microsoft ASP.NET Web API Client Libraries 4.0.20710.0 [http://nuget.org/packages/Microsoft.AspNet.WebApi.Client/4.0.20710.0] NuGet id="Microsoft.AspNet.WebApi.Client" version="4.0.20710.0" Microsoft .NET Framework 4 HTTP Client Libraries 2.0.20710.0 [http://nuget.org/packages/Microsoft.Net.Http/2.0.20710.0] NuGet id="Microsoft.Net.Http" version="2.0.20710.0" Microsoft ASP.NET Web API Self Host 4.0.20918.0 [http://nuget.org/packages/Microsoft.AspNet.WebApi.SelfHost/4.0.20918.0] NuGet id="Microsoft.AspNet.WebApi.SelfHost" version="4.0.20918.0" h2. System VM services: The plugin depends on system VM services that are coded separately. System VM services refer to CloudStack functionality that is offloaded from the management server to VMs that sit in the cloud. The services in question are template/snapshot storage, networking services, and console access. Traditionally, CloudStack has implemented these services using system VMs: secondary storage VM for image persistence, virtual router VM for networking services, and console VM for access to guest VM consoles. For the Hyper-V plugin to work properly, CloudStack has to be able to offer these same services on Hyper-V hypervisors. The image that the CloudStack database points to for Hyper-V system VM needs to be updated with a version that includes the console proxy. The latest system VM includes Hyper-V kernel modules. This provides it with the high performance network driver to act as a virtual router VM. The secondary storage VM capabilities are not required. The secondary storage VM allows CloudStack to upload images to an NFS store. However, a NFS client is not available in the "Hyper-V Server" that we are targeting. Therefore, we will not be using NFS for image storage. Finally, the console services must be updated. Hyper-V's guest consoles are accessed through RDP, which means an RDP proxy must be added to the system VM. Therefore, changes to the system VM appear to be limited to the addition of a Hyper-V compatible console proxy.
