SANTA CLARA, Calif. -- Microsoft revealed at the Open Networking Summit the virtual software-defined network that powers its Windows Azure cloud across multiple data centers.
The virtual SDN architecture of the Windows Azure network achieves scalability by massively distributing both the control plane and the data plane of the network. It has multiple federated controllers, each dedicated to specific applications, network services and functions. The data plane, as well as all Layer 4-7 services and policies, are implemented across every hypervisor host in Azure data centers.
Microsoft's virtual SDN control plane includes specialized, federated controllers, said Albert Greenberg, partner development manager for Microsoft. "There isn't just one controller," he said. "There are controllers by application, and they are federated. Coming through the northbound application programming interface (API), you have a virtual network controller, a load-balancer controller, and they have different responsibilities. Controllers work in small clusters, and then regional controllers manage those and partition the network."
Microsoft's approach to the forwarding plane in its Windows Azure network was inspired by OpenFlow. "The OpenFlow idea of match-action tables is the right idea," he said. "We built several types of match-action tables. The controller programs these tables. There are tables for [virtual networks], load balancers, NAT [network address translation], ACLs [access control lists]. We're able to light up hundreds of features and change things and implement them in ways that couldn't do if it was all cooked into hardware."
The data-forwarding plane doesn't communicate directly with Microsoft's SDN controllers. Instead, Microsoft has deployed virtual network agents on every hypervisor host. The virtual network agent sends flows onward to the vSwitch. If the vSwitch matches the flow in its match-action tables, it executes the necessary transaction with VMs on the host and the resulting data egresses through the server network interface card. If the virtual switch doesn't recognize the flow, it sends a mapping request back to the agent, which queries a mapping service in the controller cluster.
"[Azure's] data plane needs to apply per-flow policy to millions of VMs," Greenberg said. "How do we apply that to billions of flows and translate that into taking actions on packets? The host performs all packet actions on its own virtual machines. We use a tiny bit of the distributed computing power of millions of servers to solve SDN."
The southbound SDN interface in the Windows Azure network is between the agent and the virtual switch, Greenberg said. This allows Microsoft to use a high-performance, OS-level API rather than a wire protocol. The wire protocols are for the simpler, less frequent transactions between the controller, agent and related services.
Windows Azure's virtual cloud load-balancing service is an example of how this virtual SDN model can scale in a massive cloud. It is hosted on every single server in the data center. "We move the network address translation down to the virtual switch, which make the load balancers stateless and enable direct return to the client. The controller programs policies across all of this," Greenberg said.
More on Microsoft
Microsoft monitors its network with OpenFlow-based visibility network
An SDN API for Microsoft Lync can boost performance
CPU utilization was a critical barrier to success too. Microsoft gets around this issue by using the first packet of a flow as an opportunity to plumb all the table information. The rest of the packets in a flow can then "zip through efficiently through caching," Greenberg said.
Finally, the management plane for the virtual SDN is a front-end Azure portal used for self-service orchestration. Through the portal, customers can define their virtual networks and policies through their own internal enterprise addressing scheme. The portal pushes that virtual network model through a northbound API to Microsoft's cluster of SDN controllers, which sets up a virtual network.
"A virtual network is a set of mappings from the customer-defined address space to the provider's addresses to the hosts where the virtual machines are located," Greenberg said.
The controllers will also "plan appropriate policy into the nodes on Azure virtual switches, and the policy can extend back to the customer premises through a VPN gateway that will also be there," Greenberg said.
Azure's virtual SDN is the result of four years of work done by 100 developers and network engineers in Microsoft's Redmond, Calif., Mountain View, Calif., and Dublin offices. The virtual SDN powers the Azure public cloud and all of Mcirosoft's other properties, including Office365, Bing search, Skype, OneDrive and Xbox.