Part 2: The Evolution to Virtual Routing

Part 2: The Evolution to Virtual Routing

Graphic of network cables

In the prior post on virtual routing, we introduced the concept of a disaggregated router. Disaggregated routers are routers where control plane and data plane are provided by different vendors. The control plane is usually implemented in software. The data plane could be implemented in either hardware or software. For example, the use of a DPDK-based software data plane can used with various software data planes, both commercial and open-source. These can be implemented as virtual machines or Virtual Network Functions on a hypervisor.

In this series of posts, we are focusing on the service provider market, where port capacity needs, even at the edge of the network, can only be met with hardware data planes. For example, legacy router vendors license their virtualized versions of their OS which max out at the 10 to 40 Gbps range. This may be fine for a uCPE application or even inside a data center, but it would not provide enough throughput for a cell site gateway router. Thus, our focus would be then on open switches with hardware data planes implemented in hardware using switching ASICs, and the control plane is implemented in software.

Network Operating System (NOS)

A full NOS bundles both the OS and the network stack, i.e., the software that implements the routing protocols that conform the control plane, and that need to run on top of an OS. Commercial implementations of full NOS include Cumulus, IP Infusion, Infinera and Arccus.

There are other commercial options for the software from the legacy vendors. Juniper’s JUNOS operating system can be licensed as software-only for deployment on third party OCP-compliant platforms which decouples the OS from Juniper switches. Cisco’s IOS XR can be integrated with a reference third-party device.

It is important to note that some these are oriented at data center networks (like Cumulus Linux) which has different feature and management requirements. Others (like IP Infusion’s OcNOS) have the features needed for service provider networks including MPLS. These are commercially supported software, which makes use of open-source components.

Network Stack

A network stack is similar to a NOS, but it does not include the operating system itself. Network stacks usually run on top of OSs specially targeted for network service operation, such as an Open Network Linux (ONL) OS.

The protocols used in the control plane of routers are highly standardized. This is essential for multivendor interoperability. IETF specifications are the basis for all routers used in networks and conformance to these specifications is critical. There is no reason to deviate from these without going through the standards process. This situation is ideal for an open-source community where vendors can work together on these standardized protocols.

Open Source Network Stacks

Open-source software appeals to very large organizations like web scalers who have substantial technical resources. Some open-source projects have commercially supported options (like Redhat with Linux). That has not materialized for these open-source projects. Companies like AT&T and Microsoft use open source as a way to build support and expertise for this software. Neither has announced plans to support a commercial version of their projects.

Free Range Routing (FRR)

The most successful open-source community is Free Range Routing. As noted above, open-source network stacks are integrating the FRR routing protocols. FRR was a fork from the earlier open-source projects Zebra and Quagga (which itself was a fork from Zebra). FRR can be viewed as a network stack and runs on a range of operating systems.  Volta is an FRR contributor and we’ll be looking at FRR in a blog post later this month.


Software for Open Networking in the Cloud (SONiC) was developed by Microsoft for use in its Azure data centers. Microsoft made it available as open source under the Open Compute Project (OCP). SONiC runs on top of ONL and enjoys broad hardware support because of Microsoft’s position in the market. LinkedIn (which is now owned by Microsoft) is adding FRR to SONiC.

AT&T developed a network stack known as dNOS and released it as open source through the Linux Foundation open source where the project is called DANOS.

Cloud Based Control Plane

Disaggregation enables the customer to choose the data plane (in hardware) and control plane (in software) separately, which can save money, better meet requirements, and avoid vendor lock-in. However, all the models that we have defined so far still are appliances – the full software that implements the control plane, including both the OS and the network stack, runs on the hardware. However, this means that the disaggregated router is essentially the same as a legacy router with one instance of the control plane per device, and each router managed separately.

Alternatively, the control plane can be fully disaggregated from the underlying hardware by running outside the hardware, e.g., in the cloud. This opens up a number of valuable options, such as scaling the CPU and RAM resources needed to run a control plane, and the ability to run multiple control planes associated with the same hardware data plane,  enabling the introduction of multiple  (virtual) routers in the same device.

In part three of these series, we will introduce the concept of virtual router enabled by a cloud control plane, and we will explore the benefits a cloud control plane in terms of both scalability and high availability, as well as in terms of enabling virtual routers and supporting multiple virtual routers in the same physical device, that in turn enable new use cases such as RAN sharing and Soft Network Slicing in the context of 5G services, and Hard Network Slicing in the context of MEC. We will also show that, since all of the control planes run in the same cloud infrastructure, the deployment and management of virtual routers, particularly through APIs, can be simplified and automated much more easily.