Abstract
In the fast moving space of technology, everything and everyone is on the Internet accessing the world. The creation of large networks and data centers required for managing and analyzing data will increase in the future, so traditional networking technologies and skills will not be enough to solve the future complications arising in the networking industry. The present networks are more static and rigid where else they need to be more dynamic and flexible. Software Defined Networking (SDN) pledges to solve these issues and change the status quo of the industry. The key ideas behind Software Defined Networking are the separation of control and data plane, centralization of network policies and programmability of the networks. Software Defined Networking also claims to solve the perplexing network management issues with some related technologies.
This paper provides an insight into this new paradigm. In the inception, we introduce Software Defined Networking and what exactly led to SDN. The fundamental difference between SDN and traditional networking is discussed next. Then, we discuss the different definitions of SDN proposed by different communities covering every aspect of this new paradigm namely: standardization efforts, history, architectural framework with three planes of networking, northbound and southbound interfaces, SDN controllers, and network applications. We also take a look at the ongoing research work especially from the eye of industry giants like Microsoft, Google, AT&T, etc. specifically in the field of deployment and interoperability among different carrier vendors and cloud service providers. We conclude the paper by explaining how SDN has brought the whole ecosystem under one roof.
Keywords
Introduction
The first thing we need to understand is that traditional networking technologies are static, rigid, inflexible and are mostly based on standardized protocols for network communication providing almost negligible space to innovation. Most of the hardware devices which belong to traditional networking are vendor specific and have closed proprietary rights which mean they are closed source. On top of that, these networks are really complex and hard to manage [30]. In order to run these networks, the service providers and network operators need to configure each device separately with a distributed control plane. In such an ecosystem it is a really daunting task when it comes to fault management, load balancing, traffic engineering and policy enforcement [90,227,269]. In addition to these complexities, the existing networks have limited redundancy and their poor response to dynamic events have always been a concern to network engineers. This can be understood by a simple example that how much transition time traditional networks take to adapt to new protocols [90,227].
More importantly, existing network topologies are bundled with so many constraints and architectural framework that is tightly coupled. All the three networking planes that are the data plane, control plane, and management plane are coupled together reducing scalability, flexibility and performance of the network [76]. All of the above-discussed crunches brought the networking industry to a standstill from the innovation point of view. The very fundamentals of current networks are hindering innovation which is contemplated in the transition process of IPv4 to IPv6 which is still going on after almost a decade. All of this has limited the operational capabilities of the networks and incremented the Capex and Opex of the business enterprises.
Software Defined Networking can be dubbed as the generation shift in networking which breaks the status quo by opening up the existing networks to innovation; with SDN, networks would be dynamic and flexible [159,249]. Starting with the fundamentals, the SDN breaks vertically integrated network into the horizontally integrated network by decoupling the control plane and data plane. Control plane or control logic is the processing unit which frames the decision where traffic is to send, data plane (forwarding plane) or data logic is the forwarding devices like switches and routers which are responsible for moving packets from input to output. By decoupling control plane and data plane, forwarding devices are made simpler with no logical decision-making capabilities and the control is transferred to a logically centralized SDN controller. This architectural framework simplifies the network making a way for innovation [132]. Now the network control is directly programmable that allows the network administrators to dynamically adjust network-wide traffic flow to meet changing needs. SDN claims to come with open standards and vendor neutral but we will discuss its implications in the coming sections. The separation of control plane and data plane do not mean centralization of physical hardware devices, their design is still on the distributed platform, it is the logic which is centralized [122,135]. The deployment of SDN based solutions has distributed control plane but centralized control logic. An overview of SDN architecture is depicted in Fig. 1.
Overview of SDN architecture [137].
As we can visualize from Fig. 1 the logically centralized controller and forwarding elements communicate with each other by some standard communication interface. This application programmable interface (API) is used to establish communication between the SDN controller and forwarding devices of the network and grease the wheels for the controller to dynamically control the network in real time situations. The primary and most widely accepted example of this southbound API is OpenFlow [160,186]. OpenFlow based switches contain flow tables which are matched with the coming packets containing distinctive field entries according to which the specified actions are performed on these packets. Based on the matching fields the packet can be dropped, send to the controller or be kept in the memory in form of an instruction. Other than OpenFlow there are some more southbound APIs proposed by some networking enterprises in response to the monopoly of OpenFlow, most of which are the closed source [54,126]. On the top of the controller are some network applications or Northbound APIs which are responsible for the communication between the controller and the services that are active on the network. These APIs can provide services such as firewalls, load balancers and can also perform the traffic management and orchestration functions.
The introduction of software defined networking and the basic principle of separation of control and data plane, has introduced higher level of abstractions leading to innovation and evolution that was missing in the networking. Now by decomposing the network control problem into different layers we can find a more elegant solution to it with abstractions at every layer [159,249]. SDN is termed as the beginning of the software era in networking by many of its benefactors. When we talk about the evolution of SDN, it originally originated from OpenFlow in the pavements of Stanford University as a security concept but in a small period of time, it was largely accepted by the industry for solving most of the boiling issues of networking field [93]. Now significant number of Original Equipment Manufacturer (OEM) vendors, commercial hardware manufacturers, and component vendors have their product lineup which offers SDN based solutions. SDN as an idea is in its infancy period which is evolving and maturing with time through academic drilling and commercial exercises happening at both ends.
In this paper, we put all of this in a detailed manner covering every topic in different sections. Section 1 of this paper starts with the introduction of SDN and the key terms that are pivotal for better understanding of SDN. Section 2 answers the fundamental question of why SDN came into being, presents basic definition of SDN with the evolved and interpreted definitions of SDN. One part of this section throws light on standardization efforts and last part lays down the history of programmable networks. Section 3 is the core of this survey paper which broadly defines the three layer architecture of SDN. This section is subdivided into three planes of networking explaining every aspect of each plane by further classifying them into their corresponding networking layer. The idea behind following a layered approach is to bind every edge of this paradigm into a single thread which is clearly depicted in Fig. 2. Section 4 highlights some of the important pieces of software that are specially built for providing simulation and emulation environment for software defined networks. Section 5 is devoted to research work and future projects taking place in the industry and Section 6 concludes this paper.
Graphical abstract of paper.
Before we dive into the core topics of SDN, we present some terminologies for a better understanding of the topic because of their far and wide usage in this paper.
Overview of software defined networking
Software defined networking is a broad term which needs to be understood in its completeness because of the variety of networking topics that comes under its umbrella. So in this section we briefly define why do we need software defined networking at the first place with fundamental and interpreted definitions of SDN along with the difference between traditional networking and software defined networking. Then we discuss different standardization efforts carried out by several international standards organization and last but not the least we present the history of SDN in accordance to programmable networks.
Why software defined networking?
Hitting right at the heart of the traditional IP network approach where data plane and control plane are tightly coupled, decentralization of the network and corresponding devices is infrangible, SDN is clearly a disruptive concept. It has advantages over traditional networks and status quo. We are not stating that IP networks were a failure actually they were necessary and provided solutions to most of the problems the industry faced then like reliability, performance, and scalability. In fact, they are great at the layering of the network which helps in transport of packets locally and globally. It enabled innovation at each layer by decomposing delivery into fundamental components but as an academic discipline it didn’t turn out to be a success [159,249]. This resulted into a highly complex and difficult to manage networks which caused hindrance in innovation after a while because the evolution process was slow.
The traditional network architecture consists of three planes namely data plane, control plane, and management plane all contained in a single physical device. Data plane is the forwarding part of switches and routers responsible for delivery of packets, the control plane is the control unit infused with protocols that specify rules and actions to be performed on the incoming packet and management plane is the servicing part where network policy is defined. It is because of this architectural framework that traditional networks are rigid, inflexible and closed for innovation and evolution [30,90,132,210,227]. All of this can be summarized by saying, it is a vertically integrated approach.
In order to overcome the network control problem, the network engineers try to compute the configuration of each physical device, the SDN architecture is horizontally integrated providing capabilities of abstractions to the network with interfaces leading to modularity hence allowing the programmers to write millions of lines of code. These abstractions divide the control problem into tractable pieces. This gives a global network view provided through an API [132,249]. Analyzing the trend till now in networking industry every OEM vendor and hardware manufacturer provides or claims to provide solutions to the problems of network management, traffic management, and security threats by offering proprietary solutions. And to operate and manage these closed source solutions the service providers and network operators need to hire a team of specialized professionals which results in the increase in their operational cost and capital cost. On the contrary software defined networking is built on open source solutions.
SDN defined
In 2011, the first and foremost definition of Software Defined Networking was given by Martin Casado et al., at an event in Stanford University where this term came into existence to illustrate his work around OpenFlow (OF) [93]. The primary definition of SDN is referred to a network architecture where networks are programmable to enable new initiatives through flexibility, agility, and virtualization. Through a period of time the meaning and definition of SDN have changed, evolved and matured which is presented in this section of the paper, firstly let’s stick to the fundamental definition of SDN which revolves around four key ideas:
Separation of control and data plane: In SDN based networks, the data plane or forwarding elements of the network are separated from the control plane or control logic of the network to built a simpler architecture [159]. Logical centralization of control: The control logic is moved to a centralized SDN controller or Network Operating System (NOS) to present a global view of the network upon which a virtualization layer can create abstract views of the physical network to different control programs. These control programs can be created by different developers [95]. Flow based control: Flow based control means that forwarding rules are based on flow entries present in the table stored in the switch known as flow table. Each entry in a table is a flow identified by its match fields. If any entry matches the flow table entries the counter is incremented and specified action is performed with the packet. If the entry does not match with the flow table entries it is sent to the controller. So a flow is a sequence of packets between the source and destination [160]. Programmability: The network is directly programmable, that’s how APIs are developed over SDN controller that serve as network applications and services like firewall, load balancer, traffic engineering and so on. It is through these applications that SDN controller interact with forwarding devices [45,305]. With SDN and its fundamentals, networking is gradually transforming into a software discipline and becoming a part of computing. SDN is also defined by its fundamental abstraction model which was previously missing from the networking paradigm [219]. Abstractions have been an important part of computer field when we talk about operating systems, data structures, and distributed platforms, they have demonstrated tremendous growth in their respective fields by introducing abstractions [11]. In networking, layers have only dealt with the abstractions of the data plane, while SDN deals with the abstraction of control plane [249].
Three necessary abstractions
SDN can be accurately defined by these three fundamental abstractions. Abstractions divide the network control problem into tractable pieces by decomposing it and provides fundamental validity and general applicability to the idea of SDN. It is through these abstractions that no distributed control protocols are needed instead it involves designing a common distribution layer which converges into a network operating system providing a global network view and the network can be managed by control programs or network applications.
Difference between traditional networking & SDN
The disparity between traditional networking and SDN based approach can be perceived through their architectural framework. On one hand, when we talk about traditional networking the data plane, control plane and management plane are tightly coupled bringing innovation at standstill. When it comes to adding new protocols, network functionalities, policy enforcement and other network related issues traditional networks are a nightmare. Troubleshooting and reconfiguration of the network can lead to high Capex and Opex costs and deployment of revised network policies is complex [106,249]. As seen with the enterprises globally, if they want to upgrade their network systems in context with security or efficiency of the network all the network devices need to be reconfigured or even in some cases, new devices need to be installed. So in traditional networks, new policies or features are deployed into the network in form of new hardware equipment or generally known as middleboxes. These specialized hardware devices can perform the functionalities of load balancers, firewalls, intrusion detection systems but they are expensive and need specially trained professionals for operational requirements [247,304].
On the other hand, SDN pursues a whole different approach where it decouples or separates the control plane from data plane which leads to innovation in both the surfaces. A logically centralized SDN controller communicates with forwarding devices acting as network operating system which implements the network applications present on top of it which can be directly programmed with the help of abstractions. With this approach adding new protocols or introducing new network policies in the existing infrastructure becomes easier. Having a global network view helps in efficient implementation of policy decisions while troubleshooting and reconfiguration of hardware devices are not that much challenging [106]. The network applications can provide services of load balancer, firewall or intrusion detection systems eliminating the use of specialized hardware or middleboxes [138,247,304].
Evolved and interpreted definitions of SDN
When SDN was first defined by its creators in 2011, since then the definition of SDN has evolved and interpreted by different enterprises and communities according to their needs and views on what SDN is. Since SDN is a concept that tells what to do but not how to do. Therefore different enterprises said that if virtualization [253], orchestration [99], programmability [167] and, multi-tenancy is what SDN is, we can do this in our own ways which lead to an expansion in the core definition of SDN with no change in the fundamental architectural framework. The goal of SDN is to facilitate cloud providers and network operators to dynamically handle business requirements through a centralized control system. To put that into words companies like Cisco, Juniper, Nuage networks proposed different models for SDN/NFV (Network Function Virtualization) which served their interests even though remaining intact with the core definition of SDN. One of the examples is a model proposed by Nuage networks et al., known as Overlay SDN with virtualized service platform [182,183]. As claimed by the organization in this model the virtualized service directory (VSD) contains policy templates to simplify deployments and manage cloud services. The centralized controller handles devices through virtualized routing and switching. It is an open platform solution which can be integrated with existing networks. The network is dynamically programmed to work with hypervisors and manage tunnels between switches and hypervisors. A virtual network is established over the physical network and protocols like Virtual Extensible LAN (VXLAN) [153], Network Virtualization using Generic Routing Encapsulation (NVGRE) [266], Generic Network Virtualization Encapsulation (GENEVE) [94,121] are used to communicate between the tunnels that allow virtual machines (VMs) on multiple hypervisors to remain in the same domain. This SDN solution is basically implemented in data center environment because of its high scalability.
Some other attempts have been made by different communities to define SDN going beyond OpenFlow protocol. A glimpse of this is seen in the OpenDaylight project [207] where not only OpenFlow but other traditional protocols like Border Gateway Protocol (BGP) [234], Extensible Messaging and Presence Protocol (XMPP) [240], Simple Network Management Protocol (SNMP) [221] can also be used to support existing network infrastructure. There have been some legacy issues with SDN from its origin which have been undertaken by various SDN communities and working groups. So some hybrid models have come up from Internet Engineering Task Force (IETF) and others [60,247]. The broadening definition of SDN is supported by its incorporation into Cloud technologies [121], telecommunication field and extensible amount of open source projects carried out in cooperation with different IT field enterprises and networking giants. The Central Office Re-architected as a Datacenter (CORD) project [56], Open Platform for NFV (OPNFV) [149], Open Network Operating System (ONOS) project [140] and integration of OpenStack [57] with OpenDaylight are some of the prominent examples.
Standardization efforts
The standardization activities to define software defined networks and related areas are currently in progress through different Standards association, open communities or industrial frameworks. The ongoing research on SDN generally comes out in the form of open source platforms and solutions because of the efforts made by some organizations to keep this paradigm accelerating. While some of the research outcomes by these organizations are considered as standards other efforts are carried out by Standard Development Organisations whose primary activities are focused on developing and revising technical standards.
There are several standard bodies and non-profit organisations working in the field of Software Defined Networking and other related fields to make this paradigm more viable, feasible and to enhance its features. These organisations are working in different and specific areas ranging from research to standardization to educating and awaring the industry and academics about this new shift in technology which is expected to continue and is an evolving process. Some of the most prominent organizations and consortiums are Open Networking Foundation (ONF) [186], OpenDaylight project (Linux Foundation) [207], OpenStack [57] and others. So we have presented a list of leading organizations and consortiums working in this direction in Table 1. Since the focus of these organizations and consortiums is on the wide spread acceptance of SDN and related technologies most of the products excluding the commercial releases are open source which will help in accelerating the research in the respective fields.
The standardization process and history of SDN are closely related divisions because of the same organizations muddled into the process. So, first we present standardization activities by some prominent organizations followed by the history of software defined networking.
International standards organizations and communities
International standards organizations and communities
Open Networking Foundation is a non-profit, user-driven organization which initiated the SDN movement and accelerated the adoption of SDN based solutions into the real world. The trademark achievement of ONF is the introduction of OpenFlow standard protocol to enable remote programming of the forwarding plane and establish communication in the southbound interface. The technical communities of ONF are divided into three verticals namely Areas, Councils, and Groups. Areas is the community that works on the standards and certification including some other activities. The subdivision of that is Specification area which is defining standards, specifications and extensions to OpenFlow protocol for optical transport, wireless mobile, backhaul and enterprise WiFi under their respective sub groups. The first organization that was built for dedicated research on SDN and NFV is the Open Networking Foundation which works to transform networks into agile platforms for service delivery while streaming network operations by leveraging DevOps model with the cloud, open source and white box efficiencies [251]. It is funded by companies such as Microsoft, Facebook, Google, Deutsche Telekom. Some of the prominent projects under its hood are Atrium [21], ONF Libfluid [286] and SampleTap App [28] and not to forget the OpenFlow protocol. Recently it has merged with ON.Lab [187].
Linux Foundation Collaborative Projects hosts the OpenDaylight project which is an open source SDN controller. With its latest release Boron [208], it has built greater integration with larger industry frameworks from OPNFV and OpenStack to CORD and Atrium Enterprise. The OpenStack Foundation which also has some big enterprises as its members is working for NFV and SDN with its latest project OpenStack Neutron [176] focused on delivering Network-as-a-Service (NaaS).
A group called The Interface to the Routing System (I2RS) is formed by IETF to work on standards development and standardize network wide, multi layer topologies in network overlays and underlays and Routing Information Base (RIB) programming of any virtual or physical device.
European Telecommunications Standards Institute (ETSI) formed the Industry Specification Group for NFV. Industry Specification Group for Network Function Virtualization (ISG NFV) has four working groups and two expert groups which delivered white papers addressing challenges and operator requirements, as an input to standardization bodies. The ISG NFV community is working to develop required standards for NFV implementation and testing. SDN and NFV technologies are complementary and co-dependent considering that the modern telecommunication networks are based solely on proprietary hardware.
The Institute of Electrical and Electronics Engineers (IEEE) SDN standards committee established two research groups and two study groups for standardization efforts related to SDN and related areas. Some of the active standards projects include P1903.1 which is a standard for content delivery protocols of next generation service overlay networks among others. P802.1CF is the recommended practice for network reference model and functional description of IEEE 802 Access network.
All of the above discussed international standards organizations and communities that have created working groups, study groups and expert groups to analyze opportunities in the field of SDN and related areas are summarized in Table 1.
The term Software defined networking has become popular in the networking circuits over the past few years but the origins of programmable networks go back to almost twenty years [78]. There have been several efforts made by the industry to create programmable networks which started with the concept of Active Networking in mid-1990 [280]. Active networking mirrored a new approach in networking domain by introducing a programming interface or API that supported custom functionality on individual network nodes to apply to a group of packets passing through the nodes. It presented an alternative to Asynchronous Transfer Mode (ATM) [143,144] becoming the first clean slate approach to network architecture. It was further advocated by programs such as National Science Foundation’s FIND [169], European Union’s Future Internet Research and Experimentation (FIRE) [88] and Global Environment for Network Innovation (GENI) [33,214] which reflected its global acceptance.
While active networking opened the way for programmable networks and other related technologies but the problem of controlling and managing traffic over a network commonly known as traffic engineering became a major frustration for researchers opening the gates for the idea of separation of control plane and data plane. In 2001, IETF proposed standardized interface known as Forwarding and Control Element Separation (ForCES) [65] with Netlink interface to the kernel level packet forwarding functionalities in Linux. Some of the early proposals of separating the control and data plane were there in active networking and ATM but with ForCES, IETF came with an open interface for the control plane. SoftRouter [141] was proposed by IETF which allowed an external controller to install table entries in the data plane devices. IETF also provided logically centralized control of the network with Routing Control Platform (RCP) [43] and standardized protocols such as Path Computation Elements (PCE) [285]. Even after so many efforts ForCES and other open APIs were not widely adopted because of some legacy issues and distributed state of protocols so the researchers came out with another project named 4D project [78]. The 4D represented data plane, discovery plane, dissemination plane and decision plane, this high-level approach is considered as the base for the Secured Architecture for Networked Enterprise (SANE) project [48] and Ethane project [47]. These projects were carried out in Stanford University campus area as security projects where a logically centralized controller served the security policy with the help of flow tables. The roots of OpenFlow and SDN lies in the Ethane and SANE projects.
While the creation of these clean slate programs was going on which was focused at campus networks, on the other hand the government agencies were funding programs such as GENI [214] to build expensive and shared infrastructure. With all this in the scene, the researchers at Stanford University came up with pragmatic API named as OpenFlow. The creation of OpenFlow was followed by the design of control platforms such as NOX [95] which is the first logically centralized control platform. With NOX in action, the idea of network operating systems gained appraisals from the industry. OpenFlow that supported new capabilities was deployed in some campus areas with the help equipment vendors. After OpenFlow, some control platforms such as Onix [135] that had transactional persistence and ONOS [139] which is an open source controller with distributed capabilities came into existence.
With OpenFlow, the industry accelerated in an electric speed with so many open source projects surfacing the sea of networking paradigm. With widespread adoption of SDN solutions, it proved to be a problem solver for network virtualization. Virtual networks provide an abstraction of a network’s physical equipment [121]. The overlay networks or virtual networks operate through tunnels, some of the early examples of these overlay networks that created network topologies over legacy networks are MBone [152], X-Bone [213] and 6Bone [13].
Concluding the history of SDN in accordance with programmable networks which started with active networking and spanning up to OpenFlow where vision meets realism, the balance of vision with a pragmatic strategy needs to be adopted by the developers, researchers and the industry. It has been a long road with SDN solutions being adopted by the industry, this is just the starting in the field of programmable networks and network virtualization. And for better understanding of history of programmable networks and SDN, we suggest readers to refer to a paper by Nick Feamster et al. named, “An intellectual history of programmable networks” [79].
Software defined networking: Architecture
As we stated before that any SDN architecture can be split into three planes of networking, data plane, control plane, and application layer as shown in Fig. 3. Any SDN based solution deployed in the real world is characterized by different layers which are subsequent to these planes of networking but the devil lies in the details because it is not mandatory that all the layers are present in every SDN based architectural framework whereas the planes are indispensable. In this section, we introduce each plane of networking with respective layers encompassed within these planes of networking and different architectural framework proposed by different communities.
This depicts basic architecture of SDN.
Data plane is the physical network architecture that consists of all hardware devices used in the implementation of any SDN based infrastructure or traditional networks for that case. These devices are switches, routers, cables or middleboxes in the form of firewall, intrusion detection systems and so on. But in the case of SDN framework, these middlebox equipment are not present in the data plane rather the intelligence is removed from the data plane and shifted to the control plane which is discussed in the next section. The network intelligence is present inside the control plane in the form of a logically centralized controller which is governed or orchestrated through network applications built on top of that with open and standard interfaces. These interfaces introduce new levels of abstractions allowing innovation through open standards which solve the issue of interoperability and vendor lock in. The distributed nature of traditional networks and protocols have always remained a barrier to present a global view of the network.The disaggregation of data plane into Infrastructure layer and Southbound interface would help in better understanding.
Infrastructure layer
It is composed of simple forwarding devices without embedded control or software programs. These forwarding devices are specialized in packet forwarding, they can be hardware based or software based depending on the requirements of the desired network. The control is present in the centralized controller which is also known as network brain where all the logics are present. Most of these forwarding devices are OpenFlow based which is a standard southbound interface with widespread acceptability across the community because of its open standards and vendor free approach. The OpenFlow based forwarding devices work on the principle of flow tables stored in them. Each entry of this flow table consists of three parts which are: a rule for matching different field entries, the action to be performed on the incoming packet arrivals and a counter which is incremented and decremented based on the stats performed with each packet arrival and matching patterns.
Although there are other standards and protocols being developed for communication between the forwarding elements and the controller by different working groups that are in use; one of the early examples is Protocol Oblivious Forwarding (POF) [39,263] but OpenFlow being one the most influential forwarding specification is discussed in this paper.
The Switches based on OpenFlow protocol or commonly known as OpenFlow switches are those Ethernet switches that contain flow tables rather than routing tables. The basic specification architecture of SDN based OpenFlow switches as defined by ONF is shown in Fig. 4. Each flow table contains field entries for different networking functions such as MAC address, source address, destination address, etc. The flow tables are divided into three parts namely, header part which is again subdivided into differents parts which specify the rules, the action part which specifies the actions to be performed and the statistics part which handles the counter flow. The communication between OpenFlow switch and the controller occurs through a secured channel called Secure Sockets Layer (SSL) [63,235] through the exchange of OpenFlow messages.
According to one of the ONF archives, the data path presents flow table abstraction which contains field entries to match and perform actions accordingly. If it receives a packet with unknown flow entries, the packet is sent to the controller and then the controller decides whether the packet is there to reside inside the controller in form of instruction or not [190].
OpenFlow switch [160].
OpenFlow is considered as the starting point for SDN, it was through OpenFlow that the idea of Software Defined Networking came into existence because it supported most of the features which are now part of SDN and NFV [102]. Now as SDN grows OpenFlow will follow because of the parallel framework and design that they are built on. We can conclude that the future of OpenFlow relies on the future of SDN.
OpenFlow evolution summary
There are several OpenFlow enabled solutions available in the market that offer flow level control which are qualified for deployment needs of service providers and network operators. Most of these forwarding devices come enabled with Ternary Content Addressable Memory (TCAMs). We have put up a list of different software and hardware based OpenFlow switches which are in use for both research purposes and also deployed by different vendors for commercial purposes.
Software switches
Hardware switches
With this list of hardware and software based OpenFlow devices it can be detected that large-number of hardware and software manufacturers are focusing on SDN based solutions. There is a large variety of SDN or OpenFlow enabled products available in the market starting a more competitive and open source market. The rise of merchant silicon for manufacturing whitebox devices and bare metal switches is going hand in hand because it’s a necessary piece of infrastructure but non-differentiating.
Southbound interface in an SDN architecture acts as a communication link between the central controller and forwarding elements like switches and routers. Southbound interface or southbound API interacts with the forwarding elements to dynamically control the network in situations where real time guidance is vital. These APIs provide an abstraction to the infrastructure layer facilitating the developer to innovate and make enhancements to address some of the issues that service providers or network administrators face. Most of these APIs are open source but their capabilities predominantly depend on the underlying hardware and infrastructure.
The requisite for southbound APIs originated because of the limitations faced by manufacturers of commercial hardware switches. The traditional network switches face problems of upgradation and, development of new product is a complex task because of the software features related to it [129]. Whenever the network administrators want to deploy new policies to the existing network infrastructure, it is a tedious task with high Capex and Opex costs embedded with it. The development of southbound APIs deals with these issues faced by network operators and enterprises. The first proposal for a southbound API came from the Stanford University in the form of OpenFlow protocol which was accepted by the industry because it removed vendor lock in and promoted interoperability. The OpenFlow standards are regarded as a reference point for the development of other southbound APIs.
When we talk about southbound interfaces, OpenFlow is most widely accepted and deployed southbound interface. In an OpenFlow enabled device, the interaction with the SDN controller or network operating system is attained through three information sources. First, is the notification sent by the forwarding device to the controller when the link is triggered in the form of event based messages. Second, is the message sent by the forwarding device when the packet or incoming flow is unknown to it, the device sends a message to the central controller informing that no table entries were in sync with the respective packet arrival. Third, are the statistics messages generated with every flow entry which increases the monitoring capabilities. The OpenFlow enabled devices provide these flow level information to the central controller.
Even though OpenFlow standards are regarded as the reference point for southbound interfaces, there are several other participants in this field. A considerable amount of commercial vendors have come up with proprietary solutions that don’t prefer OpenFlow protocol as their southbound interface like Cisco OpFlex [261]. Other open source southbound APIs include Network Configuration Protocol (NETCONF) [70], Location Identifier Separation Protocol (LISP) [154], Open vSwitch Database Management Protocol (OVSDB) [215], SNMP [222], ForCES [65] and Programmable Abstraction of Datapath (PAD) [29].
NETCONF [70] is a network management protocol that provides a mechanism to install, manipulate and delete the configuration of network devices. The operations of NETCONF are based on Remote Procedure Calls (RPCs) which uses Extensible Markup Language (XML) based data encoding techniques for configuration and message transactions. NETCONF is an IETF initiative which was developed by NETCONF working group and standardized by IETF itself. IETF published a paper in December 2006 under the name RFC 4741 and later on revised and published it in June 2011 under the name RFC 6241.
OVSDB [215] is a simple management schema based protocol which allows network administrators to define the schema and OVSDB handles the communication part with the controller. This protocol supports all the basic database operations like delete, modify, create, select and is used to configure Open vSwitch (OVS) through these standard database commands. It was introduced with OVS created by Nicira and later acquired by VMware. OVSDB is regarded as an add on to OpenFlow because there are some functionalities which cannot be performed by OpenFlow alone so it is used to manage OVS implementations. This management protocol supports distribution across multiple physical servers by delivering configuration information to database servers.
Cisco OpFlex [261] is also a southbound policy protocol or API that communicates between the controller and underlying hardware for which Cisco has published a draft in IETF. OpFlex provides abstractions policies rather than device specific configuration providing network services with interoperability across vendors. Although OpenFlow and OpFlex look similar the difference is in their approaches, OpenFlow is application centric with imperative control whereas OpFlex is not an application centric model with declarative control. OpFlex distributes the network complexity to edges and operates in a disconnected mode from policy manager.
ForCES [65], an undertaking of IETF is considered as an alternative to OpenFlow protocol because it serves the purpose of separating the control and data plane with added data path functionality. ForCES also provide legacy support because ForCES framework backs older protocols of the existing network infrastructure. This differentiation of ForCES framework from OpenFlow can be contemplated by the fact that ForCES allows the notion of elastic routing architecture for networking devices by considering data plane and control plane part of the same network element which does not serve as a barrier in up gradation cycles.
One of the first adversaries to the monopoly of OpenFlow is POF [264]. POF is Huawei’s proposal where forwarding rules and behavior are controlled by control plane and physical infrastructure do not need to be reconstructed to adapt to new services because it can support any existing or new protocols. In POF enabled forwarding elements the network services are deployed as a third party customized software. As in the case of OpenFlow where forwarding devices match the flow entries of the incoming packets with the flow tables stored within them. In POF, the forwarding device is independent of the packet format and the flow table search keys are defined as tuples and instruction access data as tuples. Huawei has offered two prototypes for POF, one is open source software prototype and other is NP-based router platform. The company is investing in research work to come up with a perfect Flow Instruction Set (FIS) and solve some underlying technical issues related to standardization, forwarding plane virtualization and design of high-level language.
Control plane
In software defined networks, the control plane is known as the “network brain” of the architecture. This software programmable plane allows the logically centralized controller to dynamically access the network console and present a global view of network devices. This is the central part of SDN architecture which can be subdivided into three network layers namely; network hypervisors, network operating system, and northbound interface. The control plane is where logics and central policies are defined for the forwarding devices and it also directs the forwarding elements what to do with the incoming packets. The underlying infrastructure can be physical or virtual. The control plane acts as a bridge between the network applications that communicate with the central controller through northbound interfaces and the controller or NOS that directs the forwarding elements through southbound interfaces.
Network hypervisors
Network hypervisors or virtualization platforms reside between the forwarding elements of the network and central controller. To understand what are network hypervisors first we need to understand the concept of network virtualization and what role does it play in software defined networks.
Traditional networks have a strong dependency on hardware and vendor specific physical equipment. This type of infrastructure increases the Capex and Opex costs because making changes to this rigid framework and introducing new services often involves a long period of time. The standardization process between the vendors and operators is a very complex process and could take years [51]. So to tackle all these complexities, the network needs to be more flexible and adapt to new changes in a faster way which is achieved through network virtualization.
Virtualization from the network point of view means creating a virtual network over the physical infrastructure that decouples the underlying hardware from the virtual platform to allow multi-tenancy and provides isolation of services [134]. Virtualization started with cloud service providers, when they decided to decouple the physical hardware resources of servers like storage, computation and use a virtualization software to automate the slicing of these resources among multiple users [50]. So virtualization has come along years expanding its wings to networking field with the introduction of software defined networks [134].
What can be called as one of the first subsidiaries of network virtualization is topology virtualization where network administrators create virtual links among virtual nodes for the purpose of broadcast isolation by creating tunnels using the technologies like VLAN. In this, a whole network consisting of different commercial products, hardware devices are treated as a single node employing policy decisions as a whole. Next type of virtualization that exists in networking field is address virtualization where virtual address is provided to the server and the host in multi-tenant environments to prevent address collision at IP level or Media Access Control (MAC) level by using the technologies like Virtual Routing and Forwarding (VRF) [127], MPLS, Network Address Translation (NAT) and VLANs [212,312]. These two types of virtualization are supported by SDN based infrastructure in addition with network function virtualization [276].
Network hypervisors are virtual machine managers that create an abstraction layer between the network operating system and forwarding elements. The logically centralized controller depends upon the virtual global view of the network available to it. Like in the case of server virtualization the network hypervisors provide multi-tenancy support for public and private cloud service providers. They provide infrastructure-as-a-service (IaaS) or NaaS [31] regardless of the workload on their topology enabling each user to share storage resources and computational capabilities. For better resource utilization the cloud providers divide the workloads coming into their network with the help of network fragmentation. Virtualization is used in networks for simplified management as in the case of migration of network policies among different virtual machines. This means that when VMs are migrated from one physical server to another, they do not fall apart in terms of policies which is achieved by decoupling the policies from topology [212,312].
We know that current network infrastructures and virtualization technologies like VLAN, MPLS lack the basic amount of flexibility and automation required to address the issues of multi-tenancy, isolation, and migration of VMs because of their box-by-box configuration and lack of abstraction layer [51]. So to address these issues some modern tunneling technologies like VXLAN [153] and NVGRE [266] are introduced with SDN. Several open source and commercial network virtualization platforms or hypervisors are proposed by different enterprises. Prominent examples include FlowVisor [23,252], Network Virtualization Platform (NVP) [134], OpenVirteX (OVX) [8,9], Software Defined Network for Virtual Environment (IBM SDN-VE) [145,226], FlowN [66], AutoVFlow [300], Hyperflex [38], AutoSlice [40], RadioVisor [97], MobileVisor [177], Optical FV [24], etc.
FlowVisor [252] started as an academic project just as OpenFlow with the initial focus on rapid service deployment. This means that each service of the network is isolated enough that something happening in one of the network slices does not affect the other hosted virtual networks. One of the fundamental problems in network service deployment occurs when a service provider want to deploy new services to the network that require configuration changes to switches and routers, it’s really complex for network administrators to test the stability and other aspects of that particular service. FlowVisor solves this problem with the help of network slicing which is a coherent collection of resources on a network. The hypervisor intercepts control plane messages in an infrastructure where data plane is unmodified as in the case of OpenFlow. FlowVisor is conceptually based on OF where forwarding elements that are OF enabled can run FlowVisor in real production networks. A transparent virtualization layer resides between the forwarding elements and controller where every network slice believes it has complete control over every switch in the network maintaining its isolation from other slices. FlowVisor also allows the users to “Opt-In” to services in real time and add new FlowSpace to each slice policy.
Nicira/VMware NVP [134] is an open source virtualization platform which is independent of the hardware and uses OpenvSwitch to remotely login into the network . This is achieved with the help of hypervisors distributed in the network architecture to create an intelligent edge. NVP creates a virtualized view of physical infrastructure with the help of IP connectivity at the edge treating the physical network as Layer-3 fabric. NVP is deployed by many OpenStack production environments such as AT&T, DreamHost, eBay, NTT and Rackspace to accelerate their delivery mechanisms. NVP is a complete virtualization solution, with a distributed system architecture of the NVP controller cluster and security tools produced for utilization in real world deployments.
OpenVirteX [8 ,9] is a network virtualization platform based on OpenFlow which allows the user to create and design their own topology and communicate with different network operating systems to define the behavior of their network. OpenVirteX resides in the virtualization layer between the OpenFlow based forwarding devices and NOS to rewrite the messages in order to create isolation among different virtual networks. OVX is comparable to FlowVisor in terms of features provided by its virtualization tools. Multi-tenancy is supported in a way that several isolated virtual networks can have their own topology and network operating systems with the added feature of resiliency.
IBM has also introduced its solution, IBM SDN-VE [145,226] which is an overlay solution that provides a complete implementation framework for network virtualization. In the words of IBM, SDN VE is a multi-hypervisor, server-centric solution comprising multiple components that overlay virtual networks onto a physical network that provides IP connectivity. SDN VE like other hypervisors operate with no dependency on underlying network infrastructure and automate network provisioning. It enables multi-tenancy in data center networks by allowing existing network addresses to be retained. It abstracts the underlying hardware and presents it to applications as either a service or as an infrastructure.
FlowN [66] hypervisor uses the existing database technology for mapping virtual to physical topologies and is based on container based virtualization. Each container or tenant can have its own central controller that has full control over address space. FlowN contradicts the architectural framework of FlowVisor because the system design of FlowN is based on container based application virtualization where multiple tenants can run multiple application concurrently.
Huawei introduced a new platform Open Network Hypervisor (OpenNH) [111] for SDN based network virtualization. OpenNH creates a network hypervisor layer allowing tenant isolation, increasing traffic scheduling efficiency and customization for customer’s quality of service need. As claimed by Huawei the main benefit of OpenNH is network sandboxing which allows the operators to run their own applications on top of the customer’s SDN controller which enhances security and operational simplicity. One of the main use cases of OpenNH is 5G network slicing by supporting multiple virtual topologies, transparent technologies, and routing algorithms.
Another offering in the field of network hypervisors is from Cisco by the name of OverDrive Network Hypervisor [53] which is a complete virtualization solution that virtualizes network services by mapping a virtual network onto a physical network and creating subnets of addressing space. OverDrive is focused on data center environments and business semantics which follows a top-down approach in managing the network. It also provides orchestration capabilities to the network with real time automation and control.
Concluding the discussion on network hypervisors we can state that there are different types of virtualization platforms and solutions available in the market, some of them are open source and some of them are commercial multi-tenant network hypervisors with closed proprietary. An enormous amount of research work is already in the pipeline to address the issues which include improved network mapping techniques [91] and level of abstraction needed to perform virtualization in an efficient manner. The field of network virtualization is expected to grow in the same manner because of its large implications in the telecommunication field and its use among cloud service providers.
Network operating system
The conventional operating systems provide an interface between the end user and system components to perform the functionalities of memory management, file management, and other related tasks. All this is achieved by providing abstraction at almost every level of computation in the form of programs and applications. System programs continue to evolve much faster than networking because they provide abstractions and abstraction leads to innovation whereas traditional networks mostly depend on a large set of distributed protocols which are enormously complex to manage. There have been major changes in the last decade when we talk about system software because of active participation of application developers and open source codes which is fairly missing in the networking field. As we know traditional networks are broadly managed and configured by closed proprietary network operating systems provided by some big networking giants like Cisco and Juniper. They have their own operating systems to configure the devices and define policies that are to be implemented over a network but they are device specific which lacks transparency putting a barrier to innovation. So, to put all this into a place it can be concluded that network engineers have mastered the complexities of a network but there are some problems that remain and need to be addressed.
Software defined networking assures to solve the problems of distributed state algorithms and routing protocols by providing the right level of abstractions and programming capabilities in the form of a logically centralized controller or network operating system. This centralized controller provides application programming interfaces covering both southbound and northbound which speed up the innovation by facilitating developers to write application programs for functionalities such as network configuration. With NOS, the developers no longer need to care about the low-level details of the network or complexities of distributed routing protocols and topology information. All of this is handled by low-level abstractions creating an ecosystem where innovation can foster.
A network operating system can be defined as a central element of SDN architecture which is responsible for generating control decisions and communicating with southbound forwarding elements, corresponding protocols, and northbound APIs. There is a wide variety of SDN controllers that have emerged over time and diversified set of the architectural framework proposed by the creators of each controller. The classification of SDN controllers can be on the basis of two principles. First, whether they are open source or closed source, second is based on the architectural framework which means whether the controller is centralized or distributed. Apart from this, there are some controller platforms that provide hybrid architecture.
Based on the requirements of network operators and service providers both centralized and distributed platforms have their own set of specifications and functionalities. The centralized SDN controllers are mostly based on the model of parallelism with the system capabilities of multi-threaded and multi-core computer architectures. These controllers are regarded as single entities and are not enough to handle networks that contain a large amount of forwarding devices. One of the major limitation of centralized controllers is scalability. Even though there are some examples of centralized controllers which have achieved the functionalities to operate enterprise class and data center networks. Some popular examples of centralized controller include NOX/POX [82,96,157], Beacon [71], Maestro [44], Floodlight [84], Trema [275], and Meridian [26]. Some specific controllers like Floodlight and ProgrammableFlow [170,171] specifically target data center and cloud based infrastructure.
On the other hand, distributed network operating systems are much more scalable and reliable than centralized controllers. The architectural design of these controllers is in a way that the control logic is centralized but the physical infrastructure is a distributed set of elements. As in the case of data centers, where different data centers are treated as clusters of nodes spanning over a wide area network. And cloud service providers require a hybrid approach to support their diverse operational activities [122]. Example of distributed controllers include some open source and some commercial platforms such as Onix [135], ONOS [140], yanc [163], HyperFlow [281], HP Virtual Application Network (VAN) SDN [110], NVP controller [134], Distributed SDN Control Plane (DISCO) [216], and Unified controller [146]. Many distributed platforms face the problem of elasticity which means that their architecture is static in the terms nodes connected to it. Consistency semantic is a major problem with these controllers as all nodes in the same network read different values for the same property at some specified period of time. The problem of weak consistency is solved by platforms such as Onix and ONOS which provide strong consistency for all controller nodes connected in a network.
One other phenomenon in network operating systems is fault management. Whenever some fault or failure occurs at any instance in a network, by fault we mean a condition where a node in a network does not operate as it is required to, the operations of the faulty node is taken over by some other node in the network. This node can be the adjacent node to the faulty node or any other node present in the network. A centralized controller which is a single entity is not sufficient to manage large networks where crash failures can be arbitrary. Whereas in an ecosystem of distributed controllers where self-reliant controllers are distributed over the network, arbitrary or otherwise crash failures are managed in a much efficient way. Due to high scalability and resilience being the major features of distributed control platforms they reduce the impact of crash failures.
There are several components and elements present in the diversified environment of controllers. Not all components, services and properties are offered on every platform available in the market, some of the platforms have restricted or specific area of focus like cloud and telecom operators. Despite having different design and architecture, three layer design is entertained by almost every platform available till date; the application layer present on top of every architecture providing orchestration control and services, the control layer where the NOS/SDN controller is present and the lowest layer consisting of southbound devices. There are some distributed control platform where eastbound and westbound APIs are also required or implemented for better management and control functionalities.
The application or upper layer communicates with the controller through northbound APIs and programming languages. There are few options available in the section of northbound interfaces such as RESTful API [236] or commercial device/company specific interfaces developed by companies like Cisco, HP, and Juniper. Another approach for communicating with a controller is through programming languages like Frenetic [86], Merlin [265], Maple [290], and Pyretic [165]. One section of this paper focuses on northbound interfaces.
The core layer is where the controller resides and provides the basic network services offered by any network operating system like communication and execution. Other basic functionalities of controller consist of topology design, device management, configuration, memory management of statistics and event handling like notifications implemented through different algorithms. The algorithms used in these platforms can be traditional ones’ like shortest path routing algorithm [85] or specific algorithms designed for handling complexities of modern day networks like Raft [5]. Like other operating systems, a network operating system provide a mechanism for security which includes the isolation capabilities and enforcement of security policies over the services and applications available to the network.
The lowest layer of control platform communicates with the controller through southbound interfaces/APIs. The forwarding devices present in the data plane communicates through a common interface and a defined set of protocols. At this moment there are several southbound APIs available like OpenFlow, OpenDaylight, ForCES, OVSDB, ONOS, and Onix with protocol plugins and extensions to support legacy infrastructure. Most of the controllers are OpenFlow based/enabled but some platforms extend their support to old routing protocols like BGP, SNMP. OpenDaylight, Onix, and HP VAN SDN controllers offer a wide range of interfaces and protocol plugins, OpenFlow being one of them.
Other than these three layers some distributed control platforms implement special eastbound/westbound interfaces for providing functions of import and export of data between different controllers [135]. These interfaces also provide notification and monitoring capabilities, interoperability and compatibility among different platforms. Some important steps towards interoperability and increasing the diversity of an ecosystem include ForCES CE-CE interface [294], AMQP (Advanced Message Queuing Protocol) [287] by DISCO [216] and import/export functions of Onix. The eastbound/westbound interfaces are required in distributed control platforms to leverage the scalability and resiliency and not in centralized platforms [270].
For a better understanding of network operating systems we need to discuss the design and architectural framework of some diverse set of controllers and control platforms:
To summarize we present a list of some commercially available SDN controllers in Table 5 and open source SDN controllers in Table 6 with a remark that SDN controllers are crucial for the success of SDN [288]. The major issues that need to be addressed in terms of control platforms are interoperability among different vendors and standardized APIs for multi-domain deployments. In multi-domain environments, there is series of diverse northbound and southbound APIs available for both data center networks and cloud infrastructure but lack of east/westbound APIs is an issue of concern which is being tackled by the developers.
Commercially available SDN controllers
Commercially available SDN controllers
Open-source SDN controllers
Northbound Interface (NBI) is one of the two key abstractions of the SDN ecosystem. The northbound interface is still under the process of getting a standard specific architectural declaration and a final establishment which can be accepted by the industry and academics [64]. On the other hand, the southbound interface already has a widely accepted proposal like OpenFlow. The ONF has formed a working group for the development of northbound interfaces in the year 2013 which has been working on the same since then. It is still quite early to specify any particular standard for NBI. Open and standard northbound interfaces are very important for the flexibility and interoperability among different control platforms.
NOSIX [309] is regarded as the first step towards achieving both portability and controller independence. It is not fundamentally an option for northbound interfaces but rather it is a higher level abstraction for southbound interfaces. All the existing controllers which include Floodlight, Trema, NOX, Onix, and OpenDaylight have proposed different northbound APIs which have different properties and definitions. SFNet [303] is also an example of a northbound interface which is used for translating application requirements into lower level service requests. The services offered by SFNet has a limited scope.
Other proposals that use different approaches for communication between higher level applications and the centralized controller may include the yanc control platform [163] which proposes a general control platform based on Linux and abstractions such as Virtual File System (VFS).
Presently it still not likely to have a single northbound interface which may be declared as standard or reference. Different network applications have different requirements and based on those requirements the northbound APIs may also change. The network security applications work in a totally different way when compared to the financial applications. Therefore, to serve different purposes, different techniques are used for better network resource management and utilization.
There are also controllers like PANE [81] which have been proposed with the concept of participatory networking which allows network administrator to define module-specific quotas and access control policies on network resources. It provides an API which allows dynamic and autonomous requesting of network resources by the end-host applications.
Presently the paradigm which is being discussed for the northbound interfaces is the declarative Intent NBI or intent networking [55]. The intent NBI is characterized by two basic properties which are highly connected with each other, they are non-prescription and provider independence. In non-prescription, the vendor will be specified with, what are the services which can be requested by the user but the way those services would be served depends on the provider. Implementation choices like port, media, node, technology, server, path, virtual machines are left to the providers. The second characteristic, provider independence is derived largely from non-prescription only. Through this, the same intent NBI request may be presented to any provider for which consumer provider association exists. However, mappings will be generally provider-specific.
Application layer
Application Layer sits at the top in the design architecture of software defined networks. It handles all the activities associated with provisioning and monitoring of the networks. The major network services like fault management, configuration, performance management and security features are provided to the network in form of network applications. This also guarantees another layer of abstraction which is a high-level abstraction necessary for virtualization based solutions. It can also be disaggregated into two network layers namely; programming languages and network applications.
Programming languages
In computer systems, traditionally and fundamentally programming languages are used to provide abstractions and facilitate the developers or system programmers to hide the complexities of system hardware. In starting, low-level assembly programming languages were developed which were hardware specific and since then programming languages have evolved from low-level machine language to high-level programming languages like C, C++, Java and so on. This has helped the computer industry to grow faster [77,101].
Relatively, programmability in networks has always remained an issue of concern. OpenFlow, which is a low-level machine language provided abstractions for some low-level details but it is highly dependent on underlying hardware which is a barrier for developers to solve some specific network problems. The problem with low-level programming languages such as OpenFlow is that it is really difficult to read/write programs and reuse the code which makes the software development process less modular and more error-prone [164]. So some high-level programming languages are built on top of the central controller to provide higher levels of abstraction to the programmer with a runtime environment that handles the functionalities such as installing rules, responding to switches, and events that happen in the lower level of a network [109,164].
The problem with programming a network is that the software provided by equipment vendors are complex and configuring the network is expensive and error prone. The code used for creating applications is not modular therefore the resultant applications are monolithic and the code cannot be reused. Software defined networking provides an opportunity to solve these problems by creating higher levels of abstractions [231]. These abstractions help the programmer to address the core challenges of lower level instruction sets [100]. There are several challenges faced by network administrators which can be addressed by the use of network programming languages. To list some of the problems: (i) Multitasking is not possible in such a way that the functionalities and properties of one application do not interfere with properties and functionalities of other application. (ii) If there is a single controller to run multiple applications, it is difficult to manage and optimize applications accordingly.
High-level programming languages have been proposed keeping in mind the complexities and issues faced by lower level instructions sets such that they can provide a wide variety of functions and properties possible with the help of software defined networking. Some of the features of programming languages are listed below:
Creates higher levels of abstraction enabling the software programmers to innovate in a more focused environment built specifically for innovation and development of network applications.
Guarantees code re-usability and creates software programs with the additional property of modularity.
One major focus area of these programming languages is network virtualization.
Separates reading from writing by specifying queries on network state and forwarding policies.
Prevents race conditions by automatically applying the correct policies.
Reduces the latency in forwarding the packets from one controller to another and helps in defining the rules specific to the flow.
Different network programming languages which have been proposed till date are listed in Table 7. The programming languages can be classified based on the programming paradigm they are built upon. Going into the specifics, some programming languages like Kinetic [133], Pyretic [165] are imperative languages where programs are written as a sequence of statements. Functional languages such as Haskell [114], Frenetic [87] are based on mathematical computational functions. Some are functional reactive programming languages like Procera [289], Nettle [199], and Frenetic [87] where applications are written based on reactive actions triggered by events such as data and device discovery. There are some other network programming languages which are much more specific in their offerings.
Concluding with a remark that there are other several aspects where network programming languages can contribute to the development of software defined networks which need to be embarked upon by developers and researchers. For example, a recent research report published that present policy compilers in network programming languages generate unnecessary rule updates [296]. From the advances in the field of network programming languages and SDN, it is clear that most of the future work and research will be focused on network applications built on top of the SDN ecosystem which is discussed in the next section.
Network programming languages
Network programming languages
Network applications in context to software defined networks can be termed as software programs written on top of the controller for implementing various network services. Analysing the trend in SDN landscape, the dominance of application development is evident. Most of the research work is aligned towards developing network applications for different controller platforms. Considering the wide area of implementation that software defined networks cover, there are several applications for security, orchestration, monitoring and measurement, traffic engineering, routing, cloud and big data, optical and wireless networking, mobility, and virtualization. As the surge in application development is contemplated by the increased number of use cases in real world deployment of these applications, it is expected to be a major driving force in the adoption of SDN [232].
Current network applications also perform the functions of traditional network applications such as routing, security, load balancing, and monitoring but with additional features of less power consumption and flexibility. Deployment and testing of SDN applications are much more cost effective and simpler because they provide a centralized control with no hardware dependency.
The primary problem in the development of SDN based applications is portability. SDN applications are controller or platform specific, for example, if one application is implemented in one control platform, it is not possible to run the same application on any other control platform. It is also very complex to combine two applications to built a single application with the properties of those two corresponding applications. And same issues remain with debugging of SDN applications, the tools for testing and debugging are controller specific. There is some amount of research work going in this direction for creating an integrated SDN development environment, one result is the NetIDE framework [75] which claims to bring all the elements of software development to networking field.
In this section, we present a brief description of the categories under which the wide variety of SDN applications can be classified namely: Monitoring and measurement, security, optical and wireless networks, cloud and big data, and traffic engineering.
Several applications and platforms have been proposed for monitoring and measurement purposes such as OpenSample [273] which is a low latency sampling measurement platform, OpenSketch [308] separates data plane for traffic measurement, FlowSense [307] which is used for monitoring network utilization with zero measurement cost, Context Encoding enables layer 2 pathlet tracing, OpenTM [282] is a traffic matrix estimator for OpenFlow networks, OpenSample [200] which is used for monitoring extensible and scalable networks, FleXam [257] is another tool for flexible sampling of OpenFlow networks, OpenNetMon [284] for monitoring Quality of Service (QoS) parameters, and Payless [52] is a query based real time monitoring platform. Some other efforts like online measurement of large traffic through commodity switches [125] and detection of lightweight DDoS flooding attack using NOX/OpenFlow are impactful.
There are several examples of prominent steps that have been taken for securing software defined networks and also levering SDN capabilities for securing existing network infrastructures. Some of them are: Ethane [47] which is the first instance of providing security through flow-rule enforcement which was carried forward with SANE project [48] which also implemented security policies, active security is an integrated security framework which is based on feedback controls, CloudWatcher framework [254] is proposed for monitoring cloud platforms, AVANT-GUARD [256] is a specific security extension for OpenFlow networks, Cognition [277] is another effort for enhancing security mechanisms, and the list continues with FlowNAC [155], FRESCO [255], OpenSAFE [25], OrchSec [311], VAVE [302], FortNOX [220], NetFuse [293], Software Defined Remote Triggered Black-Hole (SDN RTBH) [92], Reliable multicasting [136], Resonance [89], LiveSec [291], MAPPER [241], DDoS detection and mitigation [41], SDN based elastic IP and security [268].
Therefore, several attempts have been made by the researchers and industry to leverage SDN capabilities to optical and wireless networks. Some influential examples are: Enabling optical internet with OpenFlow [258], deployment of software defined optical networks [150], Odin [245] is a Floodlight controller which is based on a framework for programmable virtualized Wide Local Area Networks (WLANs). SoftRAN [98] is a software defined radio access network with the capabilities of load balancing and interference management, SoftMoW [166] is an architecture built for recursive and reconfigurable WANs, OpenRadio [271] is a design for programmable wireless dataplane with modular interfaces, AeroFlux [243,244] is an extension for Odin which is a scalable hierarchical WLAN control plane, CROWD project [10] focuses on overlapping the functionalities of Long Term Evolution (LTE) and WLAN cell technologies, C-RAN [62] architecture is built for improved performance through RAN virtualization of mobile nets, Flexible Access Management System (FAMS) [301] is an OpenFlow based system for flexible VLAN, and so on the list continues.
One of the major focus area where both industry and academia are leveraging functionalities and capabilities of SDN is cloud data centers (CDCs) which span over a large geographical region. By leveraging SDN capabilities the CDCs and Internet Service Providers (ISPs) are engaging in much more cost effective networks which reduce the energy and bandwidth requirements to a reasonable extent [310]. In addition to this, a research has proposed an optimal pricing scheme for inter-AS traffic requests which guarantees QoS which is supposed to benefit both customers and ISPs [128]. These solutions focus on the economical effects of SDN distinctively on commercial network services.
Efforts from researchers and industry have made it possible to come out with some applicable results. Important examples of these results include: Meridian [26] which is a SDN platform for cloud service providers that supports a service-level model for application networking, FlowComb [59] is a network management framework that helps in achieving high resource utilization and low data processing times for Big Data applications, similar to FlowComb is FlowDiff [16] which detects operational problems, leveraging the SDN controller with optical switching at runtime, CloudNaaS [31] is a framework that implements networking primitives for cloud applications, NetGraph [228] is a graph based kernel networking subsystem for network management, LIME [131] model supports transparent and live migration of entire network of virtual machines and switches.
Extending the scope of above solutions, recent developments and research works are focused on providing better traffic engineering solutions necessary for better flow management, fault tolerance, topology update, and traffic analysis and characterization [6]. In this context, a research work has been done which lists the principal transformations induced by SDN and NFV by providing a better traffic matrix estimation considering both engineering and econometric prospects [83].
Till date, there are several traffic engineering applications and solutions that have been proposed by academics and industry which include: Hedera [7] which is a scalable and dynamic flow scheduling algorithm built for multi-stage switching fabric, load balancing solution Plug-n-Serve [105] minimizes server response time, ElasticTree [107] is a network-wide power manager developed for satisfying changing data center traffic loads, DEFO [2] is a central optimizer which translates high level language into compliant network configuration, wildcard rules [223] are implemented for more scalable solution to achieve a target distribution of the traffic, and the list continues on growing with solutions like ALTO VPN [242], ViAggre SDN [260], Pronto [299], QNOX [124], OpenQoS [68], Middlepipes [123], MicroTE [32], PolicyCop [27], SIMPLE [225], ProCel [168], Flow QoS [246], QoS for SDN [248], Aster*x [104], In-packet Bloom filter [151], QueuePusher [209].
Simulation and emulation
As we have specified in this paper that real world deployment of SDN can be an expensive process therefore efficient simulation and emulation tools are required to test network resources before deployment. And same is the case with academics and researchers, they need an emulation software to carry out innovation and improve the reproducibility of networking research. The most popular emulation software for testing the capabilities and resource utilization of software defined networks is Mininet [142]. Mininet is specialized network simulator which is capable of creating virtual hosts, switches, controllers, and links over standard Linux network software. The main feature of Mininet is that it provides an inexpensive network testbed for prototyping, testing and debugging OpenFlow based applications. The applications and controllers designed, developed and tested on Mininet can be deployed in real world OpenFlow enabled networks with minimal modifications.
An extension to the first version of Mininet is dubbed as Mininet HiFi [103] which transforms the Mininet from functional testing emulation software to performance testing virtualization platform. New features introduced in Mininet HiFi are the creation of virtual links with specified bandwidth, latencies and loss rate, container based virtualization, resource provisioning, and monitoring of performance. Mininet Cluster Edition (CE) [15] is another extension to Mininet which increases its functionalities but it is an experimental prototype. Mininet CE distributes Mininet over a cluster of servers to emulate massive networks with thousands of nodes. SDN DC [279] is an emulation software for developing and testing OpenFlow based controllers for cloud data centers. This framework enhances Mininet and POX controller with necessary extensions to develop cloud based applications. Several other initiatives for emulating large scale networks and SDN deployments that have distributed architecture are CityFlow [46], DOT [238] and MaxiNet [297].
EstiNet network simulator [292] and emulator is another solution with the capabilities of both emulation and simulation. In simulation environment, it includes physical layer, media access control layer, network layer, transportation, and application layer. And in emulation mode, it builds a testing network filed with both physical and virtual network devices. OFNet [185] SDN network emulator is similar to Mininet with added functionalities of traffic generation, monitoring flow, and performance evaluation. A simulation tool called fs-sdn [262] is designed to address the problems of prototyping and evaluating new SDN based applications. It is built to provide much better and accurate measurements than Mininet with scalability advantages. The adequacy of emulating the OpenFlow based SDN environment have also been added to the tried and tested ns-3 simulator [180].
Ongoing research and future expectations
Throughout this survey paper, we have highlighted on the extensive amount of research work going on, covering every aspect of SDN landscape. But for completeness, we have dedicated this whole section to the research developments taking place in different sectors of SDN pushed by different communities, vendors, and organizations working in the field of networking. We present this section through the perspective of the industry. Firstly, we explore the hardware implementation area which is touched by almost every manufacturer and then we present the research work taking place in some specific companies like Google, AT&T, Microsoft, Intel, Verizon, Ericsson, Huawei, and Equinix.
The use of TCAMs for manufacturing chipsets that are OpenFlow enabled is common with OEMs but TCAMs have some limitations regarding cost and power consumption [147] which sometimes result in power drain of the switching devices [4]. With every revised version of OpenFlow protocol and corresponding switching devices, the above-specified problem is tackled through various compression techniques, the Espresso Heuristic [239] being a prominent one. The manufacturers are working hard to reduce the cost of SDN based hardware equipment which is still a concern for the industry. One of the performance parameters for commercial switches is achieving a higher throughput of flow entries per second which is an issue that is being addressed [34,272]. Another point is CPU power consumption which is being addressed through different efforts made by the industry like the native design of SDN switches.
To improve capabilities of SDN based solutions some amount of advancement is required in the field of hardware architecture by evolving switch designs and hardware enhancements. So microchip companies like Intel and ARM are manufacturing specialized network processors with in built SDN capabilities [120]. It is not feasible to mention every effort going in this direction through this survey paper because there is a large number of small scale and large scale vendors involved in this process of evolution.
Now, we focus on recent developments happening in different companies with respect to software defined networks.
SDN at Google
Google has been one of the major contributors for supporting modern age networking or SDN. In the words of Google itself, “The Pillars of SDN @ Google” [203], consists of (i) B4 which is a wide area network interconnect for data centers built on whiteboxes. (ii) Andromeda which is a network function virtualization stack which forms the basis of Google cloud network by supporting multi-tenancy within the internal cloud network. (iii) Jupiter is termed as a software defined approach for data center networking. And their most recent offering is Espresso, Google’s peering edge architecture which presents a global view of the network and use application signals to provide real time optimization.
The future research work proposed by Google in the field of SDN is through the eye of serverless compute which provides a real-time intelligence through machine learning termed as Cloud 3.0. The forthcoming cloud infrastructure would be deeply embedded with SDN to provide high availability, modularity, scalability, security, and resiliency [203].
SDN at AT&T
AT&T terms its presence in the field of SDN as AT&T Network 3.0. Carrying some prominent SDN projects under their hood, AT&T are transforming their traditional networks to software defined networks. The Indigo project claims to connect the technologies like 5G, Big Data, AI, machine learning, cyber security with SDN [18]. Other SDN based solution include AT&T FlexWare [19] which is deployed on an integrated cloud platform that leverage SDN and NFV. By now, AT& T has converted 34% of their network functionality to SDN and by the end of 2017, 55% of their network services would be virtualized [205]. Enhanced Control, Orchestration, Management & Policy (ECOMP), the orchestration platform that provides necessary automation is transitioning into Linux Foundation’s open source project and ECOMP is integrated with another project OPEN-Orchestrator (OPEN-O) with a new name, Open Network Automation Platform (ONAP) [205].
SDN at Microsoft
Microsoft’s Azure networking team is focused on leveraging SDN through Container networking. In 2016, Microsoft announced a Linux based container networking service SONiC [162] which is an open source switch stack. Most recently in 2017, another generally available (GA) as a service in Azure is in the form of Azure container service, Kubernetes. Microsoft is working on SDN with flexible and scalable VNet over Azure platform and now they are extending Azure VNet for containers which are said to be one stop open SDN solution for VMs. Most of the research work of Microsoft team is directed towards leveraging SDN stack for cloud infrastructure and services [202].
SDN at Intel
Intel has formally announced its strategic and product plans for leveraging SDN networks. It launched SDN switch and server platform and some reference architectures for them. The offerings like Open Network Platform Switch Reference Design running over top of Intel’s Open Network Platform which is developed from Intel’s Wind River Linux operating system are considered as strong examples contemplating Intel’s interest in this novel approach [119]. Intel also acquired Fulcrum which has designed a low latency switch silicon for SDN based solution. The Data Plane Development Kit (DPDK) [118] is also included in this ecosystem as SDN is not only about software but it also includes development in the fields of merchant silicon, white boxes which are essential components of SDN based networks.
SDN at Verizon
Verizon is one of the largest telecom service provider with 24 data centers spread across the globe which needs to be managed and operated in a seamless manner with increased reliability and scalability which can be achieved by leveraging SDN. So, with solutions like Verizon Enterprise Orchestration and Virtual Network Services (VNS) [201], Verizon can get new network services to market quickly with centralized management. Verizon partnered up with Cisco for delivering solutions in several departments of network services [201].
SDN at Ericsson
Ericsson Cloud SDN [72] is a network virtualization solution for data center connectivity of physical, virtual and container based workloads. It is based on open source and industrialized version of the OpenDaylight controller. Another product by Ericsson is Services SDN [73] that enables end-user service personalization. With several other initiatives like supporting Open COMP management and orchestration, NFV interoperability testing initiative, launching a verified NFV infrastructure solution, a 5G-ready core solution which is based on SDN/NFV is taken by Ericsson to strengthen their presence. The partnership between Ericsson and Red Hat is another step for enabling open source solutions based on OpenStack, containers, NFV, and SDN [230].
SDN at Huawei
Huawei is investing a reasonable amount of cash flow into software defined networks and is offering products specific to data center, campus networks, cloud, and data center interconnection needs. The CloudFabric [113] solution leverages SDN technology to open cloud data center networks. Huawei’s wired and wireless convergence solution is tailored for networks with different scenarios. The CloudDCI is another SDN based offering focused on the interconnection of cloud and data centers. Huawei is also leveraging SDN technology in Internet of Things (IoT) with EC-IoT [112] that applies edge computing and cloud managed platforms to the IoT field.
SDN at Equinix
Equinix is the leading global colocation data center provider with more than 175 operational data centers. With this much of burden, it needs scalability and reliability to connect customers with cloud service providers which is achieved by SDN based Cloud Exchange Platform [204]. Another software based network offered by Equinix to its customers is built by using APIs from Apigee known as active switching platform [204].
Apart from these big names in the ICT domain which are contributing in the development and evolution of SDN landscape several other communities and organizations are also contributing in the research and standardization process. There are various research fields and challenge areas like design of switches, interoperability and standardization of controller platforms, resiliency and scalability issues with the SDN based real world deployment, performance evaluation of software defined networks, security of these flexible and agile networks, hybrid deployments, and migration of traditional networks to software defined networks where continuous efforts and attempts are being made to make this novel approach more acceptable.
Conclusion
After seven years when this term, software defined networking (SDN) was coined, it has gone through a phase of maturity and contradictly is still said to be in its infancy period. SDN constituted an ecosystem which opened up gates for innovation in networking by solving the problems of traditional networks. The conventional ecosystem is hard and complex to manage because it is vertically integrated and vendor specific. The data plane and control plane are tightly coupled leaving no space for innovation and evolution which is necessary for any industry to grow. Because of vendor lock-in, the products and their configuration are company specific which eventually leads to long upgradation cycles increasing the Capex and Opex costs.
Now, with SDN the above-discussed problems encountered by network engineers in traditional networks can be solved. Firstly, data plane and control plane is decoupled which means forwarding rules and control logic is separated. Software defined networks can be dynamically programmed where forwarding devices or data plane elements are converted into dumb devices with packet forwarding logic/flow tables pre-installed in them. They interact with logically centralized controller/network operating system through southbound interfaces. The central controller implements the network logic or provides the network services by establishing communication with network applications sitting on top of it through northbound interfaces. This type of architectural design which presents a global view of the network has speed up the pace of innovation in the networking industry.
With this paper, we present the readers an all-inclusive view of history, different definitions, related concepts and challenges faced by this novel approach. Trying to cover all the topics, to the best of our knowledge and literature available to us, in this paper we cover a broad range of issues following a layered approach for different planes of networking.
In the inception, the paper introduces this new paradigm by explaining why networking industry needs SDN and provides a thorough introduction to the standardization process, next it compares SDN with traditional networking techniques and the history of programmable networks. Then, it dives deep into the core architecture of SDN representing and describing three planes of networking with their corresponding subdivision layers. Namely, data plane subdivided into two layers (infrastructure layer and southbound interface); control plane subdivided into three layers (network hypervisors, network operating system and northbound interface); and application layer is categorized again into two layers (programming languages and network applications). In the end, the last section is dedicated to the ongoing research efforts and innovation by several industry players.
The journey of SDN has come a long way which started by hitting at the closed, proprietary and vertically integrated solutions of the networking industry. SDN promises disaggregation of software and hardware components, softwarization and virtualization of the networks and most importantly an open source ecosystem providing interoperability among different vendors. SDN has successfully managed to cross the barriers which were inhibiting innovation and have provided an opportunity to develop next generation network infrastructure. Areas like interoperability, carrier grade deployment, and legacy issues require further research. SDN is here to stay and will continue to grow and evolve as still there is a long journey ahead.