SDN for Cloud Networks

SDN for Cloud Networks Building the Foundation for ... r o l + M a n a g e m e n t P l a n e SBI Driver SBI Agent SBI Agent Source: Open Network Found...

0 downloads 26 Views 1MB Size
SDN for Cloud Networks Building the Foundation for “Networking as a Service”


Outline ❒ Software Defined Networking and Cloud ❒ Openflow ❒ Virtual Switching ❒ SDN Controllers

Software Defined Networking and Cloud

Networking as a Service ❒ Compute and Storage as a

Service What AWS, Azure, and Google Compute Engine provide ❍ Pay only for as much compute and storage as you use ❍

❒ What Networking as a Service is ❍ Only connect to my VMs • Looks like a LAN ❍

Only bring up network when VMs need it • Dynamic

Only pay for bandwidth when I use it ❍ Isolate my traffic from other customers ❍

❒ Goal: Offer full Infrastructure as a

Service ❍

EnterpriseLAN but entirely in the cloud


Recall: ❒ Need for isolation ❍ Broadcast/multicast ❍ Different tenants/IP addresses from same address space ❍ Between different tenant VMs on the same server ❒ Multiple tenants/dynamic creation and

destruction of tenant networks ❍

Tenant network resource usage is highly dynamic

❒ VM movement between different physical

servers ❍

DC operator manages VMs by “hot swapping” them between servers

Solution is Virtual Networks!

Key Properties of Virtual Networks ❒ Partitioning: ❍

Each physical resource can be used concurrently by multiple virtual network instances • Also true of statistical multiplexing

❒ Isolation: ❍

The clear isolation of any virtual network from all others

❒ Abstraction: ❍

Control and management of a virtual resource need not require complexity of directly manipulating underlying physical resources

❒ Aggregation: ❍

Multiple instances of physical resource grouped to obtain increased capacity Source:

Wide Area Network Characteristics ❒ Huge physical area ❍ 5,000 km from SFO to NYC ❍ 9,000 km from SFO to FRA ❍ 15-30 ms speed of light delay ❒ Sparse and minimal bandwidth

between routers/switches ❍

100 Mbps corporate VPN connections are typical

❒ Bandwidth is expensive ❍ 1G fiber connection from San Jose to Montreal - $3K per month!


❒ Simple network and edge

controlled by enterprise/retail customers ❍

But complex operator edge is trending

❒ Slow resource allocation/fast

reaction to failure Slow: routing protocol convergence times ❍ Fast: MPLS protection and failover time ❍

But in Data Centers: ❒ Small physical area ❍ Order of thousand m2 ❍ Minimal speed of light delay ❒ Lots of bandwidth

1G leaf/10G backbone links common ❍ 10G leaf/40G backbone links trending ❍

❒ Bandwidth is cheap ❍ Commodity switches inexpensive

Source: Bilal, et. al.

❒ Complex edge/simple core ❍ Virtual switch/virtual edge bridge (VEB) on server ❍ Simple L2/L3 fabric out to Internet router ❒ Fast dynamic, centralized control of

compute and storage resources ❍

Tenant resource allocation takes place through dedicated orchestration software • Like an operating system on a single server

In Addition: ❒ Natural centralization of control in data

center architecture through the Network Virtualization Authority (NVA) ❒ Software trumps hardware for fast implementation and deployment (VEPA example)

Software Defined Networks (SDN) for Data Center Network Control and Management!

Traditional Computer Networks:Data Plane

Data plane: Packet streaming Fast but Dumb

Forward, f ilter, buf fer, mark, rate-limit, and measure packets

Traditional Computer Networks:Control Plane Control plane: Distributed algorithms Slow but Smart

Track topology changes, compute routes, install forwarding rules

Traditional Computer Networks:Management Plane Management plane: Human time scale Slow but Really Smart

Collect measurements and configure the equipment

Software Defined Networking (SDN) Logically-centralized controller Slow but Smart

API to the data plane (e.g., OpenFlow)

Fast but Dumb Switches

SDN Design Principles ❒ Logically centralized but physically distributed control/management

plane Merge control and management plane ❍ Controller replicas, distributed DB protocols for scale-out and relability ❍

❒ Northbound API exposes useful abstractions for network services

programmers ❍

Hide details of: • Individual vendor equipment • Control protocols that do similar jobs • …

❒ Southbound protocols allow full control of fowarding and network

configuration Example forwarding control: OpenFlow (up next) ❍ Example network management: OF-Config (up next) ❍

❒ Switches and routers implement forwarding and network configuration

as programmed from the controller

SDN is a Key Technological Approach to Implementing Virtual Networks in Cloud

Control +Management Plane

Generalized SDN Architecture

SBI Driver SDN Southbound Interfaces (SBIs)

SBI Agent

SBI Agent

Source: Open Network Foundation



Control Path (Software) Data Path (Hardware) OpenFlow/SDN tutorial, Srini Seetharaman, Deutsche Telekom, Silicon Valley Innovation Center


OpenFlow Controller OpenFlow Protocol (SSL/TCP)

Control Path


Data Path (Hardware) OpenFlow/SDN tutorial, Srini Seetharaman, Deutsche Telekom, Silicon Valley Innovation Center

OpenFlow Switching Software Layer

Hardware Layer

Controller PC

OpenFlow Client MAC MAC IP src dst Src

IP Dst

* *


port 1


port 2

TCP TCP Action sport dport *

port 3 The Stanford Clean Slate Program,

OpenFlow Flow Table

port 1

port 4


OpenFlow Flow Table Entry Rule


Stats Packet + byte counters

1.Forward packet to port(s) 2.Encapsulate and forward to controller 3.Drop packet 4.Send to normal processing pipeline 5.… Switch MAC Port src + mask

MAC dst

Eth type


The Stanford Clean Slate Program,

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

OpenFlow Flow Table Examples Switching Switc MAC MAC Eth VLAN IP h src dst type ID Src Port 00:1f:..* * * * *

IP Dst

IP Prot

TCP TCP Action sport dport






Routing Switc MAC MAC Eth VLAN IP h src dst type ID Src Port * * * * * *

IP IP Dst Prot 5.6.7. * 8

TCP TCP Action sport dport *



Firewall Switc MAC MAC Eth VLAN IP h src dst type ID Src Port * * * * * *

IP Dst

IP Prot

TCP TCP Action sport dport





OpenFlow/SDN tutorial, Srini Seetharaman, Deutsche Telekom, Silicon Valley Innovation Center


Development of OpenFlow Specification Standards OF v0.9 developed at Stanford in CleanSlate Program

OF 1.0 Single flow table Ethernet circa 1994 IP circa 1994

OF 1.1

OF 1.2

OF 1.3

MultipleExtensible IPv6 match hdr flow extension tables Queues IPv6 MPLS Group fixed tables PBB MPLS tagging (but broken) Tunnel control

OF 1.4 Optical ports Bundled messages

OF 1.5 Meter Table Multiple Controllers

OpenFlow 1.5 Architecture

Source: Open Network Foundation, “OpenFlow Switch Specification: Version 1.5.0”, ONF TS-020, December 19, 2014

OpenFlow 1.5 Switch Model

Source: Open Network Foundation, “OpenFlow Switch Specification: Version 1.5.0”, ONF TS-020, December 19, 2014

OpenFlow 1.5 “State Machine”

Source: Open Network Foundation, “OpenFlow Switch Specification: Version 1.5.0”, ONF TS-020, December 19, 2014

Multiple Table Example:Port Based VLAN Tagging ❒ Most NICs don’t handle VLAN tags ❍ Packets are sent untagged ❒ VLAN tags are inserted at the first hop switch ❍ Based on the source port ❒ Implementing with one table results in a combinatorial explosion ❍ With one table: Nport x NMAC ❍ With two tables: Nport + NMAC



Table 0:











Push VLAN ID 42; Send to Table 1

Table 1:











Packet out Port 42

Group Table Detail Group Table: A table supporting entries with action buckets, which can be executed all or in part depending on the group type Group Table Entry: ❒ Group identifier: 32 bit integer identifying the group on

the OpenFlow switch ❒ Group type: a type identifier defining the group semantics ❒ Counters: for tracking group statistics ❒ Action Buckets: an ordered list of action buckets, where each bucket contains a set of actions and associated parameters that are always executed as an action set.

Action Bucket Semantics ❒ Indirect ❍ Execute the one bucket in this group ❍ Only a single bucket ❍ Allows multiple flow entries to point to a single group ❍ Example use: IP next hop forwarding ❒ All

Execute all buckets in the group ❍ Example use: broadcast or multicast ❍

❒ Select ❍ Execute one bucket in the group ❍ Bucket selected through a switch-specific algorithm ❍ Example use: Equal Cost MultiPath (ECMP) forwarding ❒ Fast Failover

Execute the first live bucket in the group ❍ Allows switch to change forwarding without contacting the controller ❍ Switch must implement a liveness mechanism ❍ Example use: fast failover ❍

Meter Table Details Meter Table: a collection of per flow meters used for rate limiting that can be combined with Queues to implement QoS strategies Meter Table Entry: ❒ Meter identifier: 32 bit unsigned

integer idenfitying this meter ❒ Meter bands: unordered list of bands, where each band specifies a rate and a processing method ❒ Counters: keep track of number of packets processed by this meter

Band Entry: ❒ Band type: defines how the packet is

processed ❒ Rate: lowest rate at which the band can apply ❒ Counters: keep track of the number of packets processed by this band ❒ Type specific arguments: optional arguments depending on type

Network Management for OpenFlow ❒ OF-Config is the standard Network Management

protocol for OpenFlow ❍

Others are also in widespread use

❒ Job of OF-Config ❍ Configures various switch “hard” state ❍ Query switch for capabilities ❒ Extension of IETF Netconf protocol ❍

OF-Config schema in XML

❒ Examples of hard state configuration: ❍ Virtual ports • Tunnels • Aggregated links

Queues ❍ Certificates ❍

OF-Config Architecture

Source: Open Network Foundation, “OF-Config 1.2: OpenFlow Configuration and Management”, ONF TS-016, 2014

OF-Config Tunnel UML Model

Source: Open Network Foundation, “OF-Config 1.2: OpenFlow Configuration and Management”, ONF TS-016, 2014

OpenFlow Control Plane for VxLAN Data Center Virtualization OF-Config OF-Switch Sets Sets Up Up Flow Virtual Processing Port for for VNI-1 VNI-1

OpenFlow Controller/ OF-Config Management Point

Switc MAC MAC Eth VLAN IP h src dst type ID Src Port ToR1 VNI-1 * * * * VM * *

Port Port Port


IP Dst


IP Prot

TCP TCP Action sport dport * * VNI-1 vPort out

Port vPort

Flow Table VNI-1

Port Port

Data Center L3 Network


OpenFlow and Hardware:Pica8 ❒ Example Hardware: ❍ P-3922(top) and P-3930 (bottom) ❒ Ports

64 x 10GbE ❍ 4 x 40GbE uplink ❍

❒ Either fiber (P-3922) or copper

(P-3930) ❒ Nonblocking switch fabric with low latency 1.2 Tb throughput ❍ 960 Mpps ❍

❒ Suitable for top of rack switch ❒ Conventional networking: ❍ Maximum MAC addresses: 128K ❍ Maximum VLANs: 4094 ❍ Maximum routes: 12,000 ❒ Openflow networking: ❍

Trident+ chip: maximum 8,000 flows

Limited Flow Table Size is the Problem with OpenFlow and Conventional Hardware

Virtual Switching

What is a Virtual Switch? ❒ System software that emulates a physical

switch Conceptually sits in the hypervisor ❍ Downlink: Emulates NICs toward VMs (vNICs) ❍ Uplink: Connects to hardware NICs through hypervisor ❍

❒ Provides virtualized networking support to

VMs Bridges traffic between VMs on the physical server ❍ Switches off-server traffic from the ToR to the VMs ❍

Example Virtual Switch: Open Virtual Switch (OVS) ❒ History ❍ 2007- First released by Nicira ❍ 2010 - OpenFlow 1.0 support ❍ 2011- Ported to hardware (Pica8) ❍ 2012 – VMWare acquires Nicira, commits to maintaining OVS ❍ 2012 – OVS becomes part of Linux release ❍ 2014 – OpenFlow 1.5 support ❒ Open source, maintained by VMWare ❍

Included with Xen, KVM open source hypervisors

❒ Proprietary competitors ❍ Cisco 1000V (number one) ❍ VMWare vDS (but trending downward) ❍ IBM DVS 5000V ❍ Microsoft Hyper-V vswitch

Supported Functionality ❒ IPv4 and IPv6, full 802.1, 802.3 Ethernet ❒ Standard 802.1Q VLANs with trunking ❒ Spanning Tree Protocol (STP) ❒ Multiple IP layer tunneling protocols ❍ GRE, VxLAN, and more ❒ OpenFlow 1.x, where x depends on OVS

version ❍

But not OF-Config • Older OVS-DB protocol used for configuration

❒ Many, many other features that hardware

switches support

Software Architecture User space Configuration Control Plane

Kernel space Data path (data plane) Included in Linux kernel release since 3.3


How OVS Looks to a VM


Performance v.s. Linux Bridge

OVS Linux Bridge Source:

SDN Controllers

What Should I Look for in an SDN Controller? ❒ What applications/services come with the controller? ❒ What protocols does it support on the southbound ❒ ❒ ❒ ❒

(network equipment side) interface? What kinds of abstractions does the northbound API support? Is there support for scale-out, failover, replication, reliability, high availability, in-service upgrade? Does the controller support a Web GUI, command line interface (CLI), and/or API for controller configuration? Does the controller support peering with other controllers?

Example Controller: OpenDaylight


OpenDaylight: History ❒ April 2013 – Formed as an open source

project under the Linux Foundation Cisco was the driver ❍ Other supporters were Ericsson, IBM, HP, Juniper, Arista Networks. … ❍

❒ December 2013 – Hydrogen release ❍ Three “editions” • Basic • Data Center • Service Provider

❒ October 2014 – Helium release


AAA: Authentication, Authorization &OVSDB: Open vSwitch DataBase Accounting Protocol AuthN: Authentication PCEP: Path Computation Element BGP: Border Gateway Protocol Communication Protocol COPS: Common Open Policy Service PCMM: Packet Cable MultiMedia DLUX: OpenDaylight User ExperiencePlugin2OC: Plugin To DDoS: Distributed Denial Of Service OpenContrail DOCSIS: Data Over Cable Service Interface SDNI: SDN Interface Specification (Cross-Controller Federation) FRM: Forwarding Rules Manager SFC: Service Function Chaining GBP: Group Based Policy SNBI: Secure Network DDo LISP: Locator/Identifier Separation Protocol Bootstrapping Infrastructure SDNI Network Applications S SNMP: Simple Network Wrap Orchestrations & Prot Management Protocol per Services ectio TTP: Table Type Patterns n VTN: Virtual Tenant Network AAA- AuthN Filter

Apps and API Support for Cloud Virtual Networks nothbound

“HELIU M” VTN Coordi nator


OpenSt ack Neutron

OpenDaylight APIs (REST) Base Network Service Top olog y Man ager

Stat s Man ager

Swit ch Man ager


OpenSt Functionsack Service Host Trac ker

VTN Man age r

OVS DB Neu tro n

GBP Serv ice Plug in20 C

SCF LISP Serv ice

AAA L2 Swi tch

DOCSI S Abstr action SDNI Aggr egat or

Service Abstraction Layer (SAL) (Plugin Manager, Capability Abstractions, Flow Programming, Inventory, etc.)

GBP Renderers

OpenFl ow


OpenFlow Enabled Devices





Open vSwitches




Plugi n20C

Additional Virtual & Physical Devices

Controller Platform

Support for

Openflow and OVSDB southboun Southbound d & Interfaces

Peering supporte d through BGP

Protocol Plugins

Data Plane Elements (Virtual Switches, Physical Device Interfaces)

Other Features Not Shown ❒ Apache Karaf used for OSGI support ❍ In service upgrade ❍ Allows installation of a new module without shutting down the controller ❍ Essential feature for telcom software ❒ GUI and CLI both provided ❍

CLI is through Karaf

❒ Infinispan key-value store used for storing state ❍ ACID (Atomicity, Consistency, Isolation, Durability) “hard” transction model ❍ But doesn’t support persistence ❒ No explicit support for scale-out, failover,

replication, reliability, high availability

Service Abstraction Layer (SAL) ❒ SAL is the major difference between OpenDaylight and other

controllers ❒ Objective: Write an abstract layer that can be used by various services northbound and various protocols southbound ❍ Allow different southbound protocols and northbound applications to reuse components ❍ Eliminates the problem of duplicated code for different protocols ❍

❒ Basic approach: model driven development ❍ Design model using UML ❍ Write YANG code to implement the model • YANG: Yet Another Next Generation • Data modeling language for NetConf ❍

Compile YANG code to Java

❒ Controversial ❍ Not fully realized in the Hydrogen release • Some services including cloud networking were released outside the SAL ❍

More of a research project than production ready

Summary ❒ SDN lays the foundation for Networking as a

Service (NaaS) ❍

SDN technology is a natural match for deploying virtual networks in a cloud

❒ OpenFlow is an example of a protocol that supports

the deployment of a Software Defined Network ❍

But it is not the only such protocol • Example: XMPP used by Juniper in its Contrail controller

❒ Virtual switching locates complex virtual network

configuration on the end host ❍

Where the computational resources support it

❒ SDN controllers support northbound interface to

cloud management/orchestration software simplifying virtual network provisioning

Acknowledgements ❒ Xenofontas Dimitropoulos, ETH Zurich ❒ Jennifer Rexford, Princeton