Run Juniper vMX as Contrail Gateway for IPv6 Overlay

Run Juniper vMX as Contrail Gateway for IPv6 Overlay

Overview

Juniper vMX is distributed as a compressed tar file and getting started documentation can be found at https://www.juniper.net/techpubs/en_US/vmx14.1/topics/task/installation/vmx-install-preparing.html. I explain the steps I took to launch and configure a vMX instance as Contrail L3VPN Gateway router for IPv4 and IPv6. I took the “quick-and-dirty road” to run vMX directly over the latest qemu-system-x86_64 version.
I didn’t have a proper routed IPv6 Subnet at hand, so I “carved” a /80 from the subnet the server is connected to, knowing the server is the only device on that subnet apart from the router/switch it is connected to.

Requirements

Prepare the software

Traffic will be routed between the physical server interface and the virtual bridges connecting the vMX. Therefore, IP forwarding must be enabled in the Linux kernel for both protocols:

sed -i 's/#net.ipv4.ip_forward/net.ipv4.ip_forward/' /etc/sysctl.conf
sed -i 's/#net.ipv6.conf.all.forwarding/net.ipv6.conf.all.forwarding/' /etc/sysctl.conf
sysctl -p /etc/sysctl.conf

Install the required tools to compile qemu, optionally removing apparmor (instead of adjusting the apparmor files):

apt-get update
apt-get -y purge apparmor
apt-get install -y bridge-utils git tmux build-essential pkg-config zlib1g-dev
apt-get --no-install-recommends -y build-dep qemu

Download and compile the latest version of qemu:

git clone https://github.com/qemu/qemu.git
cd qemu
./configure
make -j
make install

If all went ok, you’ll find qemu in /usr/local/bin:

/usr/local/bin/qemu-system-x86_64  --version
QEMU emulator version 2.3.91, Copyright (c) 2003-2008 Fabrice Bellard

The vMX consists of 2 virtual machines and a virtual disk to keep the configuration and log files. The images can be found in the vMX tar file under the images directory:

ls vmx-14.1R5.4-1/images/*img
jinstall64-vmx-14.1R5.4-domestic.img vmxhdd.img vPFE-20150707.img vPFE-lite-20150707.img

Copy these images to a local directory, from which they will be launched:

cp vmx–14.1R5.4–1/images/jinstallimg vcp.img
cp vmx–14.1R5.4–1/images/vPFE-lite-
img vfp.img
cp vmx–14.1R5.4–1/images/vmxhdd.img .
“`

Prepare the virtual networking

Create the tap interfaces for VCP and VFP:

ip tuntap add dev tap-vcp mode tap
ip link set tap-vcp up promisc on
ip tuntap add dev tap-vfp mode tap
ip link set tap-vfp up promisc on

ip tuntap add dev tap-vcpi mode tap
ip link set tap-vcpi up promisc on
ip tuntap add dev tap-vfpi mode tap
ip link set tap-vfpi up promisc on

Add tap-vcp and tap-vfp to the default virtual bridge virbr0, offering NAT’d Internet access via teh servers IPv4 address:

brctl addif virbr0 tap-vcp
brctl addif virbr0 tap-vfp

Create a new bridge to offer the required connectivity between VCP and VFP via tap-vcpi and tap-vfpi. Setting an IP address on brint0 is optional, but helps troubleshooting by gaining access to the VCP’s em1 static IP of 172.16.0.1 ,respectively VFP’s static IP of 172.16.0.2:

brctl addbr brint0
ip link set brint0 up
ifconfig brint0 172.16.0.10/24

echo "add tap-vcpi and tap-vfpi to brint0"
brctl addif brint0 tap-vcpi
brctl addif brint0 tap-vfpi

Create two virtual bridges and tap interfaces for ge–0/0/0 and ge–0/0/1, setting an IP address on the virtual bridge brge1, offering routed access to ge–0/0/1. ge–0/0/0 is bridged with tap0 in this example. tap0 is provided by OpenVPN and is not documented in this blog entry, offering connectivity to Contrail Control via SSL VPN ethernet tunnel.

ip tuntap add dev tap-ge0 mode tap
ip link set tap-ge0 up promisc on
ip tuntap add dev tap-ge1 mode tap
ip link set tap-ge1 up promisc on

echo "create brtap0 bridge and add tap0 and tap-ge0"
brctl addbr brtap0
ip link set brtap0 up
brctl addif brtap0 tap0
brctl addif brtap0 tap-ge0

echo "create brge1 bridge and add tap-ge1"
brctl addbr brge1
ip link set brge1 up
brctl addif brge1 tap-ge1
ifconfig brge1 inet6 add 2a01:4f8:191:24ef::2/64
ifconfig brge1 10.33.33.1/24
ip route add 193.5.1.0/24 via 10.33.33.2
ip -6 route add 2a01:4f8:191:24ef::3/128 dev brge1   # vmx1
ip -6 route add 2a01:4f8:191:24ef:1::/80 via 2a01:4f8:191:24ef::3
ip -6 route add 2a01:4f8:191:24ef:2::/80 via 2a01:4f8:191:24ef::3
ip -6 route add 2a01:4f8:191:24ef:ffff::/80 via 2a01:4f8:191:24ef::3

With all the required tap and virtual bridge interfaces up, its time to launch the vMX. VCP and VFP are launched individually using small shell scripts.

Launch vMX VCP (Junos Control plane):

# cat launch-vcp.sh
#!/bin/bash

/usr/local/bin/qemu-system-x86_64 \
  -drive if=ide,file=./vcp.img \
  -drive if=ide,file=./vmxhdd.img \
  -M pc -smp 1 --enable-kvm -cpu host \
  -m 2000 -numa node,memdev=mem \
  -object memory-backend-file,id=mem,size=2000M,mem-path=/mnt/huge,share=on \
  -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
  -netdev tap,id=tc0,ifname=tap-vcp,script=no,downscript=no \
  -device virtio-net-pci,netdev=tc0,mac=52:54:00:45:11:01 \
  -netdev tap,id=tc1,ifname=tap-vcpi,script=no,downscript=no \
  -device virtio-net-pci,netdev=tc1,mac=52:54:00:45:12:01 \
  -chardev socket,id=charserial0,host=127.0.0.1,port=8896,telnet,server,nowait \
  -device isa-serial,chardev=charserial0,id=serial0 \
  -vnc 127.0.0.1:1 -daemonize

The script will launch VCP in the background and offer access to the serial console via TCP port 8896 (use ‘telnet localhost 8896’ to access the console).

Launch vMX VFP (Forwarding plane):

# cat launch-vfp.sh
#!/bin/bash

/usr/local/bin/qemu-system-x86_64 \
  -drive if=ide,file=./vfp.img,format=raw \
  --enable-kvm -cpu IvyBridge \
  -m 5000 -numa node,memdev=mem \
  -smp 4,sockets=1,cores=4,threads=1 \
  -object memory-backend-file,id=mem,size=5000M,mem-path=/mnt/huge,share=on \
  -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
  -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
  -netdev tap,id=t0,ifname=tap-vfp,script=no,downscript=no \
  -device virtio-net-pci,netdev=t0,mac=52:54:00:11:11:02 \
  -netdev tap,id=t1,ifname=tap-vfpi,script=no,downscript=no \
  -device virtio-net-pci,netdev=t1,mac=52:54:00:11:12:02 \
  -netdev tap,id=t2,ifname=tap-ge0,script=no,downscript=no \
  -device virtio-net-pci,netdev=t2,mac=52:54:00:11:13:AA \
  -netdev tap,id=t3,ifname=tap-ge1,script=no,downscript=no \
  -device virtio-net-pci,netdev=t3,mac=52:54:00:11:13:BB \
  -vnc 127.0.0.1:2 \
  -chardev socket,id=charserial0,host=127.0.0.1,port=8897,telnet,server,nowait \
  -device isa-serial,chardev=charserial0,id=serial0 \
  -daemonize

Check both VM’s are running:

# ps ax|grep qemu
 1958 ?        Sl   1832:14 /usr/local/bin/qemu-system-x86_64 -drive if=ide,file=./vcp.img -drive if=ide,file=./vmxhdd.img -M pc -smp 1 --enable-kvm -cpu host -m 2000 -numa node,memdev=mem -object memory-backend-file,id=mem,size=2000M,mem-path=/mnt/huge,share=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -netdev tap,id=tc0,ifname=tap-vcp,script=no,downscript=no -device virtio-net-pci,netdev=tc0,mac=52:54:00:45:11:01 -netdev tap,id=tc1,ifname=tap-vcpi,script=no,downscript=no -device virtio-net-pci,netdev=tc1,mac=52:54:00:45:12:01 -chardev socket,id=charserial0,host=127.0.0.1,port=8896,telnet,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:1 -daemonize
15646 pts/0    S+     0:00 grep --color=auto qemu
18225 ?        Sl   4069:36 /usr/local/bin/qemu-system-x86_64 -drive if=ide,file=./vfp.img,format=raw --enable-kvm -cpu IvyBridge -m 5000 -numa node,memdev=mem -smp 4,sockets=1,cores=4,threads=1 -object memory-backend-file,id=mem,size=5000M,mem-path=/mnt/huge,share=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -netdev tap,id=t0,ifname=tap-vfp,script=no,downscript=no -device virtio-net-pci,netdev=t0,mac=52:54:00:11:11:02 -netdev tap,id=t1,ifname=tap-vfpi,script=no,downscript=no -device virtio-net-pci,netdev=t1,mac=52:54:00:11:12:02 -netdev tap,id=t2,ifname=tap-ge0,script=no,downscript=no -device virtio-net-pci,netdev=t2,mac=52:54:00:11:13:AA -netdev tap,id=t3,ifname=tap-ge1,script=no,downscript=no -device virtio-net-pci,netdev=t3,mac=52:54:00:11:13:BB -vnc 127.0.0.1:2 -chardev socket,id=charserial0,host=127.0.0.1,port=8897,telnet,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -daemonize

Configure vMX

Log into the serial console of the vMX, set a root password, enable ssh and configure interfaces ge–0/0/0, ge–0/0/1, BGP to Contrail and the VRF for IPv4 and IPv6 connectivity in the overlay:

## Last commit: 2015-07-25 20:42:26 UTC by mwiget
version 14.1R5.4;
system {
    host-name vMX1;
    root-authentication {
        encrypted-password "removed"; ## SECRET-DATA
    }
    name-server {
        2001:4860:4860::8888;
        2001:4860:4860::8844;
    }
    login {
        user mwiget {
            uid 2000;
            class super-user;
            authentication {
                encrypted-password "removed"; ## SECRET-DATA
                }
        }
    }
    services {
        ssh;
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
chassis {
    fpc 0 {
        pic 0 {
            tunnel-services;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description internal;
        unit 0 {
            family inet {
                address 10.8.0.101/24;
            }
        }
    }
    ge-0/0/1 {
        description external;
        unit 0 {
            family inet {
                address 10.33.33.2/24;
            }
            family inet6 {
                address 2a01:4f8:191:24ef::3/64;
            }
        }
    }
    em0 {
        unit 0 {
            family inet {
                address 192.168.122.3/24;
            }
        }
    }
    lo0 {
        unit 1000 {
            family inet6 {
                address 2a01:4f8:191:24ef:ffff::101/128;
            }
        }
    }
}
routing-options {
    rib inet6.0 {
        static {
            rib-group default2PublicV6;
            route ::/0 {
                qualified-next-hop 2a01:4f8:191:24ef::2 {
                    interface ge-0/0/1.0;
                }
            }
            route 2a01:4f8:191:24ef:1::/80 next-table Public.inet6.0;
        }
    }
    static {
        rib-group default2Public;
        route 193.5.1.0/24 next-table Public.inet.0;
        route 0.0.0.0/0 next-hop 10.33.33.1;
    }
    rib-groups {
        default2PublicV6 {
            import-rib [ inet6.0 Public.inet6.0 ];
        }
        default2Public {
            import-rib [ inet.0 Public.inet.0 ];
        }
    }
    router-id 10.8.0.101;
    autonomous-system 64512;
    dynamic-tunnels {
        contrail {
            source-address 10.8.0.101;
            gre;
            destination-networks {
                10.8.0.0/24;
            }
        }
    }
}
protocols {
    mpls {
        ipv6-tunneling;
    }
    bgp {
        group contrail {
            type internal;
            local-address 10.8.0.101;
            family inet-vpn {
                any;
            }
            family inet6-vpn {
                any;
            }
            family evpn {
                signaling;
            }
            family route-target;
            neighbor 10.8.0.1;
        }
    }
}
routing-instances {
    Public {
        instance-type vrf;
        interface lo0.1000;
        route-distinguisher 64512:1000;
        vrf-target target:64512:1000;
        vrf-table-label;
        routing-options {
            interface-routes {
                rib-group {
                    inet default2Public;
                    inet6 default2PublicV6;
                }
            }
        }
    }
}

Verify the setup

The config will set the vMX management port (em0) to 192.168.122.3. Because this port is bridged to virbr0, it is accessible via IP from the server hosting the vMX:

root@vMX1> show interfaces terse ge-0/0/*
Interface               Admin Link Proto    Local                 Remote
ge-0/0/0                up    up
ge-0/0/0.0              up    up   inet     10.8.0.101/24
                                   multiservice
ge-0/0/1                up    up
ge-0/0/1.0              up    up   inet     10.33.33.2/24
                                   inet6    2a01:4f8:191:24ef::3/64
                                            fe80::5254:ff:fe11:13bb/64
                                   multiservice
ge-0/0/2                up    down
ge-0/0/3                up    down
ge-0/0/4                up    down
ge-0/0/5                up    down
ge-0/0/6                up    down
ge-0/0/7                up    down
ge-0/0/8                up    down
ge-0/0/9                up    down

root@vMX1> show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
bgp.l3vpn.0
                       4          4          0          0          0          0
bgp.l3vpn.2
                       0          0          0          0          0          0
bgp.l3vpn-inet6.0
                       1          1          0          0          0          0
bgp.l3vpn-inet6.2
                       0          0          0          0          0          0
bgp.evpn.0
                      10         10          0          0          0          0
bgp.rtarget.0
                      15         14          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
10.8.0.1              64512      12618      13659       0       9  4d 6:25:12 Establ
  bgp.l3vpn.0: 4/4/4/0
  bgp.l3vpn-inet6.0: 1/1/1/0
  bgp.evpn.0: 10/10/10/0
  bgp.rtarget.0: 14/15/15/0
  Public.inet.0: 4/4/4/0
  Public.inet6.0: 1/1/1/0

root@vMX1>
Advertisements

One thought on “Run Juniper vMX as Contrail Gateway for IPv6 Overlay

Add yours

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

Up ↑

%d bloggers like this: