Our support

Will always help you

Uplink Subscriber Policy Engine Documentation (v.16.2)

Overview

Uplink Subscriber Policy Engine consists of kernel module (called ebt_USPE) and user-space management tool (uspe-client). Also, Uplink SPE supports pluggable modules - both kernel-space and user-space. You can find the right package for your system in Downloads section

Main concept of Uplink SPE is that it considers traffic as flowing from or towards customer. This means that network configuration of the host needs to have customer facing interface(s) and internet facing  interface(s). Typical network setup is shown on following diagram.

------Cust----------[ Host ] --------- Internet----------

However, your setup may have multiple Customer- and/or Internet- connections, even more - in case of VPN aggregation - interfaces may not yet exist in your system when module is being loaded. For these scenarios you can pass a list of customer and internet interfaces to module. These options would be discussed in Startup section in more detail.

 

Requirements

Uplink SPE kernel module is absolutely autonomous and doesn't require any other dependencies on the host. The only requirement is :

a modern 64-bit Linux kernel (3.9.0 or newer)

Even uspe-client can reside on remote server, sending commands over the network.

Uplink SPE doesn't depend on Iptables/ebtables either. We are using same system hooks as xtables subsystem and it can be used together with Uplink SPE, if you need it. In Startup section you will find options to run Uplink SPE checks before (default) or after xtables. 

 

Installation

Uplink SPE installation is very straight-forward : 

  1. Determine your Linux distribution and kernel version. Make sure your system is 64-bit.

    # uname -r

    3.13.0-36-generic

  2. Download a package for your system and kernel version from Downloads section

    # wget <package-URL>

  3. Extract the archive

    # tar -xf <package-file>.tar.gz

  4. Copy kernel to your modules directory and run depmod -a

    # mkdir -p /lib/modules/`uname -r`/extra/

    # cp ebt_USPE.ko /lib/modules/`uname -r`/extra/

    # depmod -a

  5. Copy uspe-client to a /usr/local/sbin/ (or any other location of your choice)
  6. Have fun!

System information

Uplink SPE provides quite detailed information about it's health. Most of important system counters can be found using system info command. Command and example output is shown below 

# uspe-client system info [exact]

System info: Uptime: 4 days 5:1, 8 CPUs/8 Cores, 2426MB RAM/3713MB Free, Load 2202/2159/2246, 0 processes
CPU cores stats, %:
CPU    USER NICE SYS  SIRQ HIRQ IDLE IOWT STEA GUES GUESN
CPU0:    0    0    0    0    0   95    4    0    0    0
CPU1:    0    0    0    2    0   97    0    0    0    0
CPU2:    0    0    0    8    0   91    0    0    0    0
CPU3:    0    0    0    4    0   95    0    0    0    0
CPU4:    0    0    0    4    0   95    0    0    0    0
CPU5:    0    0    0    1    0   98    0    0    0    0
CPU6:    0    0    0    1    0   98    0    0    0    0
CPU7:    0    0    0    4    0   95    0    0    0    0

Unmatched packets policy: ACCEPT

Generic counters:
packets_handled      124G
bytes_handled        114T
packet_1m_rate       1M
bytes_1m_rate        981M
packet_1m_rate_peak  1.2M
bytes_1m_rate_peak   1.1G
err_no_mem           0
num_kalloc           539K
num_vmalloc          1
bytes_kalloc         114M
bytes_vmalloc        512K
acl_lookups          108G
acl6_lookups         0
hostlist_lookups     0
session_lookups      120G
session6_lookups     0
map_lookups          13.3M
map6_lookups         0
qinqmap_lookups      0
map_items            49.4K
map6_items           0
acl_items            6.2K
acl6_items           0
hostlist_items       0
qinqmap_items        0
acl_queue            0
acl6_queue           0
hostlist_queue       0
num_profiles         4
num_policers         66
profile_items        39
policer_items        198
profile_match_items  0
policer_match_items  0
msngr_peers          0
msngr_socks          5
msngr_rx_queue       0
msngr_tx_queue       0
msngr_rx_packets     56.7K
msngr_tx_packets     336K
msngr_rx_errors      0
msngr_tx_errors      0
msngr_active_packets 14
snat4_packets_up     0
snat4_packets_down   0
err_snat4_noflows    0
snat4_pools          0
snat4_slots          0
snat4_flows          0
snat4_public_ips     0
snat4_private_ips    0
current_sessl2_cnt   0
approved_sessl2_cnt  0
slave_sessl2_cnt     0
current_sess_cnt     37.6K
approved_sess_cnt    38.2K
slave_sess_cnt       0
current_sess6_cnt    0
approved_sess6_cnt   0
slave_sess6_cnt      0
qinq_interfaces      0
qinq_subinterfaces   0
qinq_routes          0
qinq_errors          0
dhcp_drops           0
dhcp_drops_in        0
dhcp_pools           0
dhcp_exceptions      0
dhcp_leases          0
dhcp_leases_assigned 0
dhcp_relay_rx        0
dhcp_relay_tx        0
dhcp_server_rx       0
dhcp_server_tx       0
dhcp_radius_rx       0
dhcp_radius_tx       0
qinq_arp_drops       0
qinq_arp_queue       0
plugins_loaded       2

This command provides quite long listing of variables, however, all of them are very useful to monitor system status and performance.

Note: the parameter exact is used to show exact numbers. Omitting it will show more friendlier numbers, however won't be that useful for using in scripts.

Top few lines are showing generic system information like memory, CPUs, uptime and per-CPU usage, which might be useful to have on screen together with Uplink SPE system counters. Following is a description of counters.

Counter Description

packets_handled

Number of packets passed this system, counted when packet entered the system

bytes_handled

Number of bytes passed this system, counted when packet entered the system

packet_1m_rate

1 minute average packet rate. Calculated once 1 minute based on packets_handled value

bytes_1m_rate

1 minute average byte rate. Calculated once 1 minute based on bytes_handled value

packet_1m_rate_peak

maximum 1 min average packet rate system has experienced system started

bytes_1m_rate_peak

maximum 1 min byte rate system has experienced since system started

err_no_mem

Shows how many memory allocations failed

num_kalloc

Shows how many times kernel memory was allocated by SPE

num_vmalloc

Shows how many times virtual memory was allocated by SPE

bytes_kalloc

Shows how may bytes of kernel memory are currently allocated by SPE

bytes_vmalloc

Shows how may bytes of virtual memory are currently allocated by SPE

acl_lookups

Shows how many times ACL subsystem was checking IP against Access-lists

acl6_lookups

Shows how many times IPv6 ACL subsystem was checking IPv6 against Access-lists

hostlist_lookups

Shows how many times hostname ACL subsystem was checking against Access-lists

session_lookups

Shows how many times SPE was making a lookup of a IP/Q-in-Q session

session6_lookups

Shows how many times SPE was making a lookup of IPv6 session

map_lookups

Shows how many times static map was searched to lookup settings for new IPv4/Q-in-Q sessions

map6_lookups

Shows how many times static map was searched to lookup settings for new IPv6 sessions

acl_items

Shows how many items are in IPv4 ACL system

acl6_items

Shows how many items are in IPv6 ACL system

hostlist_items

Shows how many items are in hostname ACL system

map_items

Number of items in L2/IPv4 session settings maps

map6_items

Number of items in IPv6 session settings maps

qinqmap_items

Number of items in QinQ sub-interface settings maps

acl_queue

Number of requests pending to be added to the IPv4 ACL tree

acl6_queue

Number of requests pending to be added to the IPv6 ACL tree

hostlist_queue

Number of requests pending to be added to the hostname ACL tree

num_profiles

Shows how many profiles are active in the system

num_policers

Shows how many policers are active in the system

profile_items

Shows how many profile items are in total in all active profiles 

policer_items

Shows how many policer items are in total in all active policers

profile_match_items

Shows how many match rules are in total in all active profiles 

policer_match_items

Shows how many match rules are in total in all active policers 

msngr_peers

Shows number of active remote connections to Uplink SPE sockets (management, AAA)

msngr_rx_queue

Number of packets in incoming packet queue awaiting to be handled (total for all Uplink SPE sockets)

msngr_tx_queue

Number of packets in outgoing packet queue awaiting to be sent (total for all Uplink SPE sockets)

msngr_rx_packets

total number of packets received for all Uplink SPE sockets

msngr_tx_packets

total number of packets sent for all Uplink SPE sockets

msngr_rx_errors

shows number of errors detected while receiving packets

msngr_tx_errors

shows number of errors detected while sending packets

snat4_packets_up

Shows number of packets translated to outside by Carrier Grade SNAT for IPv4 module

snat4_packets_down

Shows number of packets translated to inside by Carrier Grade SNAT for IPv4 module

err_snat4_noflows

Shows how many times a packet was dropped because there was no more free flows for a session

snat4_pools

Shows number of active Carrier Grade SNAT pools in the system

snat4_slots

Shows number of active slots used by Carrier Grade SNAT. Effectively, it's a number of ip/port blocks allocated to active sessions. Currently, only one block per session is supported

snat4_flows

Number of active SNAT IP/Port mappings in the system

snat4_public_ips

Number of IPs used as public IP by Carrier Grade SNAT module

snat4_private_ips

A number of IPs allowed to be translated by Carrier Grade SNAT module

current_sessl2_cnt

Number of active Q-in-Q sessions

approved_sessl2_cnt

Number of Q-in-Q sessions authorised (statically or dynamically)

slave_sessl2_cnt

Number of Slave Q-in-Q sessions

current_sess_cnt

Number of IPv4 sessions authorised (statically or dynamically)

approved_sess_cnt

Number of IPv4 sessions authorised (statically or dynamically)

slave_sess_cnt

Number of Slave IPv4 sessions

current_sess6_cnt

Number of IPv6 sessions authorised (statically or dynamically)

approved_sess6_cnt

Number of IPv6 sessions authorised (statically or dynamically)

slave_sess6_cnt

Number of Slave IPv6 sessions

qinq_interfaces

Number of interfaces on which QinQ termination services are active

qinq_subinterfaces

Total number of QinQ sub-interfaces running on all qinq interfaces

qinq_routes

Number of routes active on QinQ sub-interfaces

qinq_errors

Number errors during QinQ de-encapsulation

dhcp_drops

number of DHCP packets dropped on QinQ interfaces, as failed in processing

dhcp_drops_in

number of DHCP packet dropped at input

dhcp_pools

number of active DHCP pools in the system

dhcp_exceptions

total number of total dhcp IPs in exceptions

dhcp_leases

total number of DHCP leases available

dhcp_leases_assigned

total number of active DHCP leases

dhcp_relay_rx

number of incoming packets for DHCP-relay

dhcp_relay_tx

number of packets sent by DHCP-relay

dhcp_server_rx

number of incoming DHCP packets received by DHCP server

dhcp_server_tx

number of DHCP packets sent by DHCP server

dhcp_radius_rx

number of incoming DHCP packets received by DHCP-Radius relay

dhcp_radius_tx

number of DHCP packets sent by DHCP-Radius relay

qinq_arp_drops

ARP packets dropped on QinQ sub-interfaces

qinq_arp_queue

Number of ARP packets in queue waiting for reply

plugins_loaded

number of plugins loaded

 

Memory information

Uplink SPE provides information about it's memory and timers usage. 

# uspe-client system memory

Module  kmem_alloc kmem_free  kmem_diff  kmem_bytes tim_cre  tim_del tim_act   
ipv4pool      2       0          2       32768      0     0        0
dhcp_process  0       0          0          0       1     0        3415
arp_process   0       0          0          0       0     0        0
DPI           0       0          0          0       0     0        0
plugin        2       0          2          208     0     0        0
ipv6pool      2       0          2       26624      0     0        0
ipv6nd        0       0          0          0       0     0        0
qinqmap       0       0          0          0       0     0        0
ipunnumb      0       0          0          0       0     0        0
snat4_proto   5       0          5       2176       0     0        0
snat4_pool    2       0          2       34816      0     0        0
snat4         0       0          0          0       0     0        0
aaa           0       0          0          0       0     0        0
system      20582   20580        2       1416       3     0      20625
messenger   519927  519902      25      156264      0     0        0
topology      7       0          7        848       0     0        0
packet_loop   0       0          0          0       0     0        0
http_redirect 2       1          1          512     0     0        0
predefined6   0       0          0          0       0     0        0
predefined  101522   50609     50913   3665736      0     0        0
policer      202      1         201     186048      1     0      13659
profile_action 0      0          0          0       0     0        0
profile_match  0      0          0          0       0     0        0
profile       45      0          45      61424      2     0      17074
hostlist      1       0          1        2640      1     0        0
netlist6      1       0          1        2640      2     0        0
netlist     6492      148        6344    256360     3     0        0
l2session     0       0          0          0       0     0        0
session6      0       0          0          0       0     0        0
session  27891467   27383122   508345   95953552   70     0     347088
hashroot    478       0          478    20099008    0     0        0
log           1       0          1       262144     1     0        0
----------------------------------
Totals    28540740   27974363   566377   120745184  84    0      401861 

 

 

Startup

To start Uplink SPE, you need to load ebt_USPE   kernel module. However, some module options might be required - they are discussed later in this section. In general it's enough to load module like this : 

# modprobe ebt_USPE 

Loading module without any parameters, will assume following : 

  • Demo licence activated ( licence_key and customer_id are empty)
  • Maximum number of profiles is 128 (max_profiles=128)
  • Module processes packets before xtables (before_iptables=1)
  • Management port is 44044 (mgm_port=44044)

Note: DEMO version runs only 24 hours - after this period it will allow all packets without any processing.

By default, USPE supports up to 128 profiles and 128 policers. If you need more - just increase max_profiles module parameter. 

Uplink SPE is managed over TCP port specified as mgm_port module parameter. By default, it's set to 44044.

Once you have obtained licence, here is how you can enable it - you need to add to the module arguments with licence information provided  licence_key=<licence_key> customer_id=<customer_id>

Using before_iptables=0 parameter you can set USPE to process packets after xtables, as by default it does packet processing before xtables.

Saving and restoring configuration

To simplify management of Uplink SPE, you can use dump and restore commands of uspe-client. Their action is very similar to iptables-save and iptables-restore in iptables world. For example, if you already have running configuration, you can save it to a file using following command : 

# uspe-client dump > config.uspe

When you want to restore your configuration (ex. right after USPE module was insmod'ed), you can use following command (assuming your configuration is in the config.uspe file in current directory) : 

# uspe-client restore < config.uspe

Configuration file format is very simple. Every line is formed of arguments you have passed to uspe-client when configuration was created. For example, let's assume we have issued following commands : 

# uspe-client profile create 1 name MyProfile1

# uspe-client profile add 1  action ACCEPT

# uspe-client profile add 1 match upload

# uspe-client profile add 1  action ACCEPT

# uspe-client profile add 1 match download

With this configuration, uspe-client dump will produce following output : 

# System
# sys dump
system unmatched-action accept
# ACL
# ACL6
# Host ACL
# Profiles
profile create 1 name MyProfile1
profile add 1 action ACCEPT
profile add 1 match upload
profile add 1 action ACCEPT
profile add 1 match download
profile commit
# Policers
policer commit
# Map
# HTTP-redirects
# AAA
# Topology
topology vfilter 0
# NAT4
nat commit
# Done

Note: every group of commands has a dump subcommand, to dump only that group's configuration. It might be useful, if you are working on new configuration script and want to dump only specific areas of configuration. 

Management settings

Uplink SPE kernel module is managed over TCP socket. You can change IP/Port it listens on and also, put an ACL list on that socket, if you want to control your BRAS remotely.

Note : By default, Uplink SPE module binds to TCP 127.0.0.1:44044, for security reasons. 

To change listen IP/port for management interface of Uplink SPE kernel module, you can use two methods. If default listen port is already taken by some other application, you can use a kernel module parameter mgm_port to listen on another port. Note that on module load you are unable to change listen address - it will always bind to 127.0.0.1.

Here is an example how to do that (as example, we make it to listen on port 44044) : 

# modprobe ebt_USPE mgm_port=44044

Uplink SPE allows to add additional TCP socket to listen for management commands. You can do dat using following command : 

# uspe-client aaa mgm <IP>:<port>

Note:  additional management socket is pre-configured with an ACL, allowing access only from 127.0.0.0/8 network

You need to add more prefixes to allow remote access.

To setup Management ACL, you need to setup an additional management socket first, and then you would be able to add prefixes to an ACL. Management ACL commands are following : 

# uspe-client aaa mgm-acl add <ipv4_prefix>

# uspe-client aaa mgm-acl list

# uspe-client aaa mgm-acl flush

Note : if you will empty management ACL, it will be unprotected and anyone would be able to connect to it. Don't flush Management ACL, unless you do know what you are doing.  

 

Topology

Uplink SPE supports Bridged (transparent), routed and mixed topology environments. Moreover, IP/IPv6 and Q-in-Q based sessions are supported at the same time. The key requirement for Uplink SPE to handle packets is to set which interfaces are Subscribers-facing and which are Internet facing. While setting which role interface has, you need specify also if this interface is Routed or Bridged and if you want Q-in-Q sessions on it. 

Note: Q-in-Q sessions are supported on Transparent (Bridged) interfaces, as on Routed interfaces are untagged by their nature. 

Note: When setting up topology, both subscriber and internet interfaces need to be set. Also, they must share environment (Routed or transparent) and Sessions type (IP-based or Q-in-Q based). Otherwise, you won't be able to handle traffic properly.

Routed environment

Syntax for setting up a routed interfaces is following : 

# uspe-client topology interface l3 internet <interface_name> ip

This command is used to setup Internet-facing Routed interface. Note last parameter - it will always be ip, as q-in-q sessions are not possible on routed interfaces.

# uspe-client topology interface l3 subscriber <interface_name> ip

This command is used to setup Subscriber-facing Routed interface.  Note last parameter - it will always be ip, as q-in-q sessions are not possible on routed interfaces.

Transparent (Bridged) environment

Syntax for setting up a Transparent (Bridged) interfaces is very similar to setting routed environment. 

# uspe-client topology interface l2 internet <interface_name> ip

This command is used to setup Internet-facing Transparent interface. Note last parameter - it's set to ip, which means that IP/IPv6 based sessions are going to be formed on this interface.

# uspe-client topology interface l2 subscriber <interface_name> ip

This command is used to setup Subscriber-facing Transparent interface.  Note last parameter - it's set to ip, which means that IP/IPv6 based sessions are going to be formed on this interface.

Q-in-Q sessions in Transparent (Bridged) environment

Syntax for setting up a Transparent (Bridged) interfaces with Q-in-Q sessions is following : 

# uspe-client topology interface l2 internet <interface_name> qinq

This command is used to setup Internet-facing Transparent interface. Note last parameter - it's set to qinq, which means that sessions on this interface are going to be formed based on Q-in-Q tag.

# uspe-client topology interface l2 subscriber <interface_name> qinq

This command is used to setup Subscriber-facing Transparent interface.  Note last parameter - it's set to qinq, which means that sessions on this interface are going to be formed based on Q-in-Q tag.

VLAN filter for Transparent environments

When used int Transparent environment, it might be useful to enable packet processing only on specific Vlans (or S-VLANs on Q-in-Q links). By default, all VLANs are processed in Transparent environment. 

This feature enables customer traffic will be processed, while service traffic will be forwarded without any processing. Following example will show how it can be done. 

Let's assume we have Transparent environment, where eth1 is facing Internet and eth2 is facing Subscribers and subscribers are distinguished by their IP. Customers are residing in Q-in-Q VLANS 10.* and 20.*, while other VLANs are used for service traffic. This scenario can be configured like this : 

# uspe-client topology interface l2 subscriber eth2 ip

# uspe-client topology interface l2 ineternet eth1 ip

# uspe-client topology vfilter 10,20

 

Sessions

A session is a base entity, which can have it's own policy. Uplink SPE supports three types of sessions : IPv4 based, IPv6-based and Q-in-Q based.  Most commonly, it's referenced as "Customer" and distinguished by unique IPv4 address (if it's a IPv4 session), IPv6 /64 prefix (if it's an IPv6 sessions) or Q-in-Q tag (if it's a Q-in-Q session). 

Note : A session has unique descriptor - IPv4, IPv6 /64 prefix or Q-in-Q tag. You aren't able to have multiple unique descriptors in one session. However, it's possible to setup a Slave Session (which will be discussed later on)

IPv4 and IPv6 sessions are supported in both Routed and Transparent environments, but Q-in-Q sessions are available only in Transparent environments. As Uplink SPE is able to work in mixed environment, all three types of sessions can be present in the system at the same time. 

Session management shares command interface and features, so it's very similar to manage all types of sessions. You can change most session parameters on the fly, so customers won't require re-authentication when it's not desired.  

Here is a list of actions that can be done with sessions (regardless of it's type) : 

  • List session(s)
  • Create session
  • Update session
  • Examine session in details
  • Clear session(s)
  • Get sessions count

Syntax for these actions is described later in this section.

 

Master and Slave Sessions

As customer may have random number of IPv4/IPv6 or QinQ resources requiring single policy, Uplink SPE implements Slave session concept, where profile and policer is attached to a single resource session (master), while others (slaves) share that setting and counters. This might be useful, when customer has a one common rate-limit for all of the resources (sessions).

Currently, only IPv4 sessions can be Master session. All of IPv4/IPv6 or Q-in-Q sessions can be slave Sessions.

You don't need to specify additionally, if session is master. However, slave sessions need to have reference to their master session.

Session can be set as slave in three ways : 

  • On session create command
  • On Session update command
  • Radius CoA request trough Radius proxy (will be discussed in Radius section of this manual)

Session control commands

All session types share common commands syntax. All the session types are also having similar output and common session state flags. 

Here are the flags and their descriptions. 

  • A - Approved (Session was created manually or approved from Radius)
  • U - Up, Online (Running session)
  • C - Counters Enabled (Byte/packets counters, profile/policer action counters enabled)
  • S - Slave (Slave session)
  • M - MAC filter present (Only these macs would be allowed on session and it's slave sessions)
  • 2 - QinQ session
  • 4 - IPv4 based session
  • 6 - IPv6 based session
  • I - Interface based session
  • N - SNAT4 active
  • n - SNAT4 static ports present
  • Q - Has quota active

Listing commands

Session listing is likely the most common command used on BRAS'es. Syntax for all of the session types is common and is following : 

# uspe-client < session | l2session | session6 >  < all | session_key | slave >

Output of these commands are also similar, check the list of output columns and their meaning.

IP / Vlan / IPv6 - it's a key for a session

Age - Session duration in seconds

Policy - Profile and policer ids

IN pkts/Drop - Number of packets passed/dropped by session as Download traffic

IN bytes/Drop - Number of bytes passed/dropeed by session as Download traffic

OUT pkts/Drop - Number of packets passed/dropped by session as Upload traffic

OUT bytes/DropNumber of bytes passed/dropeed by session as Upload traffic

Flags - Session state flags. Discussed earlier in this section.

Example commands and their output are following. 

IPv4 sessions

# uspe-client session show all

IP       Age   Policy   IN pkts/Drop IN bytes/Drop OUT pkts/Drop    OUT Bytes/Drop   Flags 
10.1.2.3     2       1/2          0/0              0/0              0/0              0/0       AUC---4-----
10.4.3.4     2       1/2          0/0              0/0              0/0              0/0       AUCS--4-----

# uspe-client session show 10.1.2.3

IP              Age   Policy   IN pkts/Drop     IN bytes/Drop    OUT pkts/Drop    OUT Bytes/Drop   Flags 
10.1.2.3     8       1/2          0/0              0/0              0/0              0/0       AUC---4-----

# uspe-client session show slave

IP              Age   Policy   IN pkts/Drop     IN bytes/Drop    OUT pkts/Drop    OUT Bytes/Drop   Flags 
10.4.3.4     5       1/2          0/0              0/0              0/0              0/0       AUCS--4-----

 

L2 sessions

# uspe-client l2session show all

VLAN      Age   Policy   IN pkts/Drop     IN pkts/Drop     OUT bytes/Drop   OUT Bytes/Drop   Flags 
 123.345  26      1/2          0/0              0/0              0/0              0/0       AUC--2------
 127.7    26      1/2          0/0              0/0              0/0              0/0       AUCS-2------

# uspe-client l2session show 123.345

VLAN      Age   Policy   IN pkts/Drop     IN pkts/Drop     OUT bytes/Drop   OUT Bytes/Drop   Flags 
 123.345    31      1/2          0/0              0/0              0/0              0/0       AUC--2------

# uspe-client l2session show slave

VLAN      Age   Policy   IN pkts/Drop     IN pkts/Drop     OUT bytes/Drop   OUT Bytes/Drop   Flags 
 127.7    9       1/2          0/0              0/0              0/0              0/0       AUCS-2------

 

IPv6 sessions

# uspe-client session6 show all

IP6                      Age   Policy   IN pkts/Drop     IN bytes/Drop    OUT pkts/Drop    OUT Bytes/Drop   Flags 
f0f0:abcd:aaab:1232::/64  9       3/4          0/0              0/0              0/0              0/0       AUC----6----
f0f0:abcd:aaab:1233::/64  9       1/4          0/0              0/0              0/0              0/0       AUC----6----

# uspe-client session6 show f0f0:abcd:aaab:1233::

IP6                          Age   Policy   IN pkts/Drop     IN bytes/Drop    OUT pkts/Drop    OUT Bytes/Drop   Flags 
f0f0:abcd:aaab:1233::/64     19      1/4          0/0              0/0              0/0              0/0       AUC----6----

 

Session details 

Session details can be obtained using following command :

# uspe-client <session | l2session | session6 > <session_key>

Session key is an IPv4 address for IPv4 sessions, IPv6 address for IPv6 sessions and Q-in-Q tag for Q-in-Q sessions. 

Note, IPv6 sessions are formed for /64 IPv6 address, so any address in desired /64 would refer to the same IPv6 session. 

L2 session

# uspe-client l2session detail 127.7

IP        Age   Policy   IN pkts/Drop     IN pkts/Drop     OUT bytes/Drop   OUT Bytes/Drop  
 127.7    16      1/2          0/0              0/0              0/0              0/0      
Profile 1
Policer : 2
Quota (In/Out) : 0/0, Quota profile/policer : 0/0
Master IP : 10.3.4.5

 

IPv4 session

# uspe-client session detail 10.2.3.2

IP              Age   Policy   IN pkts/Drop     IN bytes/Drop    OUT pkts/Drop    OUT Bytes/Drop   Flags         
10.2.3.2     22      1/2          0/0              0/0              0/0              0/0       AUCS--4-----  
VLAN tags 0.0
Quota (In/Out) : 0/0, Quota profile/policer : 0/0
Master IP : 10.3.4.5

 

IPv6 session

# uspe-client session6 detail f0f0:abcd:aaab:1235::

IP                           Age   Policy   IN pkts/Drop     IN pkts/Drop     OUT bytes/Drop   OUT Bytes/Drop  
f0f0:abcd:aaab:1235::/64     23      2/4          0/0              0/0              0/0              0/0      
Profile 2
Policer : 4
VLAN tags 0.0
Quota (In/Out) : 0/0, Quota profile/policer : 0/0
Master IP : 10.23.34.45

 

Clearing session

Clearing session will close session and free all resources attached to it. Syntax is the same for all of session types : 

# uspe-client session clear <IPv4> | all

# uspe-client l2session clear <QinQ vlan> | all

# uspe-client session6 clear <IPv6> | all

 

Session totals

To see how many sessions your system currently has you can use count command for all types of sessions. Here is the format for these commands : 

# uspe-client l2session count

VLAN Sessions (Total/Approved/Slave) :    0/0/0

# uspe-client session count

IPv4 Sessions (Total/Approved/Slave) :    0/0/0

# uspe-client session6 count

IPv6 Sessions (Total/Approved/Slave) :    0/0/0

Session update/create

Session  create command is used to create session manually and update is used to update already existing session. The actual difference between these two commands is that create will fail if session exists, while update will fail if it doesn't. These commands don't produce any output on success, while it will show an error message in case of failure. 

Create session commands are following for all session types : 

# uspe-client session create <IPv4> <profile_id> <policer_id> [<master_IPv4>] [<MAC>,...] [<NatStaticPort>,...]

# uspe-client session6 create <IPv6> <profile_id> <policer_id> [<master_IPv4>] [<MAC>,...]

# uspe-client l2session create <QinQ vlan> <profile_id> <policer_id> [<master_IPv4>] [<MAC>,...]

Update session commands are following for all session types : 

# uspe-client session update <IPv4> <profile_id> <policer_id> [<master_IPv4>] [<MAC>,...] [<NatStaticPort>,...]

# uspe-client session6 update <IPv6> <profile_id> <policer_id> [<master_IPv4>] [<MAC>,...]

# uspe-client l2session update <QinQ vlan> <profile_id> <policer_id> [<master_IPv4>] [<MAC>,...]

IPv4 sessions support setting Static SNAT4 ports, so it's possible to make static ports binding 

Note, that Maximum number of supported MAC-addresses in filter is 10

Note, that Maximum number of supported static SNAT4 port bindings is 10

Profiles and Policers

Profile and policy is a key concept in Uplink SPE. Both profile and policer can be considered as a micro-firewall or personalised firewall for group of  individual sessions. To achieve best performance and flexibility, we aren't using single plain policy to do everything in one firewall rules set, as it's traditionally used by many vendors. We use profile as a firewall rule set and policer as rate-limiting rule set. Mixing various profiles and policiers you can achieve simplicity and performance of your traffic polices for customers.

Profile and policer share rules format and structure and differ only in following :

Profile rules are checked first, and if none of them matched with action ACCEPT or DROP, it will jump to a Policer.

Policer rules are checked after Profile, and can contain actual raite-limiters set. 

Profile doesn't allow to rate-limit traffic. It's RATELIMIT action will stop Profile rules processing and jump to a Policer rules.

Note: Uplink SPE maximum number of Profiles and policers is set on module load as an argument and can't be changed while running. You will need to rmmod and insmod Uplink SPE module to increase default value of 128 using max_profiles parameter. This parameter sets maximum number of Profiles and Policers you can use

Creating your Profiles or Policers

Both Profiles and Policers consist of policy items, which can be considered as a single Iptables rule. Similarly to iptables rule, Policy item can have only one Action and zero or more matchers (match rules). However, Uplink SPE implements match rule order, which can influence performance in multi-gigiabit installations. In Uplink SPE match rules you are in control of the order that match rules are checked. Here is an example of profile and policer :

Profile: Policer:

profile create 1 name MyProfile

profile add 1  action ACCEPT

profile add 1 match upload

profile add 1 match ip remote 8.8.8.8

profile add 1  action ACCEPT

profile add 1 match download

profile add 1 match ip remote 8.8.8.8

profile commit 1

policer create 1

policer add 1  action ACCEPT

policer add 1 match upload

policer add 1 match ip remote 8.8.8.8

policer add 1  action ACCEPT

policer add 1 match download

policer add 1 match ip remote 8.8.8.8

policer commit 1

As you may noticed, these profile and policer  allow upload and download to/from remote IP 8.8.8.8

Also, same effect as these rules would be achieved using just one policy item with single match rule - "match ip remote 8.8.8.8". As you can see, following example is way much shorter, but does absolutely the same, but faster in terms of CPU cycles : 

profile create 1

profile add 1  action ACCEPT

profile add 1 match ip remote 8.8.8.8

profile commit 1

 

Profiles and policers are not activated while you configure them. You need to commit your changes to activate them. 

Note: You can commit individual profile or policer, or all policers / all profiles.  

 

Profile and policer management

Both profile and policer have absolutely the same commands and their arguments. In this section we will show examples of managing Profile, but absolutely the same can be done with Policier. 

Profile create command will initialise profile in temporary storage. If profile is already created, then it will be wiped and re-created again. 

# uspe-client profile create <id> [name <profile_name>]

Once configured, profile can be committed and enabled. It can be committed as a single profile, or to commit all configured profiles. 

# uspe-client profile commit [<id>]

Note: profile commit (without profile ID argument) will overwrite any current profile and destroy them if they were not created in temporary storage. 

If you want to recreate all profiles, it's recommended to clear temporary storage before creating new profiles. This way you will make sure that any other uncommitted changes would be cleared. Also, if new configuration doesn't have any current profile, it would be deleted on commit. You can clear temporary storage with following command :

# uspe-client profile clear

Policy items and their match rules are added using following command : 

# uspe-client profile add <profile_id> <match|action> <Match_or_Action body>

Note:  

  • match rules are attached to the last policy item (action) in selected profile (with <profile_id> field)
  • You can have up to 8 match rules in every policy item (action)
  • You can have any number of profile/policer items

Following example will show how to create two profiles and apply them in one go :

# uspe-client profile clear

# uspe-client profile create 1 name MyProfile1

# uspe-client profile add 1  action ACCEPT

# uspe-client profile add 1 match upload

# uspe-client profile add 1  action ACCEPT

# uspe-client profile add 1 match download

# uspe-client profile create 2 name MyProfile2

# uspe-client profile add 2  action DROP

# uspe-client profile add 2 match upload

# uspe-client profile add 2  action DROP

# uspe-client profile add 2 match download

# uspe-client profile commit

 

Viewing profiles/policers

To view list of current profiles you can use following command : 

# uspe-client profile list

Active profiles : 
1: MyProfile1 (2 items)

Similarly,  list of policers can be obtained with similar command : 

# uspe-client policer list

Active policers : 
1: Policer1 (2 items)

 

You can also use list profiles/policers with their counters using following command : 

# uspe-client profile counters

#   Profiles                  Packets IN            Bytes IN         Packets OUT           Bytes OUT
1 : MyProfile1                  0/0                  0/0                  0/0                  0/0      

# uspe-client policer counters

#  Policers                  Packets IN            Bytes IN         Packets OUT           Bytes OUT
1 : Policer1                    0/0                  0/0                  0/0                  0/0      

 

If you would add profile (policer) id to the counters command, it will show counters for profile (policer) items counters. 

# uspe-client profile counters 1

Profile item counters in profile #1:
#           Packets IN            Bytes IN         Packets OUT           Bytes OUT     Action
0 :            0/0                  0/0                  0/0                  0/0       ACCEPT

Similarly, for policers you can also see their counters.

# uspe-client policer counters 1

Policer item counters in policer #1:
#           Packets IN            Bytes IN         Packets OUT           Bytes OUT     Action
0 :            0/0                  0/0                  0/0                  0/0       ACCEPT
1 :            0/0                  0/0                  0/0                  0/0       ACCEPT

Another way to view generic profile/policer items you can use show command : 

# uspe-client profile show 1

Active profiles : 
Profile 1: MyProfile1
 Item #0 : Action ACCEPT
  match #0: upload

 

# uspe-client policer show 1

Active policers : 
Policer 1: Policer1
 Item #0 : Action ACCEPT
 Item #1 : Action ACCEPT

 

Profile/Policer items (actions)

Profile/Policer items (or actions) are the building blocks of Profile/Policer. It's very alike single iptables rule. As Uplink SPE follows a micro-firewall concept, there are no "jumps" to other "chains" or profiles/policiers. Whole Profile (or Policier) is a finished policy. 

Note: Uplink SPE supports plugins, so it's very easy to extend it's actions

Command format for adding new profile item is following : 

# uspe-client profile add <profile_id> action <action_body>

Command format for adding new policer item is similar : 

# uspe-client policer add <policer_id> action <action_body>

Following is a list of actions supported by Uplink SPE by both Profiles and Actions (added as <action_body parameter>) : 

  • NONE - does nothing. Might be useful to count packets matching some criteria
  • DONE - Finishes profile/policier processing. For profile it means to jump to policier, to policier - to Drop or Accept packet depending on unmatched-action setting (defaults to Accept).
  • ACCEPT - Accept packet and stop any other processing 
  • DROP - Drop packet and stop any other processing
  • FWMARK <mark> - Mark packet with a fwmark. If packet is already marked, value would be OR'ed with it.
  • LIST_ADD laddr|raddr <id> [<timeout>] -  Add Local (laddr) or Remote (raddr) IPv4 address to an IPv4 ACL, which ID is <id> with optional timeout.
  • LIST_DEL laddr|raddr <id> - Remove Local (laddr) or Remote (raddr) IPv4 address from an IPv4 ACL, which ID is <id>
  • LIST6_ADD laddr|raddr <id> [<timeout>] - Add Local (laddr) or Remote (raddr) IPv6 address to an IPv6 ACL, which ID is <id> with optional timeout.
  • LIST6_DEL laddr|raddr <id>  - Remove Local (laddr) or Remote (raddr) IPv6 address from an IPv6 ACL, which ID is <id>
  • HOSTLIST_ADD <id> - Add a hostname obtained from packet (Needs to have a DPI match like dns or http in current profile/policer item or earlier in current profile/policer) to a hostnames ACL which ID is <id>
  • HOSTLIST_DEL <id> - Delete a hostname obtained from packet (Needs to have a DPI match like dns or http in current profile/policer item or earlier in current profile/policer) from hostnames ACL which ID is <id>
  • TPROXY <local_ip> <local_port> <mark> - redirect TCP stream to a locally running transparent proxy (like SQUID) listening on <local_ip>:<local_port> and mark handled packets with <mark>. Profile needs to ACCEPT packets with this mark earlier in profie/policer.
  • SNAT4 <pool_id> - Process IPv4 Source NAT using preconfigured SNAT pool <pool_id>

 

Following is a list of actions available only in Policier : 

  • POLICE upload <rate>[/<burst>] download <rate>[/<burst>] - Per-session rate-limit. <rate> and <burst> are in Kbit/s
  • POLICE_TOTAL upload <rate>[/<burst>] download <rate>[/<burst>] - Aggregate (global) rate-limit. <rate> and <burst> are in Kbit/s

 

 

Profile/Policer match rules

Match rules are the actual checks for packets. All match rules in profile/policer action need to match for action to be executed. 

Command to add a match rule to a profile is following (note, that profile and profile item needs to be created) : 

# uspe-client profile add <profile_id> match <match_rule>

similarly, adding a match rule to a policer is following : 

# uspe-client policer add <policer_id> match <match_rule>

Note: you can have up to 8 matchers per profile/policer item

You can add your own packet match rules, as Uplink SPE supports match plugins (see Extending Uplink SPE section).

Following matchers are supported : 

  • proto <num>|<from>-<to>|<hex-num>/<hex-mask>  -  protocol match. See /etc/protocols for protocol numbers
  • port loc|remote <num>|<from>-<to>|<hex-num>/<hex-mask>  - TCP/UDP port match
  • ip loc|remote <ip>|<cidr> - match IPv4 address or prefix. Use loc to match Local address and remote to match Remote address. 
  • ip loc|remote list <list-num>  - match Local (loc) or Remote (remote) IPv4 address(or prefix) if it matches IPv4 Access list with ID <lit-num> 
  • ip6 loc|remote <ip>|<cidr>  - match IPv6 address or prefix. Use loc to match Local address and remote to match Remote address. 
  • ip6 loc|remote list <list-num> - match Local (loc) or Remote (remote) IPv6 address(or prefix) if it matches IPv6 Access list with ID <lit-num> 
  • pktlen <exact size>|<from>-<to> - match packet length. you can match exact length or allowed boundaries.
  • ifname sub|int <name>|<name-part>*  - match subscribers (sub) or internet interface name of a packet. Use * to match multiple interfaces, which names begins <name-part>
  • tcpflags <flags>|<hex-flags>/<hex-mask> - match TCP packet flags. (you can also use comma-separated list of flags : FIN, SYN, RST, PSH, ACK, URG, ECN, CWR)
  • upload|download  - match upload or download traffic. 
  • vlan <id>|<hex-vlan>/<hex-mask>  - match Vlan, or S-vlan in case of Q-in-Q packets.
  • qinq <id>.<id>|<hex-vlan-outer>/<hex-mask-outer>.<hex-vlan-inner>/<hex-mask-inner> - Match Q-in-Q packet tag  with optional  (tag mask).
  • fwmark <mark>|<hex-mark>/<hex-mask> - match fwmark with optional mask. 
  • time from <hh:mm> to <hh:mm> [on Mon[,Tue,...]] - match current time interval with optional day of week
  • tproxy  - match transparent proxy traffic (diverted to transparent proxy, ex. SQUID)

DPI match rules

Uplink SPE supports several Deep Packet Inspection (DPI) matchers. Following DPI match rules can be used just like any other match rule : 

  • dns [list <hostlist_id>]  - match DNS packet with optional record name matching hostlist with ID of <hostlist_id>
  • http [list <hostlist_id>]  - match HTTP request packet with optional virtual host name or hostname in the URL string matching hostlist with ID of <hostlist_id>

Note: DPI match rules are very expensive at CPU cycles and allocate memory for each packet. Make sure to keep number of packet checked by DPI matchers as low as possible, as DPI rules make huge performance hit.

 

Access-lists

Uplink SPE supports 3 types of access-lists - IPv4, IPv6 and Hostlist. Access-lists are designed to pre-fetch required data to accelerate matching access-lists.

IPv4 Access-lists

Note: you can create up to 60 IPv4 Access-lists.

Before starting to add items to ACL, Access-list needs to be created. It can be done with following command: 

# uspe-client acl create <id> [name <name>]

Adding items to the access-list can be done with add or load commands. Difference between these two commands is that load is optimised to load large amounts of IPv4 prefixes, skipping acknowledgement of every prefix. Note that both load and add commands have optional timeout argument, making prefix to expire in specified amount of seconds.

# uspe-client acl add <id> <ip>[/mask] [<timeout>]

# uspe-client acl load <id> <ip>[/mask] [<timeout>]

To delete a prefix from ACL you can use delete command, which has following syntax :

# uspe-client acl delete <id> <ip>[/mask]

To empty an ACL list, you can use empty command, which has following syntax : 

# uspe-client acl empty <id>

Deleting all IPv4 ACLs can be done with destroy all command : 

# uspe-client acl destroy all

To view entries in ACL, you can use show command : 

# uspe-client acl show <acl_id>

It's possible to dump configuration of a specific IPv4 ACL : 

# uspe-client acl dump [<acl_id>]

Uplink SPE features a lookup command, which will lookup a specific address against all ACLs and return which ACLs it matches. You can use lookup command with following syntax : 

# uspe-client acl lookup <ip>

 

IPv6 Access-list

Note: you can create up to 60 IPv6 Access-lists.

Before starting to add items to ACL, Access-list needs to be created. It can be done with following command: 

# uspe-client acl6 create <id> [name <name>]

Adding items to the access-list can be done with add or load commands. Difference between these two commands is that load is optimised to load large amounts of IPv6 prefixes, skipping acknowledgement of every prefix. Note that both load and add commands have optional timeout argument, making prefix to expire in specified amount of seconds.

# uspe-client acl6 add <id> <ipv6_prefix> [<timeout>]

# uspe-client acl6 load <id> <ipv6_prefix> [<timeout>]

To delete a prefix from ACL you can use delete command, which has following syntax :

# uspe-client acl6 delete <id> <ip>[/mask]

To empty an ACL list, you can use empty command, which has following syntax : 

# uspe-client acl6 empty <id>

Deleting all IPv6 ACLs can be done with destroy all command : 

# uspe-client acl6 destroy all

To view entries in ACL, you can use show command : 

# uspe-client acl6 show <acl_id>

It's possible to dump configuration of a specific IPv6 ACL : 

# uspe-client acl6 dump [<acl_id>]

Uplink SPE features a lookup command, which will lookup a specific address against all ACLs and return which ACLs it matches. You can use lookup command with following syntax : 

# uspe-client acl6 lookup <ip>

 

Hostname Access-lists

Note: you can create up to 60 Hostlist Access-lists.

Before starting to add items to ACL, Access-list needs to be created. It can be done with following command: 

# uspe-client hostlist create <id> [name <name>]

Adding items to the access-list can be done with add command : 

# uspe-client hostlist add <id> <hostname>

To delete a hostname from ACL you can use delete command, which has following syntax :

# uspe-client hostlist delete <id> <ip>[/mask]

To empty an ACL list, you can use empty command, which has following syntax : 

# uspe-client hostlist empty <id>

Deleting all hostnames ACLs can be done with destroy all command : 

# uspe-client hostlist destroy all

To view entries in ACL, you can use show command : 

# uspe-client hostlist show <acl_id>

It's possible to dump configuration of a specific Hostlist ACL : 

# uspe-client hostlist dump [<acl_id>]

Uplink SPE features a lookup command, which will lookup a specific hostname against all ACLs and return which ACLs it matches. You can use lookup command with following syntax : 

# uspe-client hostlist lookup <ip>

 

AAA and RADIUS

Static session authorisation

During session authorisation process session's parameters are set to desired values. Unauthorised sessions have default settings and can be set to drop it's traffic until they would be authorised. Unauthorised sessions use Profile 0 and Policer 0, so if you want to set any policies on them - just configure these default profile and policier. 

If you want static configuration for all or some of your sessions you can use Uplink SPE maps to set session configuration. Uplink SPE supports three types of maps - IPv4, IPv6 and Q-in-Q. 

Note:

Maps can be updated using committed changes, so there won't be missed sessions while you update your maps

Maps are having higher priority than RADIUS authorisation 

To start adding mapping for your sessions, it's advised to run clear command - to clear anything, that could be accidentally left in temporary storage : 

# uspe-client map clear 

# uspe-client map6 clear

Note: IPv4 and Q-in-Q sessions share configuration storage, so with map clear  command you clear temporary storage for both types of these sessions.

To clear both temporary and live map storage - run map clear all command : 

# uspe-client map clear all

# uspe-client map6 clear all

Adding new entries to maps is done with map add or with map load. The difference between the two is that map/map6 add adds and entry straight to to live storage, while map/map6 load adds it to a temporary location and you need to map/map6 commit your changes before they will be applied to live storage.

Following are commands to add entries to their maps : 

# uspe-client map add <prefix> <map_args> [<SNAT4_static_ports>]

# uspe-client map add vlan <tag>.<tag> <map_args>

# uspe-client map6 add <ipv6_prefix> <map_args>

# uspe-client map load <prefix> <map_args> [<SNAT4_static_ports>]

# uspe-client map load vlan <tag>.<tag> <map_args>

# uspe-client map6 load <ipv6_prefix> <map_args>

<map_args> are : <profile_id> <policer_id> [<master_ipv4>] [<MAC>[,<MAC>,...]]

<SNAT4_static_ports> are : [<port>[,<port>,...]]

Note : Maps support up to 10 MACs  in map and up to 10 static SNAT4 ports per entry.

Note: Don't forget to commit your changes if you have used map/map6 load command to add your entries

To view current entries there are list  commands (note that vlan is used to show Q-in-Q sessions) :

# uspe-client map list vlan

# uspe-client map list

# uspe-client map6 list

Also, to dump configuration there are dump commands  : 

# uspe-client map dump

# uspe-client map6 dump

 

Radius session authorisation

Uplink SPE provides Radius session authorisation using auth-proxy daemon. As implementing Radius client inside linux kernel module is extremely inefficient, we have implemented lightweight protocol to communicate from kernel module to an auth-proxy,  which will proxy requests and check their validity. If you have trusted link to your Radius server, then you can even offload auth-proxy to a remote host and balance between several nodes. 

Uplink SPE needs to know, which server(s) to ask for authorisation : 

# uspe-client aaa auth <IP>:<port>

Note: If you'll add more than one authorisation server, then requests would be sent to all of them in Round-Robin fashion

For example, we have added two auth servers - 10.23.23.1:44064 and 10.23.23.2:44064. Using aaa show  command we can check their status and statistics :

# uspe-client aaa show 

Authorisation servers    Packets sent     Errors
10.23.23.1:44064               0            0
10.23.23.2:44064               0            0

Accounting servers    Packets sent     Errors

Note: Uplink SPE uses UDP to send authorisation requests and replies

To proxy requests, you need to run auth-proxy daemon from Uplink SPE distribution. Following is a description of auth-proxy daemon arguments. 

 

auth-proxy [-d] [-l <ipaddr>:<port>] [-c <rad_conf_file>] [-r <ipaddr>:<port> -s <secret>] [-w <threads>]
Flags:
    -d    Daemonize
    -l    Listen address (default to 0.0.0.0:44064)
    -c    Radlib config file (libradius radius.conf file, defaults to NONE)
    -r    Radius remote server
    -s    Radius secret
    -w    Number of worker threads
    NOTE:    -s option overrides -r

Note: if you have several RADIUS servers and you want to spread the load among all of them, auth-proxy should be running on RADIUS servers, so Uplink SPE would be able to spread requests evenly.

Supported RADIUS attributes

Attribute Usage Example 
Acct-Interim-Interval Accounting update interval. Number of seconds between accounting updates of a session 180
Session-Timeout Session max life time. Session will need to re-authorise after this interval 3600
Idle-Timeout Session timeout if there is no traffic seen.  360
User-Name Session identifier. IPv4 address (string), IPv6 address (string) or Q-in-Q vlan (string) of format VLAN<number>.<number> "VLAN10.321"
Cisco-Service-Info Profile and Policer settings  of format : PP<profile_id>;<policer_id> "PP1;2"
Cisco-Account-Info Master IPv4 address (string). Format : MI<ipv4_addr> "MI1.2.3.4"
Cisco-Account-Info Allowed MAC-addresses. Format : MA<mac>[;<mac>....] "MA00:11:22:33:44:55"
Cisco-Account-Info SNAT4 Static Ports. Format : NP<port>[;<port>] "NP80;443;110"
Cisco-Account-Info Quota profile. Upload and download limits are in bytes. Format: 

QP<upload_limit>;<download_limit>;<profile_id>;<policer_id>

"QP50000;50000;2;2"

 

Radius session accounting

Uplink SPE provides session Radius accounting using acct-proxy userspace daemon. As Implementing Radius client in kernel is inefficient, we have implemented lightweight protocol to communicate from kernel to an acct-proxy, which will proxy requests to Radius accounting server, offloading kernel module from Radius-related tasks. 

Note: If you'll add more than one accounting server, then requests would be sent to all of them in Round-Robin fashion

Uplink SPE needs to know where to send accounting packets :

# uspe-client aaa accounting <IP>:<port>

For example, we have added two auth servers - 10.23.23.1:44054 and 10.23.23.2:44054. Using aaa show  command we can check their status and statistics :

# uspe-client aaa show 

Authorisation servers    Packets sent     Errors

Accounting servers    Packets sent     Errors
10.23.23.1:44054               0            0
10.23.23.2:44054               0            0

Note: Uplink SPE uses UDP to send accounting messages

To proxy requests from Uplink SPE to Radius, you need to have acct-proxy daemon from Uplink SPE distribution running. Following is a description of arguments accepted by daemon : 

acct-proxy [-d] [-l <ipaddr>:<port>] [-c <rad_conf_file>] [-r <ipaddr>:<port> -s <secret>] [-w <threads>]
Flags:
    -d    Daemonize
    -l    Listen address (default to 0.0.0.0:44054)
    -s    Radlib config file (libradius radius.conf file, defaults to NONE)
    -r    Radius remote server
    -w    Number of worker threads
   NOTE:    -s option overrides -r

Note: if you have several RADIUS servers and you want to spread the load among all of them, acct-proxy should be running on RADIUS servers, so Uplink SPE would be able to spread load evenly.

Supported attributes sent to an accounting server

Attribute Usage Example
User-Name  Session identifier. IPv4 address (string), IPv6 address (string) or Q-in-Q vlan (string) of format VLAN<number>.<number>  "VLAN10.321"
Cisco-AVPair

SNAT4 Public IPv4. Sent only with IPv4 Sessions enabled CG-SNAT. Format : SI<ipv4>

"SI1.2.3.4"
Cisco-AVPair

SNAT4 first Public port of port allocation. Sent only with IPv4 Sessions enabled CG-SNAT. Format : SF<port>

"SF4096"
Cisco-AVPair

SNAT4 Pool ID. Sent only with IPv4 Sessions enabled CG-SNAT. Format : SP<id>

SP1
Cisco-AVPair

SNAT4 Pool size (in bits). Sent only with IPv4 Sessions enabled CG-SNAT. Format : SS<bits>

SS10
Cisco-AVPair

Session parent IPv4. Sent only if session is Slave

PI10.2.2.1
Acct-Status-Type Accounting flag showing if session has started, ended or it's a periodic update Start or Stop or Interim-Update
Acct-Session-Id Unique session identifier "somerandomstring"
Acct-Session-Time Session duration in seconds 1234
Acct-Input-Packets 32-bit counter of input packets 12345678
Acct-Output-Packets 32-bit counter of output packets 12344657
Acct-Input-Octets 32-bit counter of input bytes  12345678
Acct-Output-Octets 32-bit-counter of output bytes 12345678
Acct-Input-Gigawords Hi-word  of  64-bit input bytes counter. 34567980
Acct-Output-Gigawords Hi-word of 64-bit output bytes counter 34566790
Cisco-Service-Info Profile and policer of a session. Format : PP<profile_id>;<policer_id>

PP1;1

Cisco-Control-Info Cisco way of sending 64-bit input bytes. Format : I<gwords>;<words> I1234456;345634567
Cisco-Control-Info Cisco way of sending 64-bit output bytes. Format : O<gwords>;<words> O1234456;345634567

 

 

Uplink SPE provides flexible solution to setup and manage CG-NAT together with BRAS functionality of Uplink SPE. Setting up CG NAT is quite simple, there are just a few concepts you need to be aware of. 

SNAT action is an ordinary action in Profiles, which basically doing - "NAT packet according rules in NAT pool number X". So NAT pool is a basic (and the only) entity to configure. NAT pool defines mappings between private and public IPs, and it's just 3 commands you operate whole pool configuration : 

# uspe-client nat create <pool_id> slot-size <slot_size> [name <pool_name>]
# uspe-client nat add public <pool_id> <prefix>
# uspe-client nat add private <pool_id> <prefix>

<slot_size> parameter is a key in Pool's configuration and it's value defines how many address translations are going to be reserved for a session and how many private IP's can be translated in this pool. In short, number of ports allocated to a session is 2^<slot_size> . Every slot is a consecutive block of ports, allocated to a session. 

Note: currently, only one slot per session is supported.

For example, if you use <slot_size> equal to 10, then number of ports allocated to a singe private IP would be : 2^10 = 1024.

Also, you can easily find how many private IP's a pool can handle per single Public IP : 2^(16-<slot_size>) = 2^(16-10) = 64

Once you are done adding public and private prefixes to the pool you need to commit your changes. You can do it like this : 

# uspe-client nat commit [<pool_id>]

Note: commit will fail, if a pool doesn't have enough public IPs to translate all private IPs. Uplink SPE doesn't allow oversubscription of public IP resources.

Viewing and managing NAT translations

To list current SNAT pools, you can use list command : 

# uspe-client nat list

# Name          Slot size(bits)     IPs Public/Private   Slots Usable/Allocated/InUse/Usage%/Effective%
1: Pool1                10                  256/4096                 16384 4096 0 0% 25%

List command provides some basic information about pool configuration and counters of current usage.

To view more details about the pool, use show command : 

# uspe-client nat show  1

# Name            Slot size(bits)      IPs Public/Private   Slots Usable/Allocated/InUse/Usage%/Effective%
1: Pool1                10                   256/4096                 16384 4096 0 0% 25% 
Flows Active/Total/Usage% : 0 4194304 0%
Private prefixes:
10.0.0.0/20
Public prefixes:
1.2.3.0/24

 

To check current SNAT translations for a specific IPv4 session, you can use nat flows command : 

# uspe-client nat flows 10.0.3.239

Flows for internal IP : 10.0.3.239
External IP : 1.2.3.15
Port External   Port Local             Age  Expire in
48128           80                      24s 4277766591s
48129           110                     24s 4277766591s
48130           443                     24s 4277766591s
48131           1035                    24s          6s
48132           1036                    24s          6s
48133           1037                    24s          6s
48134           1038                    24s          6s 
.....

Note, first 3 translations have notably long expire time - they are static translations that are set by map or RADIUS. See AAA and Radius section for more details.

It might be useful to see details of a session, to check for SNAT related information. For example, if customer has used all his translation resources. It can be easily found by session details information : 

# uspe-client session detail 10.0.3.30
IP         Age   Policy   IN pkts/Drop     IN bytes/Drop    OUT pkts/Drop    OUT Bytes/Drop   Flags
10.0.3.30  10      1/1        0/0              0/0          17717/11557    5811176/3790696 AUC---4--Nn- 

VLAN tags 0.0
Quota (In/Out) : 0/0, Quota profile/policer : 0/0

SNAT4 details: 
Pool 1, Public IP 1.2.3.12, Ports 30720-31744
Translation failures: 11557

You may want to clear translations for a specific IP, if you don't want to wait them to expire. You can do it with nat clear command  : 

# uspe-client nat clear flows 10.0.2.30

Also, you may want to clear all translation flows and slots of a certain Pool, or even all pools : 

# uspe-client nat clear [<pool_id>]

Note, when clearing flows or pools, static ports setting is wiped too. if you need to restore it - you need to clear session and re-authorise it.

QinQ Termination and IP-unnumbered 

As of version 15.08 Uplink SPE supports IP unnumbered interfaces and Q-in-Q termination.  The basic idea of this technology is a conservative use of IP addresses and simplified management, which allows to implement 'VLAN per customer' approach in ISP networks. 

Typical scenario is following:

  • A dedicated Q-in-Q vlan is allocated to customer
  • An IP address(-es) is assigned to customer via DHCP (or statically)
  • QinQ vlans are terminated at BRAS (BRAS is a gateway for the customer)
  • IP-unnumbered technology is used to use IP-address space in conservative manner, without loosing them on Network/broadcast address

Uplink SPE IP-unnumbered implementation is bound to Q-in-Q interfaces only (Including both single- and double-tagged interfaces), so there are no possibility to use it on normal IP interfaces. 

A sub-interface entity is used to describe an IP interface on a customer's Q-in-Q VLAN. Uplink SPE doesn't create virtual network interface in the system for every customer, however internally Uplink SPE has a sub-interface object and it can be managed trough uplink-spe client. 

Note: As there are no interfaces for customers qinq vlans (sub-interfaces), ARP entries are managed by QinQ termination module as well. 

The process of setting up a QinQ termination with IP-unnumbered is following :

  1. Setup a topology to configure physical interfaces to handle incoming frames the way we need
  2. Set defaults for sub-interfaces to match our needs - including flags, ARP and DHCP packet limits
  3. setup qinqmap entries for non-default settings on sub-interfaces with optional routes to avoid AAA-auth requests (optional)
  4. setup a DHCP-relay or DHCP-Radius relay settings if it's going to be used to activate sub-interfaces (optional)
  5. setup a DHCP-server to provide leases to customers from local pools (optional)
  6. setup routes for static sub-interfaces, which need to be up unconditionally (optional)

Following chapters will describe this process in details.

 

Setting up a topology

To configure an interface to be a Q-in-Q terminated with IP-Unnumbered, it's necessary to instruct Uplink SPE about a physical interface, which contains QinQ vlans to be terminated. Following command is used to do that : 

# uspe-client topology interface unnumbered <interface> <encapsulation>

<encapsulation> parameter is a VLAN protocols used on the wire. Supported values are : 

dot1q Not actually a QinQ, but a 802.1q single tagging with ethernet protocol 0x8100
double1q A double-tagged QinQ implementation where both Service and Client tags have an ethernet protocol number 0x8100
dot1ah a standard implementation of QinQ according to a 802.1AH standard. Uses ethernet protocol 0x88E7 for Service tag and 0x8100 for Client tag. 

 

Note: adding an unnumbered interface into a configuration creates a virtual device named uspeX (where X is a number), which is then used as a destination interface for routes, created for the customer ips.

 

Set defaults for sub-interfaces

Sub-interface default settings are applied to sub-interfaces on their creation time (ex. when a first packet from client arrives). Following is a command used to set these defaults : 

# uspe-clients ipunnumbered defaults <ifname> <flags> limits <arp_pps> <dhcp_pps> <nd6_pps> <max_leases> <max_v6_neigh> [arp-timeout <age>] 

<ifname> - physical interface name, which is already added to a topology as unnumbered, as described in previous section.

<flags> - is a string of flags to set to sub-interfaces. Later in this section they will be described in detail. 

<arp_pps> and <dhcp_pps> are limits of packets per seconds allowed - ARP's and DHCP requests accordingly. This is a per-sub-interface rate-limit to avoid DoS attacks.

<nd6_pps> is an amount of IPV6 ICMP neighbour discovery packets (used to map IPv6 addresses to MAC)

<max_leases> is a maximum amount of active IPv4 DHCP leases on the sub-interface

<max_v6_neigh> is a maximum number of IPv6 dynamic neighbours on a sub-interface.

 

Flag Description Master Flag
A

ARP enabled. Enable if you want Sub-interface to send ARP replies.

-
p

Proxy ARP enabled. Enable if you want Sub-interface to reply on ARP requests for IPs in same subnet, but on other sub-interface.

A
P

Private VLAN ARP enabled. Enable if you want Sub-interface to reply on ARP requests for IPs from same Sub-interface. Usually needed if a QinQ vlan has private VLANs on it.

A
R

rp_filter enabled. Check incoming packets Source IP to exist in routing table and originate from correct sub-interface. 

Note: Routes dynamically added to your system outside Uplink SPE are NOT checked and packets may be dropped, even if route exists in the system. 

-
D

DHCP relay enabled. Enable DHCP request packet processing, including unicast.

-
F

DHCP spoof protection. Enable to Drop incoming DHCP-reply packets. Useful to be enabled on customer ports.

-
r

Send DHCP requests via Radius. Instead of forwarding DHCP requests using dhcp-relay settings, use AAA auth servers to authorise clients/sub-interfaces.

D
K

DHCP option82 KEEP. Keep existing Option82 in DHCP requests. if it's not there - add it with our data.

D
8

DHCP option82 REPLACE. Replace Option82 in DHCP requests with our data.

D
d

DHCP option82 DROP. Strip out Option82 in DHCP requests if it's set. 

D
Z

Pass-trough interface. Enable to completely ignore traffic on sub-interface. Useful to skip sub-interface from IP-unnumbered processing and avoid configuration requests for sub-interface.

Note : This option overrides all other options

-
c Subinterface packets enabled. Note: To enable these counters both Subeinterface and default flags must have this flag enabled.  -
m Check Source MAC address for incomming packets and drop if route/ARP don't match -
6 IPv6 Enabled  -
T
IPv6 Stateless Address Autoconfig enabled
6
t IPv6 ICMPv6 requests limit enabled 6
Y IPV6 Neighbor passive learning enabled 6

 

Later in this chapter, in Example use-case scenarios, would be described some popular use-cases with flags used.

 

Manage individual sub-interface parameters

Sometimes, default flags are not desired on some set of sub-interfaces. There is no way to change flags on running sub-interface, however, it's possible to re-initialise sub-interface with new set of flags. qinqmap is used to store configuration for that kind of exceptions. qinqmap can be used to store routes as well, to avoid requests to DHCP or Radius servers. 

Following is a syntax to add a qinqmap entry : 

# uspe-client qinqmap add <ifname> <vlan>[.<vlan>] <flags> [limits <arp_pps> <dhcp_pps> <max_v6_neigh>] [<cidr> [<hwaddr>] [<max_age>]]

Flags string contains absolutely the same flags as in previous section. Other parameters are optional, but useful in various scenarios. 

ARP and DHCP packet limits can be set with this command to override default limits using <arps_pps> and <dhcp_pps> parameters. 

<max_v6_neigh> parameter sets limit of maximum IPv6 Neighbors allowed on sub-interface.

As optional feature of this command is adding routes and even MAC-addresses with them. Using <cidr> parameter you can add routes to a sub-interface, optionally with MAC-address to set internal ARP-entry. Multiple routes can be added to same sub-interface in qinqmap using several calls to this command. 

Note: if you need different MAC-addresses to IP-addresses in a single IP prefix, then you need to add them separately, as there are only one MAC-address allowed per <cidr> prefix.

 

Setup static sub-interfaces

Sub-interfaces can be pre-configured statically, to pre-create them during configuration load. While static sub-interfaces have no difference to sub-interfaces created using qinqmap, they are permanent and will never expire. Static sub-interfaces are pre-created and are up right away. 

Static interfaces might be useful for qinq interfaces that need to be up all the time, to avoid route flaps.

You can create that kind of sub-interface with following command (default interface flags will be used) : 

# uspe-client ipunnumbered route <ifname> <s_vlan>[.<c_vlan>] <prefix> [<mac_addr>]

To clear such route you need to clear all routes on sub-interface : 

# uspe-client ipunnumbered clear <ifname> [<s_vlan>[.<c_vlan>]]

 

QinQ map commands reference

To add a qinq sub-interface mapping you can use following command (flags are described in Set sub-interface defaults section): 

# uspe-client qinqmap add <ifname> <vlan>[.<vlan>] <flags> [limits <arp_pps> <dhcp_pps> <max_v6_neigh>]  [<cidr> [<hwaddr>] [<max_age>]]

Another way to load qinq mappings is to load them to a temporary storage and commit them all at once :

# uspe-client qinqmap load <ifname> <vlan>[.<vlan>] <flags> [limits <arp_pps> <dhcp_pps> <max_v6_neigh>] [<cidr> [<hwaddr>] [<max_age>]]

# uspe-client qinqmap commit

To clear all mapping entries or entries on specific interface, or on a specific sub-interface, you can use clear command :

# uspe-client qinqmap clear all

# uspe-client qinqmap clear  <ifname> <vlan>[.<vlan>]

To get listing of current qinq mappings you can use list command : 

# uspe-client qinqmap list [<ifname>]

Also, you can use a lookup command to find out what settings would be set for a specific QinQ vlan : 

# uspe-client qinqmap lookup <ifname> <vlan>[.<vlan>] 

 

IP-unnumbered commands reference

To get current active qinq enabled interfaces you can use list command : 

ipunnumbered list 

Following is an example output of this command : 

Current IP unnumbered interfaces :
Interface            USPE hook interface  Flags             Limits ARP/DHCP      DHCP relay information
eth1                 uspe0               q--A-P---F------     0/0                  No DHCP relay       

To view list of sub-interfaces you can use show command : 

ipunnumbered show <ifname> [<s_vlan>[.<c_vlan>]]

Following is an example output of show command : 

Current IP unnumbered interfaces on eth1 :
Sub-Interface         Route             Age    Flags          ARP (pkts/drp/limit) DHCP (pkts/drp/limit)
eth1.10.10            10.10.0.2/32      0      ---A-P---F----       0/0/0             0/0/0   
eth1.10.20            10.20.0.1/32      0      ---A-P---F-----      0/0/0             0/0/0

 

ipunnumbered detail <ifname> <s_vlan>[.<c_vlan>]

ipunnumbered clear <ifname> [<s_vlan>[.<c_vlan>]]

ipunnumbered route <ifname> <s_vlan>[.<c_vlan>] <prefix> [<mac_addr>]

ipunnumbered defaults <ifname> <flags> limits <arp_pps> <dhcp_pps> <max_v6_neigh>

 

QinQ termination and Radius 

Radius interface setup is available from version 15.8. There is no 

 

QinQ termination and DHCP server/relay

As of 15.08 release, Uplink SPE supports DHCP server, relay, and DHCP-to-Radius relay on IP-unnumbered interfaces. 

 

Note: At the moment of 15.08 release, DHCP functionality is only available on IP-unnumbered interfaces

 

Set up a DHCP-relay 

DHCP-relay settings are set per physical interface. Currently, DHCP-relay can operate in two modes - relay and radius.

In relay mode DHCP-request packets are relayed to a DHCP-server, optionally adding Option82 to the request. Setting DHCP-server address is done with following command : 

# dhcp relay <ifname> <ip:port> [src <ip>]

You can add multiple DCHP-servers to interface to load-balance requests. 

To enable DHCP requests to be handled by radius, It's sufficient to set DHCP radius flags in default interface flags or individua sub-interface flags. It's described in previous sections.

To clear DHCP-relay settings on an interface, you can use following command : 

# uspe-client dhcp disable <ifname>  [with-subs]

Using "with-subs" argument you can disable DHCP-relay settings on all existing sub-interfaces

 

Setup a DHCP relaying to RADIUS server

An unique option in Uplink SPE software is an ability to translate DHCP requests into Radius requests. So now you don't need to duplicate information about customers and their IP bindings on both RADIUS and DHCP servers, which allows you to save on infrastructure without compromising it's functionality. 

Relaying DHCP requests to a radius server is configured with a single command : 

# dhcp relay <ifname> radius

You would need to add this command for every IP-unnumbered interface in your system. 

This command instructs to spool DHCP requests to an AAA (Radius server), configured as an auth server. AAA configuration can be found in a AAA and RADIUS section of this manual.

 

Setup a DHCP-server for QinQ interfaces

Setting up a local DHCP server on IP-unnumbered interfaces is another alternative of controlling IP-address space allocation to customers on Uplink SPE. While it's configuration is just a bit more complicated than DHCP-relay, it provides more control and independence to your BRAS. 

Configuration of DHCP server consists of four  points : 

  • DHCP server operation mode
  • Lease pools
  • Lease exceptions
  • sub-interface flags and limits

As a first thing, you need to enable local DHCP server on your IP-unnumbered interface and select it's operation mode. 

Note: as of version 16.2 DHCP-pools have become independent of QinQ interface, which effectively means that single DHCP-pool can be shared between several QinQ interfaces.

# dhcp server <ifname> local [strict]

There are two modes : strict and normal.

Strict DHCP server mode distributes only statically configured IP/sub-interface mappings (configured as a QinQ map, discussed later in this chapter). This means, that all your customers need to be pre-assigned to sub-interfaces and need to have their IPs set. In strict mode leases and flags are set from qinqmap, so it's sufficient to enable DHCP on your IP-unnumbered interface and have all your IP/subinterface mappings in qinqmap. 

Note: you don't have to have MAC-addresses bound to IP/Subinterface in qinqmap. While it's preferred way to have it, it's not a requirement. Strict mode DHCP server will use qinqmap maps without MAC address as a dynamic lease and will provide it to first customer requesting it on that Sub-interface. 

Normal DHCP server operation mode in addition to strict functionality provides also dynamic allocation of IPs to customers from the pools. In this mode your qinqmap mappings can coexist with dynamically allocated IPs. 

In this mode you provide dynamic IP-addresses to customers without the need to have a strict IP/Subinterface bindings, while still have a high level of security. 

For maximum flexibility, all settings are per-pool. However, you can set defaults which will be applied to all pools in the system (while these still can be overridden by per-pool and per-prefix settings). 

System Defaults can be set in the following way : 

# uspe-client dhcp defaults system gw <ip> lease-mask <len> dns1 <ip> dns2 <ip> lease-time <seconds>

The Pool defaults can be set in this way : 

# uspe-client dhcp pool-defaults <id> gw <ip> lease-mask <len> dns1 <ip> dns2 <ip> lease-time <seconds>

Per-prefix settings can be set during prefix addition to the pool : 

# uspe-client dhcp pool add <id> <prefix> [gw <ip> lease-mask <len> dns1 <ip> dns2 <ip> lease-time <seconds>]

Note: The preference of settings is following (in ASCending order) : system defaults, pool defaults, prefix settings.

 

These defaults will be used on IP-pool if they are omitted during prefix add. Here is a more detailed description of these options :

Option Description
gw <ip> This IP will be provided to DHCP clients as Default Gateway IP.
Note: This IP will be added to DHCP exceptions automatically, however it will not appear in the exceptions list
lease-mask <len> This is a CIDR prefix len of a netmask, provided to customers. for example <len> of 24 will provide mask 255.255.255.0

dns1 <ip> and dns2 <ip> 

These IPs are DNS servers provided to customer. Two of them for redundancy
lease-time <seconds> Lease time. Duration of DHCP lease provided to customer. 

 

Following command adds more IP resources to the pool. Every prefix can have its own settings, and if they omitted - pool defaults would be used. Optional parameters are the same as described in dhcp defaults option

# uspe-client dhcp pool <id> add <cidr> [gw <ip> lease-mask <len> dns1 <ip> dns2 <ip> lease-time <seconds>]

If you want to exclude some IPs from the pool (for example, they are statically assigned with qinqmap) - you can add exceptions to dhcp server : 

# uspe-client dhcp pool <id> add-exception <cidr>

Pool needs to be committed, so all desired changes will be put to live : 

# uspe-client dhcp pool commit [<id>]

Also, QinQ interface need to be configured to use a specific pool : 

# uspe-client dhcp pool attach <id> <ifname>

 

Local DHCP-server operation and security

Service provider grade DHCP service has some challenges, which needs to be solved in every network. There is no perfect network and there is no way to guarantee correct behaviour in your network. Uplink SPE solves these issues in a simple and effective manner. 

ARP and DHCP flooding, DHCP pool exhaustion, rogue DHCP server mitigation

As most popular issue in IPoE networks is ARP/DHCP flooding, which burns CPU on BRAS, we have introduced per-Subinterface rate-limiters to limit amount of packets allowed per second. Most of the time defaults are safe, feel free to tune them to your needs. 

You can tune amount of incoming ARP and DHCP packets independently. Another issue that may happen - exhausting DHCP dynamic pool resources. This limit can also be set, to avoid dodgy wifi router to interfere to other customers connectivity. 

All these three options are set as default setting to a QinQ-IP-unnumbered interface in following way : 

# uspe-client ipunnumbered defaults <ifname> <flags> limits <arp_pps> <dhcp_pps> <max_leases> <max_v6_neigh> 

Also, these limits can be updated on per-subinterface basis : 

# uspe-client ipunnumbered update <ifname> <s_vlan>[.<c_vlan>] limits <arp_pps> <dhcp_pps> <max_leases> <max_v6_neigh>

To check current state of these settings you can use (if you don't specify s_vlan/c_vlan it will show you defaults):

# uspe-client ipunnumbered show <ifname> [<s_vlan>[.<c_vlan>]]

Uplink SPE prevents DHCP servers on sub-interfaces to forward packets by dropping incoming DHCP replies. This will prevent common situation, when customers also enabling DHCP service on their wifi routers and disrupt correct DHCP service functions. Enabling 'DHCP spoof protection' on subinterface (preferably, using default settings) will block DCHP-replies coming from customers. 

# uspe-client ipunnumbered defaults <ifname> <flags> limits <arp_pps> <dhcp_pps> <max_leases> <max_v6_neigh>

Just add a 'F' flag to <flags> string, to enable DHCP spoof protection feature. 

Note: this feature works the best if you have Vlan Per customer, or Private Vlan Per customer in your network

Source IP/Mac spoofing prevention

While isolating customers into their VLANs enable high security, sometimes it may not be enough. If an attacker manages  to connect to customer's VLAN, then he could steal customer's IP - making customers connection to be dropped. 

To avoid this situation, Static MAC addresses should be bound to IPs and dynamic ARP learning should be disabled. This can be achieved using qinqmap with mac-addr attached together with route.

Alternatively, It's also possible to use 'sticky' ARP feature - to have dynamic ARP, but infinite ARP timeout. This way first mac learnt would be allowed on the route while subinterface is up. 

To enable it, ARP timeout needs to be set to a large number : 

# uspe-client ipunnumbered defaults <ifname> <flags> limits <arp_pps> <dhcp_pps> <max_leases> <max_v6_neigh> arp-timeout <secs>]

 

Day-to-day operations

Following command shows all dhcp-enabled interfaces with their flags :

# uspe-client dhcp list

Following command displays statistics for all DHCP-enabled interfaces : 

# uspe-client dhcp stats

Following command displays extended information about dhcp server/relay on an interface : 

# uspe-client dhcp show <ifname>

Following command displays DHCP leases on a specific interface or sub-interface. Note, if you don't specify sub-interface, leases of all sub-interfaces would be shown. 

# uspe-client dhcp leases <ifname> [<vlan>[.<vlan>]]

Following command displays information about specific lease on an interface :

# uspe-client dhcp leases <ifname> <ip>

To view detailed information about specific subinterface use following command : 

# uspe-client ipunnumbered details <ifname> <vlan>[.<vlan>]

A DHCP pool can be examined to show current allocation stats with this command : 

# uspe-client dhcp pool show <id>

 

DHCP-leases protection

While BRAS meant to be rock-solid, sometimes there is a need to update software or make other service-disrupting maintenance. To avoid loosing DHCP leases information you can save current leases and import them back when needed. Dump command dumps whole DHCP service configuration, including active leases. You may want to save them using following command : 

# uspe-client dhcp dump | grep 'add-lease' > /tmp/leases.backup.txt

And later, once service is resumed, you can restore leases using following command : 

# uspe-client restore <  /tmp/leases.backup.txt

More complex scenarios, with uploading backup file to external storage is also possible.

 

QinQ termination and IPv6

As of version 16.2 Uplink SPE supports IPv6 termination on QinQ interfaces. While native Linux IPv6 stack provides complete feature set, it's not designed do suit all ISP needs. It could be used to terminate thousands of IPv6 interfaces, however amount of subscribers always put the router performance down - just by having cheap home wifi router with improper firmware they can burn router's CPU significantly. Uplink SPE combines Linux IPv6 stack possibilities with secure interface termination, providing flexible and powerful tools to protect router CPU without compromising performance. 

Uplink SPE QinQ IPv6 termination provides ways to limit a rate of incoming IPv6 ND (Neighbour Discovery) packets, to limit of number of active IPv6 Neighbour entries (IPv6 to MAC mappings) and also provides built in services of IPv6 customers stateless auto-configuration (SLAAC) using IPv6 pools.  All these services will be described in more details later on in this chapter.

 

Enabling IPv6 for QinQ interfaces

IPv6 can be enabled on QinQ interfaces in two ways - enabling on per-interface basis, or by default. To enable by default for all interfaces is sufficient to set new defaults : 

# uspe-client ipunnumbered defaults eth1 APSF6y limits 5 5 5 5 5

The flags related to IPv6 are - '6' and 'y', which represent "IPv6 Enabled" and "IPv6 Neighbour passive learning" accordingly. While "IPv6 Enabled" flag is required, the "IPv6 neighbour passive learning" allows to learn new hosts. As you may noticed limits are set with this command as well - third and fifth limit represent <nd6_pps> (maximum number of ND packets allowed per second) and <max_v6_neigh> (maximum number of IPv6 neighbours on a sub-interface). 

The second way of enabling IPv6 termination on QinQ interfaces is done on a per subinterface basis, using qinqmaps. They are discussed earlier in QinQ termination and IP-unnumbered section.  

 

IPv6 pools and Stateless auto config (SLAAC)

Stateless IPv6 auto-configuration for QinQ subinterfaces is done by setting up an IPv6 pool and enabling SLAAC configuration on a sub-interface.  To create the pool, it needs to be created, populated with prefixes with sufficient amount of prefixes and optionally exceptions. Once this is completed, a pool needs to be committed. Finally, a pool needs to be attached to a QinQ interface. 

Note: only one IPv6 pool per QinQ interface is allowed

Following is a command reference to IPv6 pools :

# uspe-client ipv6pool create <id> type <dynamic|qinq|dot1q> [name <name>]

# uspe-client ipv6pool prefix <id> <prefix>

# uspe-client ipv6pool exception <id> <prefix>

# uspe-client ipv6pool commit [<id>]

# uspe-client ipv6pool attach <infame> pool <id>

# uspe-client ipv6pool defaults <ifname> <pfx_lifetime> <ra_interval> <max_slaac>

Note: It's also required to enable SLAAC service on sub-interface by adding a 'T' flags, either to defaults settings or to a qinqmap flags for the subinterface, just like it was shown in previous section.

IPv6 pool also allows to tune the lifetime of a /64 prefix assigned to customers on a QinQ sub-interface (a <pxf_lifetime> option to the defaults command), IPv6 ND Router Advertisment interval (<ra_interval>, in seconds) and maximum number of EUI-64 generated Neighbour entries in the tree, in addition to other discovered entries. 

Note : total number in Neighbour tree on a QinQ subinterface is a sum of EUI-64 based neighbours and other neighbours.

 

IPv6 Neighbours at QinQ interfaces

IPv6 neighbour discovery in Uplink SPE works as a complementation to the static IPv6 routes and EUI-64 address mapping. Neighbours are added to the USPE tree only if they don't belong to static routes and are not the EUI-64 mapped addresses.  Later in this chapter will be discussed how to put limits on Neighbours table, to protect router from attacks on this table. 

Currently, static IPv6 Neighbours are not supported. Therefore, only viewing and flushing Neighbours table commands are available : 

# uspe-client ipv6nd show <iface> <tag>[.<tag>]

# uspe-client ipv6nd flush <iface> <tag>[.<tag>]

 

Protecting router from IPv6 resource exhaustion

The resources related to IPv6 sub-interface which potentially could be attacked are Neighbour table size and rate of incoming IPv6 ND protocol packets (which causes extra CPU load per packet). Both of these can be limited on a per- sub-interface basis.

Depending on number of customers on a sub-interface these limits should be set to reasonable values. 

You can set these limits as a default setting via defaults command (<nd6_pps> and <max_v6_neigh> parameters)

# uspe-client ipunnumbered defaults <ifname> <flags> limits <arp_pps> <dhcp_pps> <nd6_pps> <max_leases> <max_v6_neigh>

 

Example use-case scenarios

All scenarios assume a simple topology, where eth0 is an uplink interface, and eth1 is the subscribers-facing interface. 

Scenario 1: Several QinQ static sub-interfaces

This is the most basic example of QinQ termination and IP-unnumbered. This example assumes following : 

  • Clients reside in VLANS 10,20 and 30 of eth1
  • Each client has a /32 IP address within the same /24 prefix
  • eth1 is a 802.1Q interface

Configuration script would look as following : 

topology interface unnumbered eth1 dot1q
ipunnumbered route eth1 10 10.0.0.10
ipunnumbered route eth1 20 10.0.0.20
ipunnumbered route eth1 30 10.0.0.30

 

Scenario 2: Several sub-interfaces created from pre-defined config

This a basic example of QinQ termination and IP-unnumbered with dynamic sub-interfaces. This example assumes same as in Example 1, but subinterfaces need to be dynamically created on first pac

Configuration script would look as following : 

topology interface unnumbered eth1 dot1q
qinqmap add eth1 10 ARF 10.20.30.10/32
qinqmap add eth1 20 ARF 10.20.30.20/32
qinqmap add eth1 30 ARF 10.20.30.30/32

 

Some flags are set with qinq maps. Check QinQ map command reference for details.

 

Scenario 3 : QinQ maps and strict mode DHCP server

Following example enables QinQ IP-unnumbered setup, with three customers with /32 addresses, and enabling a DCHP server to provide these addresses to customers. Addresses may be used statically too, as they are bound with qinqmap to a subinterfaces 10.10, 20.20 and 30.30.

topology interface unnumbered eth1 dot1ah
ipunnumbered defaults eth1 AFc limits 5 5 20 arp-timeout 30
qinqmap add eth1 10.10 ARF 10.20.30.10/32
qinqmap add eth1 20.20 ARF 10.20.30.20/32
qinqmap add eth1 30.30 ARF 10.20.30.30/32
dhcp server eth1 local strict
dhcp defaults eth1 gw 10.20.30.1 lease-mask 32 dns1 8.8.8.8 dns2 8.8.4.4 lease-time 256

 

This example provides the easiest way to provide DHCP services to QinQ customers and providing them same allocated IP via DHCP.

Scenario 4 : QinQ maps and normal mode DHCP server with exceptions

Following example enables QinQ IP-unnumbered setup, with three customers with /32 addresses, and enabling a DCHP server to provide these addresses to customers. Addresses are allocated by DHCP servers, and customers can't use static IP addressing on their systems, as no IP-unnumbered route would be created without assigning a DHCP-lease to client. 

topology interface unnumbered eth1 dot1ah
ipunnumbered defaults eth1 R limits 5 5 20
dhcp server eth1 local
dhcp defaults eth1 gw 10.20.30.1 lease-mask 32 dns1 8.9.10.11 dns2 8.8.3.4 lease-time 600 
dhcp pool eth1 add 10.20.30.0/24 gw 10.20.30.1 lease-mask 32 dns1 8.8.8.8 dns2 8.8.4.4 lease-time 600
dhcp pool eth1 add-exception 10.20.30.0/28

 

This example provides DHCP options with the pool, overriding default ones set by dhcp defaults command. Also, a /28 network is excluded in the pool, so customers would get addresses from the rest of the pool. There are no specific VLANS set to the service and none is disabled by default configuration - so this will work with any QinQ subinterface.

 

Extending Uplink SPE and writing your own plugins

Uplink SPE supports plugins for following parts of it's code : 

  • Profile/policer match 
  • Profile/policer action
  • SNAT protocol translation
  • a log handler

As a live example of plugin, we have implemented a trpoxy plugin, which allows to redirect TCP connections to a Local Transparent socket. Most common use case for it - captive portal or notifications to a customer via web-page. 

More details can be obtained from the source code available on GitHub - https://github.com/uplink-spe/mod_tproxy