Details for traffic shaping are in http://lartc.org.

Details regarding tcp testing are here: http://www.linuxfoundation.org/collaborate/workgroups/networking/tcp_testing.

Network tests are done with a network of Ubuntu Linux computers which are connected via an ethernet switch.

Setup a server on one computer

iperf -s

Run a client on a second computer. The client will send data to the server.

iperf -c SERVERIP -i 1

Option -i will show intermediate bandwidth results every second.

In the following example an additional 100 ms delay to the outgoing traffic is added.

sudo tc qdisc add dev eth0 root netem delay 100ms

Show the current traffic shaper

tc qdisc show

Remove a traffic shaper

sudo tc qdisc del dev eth0 root

For limiting the bandwidth a token bucket shaper can be used. In the example below the packet is delayed by 100 ms with the netem shaper and the goes to the token bucket shaper. See http://lartc.org/howto/lartc.qdisc.classless.html#AEN690.

sudo tc qdisc add dev eth0 root handle 1: netem delay 100ms
sudo tc qdisc add dev eth0 parent 1:1 handle 10: tbf rate 256kbit buffer 1600 limit 300
tc -s qdisc

Ethernet card parameters can be configured with ethtool.

fritz@ubuntu:~$ ethtool eth0
Settings for eth0:
	Supported ports: [ TP ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: Yes
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	Advertised pause frame use: No
	Advertised auto-negotiation: Yes
	Speed: 1000Mb/s
	Duplex: Full
	Port: Twisted Pair
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: off
Cannot get wake-on-lan settings: Operation not permitted
	Current message level: 0x00000007 (7)
			       drv probe link
	Link detected: yes

ethtool DEVICENAME will show you the current settings of the ethernet card. In the example above the link speed is for example 1GBit/s. In order to modify the speed settings and set it to 100 MBit/s full duplex mode do:

sudo ethtool -s eth0 speed 100 duplex full

Modern network cards can do TCP segmentation in hardware. This is called TCP segmentation offloading (TSO), which offloads the packet segmentation from the cpu. To show the offload parameters do:

fritz@ubuntu:~$ ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: off
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off

In the example above the tcp segmentation offloading is activated. This will result in wireshark showing longer TCP packets than allowed according to Maximum Transmit Unit size (MTU) which is typically 1500 Bytes. You can check the MTU value with “ifconfig”.

In order to switch TSO offloading off, you can do

sudo ethtool -K eth0 tso off

The tcp_probe module can trace the state of a tcp connection.

sudo modprobe tcp_probe port=5001
sudo chmod a+r /proc/net/tcpprobe 

In order to store the log data in file test.log:

cat /proc/net/tcpprobe > test.log 

tcp_probe only adds a log entry, when the contention window size changes. In order to have a log entry for every received packet

sudo modprobe tcp_probe port=5001 full=1

The Ubuntu 12.04 LTS is based on linux kernel 3.11. The tcp_probe kernel module source code can be viewed on the linux kernel git server.

To unload the module:

sudo modprobe -r tcp_probe

The log file from tcp_probe contains lines with the log information

1           2                    3                   4  5          6          7  8          9     10
7.768727538 32 0xe656e892 0xe6568e12 16 2147483647 42248 1

1: Time in seconds
2: Source IP:Port
3: Destination IP:Port
4: Packet length in bytes
5: snd_nxt: Sequence Number of the next packet to send
6: snd_una: Sequence Number of the first unacknowledged packet 
7: snd_cwnd: Contention Window 
8: ssthr: Slow Start Threshold (-1 => not known yet)
9: snd_wnd: Send Window size in Bytes
10: srtt: Averaged round trip time estimation

The data from the log file can be plotted with gnu plot

gnuplot> plot "test.log" using 1:7 title "cwnd", "test.log" using 1:($8 > 20000000 ? 0 : $8) title "ssthr"

Local memory restrictions

Set the maximum window sizes for tcp connections to 512000 Bytes.

sudo sh -c "echo 512000 > /proc/sys/net/core/wmem_max"
sudo sh -c "echo 512000 > /proc/sys/net/core/wmem_default"
sudo sh -c "echo 512000 > /proc/sys/net/core/rmem_max"
sudo sh -c "echo 512000 > /proc/sys/net/core/rmem_default"
sudo sh -c "echo 512000 512000 512000 > /proc/sys/net/ipv4/tcp_wmem"
sudo sh -c "echo 512000 512000 512000 > /proc/sys/net/ipv4/tcp_rmem"

Initial TCP contention window size

The following section shows how to change the initial contention window size to to 1 MSS.

fritz@ubuntu:~$ ip route show
default via dev eth0  proto static dev eth0  scope link  metric 1000 dev eth0  proto kernel  scope link  src  metric 1 
fritz@ubuntu:~$ sudo ip route change default via dev eth0 proto static initcwnd 1
fritz@ubuntu:~$ ip route show
default via dev eth0  proto static  initcwnd 1 dev eth0  scope link  metric 1000 dev eth0  proto kernel  scope link  src  metric 1 

TCP Congestion avoidance algorithm selection

The linux kernel provides several algorithms for tcp congestion avoidance. The following code shows how to figure out the available algorithms (here cubic and reno) and set and control the setting. In this example the congestion avoidance algorithm is switched from “cubic” to “reno”.

fritz$ cd /proc/sys/net/ipv4/
fritz$ cat tcp_available_congestion_control 
cubic reno
fritz$ cat tcp_congestion_control 
fritz$ sudo sh -c "echo reno > tcp_congestion_control"
fritz$ cat tcp_congestion_control 
  • mscom_network_start.txt
  • Last modified: 2016/03/16 09:01
  • by beckmanf