Casual bandwidth testing is easy enough to measure. Just wget a large file from some source on the Internet and you’ll get your results. For more precise measurements, however, this will not do. Some examples of when wget is insufficient are:
- You need to measure a machine’s download speed in relation to another machine in the network irrespective of the latter machine’s configuration/hardware (e.g. Apache config, disk speed)
- You need to measure a machine’s upload speed in relation to another machine in the network.
- You need to measure DL or UL rate for UDP
In these and other cases requiring precise measurement, iperf does the job very well. In my case, I needed to test a physical machine’s ability to throttle the bandwidth of the VMs it hosts. So our set up was to use iperf between a VM in the machine and another host in the network (not a VM in the physical machine, and not the physical machine).
Testing the VM’s Download Speed
First we need to run the iperf server on the VM since it will be the server that’s always on the receiving end.
$ iperf -s -f k
The -s argument runs iperf in server mode whereas the -f k argument ensures it uses kilobits for units. This last part is important because the networking convention is 1 Kilobyte = 1024 bytes but 1 Kilobit = 1000 bits and iperf follows this same convention.
Next, I simply run the client from the external machine:
$ iperf -c 10.14.18.3 -t 20 -f k
The -t 20 argument makes the client send chunks of data to the server for 20 seconds. When the test is complete, I get the following output on the server (VM) side:
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 10.14.18.3 port 5001 connected with 10.14.18.10 port 49745
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-45.7 sec 512 KBytes 91.7 Kbits/sec
…and on the client (external machine) side:
------------------------------------------------------------
Client connecting to 10.14.18.3, TCP port 5001
TCP window size: 23.5 KByte (default)
------------------------------------------------------------
[ 3] local 10.14.18.10 port 49745 connected with 10.14.18.3 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-42.0 sec 512 KBytes 99.8 Kbits/sec
Notice that we have two rates above. One for the server (91.7 Kbps) and one for the client (99.8 Kbps). The client rate shows how fast it is able to transmit the data to the lower OSI layers without regard for how fast the server is able to receive them. Since what we want to get is how fast the VM can receive data, we use the rate on its side which is 91.7 Kbps. Since we set the limit in the VM’s host machine at 100 Kbps, our test suggests that the limiter is working. For the sake of completeness, we could perform the test multiple times each in different bandiwth limits (either higher, or lower than 100 Kbps).
Testing a VM’s Upload Speed
This is easy enough. Just swap the location of the iperf server and client in the above example and use the speed on the client (VM) side.
Testing UDP
Just as the above, but add the -u argument.
Testing Against Multiple VMs
In another test scenario, I needed to check that the bandwidth limit defined on the host machine is shared among its VMs. For example, if the host machine had a limit of 1024 Kbps and two VMs started downloading/uploading simultaneously, their aggregate speed should not go over 1024 Kbps. To test this, I have to have two VMs running iperf in server mode, and I should send data to them simultaneously. To do this, I decided to give GNU parallel a try. First, I created a text file listing the VMs I want to test against:
10.14.18.3
10.14.18.4
Next, I ran iperf -s -f k in the above VMs. Then, from my external machine, I run:
$ cat target_vms.txt | parallel iperf -c {} -t 20 -f k
This yields the following screen in the first VM:
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 10.14.18.3 port 5001 connected with 10.14.18.10 port 50963
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-48.4 sec 256 KBytes 43.3 Kbits/sec
And on the second VM:
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 10.14.18.4 port 5001 connected with 10.14.18.10 port 50964
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-47.2 sec 256 KBytes 44.4 Kbits/sec
And on the external machine:
------------------------------------------------------------
Client connecting to 10.14.18.3, TCP port 5001
TCP window size: 129 KByte (default)
------------------------------------------------------------
[ 4] local 10.14.18.10 port 50963 connected with 10.14.18.3 port 5001
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-24.0 sec 256 KBytes 87.5 Kbits/sec
------------------------------------------------------------
Client connecting to 10.14.18.4, TCP port 5001
TCP window size: 129 KByte (default)
------------------------------------------------------------
[ 4] local 10.14.18.10 port 50964 connected with 10.14.18.4 port 5001
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-25.9 sec 256 KBytes 80.9 Kbits/sec
Again, because we are concerned about the aggreate download rate of the VMs, we will use the values on the server side which are 43.3 Kbps and 44.4 Kbps or a total of 87.7 Kbps which is well below the 100 Kbps limit. Also, notice that the clients both attempted to send data at 87.5 Kbps and 80.9 Kbps which further suggests that the bandiwdth limiter on the host machine is working. Further testing on different bandwidth limits should show appropriate results.