Network testing with dd and netcat

Posted by Pavlo Khmel on Sun 23 December 2012

Network testing without special tools like iperf, netperf and other.
On 10 GigE NICs single dd + netcat are limited by one CPU core.
Also default MTU 1500 create additional load for CPU. Enabling jambo frames, MTU 9000, better transfer performance for big files.

Single thread

#host A
nc -v -l -n 2000 >/dev/null &

#host B
dd if=/dev/zero bs=1000M count=10 | nc -v -v -n b.khmel.org 2000
10485760000 bytes (10 GB) copied, 18.2118 s, 576 MB/s

Multi-threads

# host A
nc -v -l -n 2000 >/dev/null &
nc -v -l -n 2001 >/dev/null &
nc -v -l -n 2002 >/dev/null &
nc -v -l -n 2003 >/dev/null &

# host B
dd if=/dev/zero bs=1000M count=10 | nc -v -v -n b.khmel.org 2000 & 
dd if=/dev/zero bs=1000M count=10 | nc -v -v -n b.khmel.org 2001 & 
dd if=/dev/zero bs=1000M count=10 | nc -v -v -n b.khmel.org 2002 & 
dd if=/dev/zero bs=1000M count=10 | nc -v -v -n b.khmel.org 2003 &
10485760000 bytes (10 GB) copied, 33.6998 s, 311 MB/s
10485760000 bytes (10 GB) copied, 34.6415 s, 303 MB/s
10485760000 bytes (10 GB) copied, 36.3531 s, 288 MB/s
10485760000 bytes (10 GB) copied, 37.0467 s, 283 MB/s

More threads and total bandwidth is near NIC limits.

Additional options can improve performance

nice -n -20 # higher process priority
taskset -c 15 # binding process to a CPU core, better performance.

Example:

# host A
nice -n -20 taskset -c 15 nc -v -l -n 2000 >/dev/null &

# host B
nice -n -20 taskset -c 12 dd if=/dev/zero bs=1000M count=10 | nice -n -20 taskset -c 14 nc -v -v -n b.khmel.org 2000 &