This blog series consists of four parts
[NIC Teaming, Hyper-V switch, QoS and actual performance part 1 – Theory](/2013/08/28/bmd1)
[NIC Teaming, Hyper-V switch, QoS and actual performance part 2 – Preparing the lab](/2013/08/30/bmd2)
[NIC Teaming, Hyper-V switch, QoS and actual performance part 3 – Performance](/2013/09/22/bmd3)
[NIC Teaming, Hyper-V switch, QoS and actual performance part 4 – Traffic classes](/2013/10/05/bmd4)
Preparing the lab
The test lab consist of two servers (HP Proliant DL 360 G5, nothing fancy but it will give a good picture on the processor demand). Each server contains a dual port 10Gb NIC and a quad port 1Gb NIC. The NICs have RSS and VMQ support. The quad port 1Gb NIC in the servers are directly connected to each other. This will give the best picture since a switch configuration might interfere with the results.
Performance is influenced by a lot of factors. For example, copying a large file between the servers will not be very representative. Server 2012 supports SMB multichannel, whereby multiple TCP streams are used for a single file copy. This requires Physical NICs with RSS support. SMB multichannel will work with NIC teaming since RSS is exposed through the team on the default interface. The Hyper-V switch does not support RSS and does not expose it to upper level protocols. SMB Multichannel will not function for the vNICs. A file copy initiated from a vNIC is single TCP stream. NIC Teaming is designed a single TCP stream to assign to a single team member. When the file is written to the destination, disk I/O can also impact the performance.
Luckily there are some good tools available for measuring bandwidth. During the tests JPerf will display detailed information on the bandwidth, Performance Monitor shows the load distribution and Task Manager gives insight into the processor load and distribution.
NTttcp, IPerf and JPerf
In my initial test I used NTttcp, that was rewritten by Microsoft in 2008. Microsoft is using an updated version of NTttcp that enables additional parameters, but this updated version is not publicly available. Therefore I resorted to IPerf. IPerf is a commonly used network testing tool that can create multiple TCP streams and measure the bandwidth of a network connection. IPerf can run as a server or as a client. The server listens on port 5001 and one or multiple clients can send a single TCP stream or multiple TCP streams to the server. IPerf was originally created for Linux, but there are compiled version for Windows publicly available. I have used a graphical front end for IPerf called JPerf. JPerf gives some nice graphs but requires Java so I wouldn’t recommend installing it on your production servers. If you want to run the same test in your production environment you can use the compiled version of IPerf (which will leave no footprint on the server) or create two virtual machines and install JPerf inside them.
If you want to use the command line version of IPerf (no footprint) copy the content of the compiled IPerf version to your server. For JPerf you will need to install Java first. JPerf does not require a separate IPerf file. You can just copy the content of JPerf to your server. Before you can run JPerf you will need to add the path to javaw.exe to the Path variable.
In the System Properties of your server open the Advanced tab and select the Environment Variables. Search for the Path variable and (if you installed Java in the default folder) add ;C:Program Files (x86)Javajre7bin to the end of Path variable.
Now you can open JPerf by running jperf.bat located in the root of folder you copied.
To configure JPerf as receiver select Server as IPerf Mode and click Run IPerf. IPerf listens on port 5001 by default. This port should be allowed in the firewall. With the Num Connections value of 0 IPerf will keep listening on port 5001 after a successful run.
To configure JPerf as sender select Client as IPerf Mode. In the server address specify the IP address of the server where JPerf is in listening mode. During the test I concluded that JPerf will only function on interfaces with a default gateway configured.
The number of Parallel Streams represent the number of TCP streams send during the test from the JPerf client to the JPerf server.
The client needs some additional settings configured. First of all enable compatibility mode. This will give more constant values for the individual streams. Select a reasonable transmit time (30 seconds should do fine) and select an output format that is easy to understand. To get an idea, a single 1Gb NIC adapter provides a bandwidth of about 113 MBytes / sec.
When you now select Run IPerf on the client, the program will sent a single TCP stream to the server for a period of 30 second and reports the bandwidth in MBytes every second.
Performance Monitor in Windows Server 2012 provides counters to monitor the Load distribution of the TCP streams on the Physical NICs in the LBFO team. The counter Bytes Total/Sec of the performance object Network Adapter will display the Total Bytes per second for an individual physical NIC.
When you open performance monitor (perfmon.msc) and select Performance Monitor in the left menu the % Processor time counter is enabled by default. You can remove or deselect this counter.
Add a new counter by clicking on the green plus icon. Search for the Network Adapter performance object and select the Bytes Total/sec. In the instances of selected object choose all the physical NICs that are part of the LBFO team.
Performance monitor provides several views. The best view for the tests is probably Histogram bar. Open the properties of the Performance Monitor, select the Graph tab and change the view to Histogram bar. In the same tab change the vertical scale maximum to 15000 for monitoring on 1Gb Physical NICs or 150000 for monitoring on 10Gb Physical NICs.
The new Task Manager in Windows Server 2012 comes with a lot of new functionality. The performance tab shows real-time information on individual processor load.
To enable Task Manager to display the processor load, just select the performance tab, select CPU in the left menu, right click the CPU activity and select change graph to Logical processors.
In the next part of this series we will look at the actual tests.