When troubleshooting connectivity problems between hosts, one of the most effective tools is the packet capture. Software products like Wireshark and Microsoft's Network Monitor can quickly point you in the direction towards resolution in a relatively short time, but security-conscious IT organizations across a multitude of industries and markets are systematically putting the "kaibosh" on installing packet capture software on servers...which is typically where it needs to go.
Getting Around Without It
There are a couple of ways to get around the "no packet capture on my server" problem. The more involved approach involves creating a SPAN port configuration on a Cisco switch, and attaching a workstation with capture software to the SPAN port. But this requires getting network people involved, complicated or untimely request approvals, and other very inefficient activities. So, onto the leaner, meaner, more efficient way: netsh.
Using Netsh for Packet Capture
Netsh is a powerful command line utility that was introduced by Microsoft with Windows 2000 (where it was part of the Resource Kit; it has since just been installed with the operating system). At its most basic, netsh lets you look at network configurations on both local and remote systems, but it also lets you reconfigure, backup, and restore configurations as well. It also lets you capture traffic, and that's what we're going to learn about today.
If you go to Google and just type "using netsh" you will find a host of returns that showcase the utility's interface and Windows Firewall management capabilities, but very few of them talk about Trace. To begin interacting with Trace, go into an administrative command prompt (or Powershell prompt) and type netsh followed by the ENTER key. Once in the netsh shell, type trace and then the ENTER key to go into the Trace context.
Once inside, you can start to set up your trace. In general, it is bad form (unless you just need to snag a capture and get out of there) to capture everything. Typically, you only need to see the packets from either a specific host or small group of hosts, or across a specific TCP or UDP port. There are several capture filters available, and all of them can be reviewed by typing show capturefilterhelp in the Trace context. A helpful starter command within Trace is:
netsh trace start capture=yes IPv4.Address=192.168.122.2 protocol=6 tracefile=c:\temp\tcptrace.etl
This trace has two filters:
- IPv4.Address. Specifies the IP address for capture: 192.168.122.2. Specified this way, the IP address can either be the Sender or the Receiver.
- Protocol. These are the available protocols that are present in the capture. "6" indicates all TCP protocol traffic. Popular Protocol types are 1 (ICMP), 6 (TCP), and 17 (UDP). Note that Trace Capture does not filter by port.
It also directs the trace output to a file, tcptrace.etl, in C:\Temp. The directory structure (not the file) must be present prior to running the command, or it will not capture to file.
If you need to continue trace between sessions, use the persistent=yes command in the trace. This will persist tracing, even upon reboot of the host. This is helpful when attempting to troubleshoot things like DHCP addressing problems. To close the capture, type:
netsh trace stop
Advanced: Capturing on a Remote Host
Sadly, there is no way to use netsh's "remote" (-r hostname) option to capture traffic on the server from the safety and security of your Windows desktop machine. If you enable the remote context, typing "trace" will just give you an error. But with a little tenacity, there is always a way, and that way is to use SysInternal's (a tiny little division of Microsoft) PSExec utility, part of the PsTools suite. Once downloaded, the basic syntax from the local computer is:
psexec \\hostname netsh trace start capture=yes tracefile=c:\temp\tcptrace.etl
On the local computer, you will get a message saying that the netsh command exited with "error code 0" which means that it exited successfully. When the operation you're hoping to review is completed, type:
psexec \\hostname netsh trace stop
The log file will be saved to C:\Temp on the remote system.
Analyzing the Trace File
To read the produced ETL file, you can simply use Microsoft Message Analyzer or Microsoft Network Monitor. If you use Network Monitor, you must go to Tools > Options and set the Windows parser as "active". Then, just use those tools' options to further filter (for example, looking for TPC 515 traffic) and see where the problem is coming from.
If you want to use Wireshark, you have to convert the ETL file to the CAP format. You can do this in PowerShell:
$s = New-PefTraceSession -Path "C:\output\path\spec\OutFile.Cap" -SaveOnStop
$s | Add-PefMessageProvider -Provider "C:\input\path\spec\Input.etl"
$s | Start-PefTraceSession
and then open the CAP file in Wireshark.
And that's it! Happy capturing!
As a reference, and a launching point for more "Fun With Trace" start at Netsh Commands for Network Trace.