Section 4: TCP/IP Stack Implementation

The Big Picture: The Brain of Windows Networking

Everything you learned in the previous chapter about IP, TCP, UDP, ARP and routing is carried out by one kernel driver: tcpip.sys. This section looks at how it works and how to peek at its state.

4.1 The tcpip.sys Driver

What it is: tcpip.sys is the TCP/IP protocol driver. It lives in C:\Windows\System32\drivers and runs in kernel mode (the full-power world from Section 2). It loads very early at boot.

What it handles:

Settings you can look at. TCP has tuning options. To see Windows' global TCP settings, run (this only reads):

netsh interface tcp show global

Some lines you may see:

Offloads (hedge). Many NICs can do some of the TCP/IP work themselves, so the CPU does less. Two common ones:

• Checksum offload: the NIC calculates and checks the packet checksums.
• Large Send Offload (LSO): the stack hands the NIC one big chunk and the NIC cuts it into frames.

You can see which offloads your adapter offers in Device Manager → adapter → Advanced tab. Turning them on or off is a tuning and troubleshooting topic. Leave them at their defaults unless you know why you are changing them.

Two tables the stack keeps:

1. The neighbor cache (the ARP cache): recent IP-to-MAC pairs on your local network. View it with Get-NetNeighbor or arp -a.
2. The routing table: where to send packets for each destination (Section 9).

Statistics: netstat -s prints counters for IP, TCP, UDP and ICMP: packets sent and received, errors, retransmitted segments (TCP had to send data again, a sign of a lossy network), and more. Get-NetIPConfiguration and Get-NetIPAddress show the addresses the stack is using.

Windows tip: A rising number of segments retransmitted in netstat -s on a PC with a bad Wi-Fi signal is the stack quietly doing its reliability job.

4.2 Network Stack Processing of Inbound and Outbound Packets

Outbound (sending). For each packet leaving, the stack asks a few questions in order:

1. Which interface and next hop? Look in the routing table (Section 9).
2. What is the next hop's MAC? Look in the neighbor cache. If missing, send an ARP request and wait for the answer.
3. Is it allowed? WFP filters (the firewall) get a say here (Section 7).
4. Build the frame and hand it to NDIS and the miniport.

Inbound (receiving). For each packet arriving:

1. The NIC and miniport pass the frame up through NDIS.
2. tcpip.sys checks the IP header: is the destination my address? Is the checksum right?
3. WFP filters inspect the packet (the firewall may drop it).
4. The transport step looks at the port number. For TCP it finds the connection it belongs to, puts the data in order, sends an acknowledgement, and gives the data to AFD. For UDP it simply hands the message to the socket on that port.
5. AFD wakes the waiting program (recv returns).

   OUTBOUND                              INBOUND

   Application                           Application
       │ send()                              ▲ recv()
       ▼                                     │
   AFD (socket buffers)                  AFD (socket buffers)
       │                                     ▲
       ▼                                     │
   tcpip.sys: TCP/UDP -> IP              tcpip.sys: IP -> TCP/UDP
     route lookup, neighbor cache,         checks, port lookup,
     WFP filtering                         WFP filtering
       │                                     ▲
       ▼                                     │
   NDIS -> miniport -> NIC               NIC -> miniport -> NDIS
       │                                     ▲
       └──────────────► wire ────────────────┘

Why the stack uses many cores. One busy connection might be handled on one core while another is on a different core. RSS (Section 1) helps by letting the NIC choose the core for each connection. All packets of one connection go to the same core, which keeps them in order.

Time spent: Each packet needs a little CPU. Offloads and RSS exist to keep that cost low on fast links (10 Gbps and beyond).

Summary