NDIS_PACKET

NDIS_PACKET Discussion
Part 1 - NDIS Packet Data

The purpose of this topic is to provide you with some insight into the relationship between a "packet" observed a network and a NDIS_PACKET that represents the same information within a NDIS driver.

 

One often sees questions like the following asked on the newsgroups:

What Microsoft documentation specifies a Windows 2000 network packet?

Does anyone there know where I can find the specification for win2k IP packet?

I want to know basically the structure of a NDIS_PACKET. For example, I like to know which part is for source IP address, which part for portnumber and real data etc.
 

These questions need to be separated into at least two separate questions:

1. Where can I find specifications for standard network protocols, like the Internet Protocol (IP)?
2. How is network data encapsulated within a NDIS_PACKET?

The answer to the first question is that the most widely used network protocols are specified by Standards Organizations (not by Microsoft at all...). A few examples of Standards Organizations are:

	ETSI - European Telecommunications Standards Institute
	IEEE - Institute of Electrical and Electronics Engineers
	IPv6 - IPv6 Forum
	ISO - International Standards Organization
	OMG - Object Management Group Open Group
	World Wide Web Consortium

The Internet Protocol (IP) is definitely the most widely used network protocol in the world. The specifications that define IP are maintained by the Internet Engineering Task Force (IETF). The IETF is a large open international community of network designers, operators, vendors, and researchers concerned with the evolution of the Internet architecture and the smooth operation of the Internet. It is open to any interested individual.

Information about the Internet Protocol (IP) can be found at the IETF website as .

You should visit the IETF website for information about other protocols and standards that it maintains.

Of course, IP is not the only network protocol in the world, and the IETF is not the only Standards Organization in the world. So, where can you find more information?

One handy site (if only because the name is easy to remember) is Protocols.com.  They publish a handy  that includes links about a wide variety of Protocols and Standards Organizations.

So, now that you have some pointers to the specifications for network packet data - what does the data look like "on the wire". And later - how is the data encapsulated in an NDIS_PACKET.

 

The next step is to actually understand what is observed on the network. The following is a HEX dump of a simple ICMP "ping" packet:

000000:   ...c...@.I._..E.
000010:  .<.G.. ...... ..
000020:   .@..H\....abcdef
000030:  ghijklmnopqrstuv
000040:                    wabcdefghi......

The ping was initiated with the command:

C:> ping 192.168.1.64

and sent the ICMP echo request with the default of 32 bytes of data. The total length of the ping packet is 74 bytes.

The dump does not include the leading frame preamble or the trailing FCS.

The packet was captured using the PCAUSA Rawether for Windows HookPeek sample application. HookPeek isn't a network monitor by a long shot, but it has its uses...

A partial decode of this packet is provided for reference.

[  ]

 

The information of interest is, of course, the packet data. Somewhere there are 74 bytes of virtual memory that represent the 74 bytes of data that is observed on the network. At first the mechanism used by NDIS to manage packet data may seem unnecessarily complicated. However, the basic mechanism is well thought out and provides a great deal of flexibility to a protocol writer.

 

Simple NDIS_PACKET Representation

In general it is best to think of the NDIS_PACKET structure (as well as the associated NDIS_BUFFER structures) as "opaque" - except for specific "reserved" fields that are discussed later. This means that the fields that are defined in the structure should not be directly accessed and that their definition will probably change from one NDIS version to another.

Nevertheless, it is useful to have a visual idea of these structures and how they relate to the packet data.

The diagram below illustrates the simplest way that an NDIS_PACKET can be used to encapsulate packet data:

 

Figure 1. Simple NDIS_PACKET Illustration

 

In this simple case, all 74 bytes of packet data are located in a contiguous array of 74 bytes.

Here is a description of this NDIS_PACKET:

The NDIS_PACKET has one chained NDIS_BUFFER. The NDIS_BUFFER describes a 74-byte range of virtual memory that contains the complete packet data.

 

A More Complex NDIS_PACKET Representation

The diagram below illustrates another way that a NDIS_PACKET can encapsulate the same packet data.

 

Figure 2. Multi-Buffer NDIS_PACKET Illustration

 

In this case packet data is found in two disjoint ranges of virtual memory.

Here is a description of the second NDIS_PACKET:

The NDIS_PACKET has two chained NDIS_BUFFER. The first NDIS_BUFFER describes a 14-byte range of virtual memory that contains the Ethernet header. The second NDIS_BUFFER describes a 60-byte range of virtual memory that contains the Ethernet payload.

It should start to become apparent that there are no safe assumptions on how a NDIS_PACKET will be constructed. The actual construction is at the sole discretion of the software that built the packet in the first place.

 

Although the diagrams above provide a helpful picture of how an NDIS_PACKET is used, you must recall that these structures are "opaque". You should never access the fields of these structures directly. Instead, you should use access functions provided by the NDIS wrapper library.

In the following topics we'll discuss a few of the NDIS wrapper library functions. The specific focus will be on the minimum functions that you need to examine and interpret packet data if you are given an NDIS_PACKET to inspect.

There are only a few NDIS functions that you need to inspect a NDIS_PACKET and the NDIS_BUFFERs that are chained to it:

NdisQueryPacket - Returns information about a given packet descriptor.
 

	BufferCount - Count of buffer descriptors chained to the packet descriptor.
	FirstBuffer - pointer to the initial buffer descriptor chained to the given packet descriptor.
	TotalPacketLength - The total number of bytes of packet data mapped by all chained buffer descriptors.

NdisQueryBuffer - Returns information about a given buffer descriptor.
 

	VirtualAddress - Base virtual address of the virtual address range described by the buffer descriptor.
	Length - Number of bytes in the virtual address range described by the buffer descriptor.

NdisGetNextBuffer - Returns the next buffer descriptor in a chain, given a pointer to the current buffer descriptor.
 

NDIS 5.1 Note: NDIS 5.1 (Windows XP) introduces new "safe" versions of several NDIS functions. The "safe" versions should be used for NDIS 5.1 drivers:

NdisQueryBufferSafe - The safe version of NdisQueryBuffer.

Here is a code snippet that uses NdisQueryPacket to examine the NDIS_PACKET pointed to by pNdisPacket:

   PNDIS_BUFFER    pCurrentBuffer;
   UINT            nBufferCount, TotalPacketLength;

   //
   // Query Packet
   //
   NdisQueryPacket(
      (PNDIS_PACKET )pNdisPacket,
      (PUINT )NULL,           // Physical Buffer Count
      (PUINT )&nBufferCount,  // Buffer Count
      &pCurrentBuffer,        // First Buffer
      &TotalPacketLength      // TotalPacketLength
      );

The code snippet fetches the count of NDIS_BUFFERs chained to the packet and the total length of the bytes of packet data mapped by all buffer descriptors chained to the packet. It also fetched a pointer to the first NDIS_BUFFER chained to the packet.

 

The NdisQueryBuffer function can be used to extract information from a NDIS_BUFFER, as illustrated below:

   PUCHAR    VirtualAddress;
   UINT      CurrentLength, CurrentOffset;

   //
   // Query The First Buffer
   //
   NdisQueryBuffer(
      CurrentBuffer,
      &VirtualAddress,
      &CurrentLength
      );

The snippet fetches the virtual address and length of the virtual memory described by the buffer descriptor.

We can manipulate the memory pointed to by VirtualAddress as an ordinary array of unsigned characters.

Understand, however, that all the packet data may not be described by the first buffer descriptor. If the NDIS_PACKET was the simple one illustrated in Figure 1 above, then CurrentLength would be 74 and all of the packet data would actually be at VirtualAddress.

If pNdisPacket pointed to the packet illustrated in Figure 2 above, then the situation would be different. The TotalPacketLength returned from NdisQueryPacket would be 74; however, theCurrentLength returned from querying the first buffer descriptor using NdisQueryBuffer would be 14.

This means that we would have to examine other buffers chained to the packet descriptor to see the rest of the packet data. At this point one would use NdisGetNextBuffer to fetch the next buffer descriptor, etc. This iterative process (sometimes called "walking the buffer chain") would continue as necessary to examine the packet data.

The UTIL_ReadOnPacket function illustrates the technique of "walking the buffer chain" to "read" a byte range on an arbitrary packet descriptor. The function is used by PCAUSA to safely "peek" into packet descriptors of unknown construction. It may not be the optimum function to use in all situations; for example, if one intended to copy larger regions of a network packet (e.g., IP data on network with large MTU), then a more efficient function would be in order.

[  ]

Here are some additional points to consider:

 

Building A NDIS_PACKET For NdisMIndicateReceive

There are bugs in the TCP/IP stack on Windows 2000 and higher that effect use of chained NDIS_BUFFERs when calling NdisMIndicateReceive:

 The TCP/IP ReceivePacket handler does not support multiple NDIS_BUFFERs (MDLs) if the virtual memory for what would be the LookaheadBuffer spans multiple NDIS_BUFFERs.

 You need to set the packet status to NDIS_STATUS_RESOURCES if you are indicating a packet up with more than one chained NDIS_BUFFER.

The first of these is a bug that has been promoted to be a design feature. One must insure header+lookahead bytes are in the first NDIS buffer descriptor for Windows 2000 and higher.

The second of these is a bug that is anticipated to be fixed in Windows Server 2003.

 

A NDIS_BUFFER Is a Memory Descriptor List (MDL)

Well, at least this statement is true on Windows NT, Windows 2000, Windows XP and later Windows versions.

A MDL is a fairly complex structure that describes pages in a virtual buffer in terms of physical pages. One can gain quite a bit of insight into the need for buffer descriptors by reading and understanding about Windows NT memory architecture.

On all Windows platforms the NDIS_BUFFER is essential to management of memory that eventually touches hardware devices such as NICs.

The use of this sort of structure at this level of device driver programming is not at all unique to Windows. On Unix there is the "mbuf" structure and on the Macintosh there is the "BDS" structure. If you are working on a virtual memory host, then there will be a structure similar to a MDL (a.k.a NDIS_BUFFER) used for a similar purpose.

On Windows NT and later Windows versions a highest-level NDIS driver can actually use MDL and NDIS_BUFFER interchangeably. In fact, you will see DDK samples where a MDL passed to a NDIS protocol driver (e.g., "packet" driver or "NDISPROT" driver samples) is simply chained to a NDIS_PACKET using NdisChainBufferAtFront.

 Only highest-level Windows protocols can use MDLs set up by still higher-level drivers as substitutes for NDIS_BUFFER-type descriptors.

 

Don't Modify Objects That You Don't Own

 Do not modify packet data virtual memory that you did not allocate your self.

 Do not modify or adjust a buffer descriptor that you did not allocate your self.

 Do not modify or adjust a packet descriptor that you did not allocate your self (except for the ProtocoReserved of MiniportReserved areas under specific situations).

If you are writing a filter driver that encrypts or compresses data, then you must must develop a method that does not modify or alter the originals. Generally, this means that you must either make your own copy of the data to be encrypted of compressed or perform these operations "on-the-fly".

NDIS 5.1 Note: NDIS 5.1 (Windows XP) introduces new functionality called "packet stacking". This is mechanism that allows a NDIS IM driver to pass a packet to an adjacent driver without having to allocate a "fresh" packet descriptor.