What Is The Endianness of Captured Packet Headers?

asked 2020-03-31 00:05:06 +0000

trist
5 ●4 ●4 ●6

updated 2020-03-31 00:06:20 +0000

Hello,

I read here that network byte order is big-endian for TCP. This is a protocol-level property.

Two questions regarding endianness in capture files:

Is it correct that captured packet headers are written in the byte order of the host that wrote the file? In other words, what determines the endianness of the headers in a frame?
Building off #1, is it possible that, while a protocol-level property of TCP is big-endian, there is no guarantee that a packet that I inspect on Wireshark will have TCP headers that are written in big-endian?

For example,

The Ethernet header here displays type: IPv4 in big-endian (and so do the other headers). But this may not always be the case?

image description

Thanks!

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2020-03-31 04:23:22 +0000

Guy Harris
19890 ●3 ●633 ●207

network byte order is big-endian for TCP

"Network byte order" means big-endian, so it's big-endian everywhere.

Confusingly, it does not mean "the byte order for all fields in network traffic", so not all network traffic uses big-endian byte order.

TCP is a protocol that uses "network byte order", i.e. big-endian byte order, for multi-byte integral values. IPv4, IPv6, and UDP are other protocols that do.

However, SMB, the IBM/Intel/Microsoft file sharing protocol, originally ran directly on top of LAN protocols. It was mainly for IBM-compatible PCs, which have x86 processors, and those processors are little-endian, so multi-byte integral values in SMB packets are little-endian.

SMB can also run over TCP (and UDP), so that's a protocol using little-endian byte order running on top of protocols using big-endian byte order.

Is it correct that captured packet headers are written in the byte order of the host that wrote the file?

No. The packet data is a raw array of bytes, exactly as they were transmitted on the network.

For example, the Ethernet type field (the "08 00" field you've put in red) and the TCP port number field are always in big-endian order in packets, regardless of whether the machine that wrote the capture was little-endian or big-endian.

However, fields in SMB packets are little-endian, regardless of whether the machine that wrote the capture was little-endian or big-endian.

And there are some protocols - for example, the X11 display protocol and the DCE RPC/MS-RPC protocols - that can use either byte order; the byte order of the machine sending the packet is used. X11 negotiates the byte order at startup time; DCE RPC/MS-RPC puts an indication of the byte order into the packet header. However, "network byte order is big-endian for TCP" means that "multi-byte integral values in TCP headers are big-endian", and "This is a protocol-level property." means it's true of TCP but is not necessarily true of other protocols - including protocols running on top of TCP!

is it possible that, while a protocol-level property of TCP is big-endian, there is no guarantee that a packet that I inspect on Wireshark will have TCP headers that are written in big-endian?

No, it is not; there is such a guarantee (if it's not big-endian, it's not valid TCP).

However, there's no guarantee that fields in the protocol running on top of TCP are big-endian; if it's SMB-over-TCP, for example, the fields in the SMB packet will be little-endian, as per the above.

And the CME Group's iLink 3 Binary Order Entry protocol's Simple Binary Encoding encodes the Simple Open Framing Header in little-endian.

(It's "CME", all caps, which used to stand for Chicago Mercantile Exchange, which is one of the financial exchanges they run.)

So your packet has some big-endian fields and some little-endian fields in it.

(If anyone's curious, the dissectors being used there are probably the Lua dissectors from the Open Market Initiative.

edit flag offensive delete link

Comments

Wow, thank you for such a detailed and easy-to-follow answer. You never stop impressing me, Guy!

trist ( 2020-03-31 13:48:13 +0000 )edit

add a comment

answered 2020-03-31 05:15:47 +0000

cmaynard

11115 ●11 ●317 ●165 https://www.igt.com/

updated 2020-03-31 15:54:20 +0000

In most cases, the specification for the protocol, and not the host, determines the byte order for the particular protocol, so not all layers will necessarily have the same byte order.

For Ethernet, according to the 802.3-2019 - IEEE Standard for Ethernet, Section 3.2.6 Length/Type field (the field you highlighted from the Ethernet frame) clearly indicates the following regarding the byte ordering:

For numerical evaluation, the first octet is the most significant octet of this field.

In other words, "Big Endian".

Regarding TCP, the protocol is defined in RFC 793, but strangely, it's not clearly stated that "Big Endian" is the byte order for this protocol, although RFC 1700 does clarify this in the INTRODUCTION under Data Notations:

Data Notations

The convention in the documentation of Internet Protocols is to
express numbers in decimal and to picture data in "big-endian" order
[COHEN].  That is, fields are described left to right, with the most
significant octet on the left and the least significant octet on the
right.

[COHEN], by the way, references the October 1981 IEEE Computer Magazine article written by Danny Cohen. titled, "On Holy Wars and a Plea for Peace", which is an interesting read in itself.

So again, in almost all cases, the specification will define the byte order. One notable exception to this rule is ICMP, which is defined in RFC 792. It clearly states in the Introduction of the RFC that:

             ICMP, uses the basic support of IP as if it were a higher
   level protocol, however, ICMP is actually an integral part of IP, and
   must be implemented by every IP module.

Being that ICMP "is actually an integral part of IP", one would presume that it would use "Big Endian" as well, since the Internet Protocol, defined in RFC 791 very clearly specifies the data transmission order of bytes that way in APPENDIX B: Data Transmission Order. Unfortunately this isn't the case in practice. Some operating systems, notably Windows, uses "Little Endian" for ICMP. If you ever look at ICMP packets in Wireshark, it's the reason you'll see some fields (i.e., identifier and sequence number) displayed in both byte formats, because Wireshark has no way of knowing with absolute certainty which byte order was used. At least one other OS's pinger didn't add all ICMP fields as "Big Endian" either. I mentioned this in Comment 21 of an old Bug 5770, which I'll leave you to read if you wish.

In any case, I believe byte order variation, such as with ICMP, is very rare, at least for published protocols. I have encountered protocols used internally where host byte order was employed, but this is just poor design, and the only ways to figure out which byte order was used when dissecting packets of this type is to guess using heuristics or allow the user to explicitly choose the byte order.

So, to answer your final question, "is it possible that, while a ... (more)

edit flag offensive delete link

Comments

And even in the case where the byte order is the host byte order, that's typically the order of the host sending the packet, which isn't necessarily the order of the host capturing the packet.

And lastly, I don't know anything about the "Cme Futures iLink3 Sb3" protocol, nor would I know where to find the protocol specification for it, but somewhere in that document it must state that byte transmission order of the protocol, which is presumably "Little Endian", assuming whoever wrote the protocol dissector for it did so correctly.

Yes, it's little-endian. See my answer (and pointers to the Lua dissector that I suspect is being used here).

Guy Harris ( 2020-03-31 05:29:51 +0000 )edit

Thank you for your detailed response Cmaynard. Yes, I know what the endianness of CME iLink3 is. I was just wondering whether the headers of the transport, datalink, and network layers can be written in the way the host machine likes them, and you've answered that for me. Cheers!

trist ( 2020-03-31 13:54:02 +0000 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

What Is The Endianness of Captured Packet Headers?

2 Answers

Comments

Comments

Your Answer

Question Tools

Stats

Related questions

What Is The Endianness of Captured Packet Headers? edit

2 Answers

Comments

Comments

Your Answer

Question Tools

Stats

Related questions

What Is The Endianness of Captured Packet Headers?