Out-of-order EtherCAT frames with identical timestamps

asked 2025-11-19 06:53:50 +0000

Gadworx
1 ●1 ●1

updated 2025-11-19 21:13:06 +0000

I am using Wireshark on a Windows 11 PC, capturing OP cyclic frames from a TwinCAT PLC program acting as the EtherCAT master. The network has only two devices, this Master and one additional slave. Wireshark is running on the same Windows 11 PC.

The issue is random frames logged out of order and with identical timestamps Here is a snippet from a log showing this. Notice frames 5616 through 5619:

----------------------------------------
Frame | Time     | Source
----------------------------------------
5609  | 2.793484 | 10GtekTransc_22:21:a3  <--Good: Transmitted from Master
5610  | 2.793493 | BeckhoffAuto_01:00:00  <--Good: Paired with frame 5609 returned to Master
5612  | 2.794567 | 10GtekTransc_22:21:a3  <--Good: Transmitted from Master
5613  | 2.794576 | BeckhoffAuto_01:00:00  <--Good: Paired with frame 5612 returned to Master
5614  | 2.795562 | 10GtekTransc_22:21:a3  <--Good: Transmitted from Master
5615  | 2.795570 | BeckhoffAuto_01:00:00  <--Good: Paired with frame 5614 returned to Master

5616  | 2.796880 | 10GtekTransc_22:21:a3  <--Issue: From Master, has same Timestamp as #5617
5617  | 2.796880 | 10GtekTransc_22:21:a3  <--Issue: From Master, has same Timestamp as #5616
5618  | 2.796891 | BeckhoffAuto_01:00:00  <--Issue: To Master, same Timestamp as #5619
5619  | 2.796891 | BeckhoffAuto_01:00:00  <--Issue: To Master, same Timestamp as #5618

5620  | 2.797946 | 10GtekTransc_22:21:a3  <--Good: Transmitted from Master
5621  | 2.797953 | BeckhoffAuto_01:00:00  <--Good: Paired with frame 5620 returned to Master

Frames 5616 and 5618 are a matching TX/RX pair. Frames 5617 and 5619 are also paired frames. All the frames have unique payloads, so the frames can be easily matched. This example is repeated throughout the log with these anomalies occurring ~0.91% of the time (about 3,808 times out of over 150,000 frames captured).

I have read some other posts here, but none specifically like this, especially with EtherCAT where frames can only be in specific orders and period timing.

What are possible explanations for these out-of-order frames and the identical timestamps?

edit retag flag offensive close merge delete

Comments

Wireshark and presumably npcap versions?

grahamb ( 2025-11-19 09:37:57 +0000 )edit

The issue is random frames logged out of order and with identical timestamps

How is the log generated?

Chuckc ( 2025-11-19 10:13:53 +0000 )edit

Hi! Wireshark: Version 4.6.0 (v4.6.0-0-gcdfb6721e77c). Npcap 1.83, libpcap 1.10.5

Gadworx ( 2025-11-19 11:44:22 +0000 )edit

The log is a standard Wireshark log that I exported as CVS into Excel for the post.

Gadworx ( 2025-11-19 11:45:39 +0000 )edit

logged out of order

So Wireshark isn't flagging them as out of order, it's just that the log is not showing the tick tock of alternating rx and tx packet?

The Network Capture Playbook Part 1 – Ethernet Basics

If you can live with the distorted packets, do a local capture. If you can’t, and you need your capture to be precise (which is especially true for most troubleshooting cases where timings or connection aborts are involved), you need to do something better.

(The Network Packet Capture Playbook)

Chuckc ( 2025-11-19 11:51:07 +0000 )edit

see more comments

answered 2025-11-19 21:09:53 +0000

Gadworx
1 ●1 ●1

I believe this to be the culprit, but not the underlying cause:

A sanity check first…

In EtherCAT, the network cabling is daisy-chained and looped back to the Master. For every frame the Master transmits, it should receive this frame back before any other frames transmitted by the Master are received back--they will always be in order. It should be impossible for a Master to transmit two frames onto the network, and then receive the frames back in the wrong order.

An EtherCAT slave cannot initiate any frames onto the network but only can modify a passing frame as it 'flies' by, so the slave cannot change the order.

So what is probably happening is this:

Under normal expectations, the Master transmits the frame at 0ms. The return frame is received about 11 microseconds later (~0.01ms). The Master pauses for ~0.99ms and then transmits the next frame.
The Master (here TwinCAT running on Windows 11) falls behind, possibly by other processes activating in the system.
Being behind, the Master misses a scheduled frame to transmit in the given period and the frame is stuck in queue.
In the next period, the Master transmits the queued frame and the scheduled period's current frame as well. This results in two frames transmitted back-to-back from the Master with identical timestamps.
With the two Master frames in transit, they both return about ~11us later as two responses that are also back-to-back with identical timestamps.

The log shows the timestamps in only microsecond precision, while the actual timestamp is probably in nanosecond ranges which would falsely give the appearance of identical timestamps that are actually off by nanoseconds.

These steps would explain the log anomaly, but not the specific cause of the PC Master lagging. I will continue investigating this avenue and give updates.

In the meanwhile, I very much appreciate suggestions, corrections, and alternate ideas!

edit flag offensive delete link

Comments

Are you able to use Linux, macOS, or some other operating system other than Windows to do the capture? Higher resolution (not nanosecond in reality, actually 100 nsecs because that's what Windows has) will be available soon in the next release of libpcap, but Linux, macOS, and other OSes have had nanosecond timestamp resolution for years.

johnthacker ( 2025-11-19 22:55:19 +0000 )edit

add a comment

Out-of-order EtherCAT frames with identical timestamps

Comments

1 Answer

I believe this to be the culprit, but not the underlying cause:

Comments

Your Answer

Question Tools

Stats

Related questions

Out-of-order EtherCAT frames with identical timestamps edit

Comments

1 Answer

I believe this to be the culprit, but not the underlying cause:

Comments

Your Answer

Question Tools

Stats

Related questions

Out-of-order EtherCAT frames with identical timestamps