issue with Mellanox NIC on recent Ubuntu
This is not a wireshark issue.
Wireshark 2.x works with Mellanox ConnectX-4/ConnectX-5 in my Ubuntu 17.10 (kernel 4.13) PC.
However, it doesn't work in Ubuntu 18.04 (kernel 4.15) or Ubuntu 18.10 (kernel 4.18). It can capture the normal IP packets. But it won't show any RDMA (RoCEv2) packets. I also tried tcpdump and it shows the same issue. But once I replace Melloanox NICs with a QLogic NIC (41000 series), then wireshark is able to capture all the packets.
I am using inbox driver for all these NICs. There are technical reasons that prevents me from moving to Mellanox OFED driver.
I contacted Mellanox. But they didn't help and they disabled my account. :-(
So I am hoping I can get some help here.
Unlikely to be a Wireshark issue, more likely to be the NIC drivers or libpcap which Wireshark uses to capture the traffic.
As mentioned in my 1st post, this is not a wireshark issue and I do think this is a Mellanox driver issue. I just hope some wireshark users have experience as to how to get around this issue w/o using Mellanox OFED driver.
My setup is
RoCE traffic is sent between PC1 and PC2. Wireshark is running in PC3. I configure the switch so that it makes a copy of all traffic between PC1/PC2 and forward them to PC3. So for PC3, these packets are not RDMA packets. They are just the normal IP packets, which shall be passed to the kernel by the NIC.
...which would mean that PC3 is plugged into the switch. Is that the case? I.e., are PC1, PC2, and PC3 all plugged into the switch, with the port into which PC3 is plugged being configured as a "mirror port" or whatever the switch vendor calls it?
Yes, all 3 PCs are connected to the switch. Somehow after I post the message, my message format was altered a bit and now it looks as if PC3 is not connected to the switch, which is certainly not true.
The feature in the switch I use is called "port monitoring" in the switch I use. This is pretty much the standard way to capture RDMA traffic.
I fixed your ASCII diagram by using the "code" format button.