Nice to see you're jumping on the packet analysis bandwagon. It's impossible to do a troubleshooting course in one answer, but I'll try to give some hints on where to start.
- Make a network diagram of all (IP) systems involved in the communication (you were describing processes, it's another view and possible the (IP) systems follow quite neatly, but still)
- Now add information to that diagram how they are connected (all switches, routers, firewalls, etc), this could be complex (routing and firewalls involved)or as simple as they are all connected to the same switch and in the same IP subnet.
- Determine best points for capture.As a start, close to the involved systems would be perfect. Use spanports rather than capturing on a system itself, as there is some processing between the network card and the point at which a capture is made. Even better is to use TAPs, but that involves buying extra hardware, so maybe not the best way to start.
- From your description, capturing close to the system running the Laser software would be a good choice (is that where you made the 20 min capure with 150k packets?)
- In the capture, you need to start looking for patterns. Is the communication over UDP or TCP? What does a normal OK look like, what does a NOK look like. Can it be found in text, if so, filtering on
tcp contains "NOK" (or another string that matches failed attempts) could work for you (please note that 'contains' is a case sensitive filter) - Assuming the traffic is on TCP, do a
follow TCP stream on the packet that you identified as a NOK response to see the full exchange of data that led up to the NOK. - This also filters the packet list to this specific TCP conversation, so look at the delta times between packets (you might want to add an extra column in the preferences for
Delta time displayed). Also look for lines marked in black with red text, those are usually things that did not go well, like packet loss, retransmissions etc.
Good luck with the analysis. If you get stuck, you could share a pcap file and post the link here to get some help analyzing it. Do be aware of privacy concerns as indicated in the blogpost.
