"TCP Previous Segment not captured" occurs and the client stop reading the last few bytes from inputstream under high concurrency environment in android(java)
.jpg "vainquit"){kind=small}
(Updated: The SHA-1 values between the original images and the objects captured by wireshark are identical, so the issue is probably caused by the client side.)
I am writing an android app(written in Java) which will download images by POSTing http request to my SimpleHttpServer(written in Python).
The app POST text/json type data which includes some info about the requested image. After the server receives the request, it reads target images as bytes and continually writes the bytes into the established http pipe, so the app can read from its inputstream and write the bytes read to the local file.
Actually, the app doesn't only request one image.It sends multiple http requests to the server for many texts and images at a time.Under this high concurrency environment, some images cannot be properly downloaded. The error "java.net.ProtocolException: unexpected end of stream" occus, which basically means that the app read less bytes from inputstream than it should. For example, an image size has 20000 bytes (I get the file size from Content-Length in server response headers), and the app often only reads 19997(or 19998,19990,etc....) bytes, and then throws the exception. But if there are not many http requests at the same time, the bug will likely not happen.
I catch the packets both from server and the app with Wireshark (the server is deployed on a remote Windows Server 2008 R2). I get a "TCP Previous Segment not captured" warning from app side, every time, so I belive that's the breakpoint. But what should I do next to analyze the packages?
Download address: http://118.89.144.241/index2.html (The port 6000 is the port where app communicate with the server, and the port 6000 always represents the server side)
(By the way, if I run the server on my local machine, and use wireshark to capture the packets on 127.0.0.1:6000, the problem still remains but the "TCP Previous Segment not captured" warning on wireshark disappear, and I can not find any other packets that seem to cause the problem. I have uploaded the packets monitored from loopback address as well)
Is there a spot "on the wire" to capture after the server NIC has turned the large packets into network size packets?
Hi, I guess it's not the proxy's case. I have disabled the proxy, and also tried to run the server and app both on a local machine (the server is listening on 127.0.0.1:6000), but the issue still remains. And I have no idea about how to "capture after the server NIC has turned the large packets into network size packets", sorry ....
(link to the loopback capture is pointing to server_2 not loopback.pcapng)
In the loopback capture:
The graphics files all have their end of file markers:
That would seem to indicate the client is losing data.
Sorry, my fault~~ I have re-uploaded the loopback package....And yes, the client is losing data, not only at the downloading of images, but also at the requiring of texts sometimes, which results in an error parsing JSON format texts....
The JPEG and PNG end of file markers are all present in client_1 and client_2 captures.
You could export them (
File -> Export Objects -> HTTP...) and compare checksums/hashes to the original files on the server just to make sure they are making it intact from the server.If the files are complete in the capture then aren't the bytes being dropped farther up the stack or in the client app?
You might rework the question title to include Java.
That's really a good way to determine which side cause the issue! I have compared the SHA-1 values between the original images and the exported objects, and they are identical.So now the problem is basically caused by the client side. I have updated the question.