NetworkStream read skipping packet
Hi all,
I am trying to implement a TCPClient with an async reading. The code below is handling the reading:
IEnumerator ListenForData()
{
Debug.Log("Listening");
stream = client.GetStream();
cs = new ClientState(stream);
do
{
//stream.BeginRead(buffer, 0, buffer.Length, MessageReceived, client);
stream.BeginRead(cs.buffer, 0, cs.buffer.Length, new AsyncCallback(ReadCallback), cs);
yield return null;
} while (client != null);
}
void ReadCallback(IAsyncResult result)
{
ClientState cs = (ClientState)result.AsyncState;
lock (lockObj)
{
int currBytesReceived = cs.stream.EndRead(result);
Debug.Log("Network receiving: " + currBytesReceived + " " + cs.bytesReceived + " bytes. " + cs.buffer.ToString());
Array.Copy(cs.buffer, 0, cs.receiveBuffer, cs.bytesReceived, currBytesReceived);
cs.bytesReceived += currBytesReceived;
}
}
At a certain point when the server sends a series of messages, the bytes received does not match the actual bytes sent:
Wireshark log:
No. Time Source Destination Protocol Length Info
35 0.619977 127.0.0.1 127.0.0.1 TCP 64 5001 → 49872 [PSH, ACK] Seq=53 Ack=140 Win=28 Len=20
No. Time Source Destination Protocol Length Info
39 0.621735 127.0.0.1 127.0.0.1 TCP 65 5001 → 49872 [PSH, ACK] Seq=73 Ack=140 Win=28 Len=21
No. Time Source Destination Protocol Length Info
43 0.623598 127.0.0.1 127.0.0.1 TCP 65 5001 → 49872 [PSH, ACK] Seq=94 Ack=140 Win=28 Len=21
No. Time Source Destination Protocol Length Info
47 0.625630 127.0.0.1 127.0.0.1 TCP 65 5001 → 49872 [PSH, ACK] Seq=115 Ack=140 Win=28 Len=21
No. Time Source Destination Protocol Length Info
51 0.627592 127.0.0.1 127.0.0.1 TCP 65 5001 → 49872 [PSH, ACK] Seq=136 Ack=140 Win=28 Len=21
No. Time Source Destination Protocol Length Info
55 0.629683 127.0.0.1 127.0.0.1 TCP 65 5001 → 49872 [PSH, ACK] Seq=157 Ack=140 Win=28 Len=21
No. Time Source Destination Protocol Length Info
59 0.632344 127.0.0.1 127.0.0.1 TCP 65 5001 → 49872 [PSH, ACK] Seq=178 Ack=140 Win=28 Len=21
No. Time Source Destination Protocol Length Info
65 0.634544 127.0.0.1 127.0.0.1 TCP 62 5001 → 49872 [PSH, ACK] Seq=199 Ack=166 Win=28 Len=18
0000 12 00 57 c1 55 6c 01 01 03 10 02 01 00 00 00 00 ..W.Ul..........
0010 00 02 ..
Data: 120057c1556c010103100201000000000002
[Length: 18]
Unity log:
Network receiving: 20 0 bytes.
Network receiving: 21 20 bytes.
Network receiving: 21 41 bytes.
Network receiving: 21 62 bytes.
Network receiving: 21 83 bytes.
Network receiving: 21 104 bytes.
Network receiving: 21 0 bytes. 18 0 87 193 85 108 1 1 3 16 2 1 0 0 0 0 0 2 0 0 0
Network receiving: 18 21 bytes. 18 0 87 193 85 108 1 1 3 16 2 1 0 0 0 0 0 2 0 0 0 18 0 87 193 85 108 1 1 3 16 2 1 0 0 0 0 0 2
After the 20 byte message, there are supposed to be 6 21 byte messages before the 18 byte one. My receive side first processes the first 5 21 byte message, then for some reason when it is trying to process the 6th 21 byte message it is returning the correct size but it is getting the 18 byte message. Am I missing out on something fundamental here?
Answer by Bunny83 · Oct 22, 2019 at 12:42 PM
Yes it seems you're missing something fundamental here. TCP is a streaming protocol. It is not a packet or message based protocol. Yes it is transferred through a packet-based protocol (IP - internet protocol) but it's assembled on the receiving side as one continuous datastream. If you want to send "messages" you have to develop your own application layer protocol how to interpret your datastream. So you probably want to introduce some start / end marker for one message, search for those and chop up the data stream into seperate messages. Keep in mind that data generally can be fragmented when send over the internet. So a relatively large chunk of data might be send in two or multiple seperate IP packets.
That means it's not guaranteed when you receive data that all data that belongs together has been received. The only thing that TCP guarantees is that the data will arrive in the same order. So if a packet in the middle was lost the protocol would detect that and automatically resend the missing packet. The following data which might have been received already will be delayed until all missing pieces have arrived.
That's what the handshake messages ACK (acknowledge) are good for and that's also the reason why TCP can be slower on relatively bad connections. Every packet has a sequence number so the receiver knows in which order they belong and if one is missing. Every packet received is acknowledged.
It seems a bit strange that the ack number of most of your packets are the same. Are you sure that both applications are actually running properly? For any networking application you should make sure you have "run in background" active. Either set this in the player settings or set it to true from script.
Though as I said you need some kind of application layer protocol anyways if you want to build upon TCP.
edit
I just realised that your whole receiving code is pretty nonsense. When you use async reading you have to call BeginRead once. You call it every frame. The usual pattern is to call BeginRead again from inside the read callback at the end. You might want to look up some basic async socket reading example. Personally I don't really like the async mode of the sockets. It's usually easier to just start my own thread and use blocking synchronous sockets.
For more information on network protocol layers and the TCP protocol, see wikipedia
@bunny83 Yes I do have an app layer protocol and have some handling somewhere else to parse and regenerate each of the payload. The ack number the same I will have to check again, but that log is not the full one I only put the packets being sent by the custom app on port 5001. What I am wondering on this case is after it has processed the first 5 packets with a payload Len of 21, when it is supposed to be handling packet No. 59 it is getting the correct payload Len of 21, but the buffer content contains the payload for No.65 with 3 additional zeroes at the end. I also posted on StackOverflow and it seems like the ListenForData is the culprit making it run BeginRead multiple times therefore it is possible that it is overwriting the buffer. I will definitely be rewriting that function again.
Regarding the edit: Yeah that's what will happen when you code when tired. I did see that pattern earlier but have not yet implemented it that way on $$anonymous$$e, I'll try that one out.
I rewrote the ListenForData and ReadCallback to the same pattern and it looks like I am not skipping any more packets. Thanks!
Bunny83, I have something that I need help with. It is just my first project in Unity. It is a 2D scroller and basic tap and swipe actions that react differently on different devices. Would you be able to freelance for me on helping with the basic setup?
@AceEdu you can do it yourself with some plugins easily. for example, F$$anonymous$$ETP STREA$$anonymous$$ includes Networking Demo with UDP/TCP/WebSocket. you may check it out. Asset Store: http://u3d.as/1uHj forum: https://forum.unity.com/threads/670270/