- Home /
Right way to receive Images over tcp?
Greetings,
I have another application developed in C#, that sends a bunch of images, with diferent sizes (97kb, 123kb, 85kb ...).
I have this code that receive the image
byte[] bytes = new byte[180000];
if (networkStream.CanRead) // network from the socket TCP
{
byte[] myReadBuffer = new byte[1024];
int numberOfBytesRead = 0;
// Incoming message may be larger than the buffer size.
do
{
numberOfBytesRead=networkStream.Read(myReadBuffer, 0, myReadBuffer.Length);
myReadBuffer.CopyTo(bytes, numberOfBytesRead);
} while (networkStream.DataAvailable);
}
Texture2D tx = ByteArrayToImage(bytes); // transform the byte array to image
Mat.mainTexture = tx; // image goes to the material texture in a Plane gameObject
But the plane gets just a red interrogation mark as texture, I believe that indicates the texture is corrupted or missing or something like this.
What can I do to get the images right?
@Fabio_Vieira The data lost can be optimised if you split the data into chunks of packet, it helps a bit in TCP too. In UDP, it's a must due to the packet size limit.
We have a tool for local Live Stream, which was using TCP in our first release. Now, it's a cross platform Game View Sharing solution for both LAN and Internet, with different networking systems. It's called F$$anonymous$$ETP STREA$$anonymous$$, which you might be interested. http://u3d.as/1uHj
TCP is a reliable protocol. Unless you completely lost the connection all data is guaranteed to arrive and in the right order. TCP automatically retransmit packets when they are lost. Splitting the data is pointless for TCP. TCP is not a packet based protocol but a strea$$anonymous$$g protocol. Since TCP already splits the stream into relatively small IP packets (See $$anonymous$$TU). Actually trying to implement another reliable layer on top of TCP will have the exact opposite effect. See TCP "$$anonymous$$eltdown".
Agree, and thanks for your detailed explanation. But according to my strea$$anonymous$$g testing on devices, it does solve the error image(red ?) issue on mobile devices. I don't have any error issue on desktop, or good wifi environment. I understood TCP is reliable, but maybe there is other factor related to mobile devices part. or, splitting chunks manually may reduce the chance of error/retry.
I compared some live strea$$anonymous$$g testing with/without splitting chunks manually. The playback is much smoother if I split the chunks myself. Of course, the testing is between mobile devices.
$$anonymous$$eanwhile, in some extreme case, some deadlock issue happen in Unity3D when I use TCP. for example, couldn't reconnect until I restart the app. Thus, I switch to UDP for my projects with better performance.
Answer by Bunny83 · Jun 12, 2020 at 04:25 PM
Well TCP is just a streaming protocol. All it does is to ensure that no data is lost and that the send data is arrived in the same order as it was send. Note that this doesn't mean it is received in the same fragmentation that you send it. Just for example if you send 3 strings over TCP:
"Hello " // 6 bytes
"World" // 5 bytes
"!" // 1 byte
then you will most likely receive this:
"Hello World!" // 12 bytes
However it's also (theoretically) possible that you receive
"He" // 2 bytes
"llo W" // 5 bytes
"orl" // 3 bytes
"d!" // 2 bytes
In both cases we received all data in the right order. It's up to you to send the data in a way you can interpret it properly on the other side. That means especially when you want to send multiple seperate chunks of data you need a way to seperate one chunk from the other. So you have to implement your own high level / application level protocol. To avoid getting out of sync for whatever reason it's recommended to have some sort of starting sequence which marks the beginning of a "chunk" and in addition send a "size" field. So you can construct your own custom "header" which identifies one packet. For example you could set the string "Image"
followed by 4 bytes which contain the length of the image data you want to send, followed by the actual image data. That way the receiving side can actually detect the beginning of a chunk and also since we send the size of one chunk in the header we know exactly when one chunk is finished.
The most basic implementation of a simple data chunk protocol would be to just send a 4 byte length header followed by the actual data. That way we know exactly when the data of a chunk of data is done and we can read the next length header.
Note that the code you've posted has several additional flaws. First of all you quit your while loop as soon as "DataAvailable" is false. That makes no sense. DataAvailable only indicates that the socket has data available to read. That doesn't mean that there might not arrive additional data later in time. Next issue is you use two buffers and you copy the conten of your smaller buffer into the larger one. However you don't track the current position inside the larger buffer. So you always copy the smaller buffer into the first 1024 bytes of the larger buffer. You don't really need the second buffer as you can specify the offset inside the Read method of the networkstream.
Since you use synchronous read operations it's highly recommended that you run your receiving code on a seperate thread. There you could simply use the BinaryReader to read the length and the chunk of data. This would make the receiving side quite simple. You should be able to use ReadInt32 to read the size / length and then just use ReadBytes with the length you just read. Just repeat this process for every image you want to receive from the stream. Of course currently we don't really have a concept for indicating the "end" of our image series (if that is even required). You could send a length of "0" at the end to indicate that no more images will follow and the receiving side can stop reading. Though we don't know enough about your usecase to give a more detailed description or recommendation.
edit
I created a small example. Here's a standalone .NET application (pastebin) which serves as TCP server and sends every 500ms a random png images from a folder to all connected clients.
Here's a MonoBehaviour which acts as client (pastebin) and connects to the above mentioned server, receives the images, load them into a texture and replaces that in a material.
Finally here's how it looks like in action:
Answer by Viitorcloud · Apr 19, 2021 at 05:47 AM
Hi @Fabio_Vieira, Did you get it working? The answer of @Bunny83 is quite convincing but still, I am not able to get it.
The original answer is very straight forward solution, with meta data + raw byte[]. Or, you may also consider FMETP STREAM, it's C# strea$$anonymous$$g solution for many platforms. It's on Spring Sale 2021 now. http://u3d.as/1uHj
Your answer
Follow this Question
Related Questions
Tcp server 0 Answers
System.NotSupportedException: This protocol version is not supported 0 Answers
Android Tcp Socket Problem 1 Answer
Why i can't receive 2 messages in a row from my server in Unity? 2 Answers