koobas.hobune.stream
FileStream on separate Thread seems to block the main thread
When downloading something through my local server in Unity, the editor stops responding until all downloads are complete. How can that happen when it's running on a separate thread?
Here's the code:
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Threading;
using UnityEngine;
public class LocalServer
{
public bool runListener = true;
private string prefix;
private string baseDirectory = "M:/ftp backup/";
private volatile int chosenPort;
public LocalServer(int port)
{
prefix = string.Format("http://+:{0}/", port);
chosenPort = port;
Thread listenerThread = new Thread(LocalServerThread);
listenerThread.Start();
}
public void LocalServerThread()
{
HttpListener listener = new HttpListener();
listener.Prefixes.Add(prefix);
listener.Start();
Debug.Log("<color=green>Started local web server on port: " + chosenPort + "</color>");
while (runListener)
{
IAsyncResult result = listener.BeginGetContext(new AsyncCallback(ListenerCallback), listener);
result.AsyncWaitHandle.WaitOne();
}
listener.Close();
Debug.Log("<color=green>Stopped local web server</color>");
}
public void ListenerCallback(IAsyncResult result)
{
HttpListener listener = (HttpListener)result.AsyncState;
HttpListenerContext context = listener.EndGetContext(result);
string fileNameComplete = context.Request.Url.AbsolutePath.Substring(1).Replace("%20", " ");
string pathComplete = baseDirectory + fileNameComplete;
Debug.Log(pathComplete + " requested to LocalServer");
using (Stream fileStream = new FileStream(pathComplete, FileMode.Open))
{
byte[] buffer = new byte[fileStream.Length];
fileStream.Read(buffer, 0, (int)fileStream.Length);
context.Response.ContentLength64 = fileStream.Length;
context.Response.ContentType = SimpleHTTPServer.GetMimeType(fileNameComplete);
context.Response.StatusCode = (int)HttpStatusCode.OK;
context.Response.AddHeader("Cache-Control", "no-cache");
context.Response.OutputStream.Write(buffer, 0, buffer.Length);
context.Response.Close();
}
}
}
EDIT: When using an external program to download a file from the server running in Unity, Unity does not block. When downloading a file through Unity from an external program that runs the same code, Unity does not block either. So it seems like the combination of uploading and downloading inside Unity actually makes the main Unity thread block. Does someone have an idea why this happens?
EDIT 2: Only DirectShow blocks, MediaFoundation does not, this seems to be a media player related problem, odd that it's specifically happens when DirectShow downloads to the same application as the one uploading it.
Starting threads while testing inside the editor can be dangerous. You have to make sure that you properly terminate all threads. Stopping playmode does not stop other threads. There are some things a bit strange about your "HTTP server".
First of all your listener thread always waits for the current request to finish. So it can only process one request at a time. You shouldn't wait for the request to finish in the listener thread as this makes the seperate request threads pointless.
edit
Well actually waiting for the async result of "BeginGetContext" is fine and actually required. Since the handling of the request is handled by another seperate thread. When BeginGetContext "finishes" it means it has started another thread which is handling the request and we can wait for the next request. So this part was actually right.
Currently you encapculated the HttpListener completely inside the "LocalServerThread" method. However you don't have a reliable way to stop the listener. Setting "runListener" to false will not terminate the thread until another request arrived. You should call Stop on the listener when you want to stop it. In general the listener object should be a member variable of your "LocalServer" class.
I replaced WaitOne(); by Thread.Sleep(10); and now the thread is ter$$anonymous$$ated successfully. The runListener boolean is set to false on application quit and stopped local webserver is logged which means it got out of the while loop. Thanks for pointing it out. I specifically chose not to use thread.abort or listener.stop so I would not corrupt an ongoing process.
After you pointed it out I thought Wait.One would actually wait until the complete ListenerCallback would finish so I think using Thread.Sleep helps for parallel downloads. I'm not sure though I would have to test it, but before that I still need to find what's blocking the Unity main thread while the file is being downloaded.
I made the listener a member variable but I'm not actually doing anything with it.
Ins$$anonymous$$d of a Thread.Sleep you could simple do:
result.AsyncWaitHandle.WaitOne(100);
WaitOne will return false when the timeout is reached and true when it actually completed the "BeginGetContext" request. This allows it to answer multiple requests in quick succession while it keeps spinning the thread every 100 ms.
The HttpListener has a Stop method you should call when you want to stop listening. This should actually "complete" any pending "BeginGetContext". Calling "Stop" on the listener should not "corrupt" a currently running request as the request is handled on another thread entirely since you use the async variant. I just tested it by manually shutting down the listener while a download is still running. I created an artificial slow down by sending junk data in 100 seperate chunks and do a Debug.Log for each chunk. When calling Stop on the listener the listener thread will ter$$anonymous$$ate while the actual context thread for currently running requests keep running.
However keep in $$anonymous$$d that when you exit playmode the threads will keep running, however when you restart playmode or when Unity recompiles your scripts the managed environment gets reloaded which will ter$$anonymous$$ate all managed threads.
I doubt that your server code is actually responsible for blocking the editor. I've tried downloading from external application (firefox) and i tried downloading using WWW in a coroutine. In any case it did not block the editor. Also even when i stopped the listener thread running downloads will continue to run even when i'm back in edit mode. However the download does stop when i re-enter playmode or if Unity does a recompile / environment reload.
You haven't mentioned where and how you actually download the files. Since you mentioned DirectShow you probably use some native code stuff in your application.
Thank you for your advice, I improved the server code. DirectShow is the one downloading something from the server in Unity and apparently DirectShow is the one blocking the main thread until everything is downloaded. I replaced the media player/downloader with $$anonymous$$ediaFoundation and the main thread is not blocked anymore.
