koobas.hobune.stream
Loading large maps - 30000x30000 pixels (threading / parallel loading)
Hello
First off, I'm fairly new to Unity so I may be missing something obvious here.
I've been struggling with how to load large (i.e. 30000x30000 pixels) georeferenced maps fast[er]. The map is cut into 2048x2048 pixel tiles with worldfiles (.JGW). For each tile I load the JPG, instantiate a TilePrefab, set the texture and place it in the world. There is no problem in loading the tiles sequentially (with WWW and Instantiate(Resources.Load(..prefab..)) ) however that can take a long time with maps of this size.
I would love to segment the set of tiles and spawn a number of Threads to do the loading, however that doesn't seem to be possible since threads are not allowed to interfere with Unity's main loop, thus no access to WWW, Instantiate or Resources.Load.
What I'm trying to do now is to have my main 'map loader' script segment the set of tiles and instantiate a number of TileLoader prefabs to load the tiles. The problem is I can't (to my knowledge) pass any arguments when I instantiate the TileLoader prefabs and each TileLoader will need some start and end indexes for the tiles it needs to load. So I have to figure out some hacked way of instantiating prefabs, setting indexes and then running the script.
My question is if anyone has any suggestions as to how to do this kind of multi-threaded / parallel loading in Unity? Or a nice way to pass the arguments to the prefabs? Perhaps some secret ninja tricks :-)
Thanks
Edit: I accidentally a word
I would suggest time slicing. Just load one or a few chunks per frame.
Why a "hacked way"? after you instantiated a prefab you can set any values right after you instantiated it. A coroutine will do the downloading. A WWW request is already asynchron, you can start as many as you want (well, almost). Multithreading can only speed up some bottlenecks like waiting for the www request to finish. If you use coroutines (cooperative multitasking) there should be no problem and you won't get it any faster than that.
In your case i would attach a loading script to each tile. Every tile will load their own texture. Just use the sample script in WWW.texture. You can instantiate all tiles at once and right after instantiate just set the url
// C#
// TextureLoader.cs
using UnityEngine;
using System.Collections;
public class TextureLoader : MonoBehaviour
{
public string url = "http://images.earthcam.com/ec_metros/ourcams/fridays.jpg";
IEnumerator Start()
{
WWW www = new WWW(url);
yield return www;
renderer.material.mainTexture = www.texture;
}
}
// Somewhere else:
for(int x = 0; x<15; x++)
{
for(int y = 0; y<15; y++)
{
GameObject newTile = (GameObject)Instantiate(prefab, new Vector3(x,0,y),Quaternion.identity);
TextureLoader loader = newTile.AddComponent<TextureLoader>();
loader.url = GetURLForTile(x,y);
}
}
I wrote a "hacked way" since I didn't know if Start() of the TextureLoader would be called immediately after the prefab was instantiate (haven't been able to find much documentation on the control flow in Unity). I was hoping I could do it, the way you are suggesting here.
Having the tiles load their own textures is a great idea - thanks, I'll use that!
I know the Execution Order page is more confusing than explaning :D
The unify community wiki article is little bit better but also not that detailed: http://www.unifycommunity.com/wiki/index.php?title=Event_Execution_Order
Start is called at the beginning of the next frame (before Update). There are just a few events that fires immediately (Awake, OnEnable, OnDisable as far as i know)
Ah perfect! I'll look at those links as soon as I get then chance :-)
You can pass arguments to anything that has arguments, and don't need to hack anything--Instantiate returns a reference to the instantiated object, use that--however I'm not sure what you're trying to accomplish with these TileLoader prefabs. You don't need to have prefabs to have multiple instances of coroutines loading at once, if that's what you're trying to do. (Just run a loop and call the coroutine as many times as you need.) However I don't know that I'd recommend doing that anyway, since it seems to me that sequentially loading them would provide a better user experience...it will take just as long to load them simultaneously, but you'd have to sit there and wait until they're all done, whereas sequential loading at least gives you something to look at while they're loading in.
Note that running multiple instances of coroutines isn't the same as multi-threading. However, I don't see that true multi-threading would provide any benefit in this case, since it won't speed up anything that's taking time (loading JPGs--you only have so much bandwidth, applying textures--there's only one pipe to the graphics card).
You are right about the user experience. I have it working now, but as you said: Unity is unresponsive until everything is loaded. Last time I wrote something in Unity that loaded tiled maps sequentially I used an IEnumerator method and "yield return www". You could see the tiles loading one by one but Unity was still unresponsive (to camera movements etc.) while loading - will that also happen with Coroutines? (If yes - is there a way to avoid it?)
@$$anonymous$$surrow: IEnumerator and yield is a coroutine. If it was unresponsive while loading, that's only because you programmed it to be, it's not something that's inherent in coroutines.
