koobas.hobune.stream
Parsing large XML files on-the-fly, best practice?
I need to periodically (like every 10 seconds) parse an XML file I get via WWW. Using System.Xml and XmlDocument. When the file is small, it's fine, but larger ones give a noticeable pause in frame rendering. I'm doing it on an Update call. I know, I know, bad.
So what's a better way? Should I just do what I do, but in a coroutine? Different thread (if so, any good examples)? Or is there an XML parser implementation that can 'spread' its work out over multiple frames?
Well, a coroutine wouldn't make any difference since it's called within the Unity update-loop and a single task can't be interrupted. A different thread would be a way but if it's really much data it can influence the main thread as well (like every other thread).
I've never had a xml parser that support "suspending" in between. If there is one that would be great ;). Another solution would be to write your own parser. XML is not that hard to parse. What data you need? If it's just a subset, the parser don't need to "unwrap" the whole file into nodes. It would be enough to parse through the stream and just wait for the right node.
If you write your own parser you could implement your own time-watchdog that yields if a certain amount of time has passed.
When writing your own parser you have to think about if you want a "general" parser or one that fits your needs in this case. Does the xml file contain CDATA sections? Without those it gets really simple ;)
Right it's not too general, so I could write my own. I was thinking maybe a SAX parser might be targeted to this type of thing, but this data is not in need of any special love like that. Thanks.
