koobas.hobune.stream
Loading a large xml file (~200 multi-level nodes) into Unity
I'm trying to load a large xml file in for parsing from www.text, but the usual xml.LoadXml() does not work...
I've also tried:
var stringReader:StringReader = new StringReader(w.text); stringReader.Read(); // skip BOM
xml.LoadXml(stringReader.ReadToEnd());
I still get the following error:
XmlException: Text node cannot appear in this state. Line 1, position 1. Mono.Xml2.XmlTextReader.ReadText (Boolean notWhitespace) Mono.Xml2.XmlTextReader.ReadContent () Mono.Xml2.XmlTextReader.Read () System.Xml.XmlTextReader.Read () System.Xml.XmlDocument.ReadNodeCore (System.Xml.XmlReader reader) System.Xml.XmlDocument.ReadNode (System.Xml.XmlReader reader) System.Xml.XmlDocument.Load (System.Xml.XmlReader xmlReader) System.Xml.XmlDocument.LoadXml (System.String xml) gniptwitter+$fetchTweets$42+$.MoveNext () (at /gniptwitter.js:35)
The data file is from an api so I can't change its format. However, it is valid XML. It might be UTF8 or some odd encoding that's causing Unity to balk at XML-parsing it...
Does it actual have a Byte Order $$anonymous$$arker? I'd start by not doing that extra read.
i actually tried it straight on with loadX$$anonymous$$L without that .read ... but errors galore. google'd around, found that the .read worked for some, but apparently not here :-\
This does happen due to encoding issues; if you use a standard textreader and read one character at a time and then print the characters, you'll notice that the stream is littered with spaces and possibly questionmarks if the StringReader fails to decode it. If your xml is in a file on the harddrive, the usual way to load it is to call XmlDocument.Load. Not XmlDocument.LoadXml, mind you. XmlDocument.Load takes care of parsing the XML for you, while XmlDocument.LoadXml expects a string that contains raw XML to make a tree out of.
In your case, where a file is very large, calling XmlDocument.Load from the mainthread will cause the thread to block until loading completes, and you'll experience a holdup in Unity's rendering. To get around that, you can have a separate thread load the xml for you and signal the mainthread when it's complete, or you can streamread the XML (using an XmlReader), and only call its Read()-method a few times every frame.
He says "200 multi-level nodes". I assume this means each one has multiple children and grandchildren of its own.
That wouldn't be valid X$$anonymous$$L then, since there can be only one root node. (rimshot!)
1) i am not sure if it is the loading being blocked, as would loading being blocked result in a red-marked error in the console, as pasted above? 2) and, i'm not sure how to load the separate threads - would calling yield on the loadXml work? (@warwick and apologies, i often use words loosely, but the meaning is there! ;) )
