koobas.hobune.stream
Performance question - Loading large-ish amounts of text data in a mobile game?
Hello!
I'm building a trivia game for the Android platform. There's going to be a number of categories, each containing 50-100 questions (along with answer data etc). In the game, the player gets a randomized selection of 3 categories and picks one, after which a question of the relevant category is loaded and presented. This process loops until the level target of correct answers is met.
My intent was to load the questions from an XML file into a list (they are stored in separate XML files, labeled by category), so I could easily remove a question from that list after it has been used and the player wouldn't get any repeated questions should they choose the same category again. In order to save memory, I also intended to only load the relevant questions into a new list after a category has been selected instead of loading them all in at once during Start (since I may not even need some of the categories). However, I've heard many different things about lists being slow once they have a fair amount of objects in them, and I'm unsure if my loading approach is the best idea either as I don't know what the performance impact of loading data from an XML is. I know I could also avoid lists and use arrays instead, then store chosen question ID's and loop over them, but that solution seemed rather clunky to me as theoretically it's possible for the randomizer to keep getting the same chosen ID over and over.
Overall there's a lot of choices and I'm quite clueless when it comes to performance related questions, so I figured I'd try to get some opinions from more experienced people.
Considering I'm dealing with a mobile platform, what would be the most performance-friendly way to achieve this?
1) Arrays or Lists?
2) Load relevant data during gameplay when it's needed, or all at once during Start?
3) ...anything else?
I hope I was clear enough. Thank you for your time!
First of all a List just wraps around an array. So internally a List actually is an array. The only thing you might want to keep in mind if you're going to add a lot of items in a row to first set the Capacity of the List to avoid frequent reallocation of the internal array.
Text data is almost nothing compared to a single texture. There should be no problem loading all questions at once. How large is your text file? is it even over one MB? Unless you have +10k or +100k questions i wouldn't bother and just load all at once.
Gotcha! I'll keep that in $$anonymous$$d. I'm not sure what the filesizes will be since I haven't finished one properly yet, but I don't think one will ever even come close to 1$$anonymous$$B. There will be around 10-20 files with 50-100 questions each, and each question containing 6 parameters. I simply wasn't aware what kind of amounts are considered too large and hence the question. Thank you very much for the reply, guess I was worried for nothing then. :)
To conserve file size (and speed up loading time) you may want to go with a smaller format than X$$anonymous$$L (the closing tags, angle brackets, and header are all unnecessary overhead). JSON is a great format (and probably easier to code the (de)serializaion for). Binary may be even smaller, but you can't edit those files by hand. Though, 300-600 lines isn't all that big for a file (depending on the line size). Compare your data to image sizes, 1$$anonymous$$B image takes no time to load.
Yep, that's why i wrote SimpleJSON in one day and SimpleX$$anonymous$$L is still in development and currently on hold. Because after you read the X$$anonymous$$L specs 10 times your brain starts for$$anonymous$$g knots. Only the rules for entities is just pure nightmare ^^. JSON on the other hand is as simple as it can get.
The only nice thing about X$$anonymous$$L is that you can seperate metadata from data by using attributes and tag names. How would you implement a type safe serialization with JSON? (So just like binary serialization but as JSON). You somehow have to mix the actual data with the type information which is possible but ugly and usually restricts you in some way. You could use a prefix for metadata (like "" or "_") but if you do serialization of actual classes, variable names could be anything and could collide with metadata. Another approach would generally wrap each field in it's own class where there is a "data", "type" and "name" field for a single datavalue. That is basically alternating between metadata with data.
ps: binary serialization with a BinaryFormatter is most likely larger than JSON. $$anonymous$$anually packing the data with BinaryReader / BinaryWriter will be slightly smaller. JSON is the easiest way. If you fear the size you could use some sort of compression, but adding a compression library most likely adds more than it saves in this case ^^.
