koobas.hobune.stream
How to include a large dataset (dictionary is "too complex" for Unity)
Hello!
I'm building an app with a ton of quotes, so I'm using a dictionary to store the quotes / authors. However, now that I've got about 3000+ quotes, unity says:
Method loadThoughts:LoadThoughts () is too complex.
Is there a better way to include this data? In the end I'm likely to have 30,000+ entries.
Are you sure your not just running into a hardware limitation? Check how much free memory you have with as many items as possible loaded into the dictionary.
Well, I'm not sure how to check memory etc at this point. The error occurs when I run in the editor, and simply won't load any information into the dictionary.
Also, just to be clear, removing half the quotes in the dictionary load script, fixes the problem. So I'm just looking for another way, something that allow for many thousands of entries.
The thing is that there's no practicable limit to the number of items a dictionary can store (it can store something like 4 billion items provided you have the memory resources), and so it may be a hardware limitation. How many characters (including punctuation and spaces) is your average quote? If we call this average number 'q' then q 1024 1024 is how many megabyets are being used for the dictionary and if your quotes are large and your memory resources limited, this might be the cause of your issue.
I just loaded 10,000,000 strings, each 1024 characters in length into a dictionary so whilst I doubt that it's a hardware restriction, the issue isn't that the dictionary has an item limit.
Use the resource monitor (memory tab) to see how much memory the unity process is using when you run, and to see how much free memory your system has at the time.
We have just runned into the same issue when statically initializing huge dictionary (100k+ entries). We wanted to initialize it statically to make the load time as short as possible.
private static Dictionary<string, bool> _dict = new Dictionary<string, bool>() {
{"the_tag", true},
// 99k more entries here
};
It seems it doesn't work with 100k in a singe dictonary, bt 10x10k dictionaries in separate classes work fine.
The solution we decided to stick to is to divide the class into a few classes and use one as a hub for accessing inner dictionaries.
