- Home /
Split Textasset into List
Hi there,
I'm writing a Scrabble prototype for fun just to learn the ropes of scripting and I'm stuck at checking the spelled word against a dictionary.
I've got a text document with a list of words separated by a new line. My original thought was to create an array of strings and have the game check the word spelled on the game board against these strings. At first when I was researching, I thought to use Array.Contains, but that function does not exist for the built-in array as I've found out, so I decided to have it check them manually.
//Variable assignment
public static string[] wordArray = new string[];
public TextAsset dict; //assigned in inspector
//In Start()
wordArray = dict.text.Split("\n"[0]);
//In other script
if (tilesPlaced)
{
for (int i=0; i<gameMaster.wordArray.Length;i++)
{
if (gameMaster.wordArray[i] == wordOnBoard)
{
AddScore();
break;
}
}
}
This whole thing works, but is especially slow. I'm writing and testing with a computer running around 2.4GHz with four cores, and the search through the array takes almost 90 seconds. I realize it's a lot of strings to search, but I imagine there must be a better way to implement this kind of check, similar to the way Words With Friends or Scrabble itself may do it.
I've tried to use ArrayList, List, and Dictionary but I'm not sure I'm implementing them correctly. I also can't add them straight from the Split() command. The only way I can think to do it is to add an array, then transfer the information from the array to the List using a for loop in the Start function, but that would still require a lengthy initial load.
Does any code gurus out there have any ideas to help me get this together?
Something I do when searching for strings is add another if block and check to see if the length of the string is the same first. Its faster to check the length first then to check every character of every string in the array. So something like this
int wordOnBoardLength = wordOnBoard.Length;
for (int i=0; i<game$$anonymous$$aster.wordArray.Length;i++)
{
if(game$$anonymous$$aster.wordArray[i].Length == wordOnBoardLength )
{
if (game$$anonymous$$aster.wordArray[i] == wordOnBoard)
{
AddScore();
break;
}
}
}
This may not be a super crazy magical solution, but it should definitely speed things up.
If this is taking 90 seconds im assu$$anonymous$$g that you have quite a few strings as well, so you might want to go with the dictionary approach and use a dictionary of string lists.
This code is just to give you the general idea, its more or less right off the top of my head so hopefully you can just copy and paste it and it works
Dictionary<char, List<string>> m_stringDictionary
void Start()
{
m_stringDictionary = new Dictionary<char, List<string>>();
wordArray = dict.text.Split("\n"[0]);
List<string> tempList;
foreach(string s in wordArray)
{
//check to see if there is already a list belonging to the first letter, and add the string to this list if there is
if(m_stringDictionary.TryGetValue(s[0], out tempList)
{
tempList.Add(s);
}
else
{
m_stringDictionary.Add(s[0], new List<string>{s});
}
}
}
then everything will be sorted based on the first letter of the word, and you will have smaller lists to go through. Then you could use code similar to what I had posted earlier.
int wordOnBoardLength = wordOnBoard.Length;
char firstLetter = wordOnBoard[0];
for (int i=0; i<m_stringDictionary[firstLetter ].Length;i++)
{
if(m_stringDictionary[firstLetter ][i].Length == wordOnBoardLength )
{
if (m_stringDictionary[firstLetter ][i] == wordOnBoard)
{
AddScore();
break;
}
}
}
this isn't really a Unity specific question, if you branch out to C# resources you will get a lot of possibilities as well. Check out this Stackoverflow question: http://stackoverflow.com/questions/13959429/c-sharp-searching-large-text-file
Well, I kept plugging away at it last night for a few hours and managed to get it to work. It's pretty much as purdyjo said, use a dictionary.
I'll explain my findings here. Purdyjo, if you could put your comment in the answerbox, I'll mark it as correct and answered. Thanks!
What I did was keep the original string array and in the start function then write a for loop that adds each string in the array to a dictionary as both key and value.
for (int i=0;i<wordArray.Length;i++)
{
// key and value
dictionary.Add(wordArray[i], wordArray[i]);
}
There is also a function if you declare "using System.Linq" to split directly to a dictionary [Split().ToDictionary();] without the need for an array to begin with, but the time it took to populate the dictionary from the array was surprisingly milliseconds long, and the game has so low overhead that having the array in memory isn't really a concern, so I didn't bother to learn the confusing syntax for the ToDictionary() function.
Searching for a value in the dictionary is pretty much instantaneous, so if anyone is reading this wondering the same thing I was, the dictionary is the way to go.
You can search using Contains$$anonymous$$ey() or TryGetValue() and both of them work but in different ways. Contains$$anonymous$$ey() is a bit slower than TryGetValue() but it looks directly for a key, if that's what you need. TryGetValue looks for a key and returns its value in an output variable. It's faster than Contains$$anonymous$$ey() but it requires a second check to see if the output value matches the word played on the board (since in my dictionary the key and value are the same).
// key to search & value output variable
dictionary.TryGetValue(wordOnBoard, out value);
if (value == wordOnBoard)
{
AddScore();
}
It can also be used as a boolean [if (TryGetValue(wordOnBoard, out value))] so if it CAN return a value it will return true, meaning the key you're searching for is in the dictionary. This is useful if you want to keep the value free for another type of data, like a definition for your dictionary key.
For what it's worth, the speed differences were negligible for me. I have over 175,000 entries in my dictionary and using Contains$$anonymous$$ey() and TryGetValue() methods were both done at the snap of a finger, so go with what works for you.
Answer by purdyjo · Nov 17, 2013 at 08:10 PM
Alright I put my comment here as an answer like you had requested
If this is taking 90 seconds im assuming that you have quite a few strings as well, so you might want to go with the dictionary approach and use a dictionary of string lists.
This code is just to give you the general idea, its more or less right off the top of my head so hopefully you can just copy and paste it and it works
Dictionary<char, List<string>> m_stringDictionary
void Start()
{
m_stringDictionary = new Dictionary<char, List<string>>();
wordArray = dict.text.Split("\n"[0]);
List<string> tempList;
foreach(string s in wordArray)
{
//check to see if there is already a list belonging to the first letter, and add the string to this list if there is
if(m_stringDictionary.TryGetValue(s[0], out tempList)
{
tempList.Add(s);
}
else
{
m_stringDictionary.Add(s[0], new List<string>{s});
}
}
}
then everything will be sorted based on the first letter of the word, and you will have smaller lists to go through. Then you could use code similar to what I had posted earlier.
int wordOnBoardLength = wordOnBoard.Length;
char firstLetter = wordOnBoard[0];
for (int i=0; i<m_stringDictionary[firstLetter ].Length;i++)
{
if(m_stringDictionary[firstLetter ][i].Length == wordOnBoardLength )
{
if (m_stringDictionary[firstLetter ][i] == wordOnBoard)
{
AddScore();
break;
}
}
}
I find it interesting that you chose to go with a dictionary that has the word as both a key and a value. Definitely a quick true or false since dictionaries hash its keys into some number value internally, but i would think that the overhead would get very large as you add more strings.
That's what I thought too, but testing this on my phone, there were no performance issues even with a humongous dictionary.