- Home /
From two lists into a dictionary... and back?
I've been trying to wrap my head around Serialization and I'm still quite confused by data disappearing while I switch from EDIT to GAME mode.
I'm trying to get the component below to have a dictionary at its core. I've found multiple suggestions regarding using two Lists<> to load keys and values separately, to then stitch them together into a dictionary in the Awake() function. Sounds logical enough.
The problem is, if I modify the dictionary in EDIT mode, and short of keeping the two original lists synced with the dictionary at all times, how/when do I save the data of the dictionary back into the lists just before switching to GAME mode, so that Awake() can get up to date lists?
This is what I have so far:
using UnityEngine;
using System.Collections.Generic;
public class DictionaryLookAlike: MonoBehaviour
{
public List<string> keys = new List<string>();
public List<int> values = new List<int>();
// Unity doesn't serialize dictionaries even if you pray in Klingon.
private Dictionary<string, int> dictionary = new Dictionary<string, int>();
public void Awake()
{
loadData();
}
// how/when do I put the data back into the lists prior to switching to GAME mode?
public void OnWhat??????()
{
saveData();
}
// Dictionaries are not serializable in Unity so far.
// For this reason we serialize two lists and stitch
// those together during Awake().
private void loadData()
{
string aKey;
int aValue;
for(int index = 0; index < keys.Count; index++)
{
aKey = keys[index];
aValue = values[index];
dictionary[aKey] = aValue;
}
// we don't need the lists anymore, hopefully they will
// be GC'd and free some memory until we need them again.
keys.Clear();
values.Clear();
}
// As dictionaries are not serialized we must save
// all the dictionary data back into the lists.
private void saveData()
{
foreach(var entry in dictionary)
{
keys.Add(entry.Key);
values.Add(entry.Value);
}
}
}
Use OnDisable() ins$$anonymous$$d of OnWhat??????(). It'll then compile and work, I think, correctly.
It doesn't work. OnDisable() is called only when I switch back to EDIT mode. $$anonymous$$y problem is the switching from EDIT mode to GA$$anonymous$$E mode. That's when the content of the dictionary should be saved to the lists. But I cannot find a method that is executed -before- the switch to GA$$anonymous$$E mode occurs. Awake() seems to be the first method to be called, but by then I'm already in GA$$anonymous$$E mode and the dictionary has lost its data.
I had to solve this problem in a component of $$anonymous$$e, but I edit the lists in edit mode (because they serialize) and convert to dictionary for play mode (because it doesn't serialize). Don't know how possible it is for you to switch between them, but this method worked for me.
Hi everybody, and thank you so much for all your contributions. Over the past 6 weeks or so my family and I were on and off in holidays and I couldn't move this answer forward.
Now back at home, I've eventually decided to change tack. I therefore wrote the question/answer/script provided here as a new starting point, effectively bypassing unity's serialization altogether. I'm not posting it here as answer because, strictly speaking, it doesn't find the solution to the particular issue I raised in this question. However, it addresses the original problem that generated this question in the first place, that is, how to serialize (somehow!) a dictionary so that it survives going from Editor to Player mode and viceversa.
survives going from Editor to Player mode and viceversa
The "and viceversa" was not in the original question. Data doesn't get deserialized going from playmode to edit mode by design to prevent data corruption/loss.
Are you using this dictionary for save data or something? Why do you need to use data generated in play mode in edit mode? By storing data between plays you could introduce problems that will be very annoying to debug, since your game may not be starting in the same state between tests.
Answer by gregzo · Jul 23, 2013 at 10:20 PM
I'm not sure I understand where your problem lies. What's the function of your saveData method? As you have it, your two Lists will feed a dictionary in Awake, which is what you want, no?
You could prevent errors in list lengths,as well as facilitate data input in the inspector with a serializable nested class :
[System.Serializable]
public class StringIntPair
{
public int fooInt;
public string fooString;
}
Then declare a public array of StringIntPair, it will show up in the inspector :
public StringIntPair[] keysAndValues;
This would prevent potential lengths mismatches. Then build your dictionnary as you're already doing, in Awake.
Thanks for your suggestion of switching to a nested class uniting key and value, to avoid list length issues. I shall be using it. $$anonymous$$uch appreciated.
Concerning your question: no unfortunately as the code stands Awake() gets two empty lists because I cannot find a way, switching from EDIT to GA$$anonymous$$E mode, to trigger the dumping of the content of the dictionary back into the lists. That's because Awake() is called as soon as I switch to GA$$anonymous$$E mode. OnDisable() and OnDestroy() are -not- called when switching from EDIT mode to GA$$anonymous$$E mode, but only when I switch back from GA$$anonymous$$E mode to EDIT mode.
It seems that not having the line [ExecuteInEdit$$anonymous$$ode] prevented the execution of OnDisable()/OnDestroy() before entering game mode. With a glimmer of hope I tried syncing the lists with the dictionary in OnDisable() and refilling the dictionary in OnEnable(), but now it appears that the lists are no longer holding their data from the time OnDisable() is invoked to the time OnEnable() is! See the code in one of my comments to Tarlius.
Answer by Jamora · Jul 24, 2013 at 01:54 PM
To answer your question where you need to perform the serialization, the answer is in the callback of EditorApplication.playmodeStateChanged.
As to why I inherit from ScriptableObject, I could get a System.Object to serialize properly. I also don't think a dictionary should be a component in a GameObject, because it's a data structure. Instead, a Component should use it; hence a ScriptableObject.
EDIT:At some point in my coding the goal shifted from making a bare bones serializing dictionary to a serializing dictionary I can use in my projects, which is why all the unrelevant stuff in the previous code.
I will probably send this to the Asset Store when I'm finished; I've seen at least a few questions asking for a serializable dictionary.
EDIT: No, I went over the code too heavy handedly; it's not every day I get to remove functionality from my code. This code does work for me; tested.
It's also easy to break this code by changing values inside the dictionary; foreach doesn't like its lists changing on it.
The Dictionary:
[System.Serializable] //<- important
public abstract class SerializableDictionary<K,V> : ScriptableObject{
/* Lists to serialize the dictionary to.
* Notice that as they are generic fields, they will not serialize as-is.
* Instead, this class must be inherited by a non-generic class so Unity
* knows what types they are and serializes correctly. That concrete class
* is then used in applications
*
* I.E. public class StringIntDictionary : SerializableDictionary<string,int> {}
*/
[SerializeField] //<- important
public List<V> v;
[SerializeField] //<- important
public List<K> k;
public Dictionary<K,V> dictionary;
public SerializableDictionary(){
dictionary = new Dictionary<K, V>();
//Lists are null if it's the first time, otherwise they've been deserialized.
if (v == null){
v = new List<V>();
k = new List<K>();
}
}
//Does the deserializing.
void OnEnable(){
dictionary = new Dictionary<K, V>();
for(int i=0;i<k.Count;i++){
dictionary.Add( k[i],v[i]);
}
k.Clear();
v.Clear();
}
//Serializes dictionary
public void SerializeDictionary(){
v.Clear();
k.Clear();
foreach(KeyValuePair<K,V> kvp in dictionary)
{
k.Add(kvp.Key);
v.Add(kvp.Value);
}
}
}
The Editor Script
/*We need this editor script to call the Serialize function of the serializing dictionary*/
[CustomEditor(typeof(DictionaryTest))]
public class DictionaryEditor : Editor {
[SerializeField]
DictionaryTest editorTarget;
private bool unfolded = true;
private string newKey = "";
/************************************/
/* The important stuff */
/************************************/
void Awake(){
editorTarget = (DictionaryTest)target;
//Register for the scene change callback.
EditorApplication.playmodeStateChanged += PlayChanged;
}
void OnEnable(){
if(editorTarget == null) {
editorTarget = (DictionaryTest)target;
}
if(editorTarget.dictionary == null){
editorTarget.dictionary = ScriptableObject.CreateInstance<SerializableStringIntDictionary>();
}
}
//This is called whenever the playmode changes
void PlayChanged(){
editorTarget.dictionary.SerializeDictionary();
}
void OnDestroy(){
//Unregister the scene change callback
EditorApplication.playmodeStateChanged -= PlayChanged;
}
/************************************/
/* End of important stuff */
/************************************/
public override void OnInspectorGUI ()
{
DrawDefaultInspector();
EditorGUILayout.LabelField("");
unfolded = EditorGUILayout.Foldout(unfolded,"Dictionary");
if(unfolded){
foreach(KeyValuePair<string,int> kvp in editorTarget.dictionary.dictionary){
GUILayout.BeginHorizontal();
ShowDictionaryEntry(kvp.Key, kvp.Value);
if(GUILayout.Button("Remove Entry"))
editorTarget.dictionary.dictionary.Remove(kvp.Key);
GUILayout.EndHorizontal();
}
ShowAddEntryButton();
}
}
/// <summary>
/// Shows the add button.
/// </summary>
void ShowAddEntryButton(){
GUILayout.BeginHorizontal();
EditorGUILayout.PrefixLabel("Key: ");
newKey = GUILayout.TextField(newKey);
if(GUILayout.Button("Add Entry")){
editorTarget.dictionary.dictionary.Add(newKey,0);
newKey = string.Empty;
}
GUILayout.EndHorizontal();
}
/// <summary>
/// Shows the dictionary values.
/// </summary>
void ShowDictionaryEntry(string k, int v){
int oldValue = v;
EditorGUILayout.PrefixLabel(k);
v = EditorGUILayout.IntField(v);
if(oldValue != v)
editorTarget.dictionary.dictionary[k] = v;
GUILayout.Label(" ");
}
}
The test class
[System.Serializable]
public class DictionaryTest : MonoBehaviour {
[SerializeField]
[HideInInspector]
public StringIntDictionary dictionary;
void OnEnable(){
if( dictionary == null){
dictionary = ScriptableObject.CreateInstance<StringIntDictionary>();
}
}
void Start(){
dictionary.dictionary.Add("test",5);
}
void OnGUI(){
foreach(KeyValuePair<string,int> kvp in dictionary.dictionary){
GUILayout.Label(kvp.Key+" "+kvp.Value.ToString());
}
}
}
Dear Jamora, thank you so much for your effort. Indeed your solution would work. I am still hoping to avoid having to keep dictionary and lists sync'd together but I might have to accept that it is necessary, at least for the sake of simplicity. I guess there is a performance loss when adding/remove keys compared to a pure-dictionary solution, but retrieval is the operation that will be used the most and that can ignore the lists, taking full advantage of the dictionary's algorithms.
One issue I would raise about your code:
editorTarget.vh.k.Remove(kvp.$$anonymous$$ey);
editorTarget.vh.v.Remove(kvp.Value);
In a dictionary a key can point to one and only one value, but a value can potentially point to multiple keys. The second line above would remove only the first occurrence in the list of the value, but that might not necessarily be the one associated with the key you are removing. You might want to find the index of the key entry in the list and then use that index to remove the associated value. Alternatively, look at @gregzo response as it uses a single list of key/value pairs.
Similarly, as you add a key/value to the dictionary through the custom inspector, the dictionary overwrites the value of an existing key automatically. But in the lists all you do is to append a new key/value pair at the end. This can generate a scenario where the same key is present twice in the key list and the lists would no longer be in sync with the dictionary.
Again, thank you for your help. I'm going to give some more time for a solution to the serialization problem to emerge but yours is a very good solution candidate.
By the way. Why are you inheriting from ScriptableObject ins$$anonymous$$d of $$anonymous$$onoBehaviour as in my example? Is there a specific reason for it? In my original question I kind of need the script to be attached to GameObjects, which would exclude the possibility of inheriting from ScriptableObject.
Valid points. It's certainly possible to modify the code such that only the lists are being modified when in the editor and the dictionary when playing. I'll update my answer after I've done some fine tuning on the code.
Actually, I'd like to use the dictionary in both EDIT and GA$$anonymous$$E mode and use the lists only for serialization purpose. But I guess given that the performance concerns are only during retrieval from the dictionary and during the actual game experience, perhaps this way of looking at it might work. Use mainly the lists in EDIT mode, even though it's slower, and then take advantage of the dictionary in GA$$anonymous$$E mode, as the OnDisable()/OnDestroy() method ARE called in that situation.
Certainly a useful idea if all else fails.
Running into fresh problems, now with the lists themselves. See my comment/code to Tarlius.
Answer by rhys_vdw · Jul 24, 2013 at 10:16 AM
Your code is logically correct, but serialized data is only updated in the Editor. It will not be saved when you stop running your game.
If you wish to save data you may want to write and read it from a file, which is a more complex procedure. This is one approach.
The downside of this is that you'll have to do some further work if you still wish to modify the data in the Editor. You could write a custom editor that will read in the file and saves it out again.
A simpler and more idiomatic solution is to save the prefab. You'll have to give the script a reference to a prefab object in your assets folder and it can copy itself into it. Use PrefabUtility.ReplacePrefab.
In game, you will have to choose when is most appropriate to save the data out for your program. You'll at least want it in OnDestroy. OnDestroy will be called when your application quits.
Also, I'd recommend incorporating gregzo's approach for the reasons he provides.
Good luck and feel free to comment if you'd like further advice.
EDIT:
Sorry, I skimmed your question and just read your example code, guessing at what you were trying to do. Clearly I was wrong. (I'll leave the above here anyway, even though it addresses a different question).
The problem is, if I modify the dictionary in the editor, and short of keeping the two original lists up to the date every time I modify the dictionary, how/when do I save the data of the dictionary back into the lists so that as I switch to GAME mode, Awake() gets invoked and it has up to date lists to correctly refill the dictionary?
Never modify the dictionary. The dictionary should exist only at runtime. If you're working in the editor the performance of your data structure should not be important. Just do a linear search of the list.
Yes, you and Jamora are certainly leading me down the path of using only the list in edit mode and the dictionary+list in game mode.
I'm just surprised there is no other way, as it all seems to boil down, at least in the case, to the impossibility of detecting when the switch from EDIT to GA$$anonymous$$E occurs and do something before it occurs.
It seems that not having the line [ExecuteInEdit$$anonymous$$ode] prevented the execution of OnDisable()/OnDestroy() before entering game mode. With a glimmer of hope I tried syncing the lists with the dictionary in OnDisable() and refilling the dictionary in OnEnable(), but now it appears that the lists are no longer holding their data from the time OnDisable() is invoked to the time OnEnable() is! See the code in one of my comments to Tarlius!
Answer by Tarlius · Jul 24, 2013 at 12:25 PM
I think you might be looking for OnEnable and OnDisable
OnDisable gets called before a recompile, and OnEnable afterwards. You'll prolly want to add an if(Application.isPlaying) in there too, so you don't have odd things happening in game. You'll also need the [ExecuteInEditMode] tag
using UnityEngine;
using System.Collections;
[ExecuteInEditMode]
public class Test : MonoBehaviour {
void Awake() {
Debug.Log("Awake");
}
void Start () {
Debug.Log("Start");
}
void OnEnable() {
Debug.Log("OnEnable");
}
void OnDisable() {
Debug.Log("OnDisable");
}
void OnDestroy() {
Debug.Log("OnDestroy");
}
}
Output:
(On Creating the object/loading the editor) Awake OnEnable Start
(On recompile) OnDisable OnEnable
(On Enter Play Mode) OnDestroy Awake OnEnable Start
Edit: More specifically, it looks like you problem is not with going into Play mode, but with the serialization/deserialization of your class. There was a really nice video about this at Unite Nordic 2013 which I recommend watching.
Dear Tarlius, thank you for your contribution.
Especially, the list of calls in the three given situation is very helpful. But above all, a discrepancy with your On Enter Play $$anonymous$$ode list (OnDestroy() wasn't called) made me realize that I didn't have the [ExecuteInEdit$$anonymous$$ode] line in my class. Now OnDestroy() IS called before going to play mode.
Unfortunately before OnDestroy() there now is a call to OnDisable(), OnEnable() and OnDisable() again. And by the time I get to OnEnable() (never $$anonymous$$d OnDestroy()!) I loose the data in the lists too. I'm not sure what's causing it, but right when I was starting to despair your contribution gave me hope again, as I now have the chance to do things before going into Play $$anonymous$$ode.
So, allow me to investigate the issue of the data disappearing from the lists (that definitely shouldn't happen) as there might be a working no-compromises solution in sight.
By the way: you are right, it is definitely a serialization issue and I have watched the unite video you mention before I posted this question. But I am still a newbie with unity and even though I'm amassing knowledge dots the connections between them will take some time to appear.
The video was pretty heavy going, but made things a lot clearer for me. It might be helpful to watch it again, took me a few times to digest too.
Anyway- looking at your code, you are probably calling load (which clears the lists) without a matching save somewhere. I think you could solve your problem by simply changing the model to be something more like this:
void OnEnable() {
if(dictionary == null) {
// rebuild dictionary from list
}
}
void OnDisable() {
// Put the stuff into the lists
}
You might be able to make a more generic solution by making a serializable class that inherits from ScriptableObject, but I'm not too up on the details for that... $$anonymous$$ight have a go later!
Note that you probably want to be using OnEnable/OnDisable, not Awake/Destroy, since objects are not destroyed and rebuilt after a recompile, but are reserialised. That's another possible cause of the problem you describe. Now that I think about it, it might be an idea to not check whether you're in play mode or not since doing so would cause your dictionary to be destroyed if you recompile in play mode.
Not much of a chance to look at it today. I just changed a few things in the class to further simplify it and spiral toward the problem. It seems that not even the keys/values list are serialized. Check out the following code, it's short enough.
As I switch to play mode, OnDisable() is called first and in the Debug.Log() statement it correctly states the number of items in the lists (i.e., 3 and 3, depending on how many items I added manually in edit mode). Notice how the Debug.Log() statement is the last in the method.
Immediately thereafter OnEnable() is called. Notice how the very second line is another Debug.Log() statement to check the content of the lists. It turns out the lists are now empty!
I'm officially confused. Those lists should serialize automatically!
using UnityEngine;
using UnityEditor;
using System.Collections.Generic;
[ExecuteInEdit$$anonymous$$ode]
public class $$anonymous$$yCustomDict : $$anonymous$$onoBehaviour
{
public List<string> keys = new List<string>();
public List<int> values = new List<int>();
// Unity doesn't serialize dictionaries even if you pray in $$anonymous$$lingon.
private Dictionary<string, int> dictionary = new Dictionary<string, int>();
public void OnEnable()
{
Debug.Log("OnEnable!");
Debug.Log("Count:"+keys.Count+"::"+values.Count);
string a$$anonymous$$ey;
int aValue;
for(int index = 0; index < keys.Count; index++)
{
a$$anonymous$$ey = keys[index];
aValue = values[index];
dictionary[a$$anonymous$$ey] = aValue;
Debug.Log(a$$anonymous$$ey+":"+aValue);
}
keys.Clear();
values.Clear();
}
public void OnDisable()
{
Debug.Log("OnDisable!");
int index = 0;
foreach(var item in dictionary)
{
Debug.Log(index+":"+item.$$anonymous$$ey+":"+item.Value.ToString());
keys.Add(item.$$anonymous$$ey);
values.Add(item.Value);
}
dictionary.Clear();
Debug.Log("Count:"+keys.Count+"::"+values.Count);
}
public void AddItem(string a$$anonymous$$ey, int anInt)
{
dictionary[a$$anonymous$$ey] = anInt;
EditorUtility.SetDirty(this);
}
public Dictionary<string, int>.Enumerator GetEnumerator()
{
return dictionary.GetEnumerator();
}
}
I struggled with this as well. This was one of the reasons I made my serializable dictionary a ScriptableObject; It just works better there.
Sure. But then I can't attach the class to game objects anymore, and the data I'd be storing in the dictionary is proper to one game object.
Indeed I could do the same thing on a single ScriptableObject, storing the data for all the game objects this class was initially intended to. It's that from a conceptual perspective the data belongs to the objects.
Answer by christophfranke123 · Oct 16, 2014 at 05:05 PM
Hi everyone, I found a nice solution to make unity serialize dictionaries, using the ISerializationCallbackReceiver
interface, see serialization in unity for more information. You can also find an example of the serialized dictionary.