How to (de) serialize unity references at runtime?

Question

When you save scene with some MonoBehaviour that has public Texture field with texture asset assigned, it serializes in scene file as:

 testTextureField: {fileID: 2800000, guid: bcffaa4441161f641a48d989be58c7c2,
       type: 3}

I can imagine how this deserializing on loading scene in editor (asset path by guid). But scenes also can be loaded at runtime! I mean, there is all the "assets" somewhere after build, but AFAIK guid makes sense only in editor? Then how it knows which object is referenced only by this info? What is fileID in this case?

tl;dr: How to get some unique ID for asset (Object) which is constant at runtime (and editor) and between sessions, and then get asset (Object) by this ID?

inb4: "make hashtable database of all the Objects" - is this the only way? Scenes doing it somehow w/o.

Answer 1

Best Answer

Answer by Servail · May 25, 2019 at 01:56 PM

So, the main problem was that I can't retrieve any "path" information from the Object: once it's loaded, it has no information where it comes from (it was pretty logical, I guess). But it has some information about what it is! I mean hashcode. Currently my solution is ScriptableObject with Hashtable asset map, which I populate in Editor just before build with AssetDatabase.GetAllAssetPaths (not documented?!) as values and corresponding Object hashcodes as keys. Then just Resource.UnloadUnusedAssets. At Runtime, I serialize Object hashcodes, and deserialize by Resources.Load(assetMap[hashcode]). For scene objects (FindObjects(Object)) I use simple List(Object), then reinstantiate and rereference by its indexes.

UPD: Since hashes are not hashes, looks like the only reliable piece of info is Object name, which is not that useful. Solution is in the middle: it's hybrid of (Resource) AssetMap which is Dictionary/Hashtable(Object, path/address/key) or list (too unstable between versions, so I dropped) and AssetBundles or Addressables package. On deserializarion I just can load direct asset. On serialization I need to load AssetMap and look for path by Object, then unload (still weird). P.S.: I still think it can be much easier, like some direct access through Object for real.

Add comment · Show 2 · Share

Bunny83 · May 25, 2019 at 03:02 PM 0

Share

Actually GetHashCode is not a persistent reliable indicator. UnityEngine.Object derived classes like ScriptableObjects just return the instance ID as hashcode.

The InstanceID is always guaranteed to be unique but not guaranteed to be persistent across sessions or builds

Servail Bunny83 · May 25, 2019 at 05:20 PM 0

Share

Then the only 100% solution to this I see is to store actual references in list (which is hella memory consu$$anonymous$$g, bad even if used just for save/load), then serialize index by Object/deserialize Object by index (which is slow?). I'll do this if above method fails hard. Anyway, that GetHashCode override? Just to AppleInstance("red") and AppleInstance("green") were equal? Looks counterintuitive.

Answer 2

You can't create references to objects from the AssetDatabase on runtime. Then only exception are Resource assets.

The main reason is, that Unity builds exclude all non-referenced assets. So there must be a known web of references to difrerent assets at runtime. The FIleID and the way how unity assets serialize references is nothing you can affect or re-use.

You can, however, serialize 'references' to the Resources object, by serializing its path and type. And you can easily create custom serialized struct that holds these values inside and does the Resources.Load for you. You an even create editor drawers to make the inspector for this struct look like an object field.

Here's my Resource struct

     using System.Collections;
     using System.Collections.Generic;
     using UnityEngine;
     
     namespace Framework
     {
     
         [System.Serializable]
         [Serialization.Serializable]
         public struct Resource : Serialization.ISerializationCallbackReceiver, ISerializationCallbackReceiver
         {
     
             public static Resource empty
             {
                 get { return new Resource(); }
             }
     
             [System.Serializable]
             [Serialization.Serializable]
             public struct Address
             {
                 public Address(string _uri, System.Type _type)
                 {
                     uri = _uri;
                     typeName = TypeToString.GetTypeName(_type);
                 }
     
                 public static bool operator ==(Address lhs, Address rhs)
                 {
                     return Equals(lhs, rhs);
                 }
     
                 public static bool operator !=(Address lhs, Address rhs)
                 {
                     return !Equals(lhs, rhs);
                 }
     
                 public static bool Equals(Address lhs, Address rhs)
                 {
                     return lhs.uri == rhs.uri && lhs.typeName == rhs.typeName;
                 }
     
                 public override int GetHashCode()
                 {
                     return uri.GetHashCode() ^ typeName.GetHashCode();
                 }
     
                 public override bool Equals(object obj)
                 {
                     if (obj is Address)
                     {
                         var other = (Address)obj;
                         return Equals(this, other);
                     }
                     return false;
                 }
     
                 [Serialization.SaveAsValue, SerializeField] public string uri;
                 [Serialization.SaveAsValue, SerializeField] public string typeName;
             }
     
             public static bool Equals(Resource lhs, Resource rhs)
             {
                 return Address.Equals(rhs.address, lhs.address);
             }
     
             public static bool operator ==(Resource lhs, Resource rhs)
             {
                 return lhs.address == rhs.address;
             }
     
             public static bool operator !=(Resource lhs, Resource rhs)
             {
                 return lhs.address != rhs.address;
             }
     
             public override int GetHashCode()
             {
                 return address.GetHashCode();
             }
     
             public override bool Equals(object obj)
             {
                 if (obj is Resource)
                 {
                     var other = (Resource)obj;
                     return Equals(this, other);
                 }
                 return false;
     
             }
     
             public Resource(Address address) : this()
             {
                 m_address = address;
                 m_dirty = true;
             }
     
             public void SetDirty()
             {
                 m_dirty = true;
             }
     
             private UnityEngine.Object m_obj;
             [SerializeField][Serialization.SaveAsValue] private Address m_address;
             private bool m_dirty;
     
             public Address address
             { get { return m_address; } }
     
             public UnityEngine.Object obj
             {
                 get
                 {
                     if ( m_dirty )
                     {
                         m_dirty = false;
                         try
                         {
                             var type = TypeToString.ParseType(m_address.typeName);
                             // a hotfix for the new namespace
                             if(type == null)
                                 type = TypeToString.ParseType("GrayZone."+m_address.typeName);
     
                             m_obj = Resources.Load(m_address.uri, type);
                         }
                         catch
                         {
                             m_obj = null;
                         }
                     }
     
                     return m_obj;
                 }
             }
     
             public void OnBeforeSerialize()
             {
             }
     
             public void OnAfterDeserialize()
             {
                 m_dirty = true;
             }
             
     
     #if UNITY_EDITOR
             public static Address Editor_GetObjectAddress(Object obj)
             {
                 Address address = new Address();
                 if (obj)
                 {
                     var assetPath = UnityEditor.AssetDatabase.GetAssetPath(obj);
                     var split = assetPath.Split(System.IO.Path.PathSeparator, System.IO.Path.AltDirectorySeparatorChar);
                     var newPath = "";
                     var resFound = false;
                     for (int i = split.Length - 1; i >= 0; --i)
                     {
                         var part = split[i].ToLower();
                         if (part == "resources")
                         {
                             resFound = true;
                             break;
                         }
                         newPath = System.IO.Path.Combine(part, newPath);
                     }
                     if (resFound)
                     {
                         newPath = System.IO.Path.ChangeExtension(newPath, null);
                         address.typeName = TypeToString.GetTypeName(obj.GetType());
                         address.uri = newPath;
                     }
                     else
                     {
                         address.typeName = null;
                         address.uri = "";
                     }
                 }
                 else
                 {
                     address.uri = "";
                     address.typeName = null;
                 }
                 return address;
             }
     #endif
         }
     }

And the drawer

 using UnityEditor;
 using UnityEngine;
 using System.Collections.Generic;
 using System.Reflection;
 
 namespace Framework.Editor
 {
     [CustomPropertyDrawer(typeof(Resource))]
     [CustomPropertyDrawer(typeof(ResourceAttribute))]
     public class ResourceDrawer : PropertyDrawer
     {
 
         public System.Type GetResourceType()
         {
             var attrib = attribute as ResourceAttribute;
             var resType = attrib != null ? attrib.resourceType : typeof(Object);
             return resType;
         }
         
         public override void OnGUI( Rect position, SerializedProperty property, GUIContent label )
         {
             //var objProperty = property.FindPropertyRelative("m_obj");
             //EditorGUI.PropertyField(position, objProperty, label, true);
             var names = property.propertyPath.Split('.');
             var targets = property.serializedObject.targetObjects;
             var results = new List<Resource>();
             EditorReflectionUtility.GetVariable(names, targets, results);
 
             Object obj = results[0].obj;
             for ( int i = 1; i < results.Count; ++i )
             {
                 var otherObj = results[i].obj;
                 if ( otherObj != obj )
                 {
                     EditorGUI.showMixedValue = true;
                     break;
                 }
             }
 
             EditorGUI.BeginChangeCheck();
             GUI.backgroundColor = new Color(0.9f,0.9f,1f,1f);
             var newObject = EditorGUI.ObjectField(position, label, obj, GetResourceType(), false);
             GUI.backgroundColor = Color.white;
             if ( EditorGUI.EndChangeCheck() )
             {
                 Undo.RecordObjects(targets, "Setting Resource");
                 var newRes = new Resource(Resource.Editor_GetObjectAddress(newObject));
                 newRes.SetDirty();
                 EditorReflectionUtility.SetVariable(names, targets, newRes);
 
                 foreach ( var targ in targets )
                 {
                     EditorUtility.SetDirty(targ);
                     var onValidate = targ.GetType().GetMethod("OnValidate", BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Public);
                     if ( onValidate != null)
                         onValidate.Invoke(targ, null);
                 }
 
             }
             EditorGUI.showMixedValue = false;
             return;
         }
 
     }
 }

Just as an example, not all code dependencies are included

Answer 3

Answer by Bunny83 · May 21, 2019 at 09:33 AM

Unity doesn't provide any runtime function yet to translate an asset GUID to an object. The AssetGUID can be translated in editor code to an asset path with AssetDatabase.GUIDToAssetPath. The fileID usually references the YAML internal instance ID. However there are some special IDs (for those with external references). Some represent hardcoded internal Assets (usually with a fix GUID with all or most digits being 0). Others, like in your case, the fileID essentially represents the asset type (28 in your case which is a Texture2D)..

The Unity Engind does have the assetdatabase of the included asset available internally but doesn't provide any direct access to it. So yes, you would need to create your own mapping. if you really want to work with the GUIDs manually. Note that when you build your game the scene file usually isn't in YAML anymore. In a build the scenes are serialized in the binary asset format.

From your question it's not clear what you actually want to achieve.

Add comment · Show 3 · Share

Servail · May 21, 2019 at 09:52 AM 0

Share

Note that when you build your game the scene file usually isn't in YA$$anonymous$$L anymore.

That's what I missed. I thought they use some internal serialization mechanism, which I'd likely can utilize, like ResourceDatabase. I'm writing scene serialization logic (save game snapshot), the only problem left are references, and the only solution I see is to populate my own ResourceDatabase hashtable with Resources.FindObjectsOfTypeAll (as values) right before build, then serialize references as hashes (which is keys). Then get Objects by hashes on deserializatiion. Is there any better way? UPD: Looks like Resources.FindObjectsOfTypeAll returns only objects that someway "loaded"? Like, it didn't returned any AudioClips until I clicked some in Project folder... Is it expected behaviour?

Bunny83 Servail · May 21, 2019 at 03:55 PM 0

Share

FindObjectsOfTypeAll returns only objects that someway "loaded"

Yes, this is the expected behaviour ^^. Anything else would be a total killer. If you have a project with 10000 assets using FindObjectsOfTypeAll would otherwise need to load all those objects physically into memory since you expect the instances to be returned.

It's not clear what kind of references you want / need to serialize. If you only care about assets you build into your game / application and you need them all to be available for dynamic access you can use the Resources folder or just use a scriptableObject with a List where you "register" all your assets in the inspector. So the SO would act like a "database" for all those assets. Be aware that the way you setup your asset references can have a huge impact on your game loading time. If all your assets are in Resources folders or referenced by a GOD object that is loaded with the first scene, all those assets will be loaded into memory at start.

For the Resources folders I've once written this ResourceDB to actually keep track of the folder structure at runtime so you can actually query which assets are in a certain virtual folder.

If you plan to have a lot of assets you may be better of using AssetBundles which can be loaded on demand. Each Bundle could contain a "sub database" object which organises all the references to the objects in the bundle. There is no general solution to this issue. It highly depends on how many assets, what kind of assets and where and when they are needed.

ps: Here's the documentation of Resources.FindObjectsOfTypeAll:

This function can return any type of Unity object that is loaded,

Servail Bunny83 · May 22, 2019 at 07:42 AM 0

Share

scriptableObject with a List where you "register" all your assets in the inspector

That's not what I wanted, but that's what I thought I must do.

all those assets will be loaded into memory at start

That's absolutely unacceptable! I need pointer-like reference to asset, which loads only when I do something like: Texture testTexture = GetObjectByPointer(pointer); On serialization it'll be: testTexture = GetPointerByObject(testTexture); Where rough pointer example is "file path" - something that persistent everywhere. As I can assume, assets doesn't have "file path" on runtime. So, the only way to have asset pointers and free memory is to store all assets in Resources folder, and load them on demand? I'm fine with build size. (Or am I misunderstanding Resources purpose?) Still want to use editor put-asset-on-public-field functional anyway.

How to (de) serialize unity references at runtime?

3 Replies

Your answer

Follow this Question

Related Questions