How to use EditorGUI.ObjectField within a custom property drawer without resetting field each time?

I'm trying to not ask too many questions here, but I'm having a really tough time with this and am too stubborn to drop it. I'm trying to use a custom drawer with an object field, but every time the GUI updates, the field is reset. I figured out why, but I can't figure out how I'm supposed to do it. Here's my code:

 [CustomPropertyDrawer(typeof(OverlayObject))]
 public class OverlayObjectDrawer : PropertyDrawer {
     TilemapStructure ts;
     public override void OnGUI(Rect position, SerializedProperty property, GUIContent label) {
         EditorGUI.BeginProperty(position, label, property);
         {
             var serializedTilemapType = property.FindPropertyRelative("tilemapTypeOverlay");
             var tilemapHeight = EditorGUI.GetPropertyHeight(serializedTilemapType);
             var tilemapRect = new Rect(position.x, position.y, position.width, tilemapHeight);
             position.y += tilemapHeight;
             
             //Problem is here
             ts = (TilemapStructure)EditorGUI.ObjectField(tilemapRect, "Tilemap Structure", ts, typeof(TilemapStructure), true);
         }
     }
 }

Why this doesn't work is because in each frame, TilemapStructure ts is called, resetting the value to null. How can I set this to be the existing value instead of just defaulting it to null? I've tried property.serializedObject.targetObject which seemed promising but I've only got casting errors. I'm sure the answer is pretty simple, but for the life of me I can't seem to figure it out, so I would really appreciate any help. Thanks!