Unity Editor Scripting - Saving data in scriptable object problem

So I have a custom inspector, where you can add objects (meshes, audioclips, a variety of stuff) to it. Before these things get officially added (you have to click apply), they are stored in a scriptable object. When I create the scriptable object, I set all those variables' hideflags to HideAndDontSave (as well as the scriptable object itself).

However as soon as I click the object to show the inspector, it will ALWAYS ask me to save the scene if I leave. Even though if didn't change anything.

And if I do press save, it's saying that my custom inspector script has leaked 2 objects (3,4,5 depending on how many times I click on and away from it).

I am aware that this warning is not detrimental, but it is annoying and I want to get rid of it. As far as I know, there's no way to tell what exactly Unity is cleaning. I don't want to have to save my scene every time.

Are there common methods to weeding out what is causing the memory leak? Is there a step that I am missing? Or is there a better way of storing these variables (if it is the cause).