How to read the Track Binding from a Timeline Behaviour

Question

Goal

Specifically, I wrote a monobehaviour, which I named BlendShapeController, that scans an attached SkinnedMeshRenderer and stores info about any BlendShapes it finds into an array of ScriptableObjects.

So far, so good. I would like to be able to control the blend shape values from a Timeline. Where I am running into problems is getting a reference to the PlayableBehaviour (named BlendShapeControlBehaviour) for the Timeline track in such a way that the BlendShapeControlBehaviour can generate its own array of ScriptableObjects, which the user can employ to animate the blendshape values on the BlendShapeController bound to the track.

Problem

Where I am running into a snag is that, although I am able to obtain references to the BlendShapeController to the PlayableBehaviour, it seems to be at a stage where it is too late to useful. Although Debug.Log() can reference the BlendShapeController in the BlendShapeControlBehaviour or BlendShapeControlMixerBehaviour's ProcessFrame (and a few other methods) , it doesn't stick, and the fields in BlendShapeControlBehaviour are always empty. I suspect I am dealing with a clone of the PlayableBehaviour in ProcessFrame and not the original instance that actually stores the data longer term. I suspect that I need to be getting and processing the reference in the CreatePlayable method of the BlendShapeControlClip but I haven't been able to figure out a way to do this. I tried making the BlendShapeController property in the useful. Although Debug.Log() can reference the BlendShapeController in the BlendShapeControlBehaviour an exposed reference (usually used for inputs, I guess) and then assigned it using Resolve(graph.GetResolver()) after Creating the ScriptPlayable holding the Behaviour, but that does not show in the inspector. Neither does any Blendshape data I attempted to extract and store.

Question

Is there any way to accomplish this without either hardcoding the number of blendshapes in the BlendShapeControlBehaviour or assigning the BlendShapeController manually for each clip? Attached below is a screen of what the BlendShapeController inspector looks like. I would like the clip inspectors to mirror this layout, but without sharing the actual object references. I will be happy to append code, if anyone feels it might be useful, but since none of it seems to be even close to being a viable approach, I didn't think it would. Thanks for any suggestions. [edit]Perhaps I should have posted this in Default?[/edit]

Answer 1

Best Answer

Answer by mdlp · Oct 29, 2018 at 04:59 AM

Figured it out. As far as I can tell, none of the different bits can accomplish that except the TrackAsset.

I put director.SetReferenceValue(blendShapeControlBehaviour.controller.exposedName, _mTrackBinding);

into the GatherProperties Method of the BlendShapeControlTrack

Add comment · Show 1 · Share

mdlp · Nov 02, 2018 at 04:14 AM 0

Share

Completed Timeline set, including monobehaviour, scriptableObjectr for holding blendshape values, and a couple of editors for displaying and animating the data

Updated version: https://drive.google.com/file/d/1GZAvejjcr_chS$$anonymous$$lBJzYDa5JWlNDfHytS/view?usp=sharing

Answer 2

Answer by seant_unity · Oct 29, 2018 at 11:39 AM

If you are writing a custom track with the binding to your BlendShapeController, it will be passed as the playerData argument in ProcessFrame, you just need to cast it to the appropriate type.

I'd recommend doing all the work in BlendShapeControlMixerBehaviour.ProcessFrame. i.e. walk the clips to find which ones are active, do any blending (if required) and write that back to the binding.

GatherProperties is not an appropriate place to store references, as it won't be called in the player.

Add comment · Show 3 · Share

mdlp · Oct 29, 2018 at 09:48 PM 0

Share

Thanks for Replying! The problem isn't accessing the track binding during run time but during editing (writing playback control is much less vexing). The problem with Processframe is that it is (I think) evaluating a clone of the PlayableBehaviour, not the object that holds properties which can be set via the inspector and serialized for future tweaking as well as playback. I need to read BlendShape data from the $$anonymous$$onobehaviour into the BlendShapeControlBehaviour during editing, then generate controls with a CustomEditor , then be able to use the edited data stored in the BlendShapeControlBehaviour to animate the BlendShape properties in the original monobehaviour. Getting the track binding in a processFrame is easy, but the original PlayableBehaviour isn't accessible there (that I can deter$$anonymous$$e), only the copy. If I set the blendshape data in process frame, the duplicate gets the info, but the original does not.I think my scripts are now working, although I am certain that I used many sub-optimal solutions as I am not an ace at writing editor scripts, any more than I am at playables. I will tidy things up and post the script later,

seant_unity mdlp · Oct 30, 2018 at 03:24 PM 0

Share

Yes, the playable behaviour is an always instance, not the original.

I understand the issue now. Another workaround you can try is set the binding on your playableAsset (i.e. clip) derived class as a (non-serialized) property, and assign it in CreateTrack$$anonymous$$ixer() on your TrackAsset-derived class by looping through the TimelineClips. In your inspector, you can use the value, and pass it to the playable behaviour in CreatePlayable() if necessary.

TrackAsset.CreateTrack$$anonymous$$ixer() gets called before PlayableAsset.CreatePlayable(), so this tends to work pretty well, even if it's a bit hackish.

mdlp seant_unity · Nov 02, 2018 at 04:19 AM 0

Share

I ended up using CreateTrack$$anonymous$$ixer(), as you suggested, although it did not solve the main problem I had with GatherProperties, which is that it fired so often that it locked up the inspector controls, making it impossible to change any values. I ended up having to do a bit of filtering. Also, I didn't end up needing to store the actual binding. OnCreateClip sounded ideal, except it never seemed to actually be called. BlendShapeControlBehaviour, just the a clone of the Blendshape data that the binding contained. A link to the "final" (until I find a new bug) package is above.

Answer 3

Answer by ghtx1138 · Dec 12, 2018 at 02:29 AM

@mdlp Thanks for making this code available. I don't really understand what you are talking about but I can see that you've implemented a way to animate Blendshapes on timeline which I do understand (and was looking for).

I'm hoping you can help me to get this code working. I have installed Collections and Maths, changed the project to .net 4, added a track with a skinnedmesh with shapekeys, added the Controller component and added a clip. I can move the sliders in the Clip and the Blendshape movement is visible so I think everything is working to this part. But I cannot add keys and there is no record button.

I can see a Type Mismatch in the M Track Binding field. I'm trying to get my head around this code (and Playables in general) but it is mighty confusing. Would you be able to offer me some advice on how to record keys? Thanks

Add comment · Show 5 · Share

mdlp · Dec 13, 2018 at 05:02 AM 0

Share

@ghtx1138 ·I later discovered some problems with the version I posted here, and forgot to update it, sorry. I will tidy the current version up a bit, and repost, along with some explanation of how to use it tomorrow .The main problem with the current version is that the timeline clips "forget" their settings as soon as you close the scene, since the scriptable objects I was using to store the values weren't serializing correctly with the rest of the timeline. Not sure about the type mismatch, but I think the new version should fix that as well, as I haven't seen any errors of that nature in it. It also has much better handling of transitioning to settable default values.

ghtx1138 · Dec 13, 2018 at 05:29 AM 0

Share

@mdlp thanks for replying. I've dived into Playables and "sort of" have an idea what is going on. I'll look out for your code. No need to apologise please - you've given me a great head start. Cheers!

mdlp ghtx1138 · Dec 15, 2018 at 12:36 AM 0

Share

@ghtx1138 I updated the link to the improved version.

As to how to use the scripts--there is no record function--the clips are the "keyframes"

The attached image shows the timeline in two different positions, one over a part of the track without a clip, and one fully over a clip. The scene mage on the left (1) shows the default settings for the BlendShapeController. which is the monobehaviour that is attached to the Skinned$$anonymous$$eshRenderer. The default settings are what the blendshapes will revert to when the Timeline is not over-riding them with values from a clip. The scene image on the right (2) show the values being provided by a clip

About the inspector tabs, the left one shows the inspector for the monobehaviour. The greyed out "Blend Shape Value" sliders are just informational. The default value sliders are the only ones that are functional. The "Reload BlendShape Targets" button is in case you update the skinned mesh to one with additional blendshapes. It should add the new ones without erasing the existing clip values (but I haven't tested this much). The rightmost inspector tab shows the inspector for a clip. The long guid number is again just informational--it is used by the Track to distinguish when you are creating a new clip and should start with the controller's default values, and when you are duplicating an existing clip, and should start with the values of the original clip. Hope this gets you started, but happy to answer any questions.

[edit]Having difficulty figuring out how to inline this image, but the link below works. [/edit]

https://drive.google.com/open?id=1AGz3SbSimDm7wAQTRELkT5Tp6xrnA3wQ

ghtx1138 mdlp · Dec 16, 2018 at 01:25 AM 0

Share

Hi @mdlp

Thanks very much for sharing your code with me and taking the time to explain how it all works :) It is well documented and the idea of storing "key values" in SO is great. It is very helpful to get me started and to understand Timeline Playables. Cheers!

Show more comments

Answer 4

Answer by dbdenny · Dec 17, 2021 at 06:39 AM

I find a trick from the forum of unity, they assign the binding to behaviour when create the playable of the track.

     public override Playable CreateTrackMixer(PlayableGraph graph, GameObject go, int inputCount)
     {
         foreach (var c in GetClips())
         {
             (c.asset as FacialExpressionClip).parentTrack = this; // this field would be serializable in assets
             (c.asset as FacialExpressionClip).binding = go.GetComponent<PlayableDirector>().GetGenericBinding(this); // not serializable in assets
         }
         return base.CreateTrackMixer(graph, go, inputCount);
     }

Add comment · Share

How to read the Track Binding from a Timeline Behaviour

Goal

Problem

Question

4 Replies

Your answer

Follow this Question

Related Questions