koobas.hobune.stream
How can I control an AudioSource from the audio thread?
Hi. I am looking for a way to call audioSource.Play() from the audio thread, specifically from inside an OnAudioFilterRead callback. Currently, attempting to do so (understandably) causes an error stating that these methods must be called from the main thread.
Overview
I have a game that implements an audio step sequencer in OnAudioFilterRead. As you may know, to receive OnAudioFilterRead callbacks, the script must be attached to a game object with a playing AudioSource component.
The player can create many of these step sequencers and chain them together, causing one sequencer to play after another. When one sequencer completes its playback, I wish to begin playback of the next sequencer in the chain. All sequencers share the same metronome object, which can be queried each audio frame using the DSP time to retrieve the beats in the current audio buffer. This means that as long as I can ensure the OnAudioFilterRead callback of the next sequencer is rendering (i.e. its audio source is playing), it will pick up the next metronome beat in perfect beat-sync.
As such, the command to begin playback of the next sequencer is (audio) time critical – it must arrive before the next metronome beat (and ideally the same/next audio frame) in order to maintain beat-sync. It is for this reason that I would like to begin playback of the audio source with audioSource.Play() from the audio thread.
As an aside, I have tried picking up the command to play the next sequencer in the next Update cycle, and thus moving it to the main thread, however this is unreliable as to maintaining beat synchronisation. The command will sometimes arrive too late (the metronome's next beat has already occurred on the audio thread) and the sequencer will slip by one beat at the changeover point.
Current Solution
My current solution is to have all audio sources always playing and to control playback of the audio step sequencer script with its own IsPlaying flag. This removes the need to play/stop audio sources and I can control my sequencer playback from the audio thread easily by setting the flag. This works well for maintaining beat-synchronisation as the step sequencer's flag is set immediately and is guaranteed to be rendering on the next audio frame.
However, where this solution fails is that Unity (or FMOD) imposes a limit of 255 audio sources playing at once (the maximum number or real voices). The player can create long chains with potentially hundreds of sequencers and therefore hundreds of audio sources. By leaving audio sources playing and controlling playback with my own flag, I hit this limitation when the player creates more than 255 sequencers (even if none are actually playing via the IsPlaying flag). The 256th+ audio sources will not be played by the engine, and thus the OnAudioFilterRead callbacks will not be invoked. (To be clear, I'm not playing 255 sounds at once - I only have them 'playing' to ensure their OnAudioFilterRead callback is invoked.)
Desired Solution
I would like to be able to toggle each audio source's playback from the audio thread. This would allow me to have hundreds of audio sources present in the scene as outlined above, and toggle their playback whilst maintaining beat synchronisation. Is there a way I can do this?
Thanks in advance. -andy.
P.S. This is a link to my game running – it might help to illustrate the points made – https://www.youtube.com/watch?v=MpeOsIUgUoM. Each lily flower is an audio step sequencer, the lily pads are the mechanism by which players 'chain' sequencers together.
Hello there,
And first, thank you for making your post so clear. It really helps.
Second, this is my go-to solution to "work around" the threading problem in Unity:
• Create a Script called "Dispatcher"
• Throw this code in it:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;
public class Dispatcher : MonoBehaviour
{
public static Queue<Action> ExecuteOnMainThread = new Queue<Action>();
public void Update()
{
while (ExecuteOnMainThread.Count > 0)
{
ExecuteOnMainThread.Dequeue().Invoke();
}
}
}
• Throw that script on an object in the scene
• Now, anytime you have a line of code that says "can only be called from the main thread", execute it like this:
DispatcherScript.ExecuteOnMainThread.Enqueue(() =>
{
//Put your code in here
Debug.Log("This is a debug log called on the main thread!");
Debug.Log("This is another debug log!");
});
I use this to handle network events, or to know when an external EXE I've started has stopped running. So far it's always worked, but in your case I would check for potential delays.
I haven't played your video (at work atm), but if you're going for a perfect beat-sync even the slightest delay could be an issue.
I hope that helps!
Cheers,
~LegendBacon
Hi. Thanks so much for the response. A couple of points:
We have to be careful allocating memory on the audio thread, which the
Enqueuewill do. Code executing on the audio thread should be deter$$anonymous$$istic/predicatable as to its running time, which memory allocation isn't. This is to avoid starving the DSP of audio samples and thus causing clicks/pops. (If you're interested this is a good post on the topic). Ins$$anonymous$$d I tried a solution that simply toggles a flag on the audio thread, which the Update loop checks for. Which leads me to:It is not guaranteed that the main thread will pick this up in time. This solution will sometimes work, and sometimes it will miss a beat. I believe it is because if the main thread gets busy for a frame or two for whatever reason, it can cause the Update to be invoked too late (after the next audio frame). On slower machines this can also become more pronounced. I'd like to be able to guarantee my audio source has begun playing by the next audio frame, which I'm not sure how I can do without directly calling it from the audio thread.
