- Home /
Multi Channel Audio
I am trying to figure out what is the best way to have a multi channel song play in Unity.
My use case is simple: I have a song sliced up to different channels (e.g. drums, bass, lead) and I want to play them all in sync, and be able to toggle each channel on or off.
What I have learned so far:
Unity documentation talks about multi channel audio clip, but not in depth.
As far as I understand, a multi channel audio clip must be a tracker module (mod, xm, it, s3m) and I cannot mute/unmute its individual channels.
I was able to successfully play S3M and IT files, but without control over individual channels.
The only working option I found, was to have several audio sources, each with a single channel clip attached. I have some performance, memory and synching concerns with this method.
If anyone has any experience with this, it would be great if you can share.
Answer by justinbowes · Mar 18, 2015 at 02:58 PM
I had the same issues, so I brought in a native C# module player (SharpMik) and wired it up as an audio plugin.
For the code that follows, you'll need Unity Singleton, and the source of SharpMik. I put the SharpMik source in Assets/Plugins, deleted some broken module loaders, Naudio and XNA drivers, and some unused WIP source files (effects.cs). Alternately, you could use the included project files to build a DLL. I did not do this as I expect to have to expose some channel mixing functions to Unity at some point.
Save your modules as TextAssets (by adding .bytes to the filename) to bypass Unity's normal module handling.
using UnityEngine;
using System;
using System.Collections;
using System.IO;
using SharpMik;
using SharpMik.Player;
public class SharpModManager : Singleton<SharpModManager> {
internal class UnityVSD : SharpMik.Drivers.VirtualSoftwareDriver {
/**
* Note: The API for IModDriver has its origins in C APIs where
* 0 indicates success and nonzero is a specific failure code.
*
* Therefore, functions returning bool return
* - true on FAILURE;
* - false on SUCCESS.
*/
public AudioSource AudioSource {
get { return audioSource;}
set {
audioSource = value;
applyAudioSource();
}
}
AudioSource audioSource { get; set; }
sbyte[] playerBuffer = new sbyte[0];
bool playing;
public UnityVSD() {
m_Name = "Informi SharpMik VSD";
m_Version = "0.0.1";
m_HardVoiceLimit = 0;
m_SoftVoiceLimit = 255;
m_AutoUpdating = true;
}
public override void CommandLine(string data) {
ModDriver.Mode |= SharpMikCommon.DMODE_16BITS;
ModDriver.Mode |= SharpMikCommon.DMODE_STEREO;
ModDriver.Mode |= SharpMikCommon.DMODE_INTERP;
// ModDriver.Mode |= SharpMikCommon.DMODE_HQMIXER;
}
public override bool IsPresent() {
int sampleRate = AudioSettings.outputSampleRate;
if (sampleRate > ushort.MaxValue) {
Debug.LogError("Unable to use selected audio source: audio sample rate must be < " + ushort.MaxValue + " (is: " + sampleRate + ")");
return false;
}
return true;
}
public override bool Init() {
if (base.Init()) {
Debug.LogError("Underlying virtual driver failed initialization");
return true;
}
ModDriver.MixFreq = (ushort)AudioSettings.outputSampleRate;
return false;
}
public override bool PlayStart() {
playing = !base.PlayStart();
syncState();
return !playing;
}
public override void PlayStop() {
base.PlayStop();
playing = false;
syncState();
}
public virtual void Mix32f(float[] dataIO, int channelsRequired, float gain) {
// We force 16-bit mixing, so we're working in 2-byte chunks.
uint outLen = (uint)dataIO.Length * 2;
float normalize = gain / 32768f;
if (playerBuffer.Length < outLen)
playerBuffer = new sbyte[outLen];
uint inLen = WriteBytes(playerBuffer, outLen);
for (uint w = 0, r = 0; r < inLen; ++w, r += 2) {
dataIO[w] = ((playerBuffer[r] & 0xff) | (playerBuffer[r + 1] << 8)) * normalize;
}
}
void applyAudioSource() {
syncState();
}
void syncState() {
if (playing) {
if (audioSource != null) {
audioSource.Play();
}
} else {
if (audioSource != null) {
audioSource.Stop();
}
}
}
}
[Range(0f, 2f)]
public float
FinalGain = 0.75f;
public TextAsset ModuleAsset;
public bool IsPlaying { get { return player != null && player.IsPlaying(); } }
MikMod player;
UnityVSD driver;
Stream songStream;
TextAsset lastModuleAsset;
void Start() {
player = new MikMod();
bool failedInit;
driver = player.Init<UnityVSD>("command line", out failedInit);
if (failedInit) {
player.Exit();
player = null;
driver = null;
return;
}
driver.AudioSource = Instance.GetOrAddComponent<AudioSource>();
DontDestroyOnLoad(driver.AudioSource);
}
void OnAudioFilterRead(float[] dataIO, int channelsRequired) {
if (player == null || driver == null)
return;
driver.Mix32f(dataIO, channelsRequired, FinalGain);
}
void Update() {
if (lastModuleAsset != ModuleAsset) {
applyModule(ModuleAsset);
lastModuleAsset = ModuleAsset;
}
}
void applyModule(TextAsset apply) {
if (apply == null) {
unload();
return;
}
if (player == null) {
Debug.LogError("Cannot play: player failed to initialize");
return;
}
songStream = new MemoryStream(apply.bytes);
player.Play(songStream);
}
void unload() {
player.Stop();
player.UnLoadCurrent();
songStream.Close();
}
public void MuteChannel(int channel) {
if (player == null)
return;
player.MuteChannel(channel);
}
public void UnMuteChannel(int channel) {
if (player = null)
return;
player.UnMuteChannel(channel);
}
}
This should be sufficient for the use case you describe.
Answer by aka3eka · Jul 08, 2019 at 08:33 AM
@justinbowes , this is awesome! Works like magic. Thanks a lot for your wrapper code!
Btw there are some bugs in SharpMik though. For example, the method TogglePause() in MikMod.cs should call ModPlayer.Player_TogglePause();, not ModPlayer.Player_Paused();. And of course, ModPlayer.cs is a real spaghetti code ;) But it works!
I also added a few methods to control playback speed (bpm), volume, etc. and it works just fine.
Is it possible to share this working/corrected code? I'm learning program$$anonymous$$g, so I think it will be hard to figure out how to fix everything for now (but I have a huge background in audio/mixing) :) I'm struggling to have multiple audio files playing in sync. When they play in sync I have buffer problems with crackling slowed down play. When I remove the sync they play good, but don't work for my purpose. :(
Answer by mdahlgrengadd · Oct 17, 2018 at 10:30 PM
Thanks! This still works very well! Had to comment out all debug messages in the code too, though.
Answer by derekstutsman · May 02 at 07:28 PM
Does playing the audio in this fashion burn a lot of CPU?
I also want to shift between synchronized tracks but thought it might be passable to just have multiple versions of the same song mp3 and swap at the same play position.
Your answer
Follow this Question
Related Questions
Specifying OnAudioFilterRead()'s audiosource 1 Answer
GetSpectrumData over entire file 1 Answer
Can not play a disabled audio source 2 Answers
Glitched Music 0 Answers
PlayScheduled or PlayDelayed 0 Answers