- Home /
Beat detection algorithm
I'm currently working on an idea for a game i have that involves beat detction. Th engine im working with is Unity, and I've never had any experience with audio, coding wise, so be gentle :)
I've looked at several articles and tested out several algorithms including some of my own, but none we're really successful nor accurate enough, and i feel like I've been getting something wrong this entire time.
Specifically I've tried implementing the idea's presented here: http://archive.gamedev.net/archive/reference/programming/features/beatdetection/index.html
but with little success, i still think im skipping over something and i cant quite pinpoint it.
If someone could provide an explanation about how to make an actually accurate beat detector i would be very grateful.
this is my latest attempt at beat detection: http://pastebin.com/BD8y9tfz
in this i used in the link i posted above to compute the instant energy from the 1024 samples i took, and then i used to calculate the local average sound energy from the buffer containing all the previous instant energy calculations, then i checked if there is a significant rise in instant energy compared to the average local sound energy, if there is, it means there is a beat, if there isn't, the program continues as usual.
then i added the implementation for and , still not working though. added a bit of debug, and for some reason the constant is ridiculously small, numbers like:
Constant: -103416 and Constant: -54793.28, ive got no clue why im getting these numbers, any help?
@SgtPancakes. I'm interested in this as well. What type of variable are you using to hold the constants? It looks like you have wrapped around. Are you using double?
@Andy$$anonymous$$artin458 its a float, i wrapped it around because for some reason it treated it as a double.
Shouldn't affect calculations though, still dont know why its giving such bizarre results
C is your sensitivity and defines the intensity of the beat which can be detected.
It is a number you choose. Define your own C and put it back in. Vary it between 1 -> 1.5 at first to see what effect it has on your FFT.
You can compute a 'best' value but as you already know a good working range, define it yourself and vary it to see what works best for your case.
You could implement a Z-transform to make a digital filter. This could make detection easier with less sensitivity, but I'm unsure how well Unity would cope with that combined with the processing of your FFT.
@meat5000 first of all, thank you for your reply! if I set the C myselt then what does the 3rd algorithm in the gamedev article do? I thought it was to automatically calculate the C through the variance, or did i understand it wrong?
Also, im not sure if i should use getOutputData or getSpectrumData, and if spectrumdata, which FFTWindow should i use? I STILL dont know what those windows even mean.
C will be C whether its calculated or deter$$anonymous$$ed by you. In order to help with your calculation its best to find out the working range for yourself. In this case you know its not 10 million :D
The three algorithms on the site you are referencing are not meant to be used together but are three separate methods (I think, I havent read the whole thing). Pick one.
Engineers keep tests controlled by setting values so they can see whats going on, rather than having everything a dynamically changing mess. You can't know how high you are flying unless you know where the floor is :)
Plug values you expect in to your algorithm, vary them bit by bit to find out the best value and working range. You have a starting point already, as the numbers are there on the site :)
FFT is hairy stuff. Don't expect to get it right first time :)
A window is like a mask, I guess. The shape and size affects what you get out and also how that needs to be handled. I think you will need to brush up on Convolution. That part is easier than it appears.
Answer by mylovemhz · Feb 08, 2015 at 08:05 PM
In case anyone still needs help with this, I've made a simple beat detection API you can use on your GameObject. It's at my Github below:
Hey, nice work with the API. But is there a way to get if the beat is within a specific spectrum? for example a snare or a rock riff starts? im new to audio codes and stuff.. so a little help with be appreciated! :)
@Nilakash Unfortunately, the current API is extremely simple. I just wanted to detect when beat onsets happened in music. It can be extended to add more complex functionality though. I would start by taking the signal and EQing it to isolate the frequencies I want to check for. Then run the beat detection algorithm on that. In the next couple of weeks, I'll work on the detector to add some frequency isolation. However, it won't be able to isolate multiple instruments form a single source (At least not in an efficient way that won't slow your game down dramatically.) For something like that, C# is much too slow. It would have to be written in C and then compiled into a DLL file and you can reference that DLL file from visual studio.
Hey @mylovemhz I am reading your code and I see it has copyright. Do I have to buy the license in order to have the right to sell my game with your code or can I just credit your work on my game with no worries?
Answer by SgtPancakes · Jun 24, 2014 at 12:06 AM
@meat5000 MOTHER OF GOD, I've been doing it all wrong! I've been using getOutputData all this time, which from what i understand gives you the db of a certain sample length, what i've been needing to do is change it to getSpectrumData which gives the intensity of a certain sample length, thats why it hasnt been working in loud songs like the one i sent, because it was WAAAAY too loud for it to register anything, but now that it checks for intensity it doesnt even care for the volume.
Still needs some work though, although i can definitely go somewhere from here. ill edit and add the final piece of code once i get it somewhat more accurate, but for now this is a working program: http://pastebin.com/MF63GpGd
Hi,I read the code u wrote up there. Can u tell me how did this algorithm work??
Answer by Vacciones_Mad · Mar 17, 2015 at 08:30 AM
Don't know if the thread is dead or what but if anyone here stills wants it here is a slightly changed version of Sgt's script that worked for me, cudos go to SgtPancakes and whoever wrote the article posted by Sgt.
using UnityEngine;
using System.Collections;
public class SpectrumDetektor : MonoBehaviour {
//public AudioClip song;
public AudioSource song;
//public GameObject cube;
private bool Beated;
private float[] historyBuffer = new float[43];
private float[] channelRight;
private float[] channelLeft;
private int SamplesSize = 1024;
public float InstantSpec;
public float AverageSpec;
public float VarianceSum;
public float Variance;
public float Constant;
// Use this for initialization
void Start () {
Beated = false;
}
// Update is called once per frame
void Update () {
//compute instant sound energy
//channelRight = song.audio.GetSpectrumData (1024, 1, FFTWindow.BlackmanHarris); //Normal
//channelLeft = song.audio.GetSpectrumData (1024, 2, FFTWindow.BlackmanHarris); //Normal
//channelRight = song.GetSpectrumData (1024, 1, FFTWindow.Hamming); //Rafa
//channelLeft = song.GetSpectrumData (1024, 2, FFTWindow.Hamming); //Rafa
//InstantSpec = sumStereo (channelLeft, channelRight); //Normal
InstantSpec = sumStereo2(song.GetSpectrumData(SamplesSize, 0, FFTWindow.Hamming)); //Rafa
//compute local average sound evergy
//AverageSpec = sumLocalEnergy ()/historyBuffer.Length; // E being the average local sound energy //Normal
AverageSpec = (SamplesSize / historyBuffer.Length) * sumLocalEnergy2(historyBuffer); //Rafa
//calculate variance
//VarianceSum = 0;
//for (int i = 0; i < 43; i++) //Normal
//VarianceSum += (historyBuffer[i]-AverageSpec)*(historyBuffer[i]-AverageSpec);
//Variance = VarianceSum/historyBuffer.Length; //Normal
Variance = VarianceAdder(historyBuffer) / historyBuffer.Length; //Rafa
Constant = (float)((-0.0025714 * Variance) + 1.5142857); //Normal
float[] shiftingHistoryBuffer = new float[historyBuffer.Length]; // make a new array and copy all the values to it
for (int i = 0; i < (historyBuffer.Length - 1); i++) { // now we shift the array one slot to the right
shiftingHistoryBuffer[i+1] = historyBuffer[i]; // and fill the empty slot with the new instant sound energy
}
shiftingHistoryBuffer [0] = InstantSpec;
for (int i = 0; i < historyBuffer.Length; i++) {
historyBuffer[i] = shiftingHistoryBuffer[i]; //then we return the values to the original array
}
if (InstantSpec > (Constant * AverageSpec)) { // now we check if we have a beat
if(!Beated) {
Debug.Log("Beat");
Beated = true;
}
}
else {
if(Beated) {
Beated = false;
}
Debug.Log("No Beat");
}
//Debug.Log ("Avg local: " + E);
//Debug.Log ("Instant: " + e);
//Debug.Log ("History Buffer: " + historybuffer());
//Debug.Log ("sum Variance: " + sumV);
//Debug.Log ("Variance: " + V);
//Debug.Log ("Constant: " + constant);
//Debug.Log ("--------");
}
float sumStereo(float[] channel1, float[] channel2) {
float e = 0;
for (int i = 0; i<channel1.Length; i++) {
e += ((channel1[i]*channel1[i]) + (channel2[i]*channel2[i]));
}
return e;
}
float sumStereo2(float[] Channel) {
float e = 0;
for (int i = 0; i < Channel.Length; i++) {
float ToSquare = Channel[i];
e += (ToSquare * ToSquare);
}
return e;
}
float sumLocalEnergy() {
float E = 0;
for (int i = 0; i<historyBuffer.Length; i++) {
E += historyBuffer[i]*historyBuffer[i];
}
return E;
}
float sumLocalEnergy2(float[] Buffer) {
float E = 0;
for (int i = 0; i < Buffer.Length; i++) {
float ToSquare = Buffer[i];
E += (Buffer[i] * Buffer[i]);
}
return E;
}
float VarianceAdder (float[] Buffer) {
float VarSum = 0;
for (int i = 0; i < Buffer.Length; i++) { //Rafa
float ToSquare = Buffer[i] - AverageSpec;
VarSum += (ToSquare * ToSquare);
}
return VarSum;
}
string historybuffer() {
string s = "";
for (int i = 0; i<historyBuffer.Length; i++) {
s += (historyBuffer[i] + ",");
}
return s;
}
}
Answer by BadSeedProductions · Sep 23, 2015 at 01:54 AM
This was picking up some beats but not "the beat". I tried with fur elise (piano), which seemed to do the best, picking up the beat here and there and then dropping off again. fatboy slims funk soul brother picked up 1 beat when the song started and that was it, and the doors 'love her madly' picked up 2 beats during the entire song. I can see this working with a simple metronome sound possibly, but as for music, no way.