Wayback Machinekoobas.hobune.stream
May JUN Jul
Previous capture 13 Next capture
2021 2022 2023
1 capture
13 Jun 22 - 13 Jun 22
sparklines
Close Help
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
4
Question by SgtPancakes · Jun 23, 2014 at 08:56 PM · c#detectionalgorithmrhythm

Beat detection algorithm

I'm currently working on an idea for a game i have that involves beat detction. Th engine im working with is Unity, and I've never had any experience with audio, coding wise, so be gentle :)

I've looked at several articles and tested out several algorithms including some of my own, but none we're really successful nor accurate enough, and i feel like I've been getting something wrong this entire time.

Specifically I've tried implementing the idea's presented here: http://archive.gamedev.net/archive/reference/programming/features/beatdetection/index.html

but with little success, i still think im skipping over something and i cant quite pinpoint it.

If someone could provide an explanation about how to make an actually accurate beat detector i would be very grateful.

this is my latest attempt at beat detection: http://pastebin.com/BD8y9tfz

in this i used (R1) in the link i posted above to compute the instant energy from the 1024 samples i took, and then i used (R3) to calculate the local average sound energy from the buffer containing all the previous instant energy calculations, then i checked if there is a significant rise in instant energy compared to the average local sound energy, if there is, it means there is a beat, if there isn't, the program continues as usual.

then i added the implementation for R4,R5 and R6, still not working though. added a bit of debug, and for some reason the constant is ridiculously small, numbers like:

Constant: -103416 and Constant: -54793.28, ive got no clue why im getting these numbers, any help?

Comment
Add comment · Show 9
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image AndyMartin458 · Jun 23, 2014 at 09:21 PM 0
Share

@SgtPancakes. I'm interested in this as well. What type of variable are you using to hold the constants? It looks like you have wrapped around. Are you using double?

avatar image SgtPancakes · Jun 23, 2014 at 10:18 PM 0
Share

@Andy$$anonymous$$artin458 its a float, i wrapped it around because for some reason it treated it as a double.

Shouldn't affect calculations though, still dont know why its giving such bizarre results

avatar image meat5000 ♦ · Jun 23, 2014 at 10:40 PM 0
Share

C is your sensitivity and defines the intensity of the beat which can be detected.

It is a number you choose. Define your own C and put it back in. Vary it between 1 -> 1.5 at first to see what effect it has on your FFT.

You can compute a 'best' value but as you already know a good working range, define it yourself and vary it to see what works best for your case.

You could implement a Z-transform to make a digital filter. This could make detection easier with less sensitivity, but I'm unsure how well Unity would cope with that combined with the processing of your FFT.

avatar image SgtPancakes · Jun 23, 2014 at 10:52 PM 0
Share

@meat5000 first of all, thank you for your reply! if I set the C myselt then what does the 3rd algorithm in the gamedev article do? I thought it was to automatically calculate the C through the variance, or did i understand it wrong?

Also, im not sure if i should use getOutputData or getSpectrumData, and if spectrumdata, which FFTWindow should i use? I STILL dont know what those windows even mean.

avatar image meat5000 ♦ · Jun 23, 2014 at 10:59 PM 0
Share

C will be C whether its calculated or deter$$anonymous$$ed by you. In order to help with your calculation its best to find out the working range for yourself. In this case you know its not 10 million :D

The three algorithms on the site you are referencing are not meant to be used together but are three separate methods (I think, I havent read the whole thing). Pick one.

Engineers keep tests controlled by setting values so they can see whats going on, rather than having everything a dynamically changing mess. You can't know how high you are flying unless you know where the floor is :)

Plug values you expect in to your algorithm, vary them bit by bit to find out the best value and working range. You have a starting point already, as the numbers are there on the site :)

FFT is hairy stuff. Don't expect to get it right first time :)

A window is like a mask, I guess. The shape and size affects what you get out and also how that needs to be handled. I think you will need to brush up on Convolution. That part is easier than it appears.

http://en.wikipedia.org/wiki/Window_function

Show more comments

4 Replies

· Add your reply
  • Sort: 
avatar image
6

Answer by mylovemhz · Feb 08, 2015 at 08:05 PM

In case anyone still needs help with this, I've made a simple beat detection API you can use on your GameObject. It's at my Github below:

https://github.com/allanpichardo/Unity-Beat-Detection

Comment
Add comment · Show 6 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Nilakash · Jul 14, 2015 at 05:28 AM 0
Share

Hey, nice work with the API. But is there a way to get if the beat is within a specific spectrum? for example a snare or a rock riff starts? im new to audio codes and stuff.. so a little help with be appreciated! :)

avatar image mylovemhz · Jul 17, 2015 at 07:07 PM 0
Share

@Nilakash Unfortunately, the current API is extremely simple. I just wanted to detect when beat onsets happened in music. It can be extended to add more complex functionality though. I would start by taking the signal and EQing it to isolate the frequencies I want to check for. Then run the beat detection algorithm on that. In the next couple of weeks, I'll work on the detector to add some frequency isolation. However, it won't be able to isolate multiple instruments form a single source (At least not in an efficient way that won't slow your game down dramatically.) For something like that, C# is much too slow. It would have to be written in C and then compiled into a DLL file and you can reference that DLL file from visual studio.

avatar image DoctorSp8dles · Mar 30, 2018 at 04:11 PM 0
Share

Hey @mylovemhz I am reading your code and I see it has copyright. Do I have to buy the license in order to have the right to sell my game with your code or can I just credit your work on my game with no worries?

avatar image mylovemhz DoctorSp8dles · Mar 30, 2018 at 05:21 PM 0
Share

Just credit me and it's all good. The library is open source. Good luck!

avatar image DoctorSp8dles mylovemhz · Mar 30, 2018 at 05:57 PM 0
Share

Thank you mate!

Show more comments
avatar image
3

Answer by SgtPancakes · Jun 24, 2014 at 12:06 AM

@meat5000 MOTHER OF GOD, I've been doing it all wrong! I've been using getOutputData all this time, which from what i understand gives you the db of a certain sample length, what i've been needing to do is change it to getSpectrumData which gives the intensity of a certain sample length, thats why it hasnt been working in loud songs like the one i sent, because it was WAAAAY too loud for it to register anything, but now that it checks for intensity it doesnt even care for the volume.

Still needs some work though, although i can definitely go somewhere from here. ill edit and add the final piece of code once i get it somewhat more accurate, but for now this is a working program: http://pastebin.com/MF63GpGd

Comment
Add comment · Show 1 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Eric_Lin · Jan 14, 2015 at 09:24 AM 0
Share

Hi,I read the code u wrote up there. Can u tell me how did this algorithm work??

avatar image
2

Answer by Vacciones_Mad · Mar 17, 2015 at 08:30 AM

Don't know if the thread is dead or what but if anyone here stills wants it here is a slightly changed version of Sgt's script that worked for me, cudos go to SgtPancakes and whoever wrote the article posted by Sgt.

 using UnityEngine;
 using System.Collections;
 
 public class SpectrumDetektor : MonoBehaviour {
     //public AudioClip song;
     public AudioSource song;
     //public GameObject cube;
     private bool Beated;
     private float[] historyBuffer = new float[43];
     private float[] channelRight;
     private float[] channelLeft;
     private int SamplesSize = 1024;
     public float InstantSpec;
     public float AverageSpec;
     public float VarianceSum;
     public float Variance;
     public float Constant;
     
     // Use this for initialization
     void Start () {
         Beated = false;
     }
     
     // Update is called once per frame
     void Update () {
         
         //compute instant sound energy
         //channelRight = song.audio.GetSpectrumData (1024, 1, FFTWindow.BlackmanHarris);  //Normal
         //channelLeft = song.audio.GetSpectrumData (1024, 2, FFTWindow.BlackmanHarris);  //Normal
         //channelRight = song.GetSpectrumData (1024, 1, FFTWindow.Hamming);  //Rafa
         //channelLeft = song.GetSpectrumData (1024, 2, FFTWindow.Hamming);  //Rafa
         
         //InstantSpec = sumStereo (channelLeft, channelRight);  //Normal
         InstantSpec = sumStereo2(song.GetSpectrumData(SamplesSize, 0, FFTWindow.Hamming));  //Rafa
         
         //compute local average sound evergy
         //AverageSpec = sumLocalEnergy ()/historyBuffer.Length; // E being the average local sound energy  //Normal
         AverageSpec = (SamplesSize / historyBuffer.Length) * sumLocalEnergy2(historyBuffer);  //Rafa
         
         //calculate variance
         //VarianceSum = 0;
         //for (int i = 0; i < 43; i++)  //Normal
             //VarianceSum += (historyBuffer[i]-AverageSpec)*(historyBuffer[i]-AverageSpec);
         
         //Variance = VarianceSum/historyBuffer.Length;  //Normal
         Variance = VarianceAdder(historyBuffer) / historyBuffer.Length;  //Rafa
         Constant = (float)((-0.0025714 * Variance) + 1.5142857);  //Normal
         
         float[] shiftingHistoryBuffer = new float[historyBuffer.Length]; // make a new array and copy all the values to it
         
         for (int i = 0; i < (historyBuffer.Length - 1); i++) { // now we shift the array one slot to the right
             shiftingHistoryBuffer[i+1] = historyBuffer[i]; // and fill the empty slot with the new instant sound energy
         }
         
         shiftingHistoryBuffer [0] = InstantSpec;
         
         for (int i = 0; i < historyBuffer.Length; i++) {
             historyBuffer[i] = shiftingHistoryBuffer[i]; //then we return the values to the original array
         }
         
         if (InstantSpec > (Constant * AverageSpec)) { // now we check if we have a beat
             if(!Beated) {
                 Debug.Log("Beat");
                 Beated = true;
             }
         } 
         else {
             if(Beated) {
                 Beated = false;
             }
             Debug.Log("No Beat");
         }
         
         //Debug.Log ("Avg local: " + E);
         //Debug.Log ("Instant: " + e);
         //Debug.Log ("History Buffer: " + historybuffer());
         
         //Debug.Log ("sum Variance: " + sumV);
         //Debug.Log ("Variance: " + V);
         
         //Debug.Log ("Constant: " + constant);
         //Debug.Log ("--------");
     }
     
     float sumStereo(float[] channel1, float[] channel2) {
         float e = 0;
         for (int i = 0; i<channel1.Length; i++) {
             e += ((channel1[i]*channel1[i]) + (channel2[i]*channel2[i]));
         }
         
         return e;
     }
 
     float sumStereo2(float[] Channel) {
         float e = 0;
         for (int i = 0; i < Channel.Length; i++) {
             float ToSquare = Channel[i];
             e += (ToSquare * ToSquare);
         }
         return e;
     }
     
     float sumLocalEnergy() {
         float E = 0;
         
         for (int i = 0; i<historyBuffer.Length; i++) {
             E += historyBuffer[i]*historyBuffer[i];
         }
         
         return E;
     }
 
     float sumLocalEnergy2(float[] Buffer) {
         float E = 0;
         for (int i = 0; i < Buffer.Length; i++) {
             float ToSquare = Buffer[i];
             E += (Buffer[i] * Buffer[i]);
         }
         return E;
     }
 
     float VarianceAdder (float[] Buffer) {
         float VarSum = 0;
         for (int i = 0; i < Buffer.Length; i++) {  //Rafa
             float ToSquare = Buffer[i] - AverageSpec;
             VarSum += (ToSquare * ToSquare);
         }
         return VarSum;
     }
     
     string historybuffer() {
         string s = "";
         for (int i = 0; i<historyBuffer.Length; i++) {
             s += (historyBuffer[i] + ",");
         }
         return s;
     }
 }
Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image
0

Answer by BadSeedProductions · Sep 23, 2015 at 01:54 AM

This was picking up some beats but not "the beat". I tried with fur elise (piano), which seemed to do the best, picking up the beat here and there and then dropping off again. fatboy slims funk soul brother picked up 1 beat when the song started and that was it, and the doors 'love her madly' picked up 2 beats during the entire song. I can see this working with a simple metronome sound possibly, but as for music, no way.

Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

Answers Answers and Comments

12 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Multiple Cars not working 1 Answer

Distribute terrain in zones 3 Answers

how to create an outlined polygon with vector3 coordinates? 0 Answers

How to line up a certain animation to the beat of my music (C# Question!!) 0 Answers

Spawn objects with MIDI file 1 Answer


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges