What's wrong with this? (audio.timeSamples)

Basically the problem I have is that Unity doesn't sync well this.

• The audio clip starts inmediately (i.e. there's no blank sound at the beginning)

• It's an .ogg file

• Frequency 44100

So I want to make that on each beat the background music plays, the background color gets brighter by changing the alpha color of the material.

So I have this code:

     void FixedUpdate(){
         //Im always decreasing the alpha so if there's no beat, color starts to disappear
         bgMaterial.material.SetColor("_Color", new Color(bgMaterial.material.color.r,bgMaterial.material.color.g,bgMaterial.material.color.b,bgMaterial.material.color.a-0.01f));

         // With this equation, I can calculate the beats
         // 128 = BPM
         // 60 = seconds in a minute
         // 4 = to make it 1/4 beat
         // loopClip.frequency is 44100
         //(The problem is not in the if condition)
         if (mainAS.timeSamples >= (((128d/60d)/4d) * loopClip.frequency)*nextBeat){
             bgMaterial.material.SetColor("_Color", new Color(bgMaterial.material.color.r,bgMaterial.material.color.g,bgMaterial.material.color.b,0.75f));
             nextBeat++;
         }
     }

The logic is pretty simple. I get a number with that equation. When that number is the same or higher than the mainAS.timeSamples, it means that there's a beat right there. So then, set to 0.75 de alpha color. Remember that alpha is decreasing everytime, so this will make the effect that I want to reach. The thing is that this works but it's not synchronized well, and I don't know why. It's like it starts ok, but then it starts to unsync, then is synched again, and so that. I hope someone helps me!

Cheers!