Trying to use derive pitch and db from realtime microphone input
Hi, I'm new to the unity forums, I am entering my first year of high school in the fall, thanks for checking out my question. I'm trying to get values for pitch and volume from the realtime microphone input to have AI characters react to how the gamer is speaking (if you speak monotone the AI will be bored, if you speak emotionally the AI will be engaged) I need 2 things for this. 1) real time audio input. 2) something that decomposes audio and shows me the individual frequency and amplitude values real time (using an FFT for this). For 1) im using Benjamin Outram's script (link: http://answers.unity3d.com/questions/1113690/microphone-input-in-unity-5x.html ) this works. audio plays back through my headphones real time. For 2) i found a youtube tutorial on visualizing audio files. I put together a modified version of that (attached file). It works individually because I when the audio file plays in unity the pitch and volume values change. My problem is trying to combine them. The real time microphone script works by replacing the audio file that is attached to a game object with the microphone. Somehow this just seems to be bypassing the FFT part of the script, since I'm not seeing the frequency/dB values change in the inspector. Where did I go wrong? I also get an error in the console (although the name does exist and I think I have assigned the mixer): Exposed name does not exist: MasterVolume UnityEngine.Audio.AudioMixer:SetFloat(String, Single) FFTMAYBE:DisableSound(Boolean) (at Assets/FFTMAYBE.cs:131) FFTMAYBE:Update() (at Assets/FFTMAYBE.cs:89) UnassignedReferenceException: The variable masterMixer of FFTMAYBE has not been assigned. You probably need to assign the masterMixer variable of the FFTMAYBE script in the inspector. FFTMAYBE.DisableSound (Boolean SoundOn) (at Assets/FFTMAYBE.cs:131) FFTMAYBE.Update () (at Assets/FFTMAYBE.cs:89) This is my script: using UnityEngine; using System.Collections; using UnityEngine.Audio; // required for dealing with audiomixers
[RequireComponent(typeof(AudioSource))]
public class FFTMAYBE : MonoBehaviour {[link text][1]
private const int SAMPLE_SIZE = 1024;
public float rmsValue;
public float dbValue;
public float pitchValue;
public AudioMixer masterMixer;
private Transform[] visualList;
private float[] visualScale;
//Written in part by Benjamin Outram
//option to toggle the microphone listenter on startup or not
public bool startMicOnStartup = true;
//allows start and stop of listener at run time within the unity editor
public bool stopMicrophoneListener = false;
public bool startMicrophoneListener = false;
private bool microphoneListenerOn = false;
//public to allow temporary listening over the speakers if you want of the mic output
//but internally it toggles the output sound to the speakers of the audiosource depending
//on if the microphone listener is on or off
public bool disableOutputSound = false;
//an audio source also attached to the same object as this script is
AudioSource src;
//make an audio mixer from the "create" menu, then drag it into the public field on this script.
//double click the audio mixer and next to the "groups" section, click the "+" icon to add a
//child to the master group, rename it to "microphone". Then in the audio source, in the "output" option,
//select this child of the master you have just created.
//go back to the audiomixer inspector window, and click the "microphone" you just created, then in the
//inspector window, right click "Volume" and select "Expose Volume (of Microphone)" to script,
//then back in the audiomixer window, in the corner click "Exposed Parameters", click on the "MyExposedParameter"
//and rename it to "Volume"
float timeSinceRestart = 0;
void Start() {
//start the microphone listener
if (startMicOnStartup) {
RestartMicrophoneListener ();
StartMicrophoneListener ();
}
}
void Update(){
//can use these variables that appear in the inspector, or can call the public functions directly from other scripts
if (stopMicrophoneListener) {
StopMicrophoneListener ();
}
if (startMicrophoneListener) {
StartMicrophoneListener ();
AnalyzeSound ();
}
//reset paramters to false because only want to execute once
stopMicrophoneListener = false;
startMicrophoneListener = false;
//must run in update otherwise it doesnt seem to work
MicrophoneIntoAudioSource (microphoneListenerOn);
//can choose to unmute sound from inspector if desired
DisableSound (!disableOutputSound);
}
//stops everything and returns audioclip to null
public void StopMicrophoneListener(){
//stop the microphone listener
microphoneListenerOn = false;
//reenable the master sound in mixer
disableOutputSound = false;
//remove mic from audiosource clip
src.Stop ();
src.clip = null;
Microphone.End (null);
}
public void StartMicrophoneListener(){
//start the microphone listener
microphoneListenerOn = true;
//disable sound output (dont want to hear mic input on the output!)
disableOutputSound = false;
//reset the audiosource
RestartMicrophoneListener ();
}
//controls whether the volume is on or off, use "off" for mic input (dont want to hear your own voice input!)
//and "on" for music input
public void DisableSound(bool SoundOn){
float volume = 0;
if (SoundOn) {
volume = 0.0f;
} else {
volume = -80.0f;
}
masterMixer.SetFloat ("MasterVolume", volume);
}
// restart microphone removes the clip from the audiosource
public void RestartMicrophoneListener(){
src = GetComponent < AudioSource> ();
samples = new float[SAMPLE_SIZE];
spectrum = new float[SAMPLE_SIZE];
sampleRate = AudioSettings.outputSampleRate;
//remove any soundfile in the audiosource
src.clip = null;
timeSinceRestart = Time.time;
}
//puts the mic into the audiosource
void MicrophoneIntoAudioSource (bool MicrophoneListenerOn){
if(MicrophoneListenerOn){
//pause a little before setting clip to avoid lag and bugginess
if (Time.time - timeSinceRestart > 0.5f && !Microphone.IsRecording (null)) {
src.clip = Microphone.Start (null, true, 10, 44100);
//wait until microphone position is found (?)
while (!(Microphone.GetPosition (null) > 0)) {
}
src.Play (); // Play the audio source
}
}
}
public float[] samples;
public float[] spectrum;
public float sampleRate;
/// MAYBE THIS "source" INST THE RIGHT VARIABLE
/// Analyzes the sound.
/// </summary>
private void AnalyzeSound()
{
src.GetOutputData (samples, 0);
//Get the RMS Value
int i = 0;
float sum = 0;
for (; i < SAMPLE_SIZE; i++) {
sum += samples[i] * samples [i];
}
rmsValue = Mathf.Sqrt (sum / SAMPLE_SIZE);
//Get the DB value
dbValue = 20 * Mathf.Log10(rmsValue / 0.1f);
// Get sound spectrum
src.GetSpectrumData (spectrum, 0, FFTWindow.BlackmanHarris);
// Find pitch !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
float maxV = 0;
var maxN = 0;
for (i = 0; i < SAMPLE_SIZE; i++)
{
if (!(spectrum [i] > maxV) || !(spectrum [i] > 0.0f))
continue;
maxV = spectrum [i];
maxN = i;
}
float freqN = maxN;
if (maxN > 0 && maxN < SAMPLE_SIZE - 1)
{
var dL = spectrum[maxN - 1] / spectrum [maxN];
var dR = spectrum[maxN + 1] / spectrum [maxN];
freqN += 0.5f * (dR * dR * -dL * dL);
}
pitchValue = freqN * (sampleRate / 2) / SAMPLE_SIZE;
}
}
[1]: /storage/temp/98256-decompressaudio.zip
Your answer
Follow this Question
Related Questions
Trying to use derive pitch and db from realtime microphone input 0 Answers
Detecting when two different audio sources are emitting sound in Unity 0 Answers
Can't get an AudioSource to play on awake 2 Answers
oneshot not working 0 Answers
How to stop current audio before another audio starts? 0 Answers