koobas.hobune.stream
Unity Microphone downsampling audio
Hi, I am trying to downsample audio that comes from using Microphone.Start and OnAudioFilterRead.
It seems to always record in 44100hz stereo but I need 16000hz mono. I am able to merge the stereo channels to mono fairly easily but downsampling is harder. I tried using c# lib's like alvas but they do not work on unity due to other dependancies. I am likely going to need to write a low pass filter and then interpolate/decimate the audio to get my clean downsampled audio.
I was hoping if there is a easier way to do this in unity. I am attaching my code so far. It was inspired by some existing code I found on the unity forums.
using UnityEngine;
using System.Collections;
using System.IO;
using System.Collections.Generic;
using System;
public class LiveRec : MonoBehaviour {
FileStream file;
int microphoneMaxRecordingTime;
void Start ()
{
StartWriting();
microphoneMaxRecordingTime=1;
audio.clip = Microphone.Start (null, true, microphoneMaxRecordingTime, AudioSettings.outputSampleRate);
audio.loop = true;
while (!(Microphone.GetPosition(null) > 0))
{
Debug.Log ("Microphone.GetPosition(null)");
}
audio.Play();
}
void OnAudioFilterRead(float[] data, int channels)
{
Debug.Log("channels=" + channels.ToString());
Debug.Log("floats=" + data.Length);
ConvertAndWrite(data);
Array.Clear(data, 0, data.Length);
}
void ConvertAndWrite(float[] dataSource)
{
Int16[] intData = new Int16[dataSource.Length];
//converting in 2 steps : float[] to Int16[], //then Int16[] to Byte[]
Byte[] bytesData = new Byte[dataSource.Length];
//bytesData array is twice the size of
//dataSource array because a float converted in Int16 is 2 bytes.
int rescaleFactor = 32767; //to convert float to Int16
for (int i = 0; i<dataSource.Length/2;i++)
{
//intData[i] = (short)(dataSource[i]*rescaleFactor);
Int16 leftChannel = (short)(dataSource[i*2]*rescaleFactor);
Int16 rightChannel = (short)(dataSource[i*2+1]*rescaleFactor);
Byte[] byteArr = new Byte[2];
short monoChannel = (short)Math.Round(((int)leftChannel + (int)rightChannel) / 2F);
//byteArr = BitConverter.GetBytes(intData[i]);
byteArr = BitConverter.GetBytes(monoChannel);
byteArr.CopyTo(bytesData,i*2);
}
Debug.Log("bytes="+bytesData.Length.ToString());
file.Write(bytesData,0,bytesData.Length);
}
//Create and start writing the file
void StartWriting()
{
string filePath = Application.dataPath + "/zz.raw";
file = new FileStream(filePath,FileMode.Create); //Create or open our file if exists
}
}
Hi Elias,
Very inteesting question, which raises many different issues:
1) Getting Microphone data
No need to use OnAudioFilterRead there! As you've noticed, it will return data that contains the number of channels your project is set to, 2 in most cases. Plus, OnAudioFilterRead's data is post spatialisation, i.e. data from a gameObject far away from the scene's listener will be less loud. What you need is to grab the float[] directly from the audioclip Microphone.Start creates for you, through AudioClip.GetData(float[] data, int offset).
In your code, you are setting up Microphone.Start to create an audioclip of length microphoneMaxRecordingTime. This is a waste of ram, as Unity will allocate that clip however long it is. Better to setup a short microphoneClip (2 seconds), and to set it to loop. Simply grab the data in Update, being mindful of the special case when the clip loops. You'll grab mono data, and won't need to bounce OnAudioFilterRead stereo data to mono as you're doing here, which by the way incurrs a strong risk of saturating.
2) Frequency
Microphone.GetDeviceCaps allows you to check which recording frequencies are available for a particular recording device (Microphone.devices). If yu need to record at 16000 hz, and your mic allows it, no need to resample ! In line 14 of your script, you ask for a rec frequency equal to AudioSettings.outputSampleRate. This does not need to be the case.
If the mic doesn't support 16000 hz recording, then you'll have to record first, and resample after using a plugin. Can I ask why you'd want to do that? If the only reason is to save space, make sure through Microphone.GetDeviceCaps that the available mics don't support low frequencies first.
There's a lot to digest here, do let me know how you're faring!
Gregzo
Doesn't this work?
void Start { AudioSettings.outputSampleRate = 16000; //Rest of your code }
Hi, i want to make talking tom cat in unity. i used this script for this purpose.
var audioSource : AudioSource;
var power : float;
private var samples : float[];
private var audioClip : AudioClip;
function Start()
{
Screen.sleepTimeout = SleepTimeout.NeverSleep;
audioClip = $$anonymous$$icrophone.Start("", true, 1, 44100);
audioSource.clip = AudioClip.Create("", 44100, 1, 44100, false, false);
audioSource.Play();
samples = new float[audioClip.samples * audioClip.channels];
}
function Update()
{
audioClip.GetData(samples, 0);
//if you want to do something with the voice, use this
for (var i = 0; i < samples.length; i++)
{
samples[i] *= power; //for example this line will make the volume higher or lower (adjust the power value to change it)
}
audioSource.clip.SetData(samples, 0);
}
when i run it on my computer. it starts repeating after 2 seconds, it does not wait me to complete my long sentence. when run this same code on device. it repeats again and again and this repeating sound converts to noise. please help me. Thanks.
