koobas.hobune.stream
Playing wav audio with integer samples
I am trying to request wav audio from Nuance NDEV TTS inside Unity and play back the audio. I was hoping that WWW.GetAudioClip(false, false, AudioType.WAV) would work, but it seems that it does not. There is no error when I try to play it (I am checking the isReadyToPlay property of the AudioClip), but I hear nothing and the length of the AudioClip is 0.
I have a suspicion about what the problem is. Previously, I was able to convert the response from Nuance into an array of integers and then to an array of floats within a Python script (with the aid of SciPy: http://nbviewer.ipython.org/github/mgeier/python-audio/blob/master/audio-files/audio-files-with-scipy-io.ipynb). I then sent this array of floats over HTTP to Unity and used AudioClip.SetData. This worked fine. So, I'm thinking maybe GetAudioClip only works with wav files in which the samples are floats, which seems odd since apparently wav files most commonly use integers (maybe floats are the norm for sound for games?).
The next step, then, is for me to figure out how to get an array of floats from a wav file in C#. It looks like NAudio might be a good possibility (http://naudio.codeplex.com/).
It turns out I made a wrong assumption, which was that the response from Nuance was a WAV file. In fact, it's simply raw 16-bit PCM audio (no header). That is, every 2 bytes in the response is a 16-bit signed integer representing a sample. I wrote the following method to convert the binary response to an array of shorts and then to an array of floats. The audio plays back fine in Unity.
float[] PCM2Floats(byte[] bytes)
{
// See pcm2float in https://github.com/mgeier/python-audio/blob/master/audio-files/utility.py
float max = -(float)System.Int16.MinValue;
float[] samples = new float[bytes.Length / 2];
for (int i = 0; i < samples.Length; i++)
{
short int16sample = System.BitConverter.ToInt16(bytes, i * 2);
samples[i] = (float)int16sample / max;
}
return samples;
}
