OnAudioRead Buffer Changing Sizes?

For a variety of reasons on my project (the main one is having over 10000+ audio assets), I'm loading ogg files into memory from an external source, and using the NVorbis library to stream samples to unity through the callback functions available using AudioClip.Create. The code looks roughly like this:

 AudioClip InitClip(byte[] inbytes)
 {
     stream = new MemoryStream(inbytes);
     vorbis = new VorbisReader(stream, true);
     
     samplecount = (int) (vorbis.SampleRate*vorbis.TotalTime.TotalSeconds); //probably close enough
 
     return AudioClip.Create("clip", samplecount, vorbis.Channels, vorbis.SampleRate, false, true, OnAudioRead, OnAudioSetPosition);
 }
 
 void OnAudioRead(float[] data)
 {
     var f = new float[data.Length];
     vorbis.ReadSamples(f, 0, data.length);
 
     for (int i = 0; i < data.Length; i++)
     {
         data[i] = f[i];
     }
 }
 
 void OnAudioSetPosition(int newPosition)
 {
     vorbis.DecodedTime = new TimeSpan(0); //Only used to rewind the stream, we won't be seeking
 }

This actually works pretty well, in most cases it streams properly. I simply attach it to an AudioSource and it's good. The reason for the for loop in the OnAudioRead section is because while OnAudioRead is supposed to be passed a float[], it's actually getting a Single[][], which I can't pass directly to NVorbis. Copying over the buffer like this seems to resolve the issue.

Main problem though is that periodically I will receive error messages and garbled segments of audio. The error is "System.IndexOutOfRangeException: Array index is out of range. at AudioAsset.OnAudioRead (System.Single[] data)". The error to me doesn't make much sense, because there shouldn't be any way the index could be invalid.

So I rigged up the following code within OnAudioRead to test it.

 var length = data.length;
 vorbis.ReadSamples(f, 0, data.length);
 var length2 = data.length;
 
 if(length != length2)
     Debug.LogError("Warning! Buffer size changed!? Was " + length + ", now is " + length2);

Running this code, lo and behold, it would sometimes generate the error. I would get this in my error log, interspersed with IndexOutOfRangeExceptions.

 Warning! Buffer size changed!? Was 928, now is 4096
 Warning! Buffer size changed!? Was 928, now is 4096
 Warning! Buffer size changed!? Was 928, now is 4096
 Warning! Buffer size changed!? Was 4096, now is 2512
 Warning! Buffer size changed!? Was 2512, now is 4096

So the question is, why in the world would the length of the buffer be changing while I'm reading? I can't make any sense of it. In order to rule out NVorbis causing the issue, I did another test, excluding it entirely from the OnAudioRead function.

 public void OnAudioRead(Single[] data)
 {
     for (var i = 0; i < data.length; i++)
         data[i] = Mathf.Sin((float)i/100);
 }

Still periodically receiving IndexOutOfRangeExceptions. There's so little documentation on these functions that there's nothing really to help me figure out why it's not working. Locking doesn't seem to help either.

Anyone have any clue?