PocketSphinx Implementation in Unity for Android

Question

Hello everyone,

I'm a PhD student and I have a Unity game that uses Pocketshinx and I want to export it for Android. First of I'm applying reverse engineering on a game app that uses Pocketsphinx in Unity. I also have limited coding skils but I need to make it work in 15 days otherwise my whole PhD will go down the drain. When I export the project on Android it kind of skips the speech recognition part. In the project I have a plugin CCIRPocketSphinx.DLL, which is custom code not written by me ( I cannot contact the one who did ) compiled for Windows as a bridge to PocketSphinx.

Also, in the root of the project, Pocketsphinx.DLL and Sphinxbase.DLL - these are the Windows DLLs for PocketSphinx and come with the SDK. For Android, they need to be .a or .so files, compiled for that platform. By asking around they gave me these options:

For moving to Android you would have to do something like we did for Windows, writing a bridge from the C# Unity scripts to PocketSphinx. It looks like PocketSphinx is available on Android at http://cmusphinx.sourceforge.net/wiki/tutorialandroid but I had a quick look at it and there seems to be quite a lot of steps to setting it up. Perhaps you might be better using the built-in Android speech kit: http://developer.android.com/reference/android/speech/SpeechRecognizer.html – it seems to stream the audio for recognition, but perhaps that can be hooked into Unity a lot easier? There’s also now a Unity Android speech recognition asset: https://www.assetstore.unity3d.com/en/#!/content/13882

Is it possible for me to make Pocketsphinx- Unity work for Android? If yes can I alter the code that I already have somehow or it would be easier for me to use the speech recognition asset?

Thanx!

Answer 1

Answer by nshmyrev · Jan 27, 2017 at 06:25 PM

Pocketsphinx can work as a unity plugin, not necessary on Android.

We have published a sample demo here:

https://github.com/cmusphinx/pocketsphinx-unity-demo

Add comment · Share

Answer 2

Answer by WrightE · Feb 24, 2020 at 05:45 PM

Sorry to necro an old thread, but since this is the first result on Google I wanted to chime in.

I've created a newer, easier to understand, version of the above demo for everyone to use: https://github.com/Funnyguy77/PocketSphinxUnityDemo

It also includes pre-compiled binary's!

Add comment · Show 5 · Share

xxmariofer · Feb 25, 2020 at 08:55 AM 0

Share

Hello! first thanks for your work, i was trying to test your project but in the method $$anonymous$$oveNext, when calling inStream.Read it never reads the last bytes and i recieve the "Can not read header" do you know what could be causing it? tested in windows and android with the same unity version (i always have 379 bytes remeaing) thanks!

also, it is posible to have the voice recognition running in the background so i can use it as a "key phrase" detector? similar to "$$anonymous$$a" or "Ok google"

WrightE xxmariofer · Feb 25, 2020 at 02:52 PM 0

Share

Hi!

Thanks for checking out the demo! That error occurs when your language model (.tar) file doesn't contain any data. The repo is using Git LFS, so you will need to clone the repo ins$$anonymous$$d of downloading the zip.

Edit: Yes it is possible to add functionality for something similar to "Ok Google" or "$$anonymous$$a". Essentially you would have to listen to the microphone at all times and record when a user starts speaking. Once you have the audio file, you can simply check it with Sphinx and deter$$anonymous$$e what the user said. If they say "Ok Google" for example, you can have the program respond with something.

Hope that helps!

xxmariofer WrightE · Mar 02, 2020 at 11:13 AM 0

Share

hello sorry for the really late response but i was unable to test it before, i did clone the repository, i didnt download it, i have test twice without luck, i have tried to uncompress the en-us.tar file manually using z-zip and it sais the file is corrupted, i have tried to open other tar files in case it was a problem with my pc and it worked just fine, yoou think that could be the problem?

i have checked and when i read the tar it says its lenght is 133, but the header.headerSize is bigger (512)

any help is appreciate it!

Show more comments

PocketSphinx Implementation in Unity for Android

2 Replies

Your answer

Follow this Question

Related Questions