- Home /
Can Google cloud services work with Unity?
I am trying to include Google speech api in my unity app.
I have followed all the steps in the api documentation for c# https://cloud.google.com/speech-to-text/docs/quickstart-client-libraries#client-libraries-install-csharp . I installed this package "Install-Package Google.Cloud.Speech.V1 -Pre" through NuGet package manager but no google references are being shown. This results to the error "The type or namespace name 'Google' could not be found (are you missing a using directive or an assembly reference?)". I checked these possible duplicates but to no avail "The type or namespace name 'Google' could not be found", "The type or namespace could not be found". Switching platform within unity from android to windows shows the google references but without resolving the error. using Google.Cloud.Speech.V1; using System;
namespace GoogleCloudSamples
{
public class SpeechGoogle
{
public static void Main(string[] args)
{
var speech = SpeechClient.Create();
var response = speech.Recognize(new RecognitionConfig()
{
Encoding = RecognitionConfig.Types.AudioEncoding.Linear16,
SampleRateHertz = 16000,
LanguageCode = "en",
}, RecognitionAudio.FromFile("audio.raw"));
foreach (var result in response.Results)
{
foreach (var alternative in result.Alternatives)
{
Console.WriteLine(alternative.Transcript);
}
}
}
}
}
The error occurs from the first code line "using Google.Cloud.Speech.V1;". Is it possible that unity cannot support google cloud services or am I missing a step?
Answer by G-Reusch · Mar 29, 2019 at 09:48 PM
I'm running into similar issues, and unfortunately there doesn't seem to be an easy solution.
I'm guessing there's no error in Visual Studio, but it does show up in the Unity console? If so, the error you're getting is because Unity doesn't play well with NuGet. Unity generates its own Visual Studio solutions which tends to mess up NuGet references.
I've looked into a number of solutions to try and get NuGet to work with Unity, but usually it boils down to manually adding the DLLs to the "Plugins" folder in your project. That usually works for simple libraries, but the Google libraries have some outside dependencies that, as of right now, I can't find a way to import.
The Speech DLL is no exception and has a bunch of dependencies on other Google APIs. Specifically, this chain of dependencies will cause problems for you, as it is with my project: Google.Cloud.Speech.V1
-> Google.Api.Gax.Grpc
-> Google.Api.Gax
-> System.Interactive.Async
System.Interactive.Async
is also on NuGet, but it has dependencies on some System functions that Unity simply doesn't have. Specifically, it references System.Linq.Async
and System.Threading.Tasks.Extensions
. Neither of those are in Unity's version of the System DLLs, and from my research so far I can't find any way to import them in. It's either a C# versioning issue or something to do with Reactive .NET, which is practically a whole other version of the System libraries. We very well may be stuck unless Google decides to remove that dependency on System.Interactive.Async
, but someone else may have more knowledge on the subject than I do.
Answer by solarisn · Sep 16, 2019 at 01:20 AM
@G-Reusch
I can confirm that this IS possible with Unity 2018.2 (or 3?) and above if you fix some dependencies manually. You can grab a .unitypackage of gRPC here. Assuming you want to use streaming/realtime speech recognition, you must use gRPC.
Another challenge is creating a way to encode microphone input to send through the client library. If you want to do streaming recognition, you'll need to create your own method for encoding data from unity's Microphone class in chunks (likely LINEAR16 encoded as a base64 string). You can reference the IBM SDK for Unity because they encode using the Microphone class in this way for their speech recognition API.
I recommend creating a .NET Framework 4.x class library project in Visual Studio to generate all the artifacts (.dll dependencies). This will allow you to use NuGet to import and resolve dependencies. When built, all the dependent .dlls will be put in the output directory and you can import these into Unity as plugins.
There will likely be output artifacts that target different architectures. You'll need to make sure that plugins imported to Unity have correct import settings to make sure Unity doesn't load two versions of the same plugin for different architectures because you will get all sorts of issues resulting in the plugins failing to load.
If you have any issues with the plugins not loading in Unity, you can use dotPeek to inspect the .dlls to determine which plugins are failing to resolve references to other plugins.
Answer by maurosso · Apr 21, 2020 at 09:35 AM
@solarisn Have there been any developments in this regard? Has there appeared a more streamlined way to integrate the Cloud API and in particular Speach-to-text into Unity?
I have tried putting the libraries into the /assets/Plugins, but it still resolves with hundreds of errors
EDIT:
Good news!
With some more lurking around on the internet, I did find a plug and play solution :)
https://github.com/oshoham/UnityGoogleStreamingSpeechToText#readme
Oshoham is a God amongst men...With very little effort, the implementation indeed works. Tested on Unity 2019.3.9f1
Unfortunately I haven't seen any publicly available off-the-shelf solutions yet. I would provide it myself but my company was contracted to build it and we don't have licensing for redistribution. Sorry I can't be of more help!
Good news!
With some more lurking around on the internet, I did find a plug and play solution :)
https://github.com/oshoham/UnityGoogleStrea$$anonymous$$gSpeechToText#readme
Oshoham is a God amongst men...With very little effort, the implementation indeed works. Tested on Unity 2019.3.9f1