koobas.hobune.stream
How to tie Unity app to Windows event
I'd like to build a Unity app which runs in the background on a Windows desktop, and does something - let's say, pops up a picture of a poodle - whenever somebody, say, waves their hand at the webcam.
Now let's assume I already have the gesture recognition software that knows when I wave at the webcam, my question is about the rest. How do I create a unity app which, once activated, sits in the background of the windows desktop until the gesture recognition event takes place, and then suddenly pops something up on the desktop.
Is this even possible in Unity?
It depends. Are you okay with the Unity App still having a window, or do you need it to render in a way that integrates with the desktop, e.g. have parts of the Unity App be transparent against the windows desktop?
I have an idea how to implement the first, but I don't think it's possible to have Unity render "natively" onto the windows desktop outside the confines of its window.
I'm okay with the Unity App having a window. But the app would have to run when $$anonymous$$imized, constantly listening and waiting until the gesture is made, at which point an event in the app triggers an image to pop up.
Okay, then I think I have a solution. Answer co$$anonymous$$g up.
Okay, so basically, you need to programmatically control whether the window that houses the Unity application is visible on screen or not based on some event, and it should pop up automatically when this event occurs, i.e. the camera movement.
It's possible to control the visibility of a window by accessing the OS-level window id (the HWND) of the Unity App and then setting its visibility depending on this event. This means you're simply determining on the OS level whether to paint the window or not, including everything within it (Unity). The way this is done is with the Win32 API and some functions in User32.Dll which you access with the extern keyword. The functions you need are GetActiveWindow and ShowWindow, respectively:
HWND WINAPI GetActiveWindow(void);
BOOL WINAPI ShowWindow(_In_ HWND hWnd, _In_ int nCmdShow);
Don't be alarmed by the weird data types. Those are just Win32 API versions of integers. A HWND is a handle to a window which is implemented in C# as a pointer to an integer, i.e. an IntPtr. Make a class to house these functions called "WindowControlNative.cs" or something like that, and declare that you wish to use the functions like this:
[DllImport("user32.dll")]
static extern IntPtr GetActiveWindow();
[DllImport("user32.dll")]
static extern bool ShowWindow(IntPtr hwnd, int nCmdShow);
You call GetActiveWIndow first to obtain the handle to Unity's OS window. You then pass the returned IntPtr to ShowWindow along with an integer that represents the command you wish to give it. See the list in the MSDN documentation I linked to for a thorough explanation. In short, though, hide and show would be 0 and 5, respectively, while maximize and minimize are 3 and 6. Experiment at your leisure.
The next task is to hook it up to this camera-hand-wave code you've got going on. I'm assuming you have a framework that triggers some kind of callback or event from this camera? In a handler for that event, you'd just call ShowWindow with your window handle and give it the integer command for Show or Maximize, depending on what behavior you want. I'm hoping this all makes sense? Let me know if you have more questions.
Edit:
I forgot mentioning that if your webcam code runs in sync with the Unity API and depends on its continuous updating to run, then you need to set Application.runInBackGround to true to make sure Unity is still invisibly rendering frames while the window is hidden.
