- Home /
How to display Unity scene in a Qt window
Good evening,
In my current project , I have to find a solution for integrating the 3d engine Unity into a QT simulator. The simulator concerns a product we develop in my company. This product generates incrusts over the image of a viewed landscape. A simulator already exists with an artificial landscape constructed with photos. We have decided to replace the landscape with a 3d world generated by Unity.
Our needs is then to first be able to catch the Unity output in a specific window (other than the default one), and second to be able to give orders to the engine like movements from the Qt application. In other words, how could I create a data channel between Unity engine and an external application.
We are opened to buy the pro solution.
My current results leads me to two partial solutions. If we build a standalone solution, it is possible to give information from Qt via Unity plugins and build an ethernet communication between Qt and the Unity plugin. Therefore I can manage movements from Qt but I still do not know how I could get the image back into Qt. The other solution proposed by a collegue is to use the Webplayer and the Qt webviewer component. In this case, we will be able to catch the 3d image, but in this case, I do not see any means to give orders to the engine (plugins are only available in standalone compilation).
I hope my question is well detailed enough.
I you find some misunderstanding in my question, do not hesitate, I am far from an expert. Your remarks will be welcome.
Thank you in advance
Hello, I encounter the exact same problem as you so I hope that my answer will bring more answers. If you have found any solutions I would be glad to know them.
Answer by ahrakos · Jan 22, 2015 at 03:56 PM
I tried the same thing today and found a working solution based on the example from unity command line options page: http://docs.unity3d.com/Manual/CommandLineArguments.html.
My Code:
//GET FILE
QString fileName = QFileDialog::getOpenFileName(this,
tr("Open Unity Build"), "D:/tmp/", tr("Unity Build (*.exe)"));
// PASS ARGS
QString _args = QString("-parentHWND %1 -force-opengl").arg(ui->unityWidget->effectiveWinId());
//
PROCESS_INFORMATION procInfo = {0};
STARTUPINFO startupInfo = {0};
startupInfo.cb = sizeof(startupInfo);
startupInfo.dwFlags = STARTF_USESTDHANDLES;
startupInfo.hStdInput = GetStdHandle(STD_INPUT_HANDLE);
startupInfo.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
startupInfo.hStdError = GetStdHandle(STD_ERROR_HANDLE);
//
HWND hwnd = (HWND) ui->unityWidget->effectiveWinId();
LPWSTR cmd = (LPWSTR) fileName.utf16();
LPWSTR args = (LPWSTR) _args.utf16();
BOOL bProcessStarted = CreateProcess(cmd,args , NULL, NULL, TRUE, CREATE_NO_WINDOW,
NULL, NULL, &startupInfo, &procInfo);
if(!bProcessStarted) ;// TODO: show error
//WAIT FOR START
qint64 before = QDateTime::currentMSecsSinceEpoch();
Sleep(100);
// WaitForInputIdle(ProcessInfo.hProcess, INFINITE);
qint64 after = QDateTime::currentMSecsSinceEpoch();
qDebug() << "Waited " << (after-before);
//GET UNITY WINDOW HANDLE
EnumChildWindows(hwnd, MainWindow::EnumChildGet, 0);
It Only works on Windows and Unity 4.5+. I hope it helps! I needed a whole workday to make it work.
You don't need most of these hacks any longer. Unity Standalone players already have command line option to initialize their renderer into specified WinId() [or well, HWND]
-parentHWND (Windows only) - Embeds Windows Standalone application into another application, you have to pass parent application’s window handle to Windows Standalone application. See this example EmbeddedWindow.zip for more information.
Answer by torus · Jan 30, 2015 at 04:02 PM
Hi ahrakos,
thanks for your answer. Sorry for beeing late, I have made some tries to get to my solution. Actually, I think that I am very close to what expected, just one little thing to solve, and the solution will be perfect. After reading my question, I recognize that there is something important on which I haven't been clear enough, that is why you gave me this answer. When I talk about catching the unity output, I mean that Qt must not only know the image handle, but it must also be the window itself. So, a pixels copy is necessary (I suppose...). Anyway ahrakos, I am sure some people will be very interested by your solution which is simpler and more natural.
So here are a few hints for people with the same issue : - First, I implemented a communication channel between unity and Qt processes via a shared memory with a plugin (http://qt-project.org/doc/qt-4.8/ipc-sharedmemory.html) - Then, I implemented in another plugin a command to get the handle of unity window and hiding it (http://stackoverflow.com/questions/1888863/how-to-get-main-window-handle-from-process-id) - Now I can hide the unity window, send it some commands and kill it at the end of the Qt process.
Last thing that I need, is to be able to retrieve pixels matrix from unity. I have succeeded to get the rendering texture to create a png file (using Texture2D.ReadPixels). Now, instead of getting the file, I have to copy the unity image in the shared memory which will be read back by Qt for displaying. I guess that I will have to use the functions Texture.GetNativeTexturePtr() and IntPtr.ToPointer, but I am a bit stuck for completion.
Honestly, I have no idea of the output performance. So many copies to do is not what we could call an optimized process, but for a prototype, we can live with that.
Answer by petersvp · Jun 10, 2015 at 08:52 AM
UPDATE: Corrected the link
For displaying, you need to use following:
-parentHWND (Windows only) - Embeds Windows Standalone application into another application, you have to pass parent application’s window
handle to Windows Standalone application. See this example for more information. http://docs.unity3d.com/uploads/Examples/EmbeddedWindow.zip
For communication, you can use Shared Memory via QSharedMemory, or better QLocalSocket with native plugin. You can also use loopback TCP/IP or Registry Hives. It's up to you.
hi petersvp, the example that you posted is not available anymore unfortunately. could you re-post it? I would like to have a look..
UPDATE: I found the working link, in case anyone is interested: http://docs.unity3d.com/uploads/Examples/EmbeddedWindow.zip
do you by any chance have/know of an example project that shows how Unity and Qt communicate via QShared$$anonymous$$emory? I wasn't able to find any example code anywhere..
more presicely I would like to know, how do I share the memory on unity side & how do I access it on the Qt side..
Answer by gambr · Jun 22, 2018 at 12:56 PM
Interesting topic. I implemented a Qt application in C++ with embedded unity player. There is just one point not working and I see the same problem in EmbeddedWindow.zip C# application: mouse interaction with embedded unity player is not working at stat-up. Just after the application window resize the mouse interaction works. Any idea?
Regards Gianni
Hi, could you explain, how did you do? $$anonymous$$any thanks Regards, Clement
Answer by DiegOne · Jul 29, 2021 at 01:04 PM
Good morning, could someone explain better how to integrate Unity in a QT window? I am new in using Qt and I don't know where to start. Thanks.