Wayback Machinekoobas.hobune.stream
May JUN Jul
Previous capture 13 Next capture
2021 2022 2023
1 capture
13 Jun 22 - 13 Jun 22
sparklines
Close Help
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
0
Question by NameZero912 · Oct 08, 2013 at 09:11 AM · synchronizationnative pluginthreading

How to synchronize native plugin texture update with orientation change from script

Hi,

I have a program which retrieves a 360-degrees panoramic image from a camera, as well as an orientation quaternion from a orientation-sensor mounted to that camera. The information is retrieved in real-time and written to shared memory. In Unity I have a sphere with inverted normals which acts as skydome, and a camera in the center of the sphere. I've created a native C++ DLL plugin and a C# script. The plugin reads the orientation quaternion and the image from shared memory and updates the texture, while the script assigns the orientation values to the skydome Transform component. My solution is similar to the example project described on the native plug-in interface page.

The problem is that I observed that the point of time where the texture content changes is not in sync with the point of time where the orientation of the skydome-sphere is applied. This leads to small jitter-noise, which is, unfortunately, very apparent to the viewer. Please take a look at this short video demonstration (password = jitter) of the problem. By replaying the image/sensor data more slowly and taking a slow-motion video of my computer screen, I observed that first the texture is updated (while some of the orientation is applied), and then in the next frame the rest of the orientation-change is applied

I'd like to mention that the orientation sensor data comes it at ~60 Hz, while the image data comes in at 28-32 Hz. My approach is to update the skydome texture whenever there is a new camera image available, but to only update the skydome orientation at these very same moments, not at the 60Hz it could provide. Were I to update orientation readings at 60 Hz, I would get jitter for sure (sampling problem). I'd also like to note that I've taken care of the different sensor delays (the difference between delays of < something happens in real world > to < camera image reflects this change >, or < something happens in real world > to < orientation sensor readings reflect this change >), by artificially delaying the data of faster of the two sensors.

Below you see an extract of my code:

 [DllImport ("UnityPlugin")]
 private static extern void SetSkydomeTextureFromUnity(System.IntPtr texture);
 
 [DllImport("UnityPlugin")]
 private static extern bool isNewCameraImageAvailable();
 
 private bool updateOrientation;
 
 IEnumerator Start() {
     updateData(); //copy sensor/image content from shared memory to memory of the C++ plugin
     //Create the skydome texture
         Texture2D tex = new Texture2D(1920, 1080, TextureFormat.ARGB32, false);
     tex.Apply(); //actually uploaded to the GPU
     GameObject sphere = GameObject.Find("pSphere1"); //skydome mesh
     sphere.renderer.material.mainTexture = tex;
     SetSkydomeTextureFromUnity(tex.GetNativeTexturePtr()); //pass texture pointer to the plugin
         
     yield return StartCoroutine("CallPluginAtEndOfFrames");
     }
 
 private IEnumerator CallPluginAtEndOfFrames ()
 {
     while (true) {
         //Do work inbetween frames
         yield return new WaitForEndOfFrame();
             
         updateData();
             
         //now apply the values to the different gameobjects
         updateHeadTrackerOrientation(); //no more details necessary
         updateOrientation = isNewCameraImageAvailable();
         updateSkydomeOrientation(); //simply gets w, x, y, z from native plugin and assigns values to skydome transform object, but only if updateOrientation == true
             
         //Update skydome Texture from C++ code immediately in case there is a new image available
         if (updateOrientation) {
             GL.IssuePluginEvent(1); //note: the parameter is an unused ID (ignored)
         }
     }
 }
 
 //note: nothing relevant in Update() or LateUpdate()

The update of the skydome texture itself takes place in C++ code in EXPORT_API void UnityRenderEvent(int eventID).

It seems to me that the maybe the rendering thread (which steps into EXPORT_API void UnityRenderEvent(int eventID)) and the thread that executes the CallPluginAtEndOfFrames() co-routine (updating the skydome orientation) are not properly synchronized. However, I don't know how to achieve this synchronization... :( Any help would be appreciated!

Best regards!

Comment
Add comment
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

0 Replies

· Add your reply
  • Sort: 

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

Answers Answers and Comments

15 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Are calls to C++ DLL handled asynchronously? 1 Answer

functional multithreaded c++ code crashes if executed in Unity as native plugin 1 Answer

Cannot make native plugin working on OS X. 0 Answers

How to create a native OS X plugin from a static C++ .a library 0 Answers

Using CVOpenGLTextureCache in Plugin on OSX - Get correct OpenGL context? 0 Answers


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges