koobas.hobune.stream
Showing BoundingBoxes on the Microsoft Azure Object detection tutorial
Hey Guys,
I'm currently building the Microsoft and Azure Object detection tutorial (https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-310) on my hololens and everything works pretty well by now. I'm currently at the Exercise 1:
Exercise 1 Adding to the Text Label, use a semi-transparent cube to wrap the real object in a 3D Bounding Box.
And I can't seem to find the right solution here. I think the solution is pretty easy, because you get the BoundingBox left and BoundingBox top, height and width back from the AI and as a Debug.log in the SceneOrganiser class, but I still cant seem to figure out how to draw a BoundingBox there. I wrote a method that takes 4 Vector3 points and draws a Rectangle from that. I can't seem to figure out how to transform the restults from the method public Vector3 CalculateBoundingBoxPosition(Bounds b, BoundingBox boundingBox) correctly. For example a return is the following:
BB: left 0.3818852, top 0.293549, width 0.111510605, height 0.336301029
BB CenterFromLeft 0.4376405025, CenterFromTop 0.4616995145
Quad Width 0.7847441, Quad Height 0.4862716
I even got the Center of a quad (which represents the taken image that is send to the ai) as real 3D coordinates:
Center: (-1.3, -0.4, 3.1), Extents: (1.5, 0.9, 0.7)
I can't seem to figure out what to do with those float values and how to transform them to 3D Vectors. Do I need to do something like CenterFromLeft * Quad.size.x and then subtract that from the Center?
I know that this forum is not about you guys getting my job done, but I can't seem to figure out the solution and I think its pretty much in the code but I can't get it to work for hours now.
Edit: I found out that this: https://stackoverflow.com/questions/50794707/how-to-use-azure-custom-vision-service-response-boundingbox-to-plot-shape descripes by problem pretty well but only in 2D Space. Whats the best way to convert that to 3d? At the moment my X values are always off, thats probably because of the camera rotation and axis rotatio because on a static 2d plane the values are good and a good rectanlge is painted
Thanks!
