Aligning gameobject with image,Aligning gameobjects with 2d image

Hello everyone,

I am note sure if this is the correct place to ask this question but this is my best guess for now as I am new to Unity. (If I am in the wrong place, I'll be glad to learn how to post such a question)

I was thinking of an application that allows the user to move a gameobject and align it with a 2d picture (later a videostream) of said object in a scene. You can imagine it something like what the person has shown here

To test this I took a photo of a box on a table which I then imported to Unity.

I measured the box, where the camera was placed and calculated what the coordinates of the box relative to the camera (which I used in coordinates 0,0,0). I then tried to reconstruct the scene in Unity placing the camera and box in on the coordinates I calculated.

Now for the picture. I imported it in the scene but I am unsure how one would logically place it, so that wherever the box is placed by the user to align with a photo, the Box will have the correct coordinates. The coordinates don't have to be perfect but in the end I would like to get a somewhat right 6dof pose of the placed object.

However right now looking through the camera the object does not align with the reference image at all. One thing I don't know is how one would initally position the image in 3D space? I tried to modify the camera a bit, fitting the FOV and focal length to the ones of the camera I used to take the reference picture. Are there maybe other things I have to take care of?

I am unsure what I am doing wrong or if my initial approach is good at all. Therefore I would be glad about any help regarding this problem!

,

 Hello everyone,

 I am note sure if this is the correct place to ask this question but this is my best guess for now as I am new to Unity. (If I am in the wrong place, I'll be glad to learn how to post such a question)

 I was thinking of an application that allows the user to move a gameobject and align it with a 2d picture (later a videostream) of said object in a scene. You can imagine it something like what the person has shown [here][1]


 To test this I took a photo of a box on a table which I then imported to Unity.

 I measured the box, where the camera was placed and calculated what the coordinates of the box relative to the camera (which I used in coordinates 0,0,0). I then tried to reconstruct the scene in Unity placing the camera and box in on the coordinates I calculated.

 Now for the picture. I imported it in the scene but I am unsure how one would logically place it, so that wherever the box is placed by the user to align with a photo, the Box will have the correct coordinates. The coordinates don't have to be perfect but in the end I would like to get a somewhat right 6dof pose of the placed object.

 However right now looking through the camera the object does not align with the reference image at all.
 One thing I don't know is how one would initally position the image in 3D space?
 I tried to modify the camera a bit, fitting the FOV and focal length to the ones of the camera I used to take the reference picture. Are there maybe other things I have to take care of?

 I am unsure what I am doing wrong or if my initial approach is good at all. Therefore I would be glad about any help regarding this problem!