Plot real objects in unity and make them coincide with live picture from smartphone camera

Hi,

I'm working on app https://github.com/ssnake/contrails. The idea is to get long, lat coordinates of aircrafts, plot them in unity and make them coincide with image stream from webcam. So, in a nutshell, you get your smartphone, open the app, points your smartphone camera into air and you can see aprx. position of airtcrafts on your screen . What I have covered in the app.

1) Converting lat/long position into unity x,y,z position.

I recon to scale should be 1 unity meter : 1 real meter. I used Mercator projection (look at class Mercator1 in Assets\MapController.cs. There is WebMercator1 class as well)

2) Converting accelerator and compass movement into MainCamera movement. It's done in Assets\CameraController.cs

3) Getting smartphone camera image. It's implemented in Assets\WebCamera.cs 4) Getting gps info. Look at Assets\GpsController.cs 5) Get aircraft data. So far it's simulated. Assets\AircraftController.cs

Were have I ended up.

To test things I came up with idea to put aircrafts(now it is just a sphere) above buildings surrounded my gps position. I expected that looking via smartphone I can see these spheres right above building, but it turned out this is not true.

I wonder if someone could point out what I missed. Thanks in advance.