Hands over 3D object - Vuforia AR - Unity

I have finally done with the help of an other user. Here you can see how can be done: https://developer.vuforia.com/forum/unity-extension-technical-discussion/hand-over-3d-object-help#comment-2062044

OK.

I try to do the same thing and I think I'm pretty close, for what I have learn and ignore till now (ignorance is like an invisible wall, you see the other side but you cannot find the door to go there, especially where there is no door).

I haven' t accomplished it yet, cause i am not that good programmer but I have ideas to share, and I am pretty welcome to code solutions and development disquation..

What I think, is pretty close to the video with the "holding wooden cube" and an instance of a step furthermore...

In the video they use the green screen trick (alpha removal / chroma key). That is why the green fabric.

Both background and real cube are in colour tones of green.

In the unity setup inside the ARCamera prefab there is a plane (AARCamera --> Camera --> plane) that renders the webcam / smartphone camera input video (what you see of the real world, in your screen, is a simple video projection). What we can do is repeat this video, before the augmentation repeat this video with the alpha channel removed. What it's left is the hand.

Possible problems to take under consideration:

1) Scale problems. The hand might be bigger. <--> Maybe ortho camera is a solution.

2) Mark detection and alpha removal. There is a thin line between doing everything in tones of green and computer vision reads clear the mark identifier. <--> This is a try-fail-repeat procedure. What I think is creating / using a VUMARK in two green tones and check it in the vuforia target manager. What tones of green can be "deleted" needs further research. But if you achieve create the right VUMARK then you have a AR logo to print on stickers and stick them everywhere!

3) In real everyday use we cannot have a chroma key fabric with us, so to play. This is the real hard part. <--> What I am thinking, but still no results (not that I have done the above, just being pretty exited) is a hard code mixture of stencil shaders and chroma key removal. Somehow to repeat part of the background video input. The part that is being hidden by the augmentation (fingers / hands) with part of the real background that hopefully is chroma key masked by VUMARK. I haven't think any solution for the part of hand that cannot be masked by the VUMARK but I 'll come to a solution. Maybe skin recognition colour algorithms, but I have no idea if this do exists (just read it in a post). I also have found no tutorial / information / prefab and instructions on the use of stencil shaders.

4) All the above maybe easily run on the pc / unity editor, but it is not sure that our options will run on a mobile device. ex.: I have read somewhere that you maybe have to manually set smartphone to 24bit buffer something, for stencil shader to work, etc.

That's all till now,

Any ideas or implementations, even small parts of a single solution, please share....