Help to design an communicating character with user via Kinect !

Hello, new beginning with Unity, I choose this game engine to create an animated avatar that can communicate with the player via a kinect. It is a communication "speech-gesture" Scence is sample : - the avatar ask player by vocal if he want to joint to a game - the player answer by gestures - the system detect gestures to recognize the answer, either yes or no - If yes, the system run the game - Else, the avatar say goodbye and close de windows

Could you suggest a plan for archieving this scenarios ? Knowing that I think to use Kinect SDK et des 3D Models.

Thanks a lot.