Sensor Fusion of Accelerometer and Gyroscope

I am trying to use the phone as a gun aiming with the gyroscope. I calibrate with the phone or tablet in a certain orientation. This will shoot straight. Then depending on the direction the phone is turned (left/right/up/down.), the gun shoots in that direction.

I am using the gyroscope. And all this works. Except after shooting for about 30 secs, the gyroscope slowly starts drifting towards left or right. So when I go back to the orientaion I calibrated with, it doesn't shoot straight anymore. Does anyone have any experience writing a Complementary or Kalman Filter to fuse gyro and accelerometer data to give better results in Unity 3D?

I've found this online - http://www.x-io.co.uk/open-source-ahrs-with-x-imu/. It seems to do exactly what I want. But I am using it wrong. I sometime get better and sometimes get worse results with. Anybody have any experience with it ?