Pickup a rigidbody cube over network

In my game I have players capturing a cube with a rigidbody and picking it up. The script works when the host picks it up but when clients pick it up it just disappears and reappears when it's dropped.

Here's my code

 public GameObject takenobject;
 
 void Update() {
 if (hit.collider.CompareTag ("Flag-Gold")) {
         takenobject = hit.collider.gameObject;
         networkView.RPC ("TakingFlag",RPCMode.AllBuffered);
         }
 }
 
     [RPC]
     void TakingFlag() {
         takenobject.transform.parent = this.transform.parent;
         takenobject.rigidbody.isKinematic = true;
         takenobject.transform.position = this.transform.parent.position + this.transform.parent.forward;
         takenobject.transform.localRotation = Quaternion.Euler(new Vector3(0,0,0));
     }