Detecting when the other player presses a button

I am making a 2 player multiplayer game and I need to detect when the other player presses a button, and when he presses the button it needs to change my current position, however it doesn't work and i can't figure why.

Here is the script i wrote:

 public class Change : NetworkBehaviour {
 
     public GameObject player; // reference to the other player
 
 
     [SyncVar(hook="Activate")] bool activate=false;
 
 
     void Start()
     {
         if(!isLocalPlayer)
         {
             //Destroy(this);
             return;
         }
 
          if (this.name == "Red(Clone)")
         {
             player = GameObject.Find("Green(Clone)");
         }
         else
         {
             player = GameObject.Find("Red(Clone)");
         }
 
     }
 
     [RPC]
     void Activate(bool act)
     {
         activate = act;
     }
 
 
     void FixedUpdate()
     {
       
            if(Input.GetKey(KeyCode.E))
         {
             player.GetComponent<Change> ().activate = true;
             player.transform.position = new Vector3 (20f, 20f);
     } 
         else
         {
             player.GetComponent<Change> ().activate = false;
          }
             
        
     }
 

The player seems to receive the button press only when it is received from the client, however the position is not updated. I would appreciate any help. Thanks