Using joints with UNet Not Working Correctly

Hello. I am using UNet and have a scene in which each player can see one another and move around. I have an object in the scene with a Network Identity and Network Transform component. Each player can move these around by using a script i have set up which uses FixedJoints to move the objects around.It works fine on the server, as the client can see the servers changes, but the server cannot see the clients. The objects have a Rigidbody each. The script is below:

 using System.Collections;
 using System.Collections.Generic;
 using UnityEngine;
 using UnityEngine.Networking;
 
 public class PlayerPickupManager : MonoBehaviour {
 
     public Camera camera;
 
     private bool pickedup = false;
     private FixedJoint joint;
 
     private GameObject hitReference = null;
 
     void Start () {
         joint = GetComponentInChildren<FixedJoint>();
     }
     
     void FixedUpdate () {
 
         Ray ray = new Ray(camera.transform.position, camera.transform.forward);
         RaycastHit hit;
 
         Debug.DrawRay(ray.origin, ray.direction * 50);
 
         if (Physics.Raycast(ray, out hit, 50) && hitReference == null)
         {
             if(hit.collider.tag == "Thing")
             {
 
                 if (Input.GetKey(KeyCode.Mouse0))
                 {
                     Debug.Log("Found Item.");
 
                     hitReference = hit.collider.gameObject;
                     hit.collider.gameObject.GetComponentInParent<Rigidbody>().useGravity = false;                    
                     joint.connectedBody = hit.collider.GetComponentInParent<Rigidbody>();
 
                 }
             }
         }
 
        
        if (Input.GetKeyUp(KeyCode.Mouse0))
        {
              if (hitReference != null)
                 {
                 Debug.Log("Left Mouse Up.");
 
 
                 //hitReference.GetComponent<Rigidbody>().velocity = GetComponentInChildren<Rigidbody>().velocity;
                 hitReference.GetComponentInParent<Rigidbody>().useGravity = true;
                 joint.connectedBody = null;
                 hitReference = null;
 
 
             }
        }
 
 
     }
 }
 

I just presumed the transform would be updated for each of these items and that the fixed joint would allow this? How can i set it up so these can update?

Thanks! :D, sorry, i am new to UNet!