Allow picked up object to collide with game level GameObjects

I'm currently producing a gravity gun similar to that in half life in C#. The player right clicks on an object to pick it up and it will make itself a child of an GameObject which will act as the state for carrying the object.

The parent that the object is assigned to has only a box collider and is essentially an invisible cube that the object being picked up sticks to.

The problem I'm experiencing however is that even though the object being picked up has a box collider and a rigidbody, the block is still able to be pushed through the Game level GameObjects and I have no idea how I can stop this from happening.

When the player chooses to drop the picked up object via clicking the right mouse button again then it will collide with the game level GameObjects just not when its being picked up and made an child.

I created a small workaround in which I previous had pickup.collider.isTrigger set to false on everything but since removed that as it was useless. And currently at the moment in time if you push the picked up object through an gameObject in the game level it will simply bounce out. However the way I want it is the player is unable to push the carried object through the game level GameObjects in the first place and just simply collides with it.

Any ideas?

pickUp Code

 public Quaternion   objectRot;
 public Vector3      objectPos;

 public GameObject   pickObj;
 public GameObject   pickref;

 public bool         canpick = true;
 public bool         picking = false;

   // Use this for initialization
 void Start ()
 {
     pickref = GameObject.FindWithTag("pickupref");
     pickObj = pickref;
 }

 // Update is called once per frame
 void Update ()
 {
     objectPos = transform.position;

     objectRot = transform.rotation;

     //Method here picks up the object with the right mouse button
     if (Input.GetMouseButtonDown (1) && canpick)
     {
         Debug.Log ("Object Picked Up");
         picking = true;

         Ray ray = Camera.main.ScreenPointToRay (Input.mousePosition);

         RaycastHit hit;

         if (Physics.Raycast (ray, out hit, 10) && hit.collider.gameObject.tag == "pickup")
         {
             pickObj = hit.collider.gameObject;
             hit.rigidbody.useGravity = false;
             hit.rigidbody.isKinematic = true;
             hit.transform.parent = gameObject.transform;
             hit.transform.position = objectPos;
             hit.transform.rotation = objectRot;
         }
     }
     
     if (Input.GetMouseButtonUp (1) && picking)
     {
         picking = false;
         canpick = false;
     }

     //This acts as a small work-around by reclicking and detecting that you haven't picked anything it will
     else if (Input.GetMouseButtonUp(1) && !picking)
     {
         canpick = true;
     }

     //Method here drops the object if it has been picked up by clicking the right mouse button
     if (Input.GetMouseButtonDown (1) && !canpick && pickObj.GetComponent<PickedUpObject> ().refuseThrow != true)
     {
         canpick = true;
         pickObj.rigidbody.useGravity = true;
         pickObj.rigidbody.isKinematic = false;
         pickObj.transform.parent = null;
         
         pickObj = pickref;

    

I've also included the code for what goes on the object being picked up in case people want to see what I'm applying to objects that are being picked up.

PickedUpObject Code

 public bool refuseThrow = false;

 void OnTriggerEnter(Collider other)
 {
     if (other.gameObject.tag != "Player" && other.gameObject.tag != "pickto")
     {
         refuseThrow = true;
     }
 }

 void OneTriggerExit(Collider other)
 {
     if (other.gameObject.tag != "Player" && other.gameObject.tag != "pickto")
     {
         refuseThrow = false;
     }
 }