Add force only to specific layer?

I have a raycast that applies forces to rigidbodies but I'd like it to apply different amounts of force depending on what specific object its hitting. In my case, I'd like it to apply a specific amount of force when its hitting an NPC as opposed to level props.

I can't use CompareTag because the NPC has several different colliders attached to its bones, each with their own tag. I was thinking I could just group all of those colliders into one layer and have the raycast check for that. So how do I do something like (pseduo code):

 RaycastHit hit;
 if (Physics.Raycast(ray,out hit)){
 
     if (hit.rigidbody!=null || EnemyColliders Layer){
        apply specific force
 
    }
 
 }