Mouse Cursor is flashing

I'm trying to make my mouse cursor change depending on what it is hovering over based on the tag of the gameobject. At the moment I'm just using two tags, enemy and npc. I've read all the docs that I can find and it looks like I'm doing everything correctly but everytime I run the game the mouse cursor flickers between two of the cursors for about 10-15 seconds and then it'll settle down. Even after that though I'm only getting mouse click responses sporadically. This is what I have for code at the moment. Edit: don't know if this has anything to do with the problem, but my cursor script is attached to my main camera and the main camera is attached to the player. There is 6 different players each with a main camera attached to them, but only one player is ever activated at a time.

public class CursorAffordance : MonoBehaviour {

 [SerializeField] Texture2D lootCursor = null;
 [SerializeField] Texture2D unknownCursor = null;
 [SerializeField] Texture2D attackCursor = null;

 void Start() {
     Cursor.SetCursor(unknownCursor, Vector2.zero, CursorMode.Auto);
 }

 Ray ray;
 RaycastHit hit;

 void Update() {
     ray = Camera.main.ScreenPointToRay(Input.mousePosition);
     if(Physics.Raycast(ray, out hit)) {
         if(hit.collider.tag == "Enemy") {
             print("Enemy");
             Cursor.SetCursor(attackCursor, Vector2.zero, CursorMode.Auto);
         }else if (hit.collider.tag == "NPC") {
             print("NPC");
             Cursor.SetCursor(lootCursor, Vector2.zero, CursorMode.Auto);
         }else {
             Cursor.SetCursor(unknownCursor, Vector2.zero, CursorMode.Auto);
         }
     }    

     if (Input.GetMouseButtonDown(1) && !UnityEngine.EventSystems.EventSystem.current.IsPointerOverGameObject()){
         GetInteraction();
         print("Clicked something");
     }
 }