Need help with Raycast2D

Hi everyone, here's my setup. I have a 2D side-scrolling game where movement is determined by the position returned by a mouse click. I'm trying to use a Raycast 2D to determine if the tap is on a UI object or not. If the Raycast doesn't hit a UI object, then the click location is set as the target location and the player moves to that target. Right now, the character moves to the target when the layerMask is set to Everything, but no other setting. Also, I added a button and the player moves to the button location when it's clicked when it should be ignoring that click. If you could offer some advice on what the settings should be there, that would be wonderful. The player also doesn't stop when it's within range of the target. I messed with the distance tolerance and the player only stops if you click a target near it's current position while it's currently moving.

Here's my script: using System.Collections; using System.Collections.Generic; using UnityEngine;

public class YoloMates : MonoBehaviour {

 public Animator animator;
 public LayerMask layerMask;
 public Vector2 target = new Vector2();
 public Vector2 direction = new Vector2();
 public Vector2 position = new Vector2();
 public Vector2 movement = new Vector2();
 Rigidbody2D rb2D;
 public float speed = 2.0f;

 void Start()
 {
     animator = gameObject.GetComponent<Animator>();
     rb2D = GetComponent<Rigidbody2D>();
 }
 void FixedUpdate()
 {
     Vector3 m = Input.mousePosition;
     m = new Vector3(m.x, m.y, transform.position.z);
     Vector3 p = Camera.main.ScreenToWorldPoint(m);

     position = gameObject.transform.position;

     if (Input.GetMouseButtonDown(0))
     {
         Debug.Log("Click");
         RaycastHit2D hit = new RaycastHit2D();
         Ray2D ray2d = new Ray2D(new Vector2(p.x, p.y), Vector3.down);

         hit = Physics2D.Raycast(new Vector2(p.x, p.y), Vector3.down, Mathf.Infinity, layerMask);

         Debug.Log(hit.collider.gameObject.name);

         if (hit)
         {
             {
                 target = Camera.main.ScreenToWorldPoint(Input.mousePosition);
                 Debug.Log("Click");

                 if (target != Vector2.zero && (target - position).magnitude >= 1)
                 {
                     direction = (target - position).normalized;

                     movement.x = (direction.x * speed * Time.deltaTime);
                     movement.y = (direction.y * speed * Time.deltaTime);

                     rb2D.velocity = movement;

                     animator.SetBool("isMoving", true);
                     animator.SetFloat("moveX", direction.x);
                     animator.SetFloat("moveY", direction.y);
                 }
                 else
                 {
                     direction = Vector2.zero;
                     rb2D.velocity = Vector2.zero;
                     animator.SetBool("isMoving", false);
                 }
             }

         }
     }
 }

}