creating 3d melee cone splash attack

Hello! Trying to learn the ropes around Unity3d, so please dont judge me to hard. I've been struggling around with one problem that I couldnt find a good solution for. I need to make a melee cone splash attack that player's character is performing on left click. For that I need to find all the enemies in front of the player in a particular range and reduce their health. First I'm collecting enemies into an array with Physics.OverlapSphere and then I check that only those in front of the player are hit with Vector3.dot. So first part about getting all the enemies in range around the player is going smoothly, but once I get to the part where I check their location with Vector3.dot, I start to get pretty odd behaviour and weird angles. Where I would expect an angle of 60 degrees or even less, I get angles 120 degrees and even more and therefore in situations where hit is supposed to be scored - it doesnt happen, and when there is no way it could happen (character is standing with it's back to the enemy) - it happens. Here's the code of player's attack script:

  using System.Collections;
     using System.Collections.Generic;
     using UnityEngine;
     
     public class PlayerAttack : MonoBehaviour {
          
         public int damagePerSwing = 20;
         public float timeBetweenSwings = 0.15f;
         public float SwingRange;
     
     
         float timer;
         Ray swingRay;
         RaycastHit swingHit;
         int shootableMask;
         int layertest = 1 << 9;
     
         void Awake(){
         
             shootableMask = LayerMask.GetMask ("Hittable");
         }
 
     void Update() {
             timer += Time.deltaTime;
             if (Input.GetButton ("Fire1") && timer >= timeBetweenSwings) {
             
                 Swing ();
             }
         }
 
         void Swing (){
             timer = 0f;
     
             swingRay.origin = transform.position;
             swingRay.direction = transform.up;
 //that's where I check who's nearby
                Collider[] Enemy_col = Physics.OverlapSphere(transform.position, SwingRange, layertest );
               int i = 0;
             float inRange = 0f;
             while (i < Enemy_col.Length) {
 //there I get vector3.dot from player's front vector and player-to-enemy vector (which I get by substracting as follows)
                  inRange = Vector3.Dot (swingRay.GetPoint(SwingRange).normalized, Enemy_col[i].transform.position.normalized - transform.position.normalized);
                    //that's where I check that I can hit those enemies
                 if (inRange > 0f && inRange < 1f) {
                      //and there's an actual hit happens  
                     swingRay.direction = Enemy_col[i].transform.position;
                     Physics.Raycast (swingRay, out swingHit, SwingRange, shootableMask);
                     EnemyHealth enemyHealth = Enemy_col[i].GetComponent<EnemyHealth> ();
                     if (enemyHealth != null) {
                         enemyHealth.TakeDamage (damagePerSwing, swingHit.point);
                     }
                     swordLine.SetPosition (1, swingHit.point);
                 }
                 i++;
             }
     
         }
         
     }
 

I have a feeling that if I dont find any solution to this I'll just have to redesign whole script and prehab for an additional gameobject "Cone" with collider triggers and etc and my guts tell me it's not that elegant to go this way.