ScreenPointToLocalPointInRectangle not accurate

I have the following code to see if a position on screen is on the edge of a ui panel and resize it when it is.

currentPointerPosition is PointerEventData.position from OnDrag()

 float edgeSize = 10;
 Vector2 pos;
 RectTransformUtility.ScreenPointToLocalPointInRectangle(panelRectTransform, currentPointerPosition, data.pressEventCamera, out pos);
 Rect rect = new Rect(0, 0, panelRectTransform.rect.width, panelRectTransform.rect.height);

 bool output = false;
 if (pos.x > rect.xMax - edgeSize) {
     resizeX = 1;
     output = true;
 }
 else if (pos.x < rect.xMin + edgeSize) {
     resizeX = -1;
     output = true;
 }

Unfortunately when I place my cursor all the way on the left edge of the panel. The lowest x value I get is 8 but it could also go up to 12. This means I have to add or subtract from the x value and hope that it is accurate enough to grab the edge every time.

Is there something I can do to make it more accurate?

Update ##

I've resorted to using 8 buttons now. In my opinion not an ideal solution but I can't use this function without being uncertain if the user will be unable click on an edge to resize.