Unity Touch sensitivity issue on Samsung Devices

We are working on a touch based app for Android in Unity3D and C# which colors part of an image with Flood Fill as per touch.

Problem is, while on most of the devices the touch works smoothly, on Samsung Note 5 and S6 specially, it takes a few taps to register a single touch. To extend the confusion, when we try other similar apps on these devices, they have no problems at all.

Here's the code we used for the touch. It's basically a simple raycast on tapping the screen. We tried with both, mousePosition as well as GetTouch.

 if (Physics.Raycast (Camera.main.ScreenPointToRay (Input.GetTouch(0).position), out hit, Mathf.Infinity)) {
 
         Renderer rend = hit.transform.GetComponent<Renderer> ();
         MeshCollider meshcol = hit.collider as MeshCollider;
 
 
         myImage = Instantiate (rend.material.GetTexture ("_MainTex")) as Texture2D;
         Vector3 texCoord = hit.textureCoord;
         float scaleX = rend.material.GetTextureScale ("_MainTex").x;
         float scaleY = rend.material.GetTextureScale ("_MainTex").y;
         float deg = rend.material.GetFloat ("_RotationDegrees");
         deg *= Mathf.Rad2Deg;
         texCoord.x = (((texCoord.x * scaleX + rend.material.GetTextureOffset ("_MainTex").x)));
         texCoord.y = (((texCoord.y * scaleY + rend.material.GetTextureOffset ("_MainTex").y)));
         #region ROTATION
         float s = Mathf.Sin ((deg * Mathf.PI) / 180);
         float c = Mathf.Cos ((deg * Mathf.PI) / 180);
         Vector2 temp = new Vector2 (texCoord.x, texCoord.y);
         temp.x -= 0.5f;
         temp.y -= 0.5f;
         float xnew = temp.x * c + temp.y * s;
         float ynew = -temp.x * s + temp.y * c;
         temp.x = xnew + 0.5f;
         temp.y = ynew + 0.5f;
         texCoord.x = temp.x;
         texCoord.y = temp.y;
         #endregion
         texCoord.x *= myImage.width;
         texCoord.y *= myImage.height;
 
 
         FloodFiller.RevisedQueueFloodFill (myImage, (int)texCoord.x, (int)texCoord.y, GameplayConstants.currentColor, false);
         //  penSprite.GetComponent<Animation> ().Play ();
         rend.material.mainTexture = myImage;
     }

We have even used a script that detects DPI and makes changes to the Unity UI sensitivity on the fly, but that too doesn't help. In fact our other devices have similar DPI levels.

Will be thankful for any help!