koobas.hobune.stream
Automated testing webgl mouse events not working
I'm working on end-to-end automated tests with a webgl build using the Cypress framework.
The purpose is to test from the perspective of an end user (BDD testing), so the tests need to simulate interaction at the browser level.
It works great for a number of things, but it is failing to simulate mouse events properly.
The mouse events are sent to the canvas, but they do not register with the unity client.
One of the major differences between my simulated clicks and a real click is the event property 'isTrusted', which is true for a real event, but is designed specifically to be false for simulated events. Is this the source of my headache?
simulated click:
MouseEvent {isTrusted: false, stopPropagation: ƒ, screenX: 0, screenY: 0, clientX: 70, …}
altKey: false
bubbles: true
button: 0
buttons: 0
cancelBubble: false
cancelable: true
clientX: 70
clientY: 20
composed: false
ctrlKey: false
currentTarget: null
defaultPrevented: false
detail: 1
eventPhase: 0
fromElement: null
isTrusted: false
layerX: 20
layerY: 20
metaKey: false
movementX: 0
movementY: 0
offsetX: 20
offsetY: 20
pageX: 70
pageY: 20
path: (7) [canvas##canvas, div#gameContainer, div.webgl-content, body, html, document, Window]
relatedTarget: null
returnValue: true
screenX: 0
screenY: 0
shiftKey: false
sourceCapabilities: null
srcElement: canvas##canvas
stopPropagation: ƒ ()
target: canvas##canvas
timeStamp: 2080510.8000000035
toElement: canvas##canvas
type: "click"
view: Window {postMessage: ƒ, blur: ƒ, focus: ƒ, close: ƒ, parent: Window, …}
which: 1
x: 70
y: 20
__proto__: MouseEvent
real click:
MouseEvent {isTrusted: true, screenX: 3355, screenY: 162, clientX: 75, clientY: 31, …}
altKey: false
bubbles: true
button: 0
buttons: 0
cancelBubble: false
cancelable: true
clientX: 75
clientY: 31
composed: true
ctrlKey: false
currentTarget: null
defaultPrevented: false
detail: 1
eventPhase: 0
fromElement: null
isTrusted: true
layerX: 25
layerY: 31
metaKey: false
movementX: 0
movementY: 0
offsetX: 26
offsetY: 32
pageX: 75
pageY: 31
path: (7) [canvas##canvas, div#gameContainer, div.webgl-content, body, html, document, Window]
relatedTarget: null
returnValue: true
screenX: 3355
screenY: 162
shiftKey: false
sourceCapabilities: InputDeviceCapabilities {firesTouchEvents: false}
srcElement: canvas##canvas
target: canvas##canvas
timeStamp: 25545.880000005127
toElement: canvas##canvas
type: "click"
view: Window {postMessage: ƒ, blur: ƒ, focus: ƒ, close: ƒ, parent: Window, …}
which: 1
x: 75
y: 31
__proto__: MouseEvent
Hi, I managed to reproduce your issue. It seems like the position of the simulated click events is not being used when the click is processed. Ins$$anonymous$$d, it uses the current mouse position. If you hover the button you want to click when the simulated event happens, the click will succeed. We're going to continue looking into this issue and try to confirm with the developers if this is expected behaviour.
Whoa is this an actual Unity response to an Answers post? Great to see!
Because Cypress doesn't actually move the mouse, the simulated clicks do not happen at the position you require. I managed to find out a workaround for your specific case, which is to do a slight modification to StandaloneInputModule.cs. I've tested this only in WebGL and might not work or might require modifications for different platforms.
You can create a copy of StandaloneInputModule and modify the ProcessMousePress function to do an explicit raycast with the new click coordinates, instead of relying on the ones from the mouse position.
if (Event.current != null)
{
// Y axis is inverted in the Event coordinates
pointerEvent.position = new Vector2(Event.current.mousePosition.x, Screen.height - Event.current.mousePosition.y);
// Raycast the received click position
eventSystem.RaycastAll(pointerEvent, m_RaycastResultCache);
var raycast = FindFirstRaycast(m_RaycastResultCache);
// Save the raycast result and replace the "clicked" gameObject with the new result
pointerEvent.pointerCurrentRaycast = raycast;
currentOverGo = pointerEvent.pointerCurrentRaycast.gameObject;
}
You can then remove the StandaloneInputModule script from the EventSystem object in your scene and replace it with you new CustomStandaloneInputModule. An example of how this might be done is attached here.
The way the UI input system works is to detect GameObjects underneath the mouse pointer and whenever a click event happens, it propagates the event to the corresponding GameObjects.
You can browse through how all this works in the public repository.
