Wayback Machinekoobas.hobune.stream
May JUN Jul
Previous capture 12 Next capture
2021 2022 2023
1 capture
12 Jun 22 - 12 Jun 22
sparklines
Close Help
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
1
Question by Doodums · Dec 22, 2016 at 02:58 PM · mouse clicksimulate

Test Automation: Simulate mouse down event at Vector2 location during runtime

I have a script that records the mouse position every time an Input.GetMouseButton(0) event occurs. I store the time of the click, and the current cursors position. I want to take that information, and playback the inputs on request.

I want to take the Vector2 coordinates and tell Unity to "Simulate a mouse down event at this Vector2 location at this time".

Current attempted solutions:

  1. The only solutions I have found for this problem talk about using ExecuteEvents.Execute to fire off an event, but that requires you to specify a GameObject in which to trigger the event on, which I don't want to store.

  2. The other solution was to implement methods from user32.dll and move the actual windows cursor and trigger mouse clicks from there. That currently only works if the user running the simulation again has the game window in the exact same location, and same screen as when the simulation was recorded.

  3. A 3rd solution was to have a script on every intractable object that monitors for a MouseDown event, but I want to avoid this solution at all costs.

This is what my class currently looks like:

     public class IntegrationTestBehaviour : MonoBehaviour
     {
         [SerializeField]
         public MouseInputInstance[] Inputs;
         private List<MouseInputInstance> InputRecord;
         private bool _recording;
         private float _recordingStartTime;
     
         public void RecordInputs()
         {
             _recording = true;
             _recordingStartTime = Time.time;
             InputRecord = new List<MouseInputInstance>();
         }
     
         public void PlaybackInputs()
         {
             for (int i = 0; i < Inputs.Length; i++)
             {
                 StartCoroutine(HandleInputInstance(Inputs[i]));
             }
         }
     
         void Update()
         {
             if (!_recording)
                 return;
             if (Input.GetKeyDown(KeyCode.Return))
             {
                 _recording = false;
                 Inputs = InputRecord.ToArray();
             }
     
             if (Input.GetMouseButtonDown(0))
             {
                 InputRecord.Add(new MouseInputInstance()
                 {
                     ClickTime = Time.time - _recordingStartTime,
                     CursorPosition = Input.mousePosition
                 });
             }
         }
     
         private IEnumerator HandleInputInstance(MouseInputInstance mouseInput)
         {
             yield return new WaitForSeconds(mouseInput.ClickTime);
             // TODO: Simulate mouse click at mouseInput.CursorPosition
         }
     
         [Serializable]
         public class MouseInputInstance
         {
             [SerializeField]
             public Vector2 CursorPosition;
             public float ClickTime;
         }
     }

With the following CustomInspector

 [CustomEditor(typeof(IntegrationTestBehaviour))]
 public class IntegrationTestBehaviourCustomInspector : Editor
 {
     private IntegrationTestBehaviour _target;
 
     public override void OnInspectorGUI()
     {
         base.OnInspectorGUI();
         _target = (IntegrationTestBehaviour)target;
         if (!EditorApplication.isPlaying)
             return;
 
         if (GUILayout.Button("Start Recording"))
         {
             _target.RecordInputs();
         }
 
         if (GUILayout.Button("Playback Recording"))
         {
             _target.PlaybackInputs();
         }
     }
 }

Comment
Add comment · Show 1
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image asger60 · Sep 18, 2019 at 06:22 PM 0
Share

I'm also very interested, if anybody has a solution, or the author figured something out

1 Reply

· Add your reply
  • Sort: 
avatar image
1

Answer by Spree_User · Sep 18, 2019 at 06:43 PM

I recently ran in to a similar situation. As long as you are doing this on a PC, you can import user32.dll to fake the mouse position and mouse events.

Here's a class that I found while looking for a solution to this.

 using System;
 using System.Runtime.InteropServices;
 
 public class MouseOperations
 {
     [Flags]
     public enum MouseEventFlags
     {
         LeftDown = 0x00000002,
         LeftUp = 0x00000004,
         MiddleDown = 0x00000020,
         MiddleUp = 0x00000040,
         Move = 0x00000001,
         Absolute = 0x00008000,
         RightDown = 0x00000008,
         RightUp = 0x00000010
     }
 
     [DllImport("user32.dll", EntryPoint = "SetCursorPos")]
     [return: MarshalAs(UnmanagedType.Bool)]
     private static extern bool SetCursorPos(int x, int y);
 
     [DllImport("user32.dll")]
     [return: MarshalAs(UnmanagedType.Bool)]
     private static extern bool GetCursorPos(out MousePoint lpMousePoint);
 
     [DllImport("user32.dll")]
     private static extern void mouse_event(int dwFlags, int dx, int dy, int dwData, int dwExtraInfo);
 
     public static void SetCursorPosition(int x, int y)
     {
         SetCursorPos(x, y);
     }
 
     public static void SetCursorPosition(MousePoint point)
     {
         SetCursorPos(point.X, point.Y);
     }
 
     public static MousePoint GetCursorPosition()
     {
         MousePoint currentMousePoint;
         var gotPoint = GetCursorPos(out currentMousePoint);
         if (!gotPoint) { currentMousePoint = new MousePoint(0, 0); }
         return currentMousePoint;
     }
 
     public static void MouseEvent(MouseEventFlags value)
     {
         MousePoint position = GetCursorPosition();
 
         mouse_event
             ((int)value,
              position.X,
              position.Y,
              0,
              0)
             ;
     }
 
     [StructLayout(LayoutKind.Sequential)]
     public struct MousePoint
     {
         public int X;
         public int Y;
 
         public MousePoint(int x, int y)
         {
             X = x;
             Y = y;
         }
     }
 }


Edit:

To do a mouse click, call MouseEvent like this: MouseOperations.MouseEvent(MouseOperations.MouseEventFlags.RightUp | MouseOperations.MouseEventFlags.RightDown);

Of course, set the position first.

Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

Answers Answers and Comments

63 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Trajectory prediction basketball 1 Answer

How to make a mouse click selection for in game use 1 Answer

How to do a Burst Particle effect on Mouse Click? 2 Answers

C# Script Simulating Input.inputString 0 Answers

How to disable Touch to Mouse Click translation 2 Answers


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges