koobas.hobune.stream
Integration tests fail on 'Run All'
I have a selection of three integration tests on my scene. When running the tests individually, each of these pass. However, when I choose 'Run All', two of these tests fail.
The tests that are failing are the ones with a script to call a 'Move' method on the player object, moving the player to the right.
Does anyone know what might cause testing to fail when run together?
I had the exact same problem. It didn't seem to be a problem of my making with shared states. I was able to solve it by slowing the test down.
I was doing the move right at the beginning of the test then the assertion component was checking right after in the FixedUpdate method. I changed it so the move happened 0.5seconds into the test, and the assertion component set to check after 1 second.
This seemed to fix my issue- seems like it could be a problem with the framework potentially.
How were your tests failing? Were they odd Null reference exceptions?
I've also noticed view weird behaviour if I duplicate tests to make more- the test components seem linked somehow in weird undefined ways, so that's something to check also- try creating the problem tests from scratch if you duplicated them to create them initially.
EDIT: I think I've found a more concrete cause of this. It seems that tests carry on in the background even after they've passed. If that test happens to hit a failing condition it will fail the test that's currently running. I was trying to debug why a test was failing when I realised the call to IntegrationTest.Fail belonged to a game object, whose root test component was for an already completed and passed test! This makes it really awkward but I was able to fix it with an if (transform.root.gameObject.activeSelf) around the failing condition. I hope this is able to help someone.
I had the same problem, but it wasn't that the tests were running in the background so much as it was that I had failed to clean up after myself. For example, one of my tests checks for the existence of an object I've created, expecting to find just one object. When the assertion component performs the check, the test passes and the test runner moves on to the next test, which also expects just one object. However, the scene wasn't cleaned up at that point -- my original test object still existed and the test found two.
Since the test runner enables the TestComponent game object before starting a test and disables it before finishing, my solution was to do a full-scale tear down / clean up on the OnDisable call. (I also do set up on the OnEnable call, but that's just to make it all symmetric really.) That seems to have worked.
