After we upgraded the bots from Xcode 8.1-beta1 to 8.3 we started seeing this test randomly failing. It looks like the 400ms timeout is suddenly not enough anymore.
A similar test that started failing was MonoTests.System.Runtime.Remoting.SynchronizationAttributeTest.TestSynchronizationReleasedOnMultipleAcquire which was "fixed" in https://github.com/mono/mono/pull/5373 by drastically increasing the timeout.
This looks like some part of the runtime got much slower due to the compiler upgrade.
The Delay_Simple test is much smaller so probably easier to investigate.
The XM team hit this failure too, see discussion on https://lists.dot.net/pipermail/mono-devel-list/2017-August/044502.html
Bumped the timeout in Delay_Simple for now with https://github.com/mono/mono/pull/5410
@Ludovic: can you please route this to someone who can take a look at it? :)
Its possible that machine load, gcs, other tasks, etc. would cause a 100ms delay, so this might not be a bug.
@Zoltan in case of the TestSynchronizationReleasedOnMultipleAcquire test the timeout had to be bumped by 8 *seconds*.
Since this never occurred before the compiler upgrade it still needs to be investigated whether this is hiding an underlying issue that is more widespread.
I can confirm the test takes 3 seconds on my machine.
I profiled TestSynchronizationReleasedOnMultipleAcquire with xcode 8.3 and could not find anything out of ordinary. Those tests are really bad and are mostly wasting cpu time.
It would be great to have a Remoting maintainer to take a look and sanitize them.
I could not repro it taking more than its expects 2.4s.
I did the same with the other test. There's no particular performance woes.
If anything, this is due to timing differences, CI compiler flags or an unwanted side effect of callconv breakage.