Running/Debugging of Xamarin.UITest projects is broken in VSMac 15.4 previews. The issue appears to be caused by an underlying Mono change (see https://bugzilla.xamarin.com/show_bug.cgi?id=59302). Not 100% sure, but this is manifesting itself in VSMac users not being able to run/debug Xamarin.UITest projects.
Whilst I do not have exactly the same bits as the current beta channel installed I can repro this error with the following repro steps:
1) Create a new iOS - App - WebView.
2) Update Xamarin.TestCloud.Agent NuGet package in iOS project to latest stable. Currently version 0.21.1
3) Update Xamarin.UITest NuGet package in UITests project to latest stable. Currently 2.1.3
4) Build the solution and run the tests from the Unit Tests window.
App opens in simulator and tests run successfully.
App opens in simulator but tests fail with:
ArgumentOutOfRangeException : Length cannot be less than zero.
Parameter name: length
You can reproduce the error from the command line just using nunit-console once the path to the .app is added to the StartupApp call:
Tried a few different Mono versions and ran the UITests from the command line:
Mono 126.96.36.199 (d15-3/14f2c81) - OK
Mono 188.8.131.52 (2017-06/2f1b9e8) - Fail
Mono 184.108.40.206 (2017-06/6b8abfeb7cc) - Fail
Mono 220.127.116.11 (2017-06/71277e78f6e - Fail
Mono 18.104.22.168 (2017-06/cb4c3ca997f) - Fail
Mono 22.214.171.124 (2017-10/8a2a0628624) - OK
## Additional testing to answer what fixed the issue
The fix that is present in Mono 5.8 is also present in Mono 5.6. I performed a bisect using the available CI builds, and found that one of the following 2 commits fixed the issue:
https://github.com/mono/mono/commit/b08c54265b20d914bd3a5f1c0f7a8ad606411c6e "Bump API snapshot submodule"
https://github.com/mono/mono/commit/efc7e54e149d5b4642bc3b89d7d963c98ec84561 "Bump CoreFX"
"Bump CoreFX" is likely the fix (corefx does appear in the stack trace), but unfortunately that bump is a large change, so backporting it would carry risks.
I can attempt a similar bisect to find the point of breakage tomorrow. My suspicion would be that it's another earlier CoreFX bump.
I will mark this bug as a duplicate of Bug 59302 to consolidate tracking of the issue. I will leave this bug categorized under the "Xamarin Studio > Test Cloud" component to help with searchability in case any users search are searching for the issue primarily in the context of Test Cloud.
*** This bug has been marked as a duplicate of bug 59302 ***