Bug 36064 - [iOS] Running similar Unit Test - TestFixtures will cause the second set to fail, with unexpected results
Summary: [iOS] Running similar Unit Test - TestFixtures will cause the second set to f...
Status: CONFIRMED
Alias: None
Product: iOS
Classification: Xamarin
Component: Xamarin.iOS.dll (show other bugs)
Version: XI 9.2
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: Future Cycle (TBD)
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2015-11-19 15:08 UTC by Matthew Regul
Modified: 2016-07-14 23:29 UTC (History)
4 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Sample App (6.30 KB, application/zip)
2015-11-19 15:08 UTC, Matthew Regul
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report for Bug 36064 on Developer Community or GitHub if you have new information to add and do not yet see a matching new report.

If the latest results still closely match this report, you can use the original description:

  • Export the original title and description: Developer Community HTML or GitHub Markdown
  • Copy the title and description into the new report. Adjust them to be up-to-date if needed.
  • Add your new information.

In special cases on GitHub you might also want the comments: GitHub Markdown with public comments

Related Links:
Status:
CONFIRMED

Description Matthew Regul 2015-11-19 15:08:21 UTC
Created attachment 13896 [details]
Sample App

**Overview:**

Running similar TestFixtures will cause the second set to fail, with unexpected results

Screencast - http://screencast.com/t/b1X37CRvA6T
Sample App - *Attached Above*
TestFixture Code Snip - https://gist.github.com/mattregul/30ad4834358f7f845d76#file-myunittest-cs
Error Screenshot - https://gist.github.com/mattregul/30ad4834358f7f845d76#gistcomment-1625614
Application Output Log - https://gist.github.com/mattregul/30ad4834358f7f845d76#file-appoutput-cs
Version Info - https://gist.github.com/mattregul/30ad4834358f7f845d76#file-versioninfo-cs

**Steps to Reproduce:**

1. Create a new iOS UnitTest App project in Xamarin Studio 
 (Guide - https://developer.xamarin.com/guides/ios/deployment,_testing,_and_metrics/touch.unit/)
2. Add a new iOS UnitTest file
3. By default, the UnitTest template contains a TestFixture with 3 individual Tests
4. Rename the TestFixture to something unique (ex: aTests)
5. Make a duplicate of the entire TestFixture
6. Paste and rename the new TestFixture to something else (ex: bTests)
7. Run the Unit Test app in simulation
8. Click into the first TestFixture (aTests)
9. Select Run All
10. Make note that everything returned with their expected results
11. Go back and enter the second TestFixture (bTests)
12. Select Run All
13. Make note that each of these tests returned errors
14. Now that the fault occurred, trying to run aTests again, or backing to the start and trying "Run All Tests" will result in an error
15. Close the App and rerun it again, fresh
16. Running each test individually (each of the three inside aTest and bTest), they will all function properly.


**Actual Results:**

The second TestFixture fails, making mention of the first TestFixture
"System.Reflection.TargetException: Object of type 'iOSUnitTest.aTest' doesn't match target type 'iOSUnitTest.bTest'"

See Screenshot - https://gist.github.com/mattregul/30ad4834358f7f845d76#gistcomment-1625614
See Application Output Log - https://gist.github.com/mattregul/30ad4834358f7f845d76#file-appoutput-cs


**Expected Results:**

Both TestFixtures to function as expected   


**Build Date & Platform:**

Version Info - https://gist.github.com/mattregul/30ad4834358f7f845d76#file-versioninfo-cs
Comment 2 Rolf Bjarne Kvinge [MSFT] 2015-11-20 05:04:21 UTC
This is probably related to bug #22606.
Comment 3 Jon White 2016-07-14 18:20:39 UTC
This still seems to be an issue.  Is there a pending fix or workaround?
Comment 4 Rolf Bjarne Kvinge [MSFT] 2016-07-14 23:29:20 UTC
This is still an issue, and there's no pending fix. It seems to be simple to fix once someone has the time to look at it.