Bug 20623 - Test instrumentation is not displaying the same number of tests ran as logcat
Summary: Test instrumentation is not displaying the same number of tests ran as logcat
Alias: None
Product: Android
Classification: Xamarin
Component: General ()
Version: 4.14.0
Hardware: Macintosh Mac OS
: Normal normal
Target Milestone: ---
Assignee: Jonathan Pryor
Depends on:
Reported: 2014-06-16 12:17 UTC by Peter Collins
Modified: 2017-08-23 21:11 UTC (History)
4 users (show)

Tags: bb
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Peter Collins 2014-06-16 12:17:45 UTC
Now that the web-tests are running reliably (fingers crossed...), they can be used as a repro project for this issue. Previously, this project ran around 440 tests, and the instrumentation only reported 330. Now that a smaller number of tests are being ran the difference has also reduced, but is visible none the less.

Here's a full gist of the logcat from that run:

The issue can be seen in the last three lines, right now there should be 241 tests executed, and only 234 are reported by the instrumentation. It should also be noted that when this suite is ran via the nunitlite GUI, it also shows that 241 tests ran.

I/mono-stdout( 1743): Tests run: 241, Passed: 241, Failed: 0, Skipped: 0, Inconclusive: 0
I/NUnitLite( 1743): NUnit automated tests completed
I/NUnitLite( 1743): Passed: 234, Failed: 0, Skipped: 0, Inconclusive: 0

I've added a workaround to my result parsing to listen for the mono-stdout test count for the time being.
Comment 1 Tom Opgenorth 2017-06-28 18:12:11 UTC
Not to sure how to duplicate or repro this issue. Can you provide reproductions steps?
Comment 2 Cody Beyer (MSFT) 2017-08-23 21:11:12 UTC
Because we have not received a reply to our request for more information we are closing this issue. If you are still encountering this issue, please reopen the ticket with the requested information. Thanks!