Bug 29220 - After latest update Test Cloud throws error in output window of Visual Studio
Summary: After latest update Test Cloud throws error in output window of Visual Studio
Alias: None
Product: Visual Studio Extensions
Classification: Xamarin
Component: Test Cloud (show other bugs)
Version: 3.11 (C5)
Hardware: PC Windows
: --- normal
Target Milestone: 4.1.0 (C7)
Assignee: Bugzilla
Depends on:
Reported: 2015-04-18 01:17 UTC by valdetero
Modified: 2016-06-02 06:08 UTC (History)
13 users (show)

Is this bug a regression?: ---
Last known good build:

Screenshot of error in Visual Studio (18.98 KB, image/png)
2015-04-18 01:17 UTC, valdetero
Logs from %LOCALAPPDATA%\Xamarin\Log (21.75 KB, application/zip)
2015-04-28 10:49 UTC, valdetero
Solution that causes the error. (57 bytes, text/plain)
2015-04-28 10:51 UTC, valdetero
Android build tools screenshot. (9.90 KB, image/png)
2015-04-28 10:56 UTC, valdetero
After latest update (16.45 KB, image/png)
2015-04-30 17:34 UTC, valdetero
Test cloud output window. (3.54 KB, image/png)
2015-07-07 12:15 UTC, valdetero

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description valdetero 2015-04-18 01:17:35 UTC
Created attachment 10809 [details]
Screenshot of error in Visual Studio

Windows 8.1 
Visual Studio 2013 Update 4 
Xamarin 3.11.431.0
Xamarin.TestCloud.Integration 1.0
Beta Channel

After the latest update, I know have an option for "Test Cloud" in the Output window in Visual Studio. Without doing anything other than loading my project, the output from "Test Cloud" is 
ERROR: Value cannot be null.
Parameter name: project
This is causing an error (build error) in the Error List window.
Comment 1 Brendan Zagaeski (Xamarin Team, assistant) 2015-04-28 06:44:24 UTC
I tried a few experiments with iPhone and Forms template projects created using either the new project dialog in Xamarin Studio (that automatically adds references to the "Xamarin Test Cloud Agent" NuGet Package).

When I have a project that includes a "Xamarin Test Cloud Agent" package installed, I can produce some output in the "Output" -> "Test cloud" window if I use the "Run in Test Cloud" context menu option on the UITest project.

But so far I have not been able to reproduce the "ERROR: Value cannot be null. Parameter name: project" message.

## Possible next steps

If you can reproduce the problem with a brand new project (created using one of the built-in templates), that would be good to know.

If you _cannot_ reproduce the problem with a brand new project, then the most direct way to proceed would probably be to zip up a project that demonstrates the problem and attach it to the bug report. The last couple of "Additional details" from [1] might be helpful in creating a test case. The information on that article about contacting support might be helpful too.

> [1] https://kb.xamarin.com/customer/portal/articles/1910343-when-and-how-should-i-file-a-bug-report-#note-5
(If this link redirects to the top-level kb.xamarin.com/ page the first time you click it, try clicking it once more.)

Thanks in advance!
Comment 2 valdetero 2015-04-28 10:49:16 UTC
Created attachment 10936 [details]
Logs from %LOCALAPPDATA%\Xamarin\Log
Comment 3 valdetero 2015-04-28 10:51:43 UTC
Created attachment 10937 [details]
Solution that causes the error.
Comment 5 valdetero 2015-04-28 10:55:51 UTC
I tried making a new project and it did not give the error. I currently don't have any UI Test projects and am not using Test Cloud. I rolled back to the stable channel and had no error. I switched back to Beta and had the error again.

I have attached my project. It can also be found at: https://github.com/valdetero/SevenDays.
Comment 6 valdetero 2015-04-28 10:56:13 UTC
Created attachment 10938 [details]
Android build tools screenshot.
Comment 7 Brendan Zagaeski (Xamarin Team, assistant) 2015-04-28 19:12:41 UTC
Many thanks for all the additional details and the sample project.

Just to confirm the steps to reproduce if you get a chance, is it sufficient to open "SevenDays.sln"? After you open the solution, it sounds like you immediately see the error in the Test Cloud output about "Value cannot be null. Parameter name: project". Correct?

Thanks again.
Comment 8 valdetero 2015-04-30 17:34:54 UTC
Created attachment 10999 [details]
After latest update
Comment 9 valdetero 2015-04-30 17:36:48 UTC
Yes that is correct. Just opening the solution causes the error. However, I just realized that everytime I right-click on an unloaded project, it generates another error. I tried this with winphone, ios, and pcl projects.
Comment 10 Brendan Zagaeski (Xamarin Team, assistant) 2015-04-30 22:14:31 UTC
I'm updating the target milestone to ensure an initial assessment before service release 1. Since we have seen a few reports of the problem already, there is a fair chance we will start seeing more reports as customers continue to upgrade to 3.11.

I'm also updating the severity to reflect that this problem blocks development for the users affected by it.
Comment 11 Brendan Zagaeski (Xamarin Team, assistant) 2015-05-01 13:51:23 UTC
Adjusting the severity back to Normal based on observations that these errors do not block building, deploying, or debugging (as it turns out, comment 10 was inaccurate).

The errors are quite confusing when building because they appear after the build completes, but the build apparently completes successfully despite these error messages.

I'd still say this is worth an assessment before service release 1 is finalized because the error messages are quite confusing.
Comment 12 Brendan Zagaeski (Xamarin Team, assistant) 2015-05-03 00:35:55 UTC
## Possible partial workaround

Assuming this error is indeed originating from the new Test Cloud integration feature, I believe uninstalling the Test Cloud integration (see Bug 29680, Comment 2) will stop these error messages.

If any 1 person who's seeing this error would like to verify that that partial workaround is successful, that would be perfect. Thanks!
Comment 13 Daniel Dickey 2015-05-04 09:11:13 UTC
I can verify that the partial workaround from Brendan was successful for me. It removed the Test Cloud integration and I stopped receiving the errors.

Comment 14 Christian Meneghini 2015-05-04 09:17:04 UTC
Same here: creating a new Xamarin project from Visual Studio (2013 sp4) on Windows 8.1 causes the error.
Solved by deleting testcloud references as describe in Bug 29680.
Comment 15 valdetero 2015-05-04 09:54:12 UTC
I confirm the workaround caused the error to stop showing for me.
Comment 16 Luke Watsn 2015-05-05 09:34:05 UTC
I dont have a reference to Test Cloud integration. also this is affecting every solution i have not just xamarin projects. I can not build so this is affecting my job.

I rolled back to a previous version of xamarin and  the error went away, but now all of my nuget references are broken and nothing builds in visual studio. i reinstalled visual studio and 're-got' all of my code from source control and they still do not build.

Currently i have the lastest version of Xamarin again - error has come back, but i dont know how to disable test cloud.
Comment 17 Rogier van der Hee 2015-05-06 06:56:20 UTC
I can confirm too that the workaround (delete 2 TestCloud files, re-run devenv setup is working to remove the error
Comment 18 Luke Watsn 2015-05-06 06:59:31 UTC
I ended up rebuilding my machine. fresh install on windows 8.1 and now I don't have that issue.
Comment 20 info 2015-05-08 03:38:13 UTC
Hello, I also get exactly that message on the "Test Cloud" output, although I am not using Test Cloud at all. I am pretty sure that the error message and problem is related to the Components that I have included my project which are

Android Support Library v7 AppCompat
Android Support Library v7 CardView

After loading the solution the latter is actually shown as

Android Support Library v7 CardView (Error)

Opening the component by clicking on it shows exactly the same message at the top. I uploaded a screenshot for your convenience at


Surprisingly the project just builds and runs, so it doesn't seem to be a real problem. Hope this helps to find and fix the bug.
Comment 21 valdetero 2015-06-01 13:42:03 UTC
I still get this error after beta update to 3.11.584.
Comment 22 valdetero 2015-07-07 12:14:29 UTC
I am still having an issue with this. Now the Test Cloud output window is is giving the error: "ERROR: Object reference not set to an instance of an object." This is now causing a BUILD ERROR for my ios project.
Xamarin VS 3.11.666.0
Comment 23 valdetero 2015-07-07 12:15:50 UTC
Created attachment 11912 [details]
Test cloud output window.
Comment 27 Arpit Jha 2016-06-02 06:08:24 UTC
I have checked this issue with C7 build XVS +X.UITest 1.3.8 and able to upload app successfully to TestCloud.

I am not observing any error in Output of TestCloud.

For now I am closing this issue.Please feel free to reopen this issue if you are getting this again in future.

Screencast : http://www.screencast.com/t/lw6m0uGMH
Environment Info: https://gist.github.com/Nischalk/75dd14178f442ded4d0f431bc01af24a