Bug 35424 - UITest in Visual Studio is unable to detect Android Package
Summary: UITest in Visual Studio is unable to detect Android Package
Alias: None
Product: Visual Studio Extensions
Classification: Xamarin
Component: General ()
Version: 4.0.0 (C6)
Hardware: Macintosh Mac OS
: High critical
Target Milestone: 4.0.1 (C6SR1)
Assignee: Bugzilla
Depends on:
Reported: 2015-10-30 17:10 UTC by James
Modified: 2015-12-17 14:47 UTC (History)
6 users (show)

Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description James 2015-10-30 17:10:42 UTC
# Steps to reproduce
1. Create new Xamarin.Forms project in Visual Studio
2. Add UI Test Project
3. Configure and set up to run in Visual Studio here:
- Add Android reference project
- Open Test Explorer, Compile, try to run

# Expected behavior
- Automation runs on device

# Actual behavior
- Errors out with the following:
Test Failed- Message: Setup: System.Exception: ApkFile or InstalledApp has not been Configured:

Result StackTrace:	
at Xamarin.UITest.Configuration.AndroidAppConfiguration.Verify()
at Xamarin.UITest.Android.LocalAndroidAppInitializer.VerifyConfiguration()
at Xamarin.UITest.Android.AndroidApp..ctor(IAndroidAppConfiguration appConfiguration)
at Xamarin.UITest.Configuration.AndroidAppConfigurator.StartApp(AppDataMode appDataMode)
at testing.UITests.AppInitializer.StartApp(Platform platform) in \\psf\Home\Documents\Projects\testing\UITests\AppInitializer.cs:line 14
at testing.UITests.Tests.BeforeEachTest() in \\psf\Home\Documents\Projects\testing\UITests\Tests.cs:line 24
Result Message:	SetUp : System.Exception : ApkFile or InstalledApp has not been configured.

#Output results/failures

------ Run test started ------
NUnit VS Adapter executing tests is started
Loading tests from \\psf\Home\Documents\Projects\testing\UITests\bin\Debug\testing.UITests.dll
Run started: \\psf\Home\Documents\Projects\testing\UITests\bin\Debug\testing.UITests.dll
Full log file: C:\Users\James White\AppData\Local\Temp\uitest\log-2015-10-30_14-07-30-477.txt
Attempting IDE connection through pipe 8344288555b64c018e9298c27ed2352d.
Could not retrieve host endpoint from host url ipc://8344288555b64c018e9298c27ed2352d/Xamarin.UITest.IdeHost: System.NullReferenceException: Object reference not set to an instance of an object.

Server stack trace: 
   at Microsoft.VisualStudio.Extensions.GetAssemblyPath(Project vsProject) in C:\Users\builder\data\lanes\2102\5abb0de8\source\XamarinVS\src\Features\VisualStudio.TestCloud\UITest\Extensions.cs:line 20
   at Microsoft.VisualStudio.Extensions.GetAssemblyPath(IProjectNode vsProject) in C:\Users\builder\data\lanes\2102\5abb0de8\source\XamarinVS\src\Features\VisualStudio.TestCloud\UITest\Extensions.cs:line 58
   at Microsoft.VisualStudio.Extensions.<>c__DisplayClass0.<GetProjectFor>b__1(IProjectNode x) in C:\Users\builder\data\lanes\2102\5abb0de8\source\XamarinVS\src\Features\VisualStudio.TestCloud\UITest\Extensions.cs:line 31
   at Clide.Solution.ISolutionNodeExtensions.FilteringProjectsVisitor.VisitEnter(IProjectNode project)
   at Clide.Sdk.Solution.SolutionVisitable.Accept(IProjectNode project, ISolutionVisitor visitor)
   at Clide.Sdk.Solution.ProjectNode.Accept(ISolutionVisitor visitor)
   at Clide.Sdk.Solution.SolutionVisitable.Accept(ISolutionNode solution, ISolutionVisitor visitor)
   at Clide.Sdk.Solution.SolutionNode.Accept(ISolutionVisitor visitor)
   at Clide.Solution.ISolutionNodeExtensions.FindProject(ISolutionNode solution, Func`2 predicate)
   at Microsoft.VisualStudio.Extensions.GetProjectFor(ISolutionExplorer solutionExplorer, UITestSession testSession) in C:\Users\builder\data\lanes\2102\5abb0de8\source\XamarinVS\src\Features\VisualStudio.TestCloud\UITest\Extensions.cs:line 31
   at Xamarin.VisualStudio.UITest.VSIdeHost.CanHandle(UITestSession testSession) in C:\Users\builder\data\lanes\2102\5abb0de8\source\XamarinVS\src\Features\VisualStudio.TestCloud\UITest\VSIdeHost.cs:line 46
   at Xamarin.UITest.Utils.Integration.IdeHost.Xamarin.UITest.Utils.Integration.IIdeHost.GetHostEndPoint(String sessionString) in C:\Users\builder\data\lanes\2102\5abb0de8\source\XamarinVS\src\Features\VisualStudio.TestCloud\UITest\Services\IdeHost.cs:line 100
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Xamarin.UITest.Utils.Integration.IIdeHost.GetHostEndPoint(String sessionString)
   at Xamarin.UITest.Utils.Integration.IdeConnection.Connect(String testAssemblyPath, Platform platform)
Skipped IDE integration, connection failed.
NUnit VS Adapter executing tests is finished
========== Run test finished: 1 run (0:00:03.8627222) ==========

# Version Information
Microsoft Visual Studio Enterprise 2015
Version 14.0.23107.0 D14REL
Microsoft .NET Framework
Version 4.6.00079

Installed Version: Enterprise

Architecture and Modeling Tools   00322-80000-00000-AA005
Microsoft Architecture and Modeling Tools
UML® and Unified Modeling Language™ are trademarks or registered trademarks of the Object Management Group, Inc. in the United States and other countries.

Visual Basic 2015   00322-80000-00000-AA005
Microsoft Visual Basic 2015

Visual C# 2015   00322-80000-00000-AA005
Microsoft Visual C# 2015

Visual C++ 2015   00322-80000-00000-AA005
Microsoft Visual C++ 2015

Windows Phone SDK 8.0 - ENU   00322-80000-00000-AA005
Windows Phone SDK 8.0 - ENU

Application Insights Tools for Visual Studio Package   1.0
Application Insights Tools for Visual Studio

ASP.NET and Web Tools   14.0.60814.0
ASP.NET and Web Tools

ASP.NET Web Frameworks and Tools 2013   5.2.30624.0
For additional information, visit http://www.asp.net/

Common Azure Tools   1.6
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

GenerateUnitTest   1.0
Generates unit test code for methods in classes under test.

Microsoft Azure HDInsight HQL Service   2.0.2600.0
Language service for Hive query

Microsoft Azure HDInsight Tools for Visual Studio   2.0.2600.0
An integrated development environment for HDInsight application development.

Microsoft Azure Mobile Services Tools   1.4
Microsoft Azure Mobile Services Tools

Microsoft Azure Tools   2.7
Microsoft Azure Tools for Microsoft Visual Studio 2015 - v2.7.30818.1601

Microsoft Code Digger   0.9
Microsoft Code Digger

Microsoft.Pex.VisualStudio   1.0

NuGet Package Manager   3.2.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

PreEmptive Analytics Visualizer   1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

SQL Server Data Tools   14.0.50730.0
Microsoft SQL Server Data Tools

Visual Studio Tools for Universal Windows Apps   14.0.23309.00 d14oob
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.

Xamarin (5abb0de)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin.Android (f94dc5b)
Visual Studio plugin to enable development for Xamarin.Android.

Xamarin.iOS (b5396c2)
Visual Studio extension to enable development for Xamarin.iOS.

Tested with UITest 1.2 and
Comment 1 Arpit Jha 2015-11-02 06:23:58 UTC
I have checked this issue with latest Cycle 6 build  XVS and observed that its working fine.Automation Test runs on Physical device

Steps I have followed to reproduce this issue:
1.Create a CrossPlatform App in VS.
2.Add a UITest project in Solution.
3.Add reference of Android project to UITest Project.
4.Update Xamarin.UItest to 1.2.0
5.Comment Xamarin.IOS attribute in Test.cs
6. Build UITest App.
7.Run app from Test-> Windows Explorer -> Run All

Observed that Automation test runs on device

Supplement Info: 
Screencast : http://www.screencast.com/t/P4YTj8vp0IH
VS IDE log: https://gist.github.com/Abhishekk360/119e1f4e03318255c989
Output: https://gist.github.com/Arpit360/0afef30df526c1902220

Environment Info: 
Xamarin.UITest: 1.2.0

Please let me know If I am missing anything to reproduce this issue.
Comment 2 James 2015-11-02 13:29:53 UTC
I looked into this. It appears this issue is specific to SHARED projects and not PCL. As a confirmation, I tested a PCL like you did and everything ran fine. Shared projects exhibit the issue.
Comment 3 Arpit Jha 2015-11-03 03:29:27 UTC
I have checked this issue and able to reproduce this issue with the help of comment 2.

 I am getting same error mentioned in bug description.

Supplement Info:
Screencast:  http://www.screencast.com/t/KSifLWwpya
Output: https://gist.github.com/Abhishekk360/84b680c57340291dc125
VS Ide log : https://gist.github.com/Abhishekk360/aebd73c5c1bc5b074696
Environment Info: https://gist.github.com/Abhishekk360/84b6e6f1896744c513e5
Comment 4 Arpit Jha 2015-11-03 04:47:00 UTC
An update to comment 3 

REGRESSION : Not a REGRESSION we are getting same behaviour with  Xamarin.VisualStudio_3.11.1589
Comment 5 Arpit Jha 2015-11-03 05:09:15 UTC
As per comment 4, This is not a REGRESSION  

So I am changing Target Milestone to C6SR1
Comment 6 xamarin-release-manager 2015-11-05 10:22:36 UTC
Fixed in version (master)

Author: jmt
Commit: 10a1765c71d4c1875f06c0d3081ec68466988c6c (xamarin/XamarinVS)
Comment 7 xamarin-release-manager 2015-11-26 10:40:11 UTC
Fixed in version (cycle6)

Author: jmt
Commit: 10a1765c71d4c1875f06c0d3081ec68466988c6c (xamarin/XamarinVS)
Comment 8 Arpit Jha 2015-12-17 14:47:52 UTC
I have checked this issue with latest C6SR1 XVS (889e86b)  and its working fine.

Observed that Test case getting passed for Xamarin.Forms Shared project.

We are able to test on Physical device.

Screencast : http://www.screencast.com/t/e0t303iTWk
Test Output: https://gist.github.com/saurabh360/b41f9fff5c1f3b511320
Environment Info:

Hence closing this issue.