Bug 52760 - [Cycle 9] Breakpoints non-functional and exception call stacks show user code as "External Code" for PCLs referenced by PCLs on both Android and iOS
Summary: [Cycle 9] Breakpoints non-functional and exception call stacks show user code...
Status: VERIFIED FIXED
Alias: None
Product: Visual Studio Extensions
Classification: Xamarin
Component: Debugger (show other bugs)
Version: 4.3.0 (C9)
Hardware: PC Windows
: Highest critical
Target Milestone: C9SR0
Assignee: Bugzilla
URL:
: 52751 53281 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-02-24 19:20 UTC by Brendan Zagaeski (Xamarin Team, assistant)
Modified: 2017-06-21 15:52 UTC (History)
20 users (show)

Tags: Cycle9R
Is this bug a regression?: Yes
Last known good build: Cycle 9 RC (XamarinVS 4.3.0.738)


Attachments
Test case (46.26 KB, application/zip)
2017-02-24 19:20 UTC, Brendan Zagaeski (Xamarin Team, assistant)
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
VERIFIED FIXED

Description Brendan Zagaeski (Xamarin Team, assistant) 2017-02-24 19:20:01 UTC
Created attachment 19953 [details]
Test case

Breakpoints non-functional (debug symbols not loaded?) for PCLs referenced by PCLs, possibly Android specific




## Steps to replicate

1. Open the attached test case in Visual Studio.

2. Set a breakpoint on PortableClassLibrary2\Class1.cs line 14 (or one of the other lines).

3. Run the app in the debug configuration. 




## BAD Results with Cycle 9

The app loads, displays the "Hello World" button on the device, and shows "z: 22" in the debug output, but it never pauses on the breakpoint.




## Expected results

The debugger should pause on the breakpoint.




## Testing environment info (brief)



### Testing device

LG Optimus L9, Android 4.1.2 (API 16)



### Windows computer

XamarinVS 4.3.0.784 (73f58d6)

Microsoft Visual Studio Enterprise 2015
Version 14.0.25431.01 Update 3
Microsoft .NET Framework
Version 4.6.01586

Windows 10 (64-bit) Version 1607 (OS Build 14393.321)
US English locale, US eastern time zone
Comment 1 Brendan Zagaeski (Xamarin Team, assistant) 2017-02-24 19:35:27 UTC
## Bookkeeping note

I will assign this bug to myself for a moment for a little bit of additional testing to determine whether the issue:

- Is a regression?
- Is specific to Android?
- Only happens with "second-order" PCL references?
- Only happens in Visual Studio?
- Also affects some other Android device?
Comment 2 Brendan Zagaeski (Xamarin Team, assistant) 2017-02-24 20:14:23 UTC
## Regression status

BAD:  XamarinVS 4.3.0.784 (73f58d6)
GOOD: XamarinVS 4.2.2.11  (00fa5cc)

The breakpoint in the second-order PCL causes the debugger to break successfully in XamarinVS 4.2.2.11.




## Is specific to Android?

No, the issue also affects iOS.

I added a new "Single View App (iPhone)" to the project, added the reference to PortableClassLibrary, and added the call to the constructor in the `ViewDidLoad()` override.

As on Android, the debug output successfully shows "z: 22", but the debugger never pauses on the breakpoint.




## Only happens with "second-order" PCL references?

Yes, it only happens with second-order PCL references.

I changed the reference in each of the app projects to reference the "second order" PCL directly, manually deleted all the `bin` and `obj` folders, rebuilt, and then the debugger broke successfully on the breakpoint (and the debug output showed the expected "Resolved pending breakpoint" message).




## Only happens in Visual Studio?

Yes, it only happens in Visual Studio.

I tested the same test case with the matching Cycle 9 versions of Xamarin Studio, Xamarin.Android, Xamarin.iOS, and Mono on Mac.  The debugger paused successfully on the breakpoints there (for both Android and iOS).
Comment 3 Brendan Zagaeski (Xamarin Team, assistant) 2017-02-24 20:17:27 UTC
*** Bug 52751 has been marked as a duplicate of this bug. ***
Comment 4 Brendan Zagaeski (Xamarin Team, assistant) 2017-02-24 20:38:50 UTC
Additional result: The issue also affects symbol resolution for exceptions thrown within the second-order PCL, so this seems to be a general issue with the debug symbols not being loaded as expected for that second-order library.
Comment 5 Brendan Zagaeski (Xamarin Team, assistant) 2017-02-24 20:42:12 UTC
## Area of impact



### Has the issue already taken up time for many users?  For example are many users already CC'd on this bug report?

Not yet.  So far there has been one report of this from the reporter of non-public Bug 52751 (marked as a duplicate of this bug).



### If not, based on the available information so far, is it likely the issue will be reported by many users in the near future?

Yes, very much so.  Incorrect debugger symbol behavior in second-order references has come up before, and it was a highly visible problem in those earlier instances.



### For users who encounter this issue, does it make proceeding with development (a) difficult, impossible, or potentially hazardous, (b) moderately inconvenient, or (c) mildly inconvenient?

(b) It is _possible_ to continue development, but proper breakpoint (and other debugger symbol) functionality is one of the central features of the IDE development experience.  Working without that functionality is a significant inconvenience.
Comment 6 Björn Bentmar 2017-02-25 12:28:50 UTC
Thanks for the work on the bug Brendan, this is a regression. Before i updated to stable i was 2 or 3 beta releases behind, and it worked on that version.
Comment 7 Robby Groot 2017-03-01 12:45:43 UTC
I want to report that I'm facing the mentioned issue in my company's project. It makes it a lot more difficult to debug than necessery.

The project structure is as follows:

- Core project (PCL)
- Android project  (Relies on Core)
  - 1 Binding Library
- iOS Project  (Relies on Core)
  - 1 Binding library

The only thing the call stack lists is [External Code].

I can confirm this definitely happens with the Android project, but I have not faced or tested this issue on iOS as of now.
Comment 8 Björn Bentmar 2017-03-01 13:12:51 UTC
After using this a few days im having issues to breakpoint in any of the standard projects aswell... Its very rare to have a breakpoint hit.
Comment 9 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-01 18:37:57 UTC
Follow-up checks thanks to the hints about regression status in Comment 6.




## Regression status (more precise)

Using the test scenario in Comment 0, only the latest version of the Cycle 9 shows this problem in my test environment.  Neither of the preceding 2 Beta builds shows this issue on either iOS or Android.

GOOD: XamarinVS 4.3.0.738 (9473a85)
GOOD: XamarinVS 4.3.0.636 (15692db)




## Optional downgrade links

If any users would like to use one of these older RC builds temporarily, here are the download links:

https://dl.xamarin.com/XamarinforVisualStudio/Windows/Xamarin.VisualStudio_4.3.0.738.msi
https://dl.xamarin.com/XamarinforVisualStudio/Windows/Xamarin.VisualStudio_4.3.0.636.msi
Comment 10 Björn Bentmar 2017-03-01 22:11:35 UTC
perfect thank you!
Comment 12 Daniel Cazzulino 2017-03-03 15:57:15 UTC
We have tracked the root cause of this regression to a fix that was put in place to enable multi-targeting class libraries (SDK-style, under VS2017) project references to work.

The linked versions with the prior bits will not work with .NETStandard class libraries as project references if they multi-target more than one target framework and generate portable PDBs.

Working on a solution that fits both scenarios right now.

Thanks for reporting!
Comment 13 Daniel Cazzulino 2017-03-03 17:13:16 UTC
Ok, we have a fix under way.

The workaround to unblock users is to open the [MSBUILD]Microsoft.Common.Targets\ImportAfter\Xamarin.Common.targets file and modify the DependsOnTargets for the GetBuiltProjectOutputRecursive target in line 10 as follows:

from: 

DependsOnTargets="AssignProjectConfiguration;AllProjectOutputGroups"

to:

DependsOnTargets="AssignProjectConfiguration;_SplitProjectReferencesByFileExistence;AllProjectOutputGroups"



[MSBuild] would be different depending on the VS version:

- VS2017: C:\Program Files (x86)\Microsoft Visual Studio\2017\[EDITION]\MSBuild\15.0\
- VS2015: C:\Program Files (x86)\MSBuild\14.0\
- VS2013: C:\Program Files (x86)\MSBuild\12.0\


The root cause was that the new %(SetTargetFramework) item metadata in VS2017 for multi-targeting projects is added to the @(_MSBuildProjectReferenceExistent) items, rather than the @(ProjectReferenceWithConfiguration) we were using previously. To get the new metadata, we switched the item group being used, but missed adding the dependent target so that it would be populated before our target runs. For first-level project references, by the time our target was run, the dependent target we missed had already run, so the problem didn't manifest with first-level PCL dependencies. It does, as noted in this bug, break for second-level.

By adding the _SplitProjectReferencesByFileExistence target as a dependency, we force it to run before we do, therefore ensuring the @(_MSBuildProjectReferenceExistent) item group is always populated regardless of the level of the project reference.
Comment 14 Daniel Cazzulino 2017-03-03 17:40:54 UTC
If you have the file on this path on your machine too, you should also update it for the workaround to work on VS2017:

- VS2017: C:\Program Files (x86)\MSBuild\15.0\
Comment 15 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-03 18:24:04 UTC
## Extra test of Comment 13 using the original testing environment from Comment 0

1. Manually apply the candidate change from Comment 13 in the .targets file.

2. Quit Visual Studio.

3. Manually delete all of the bin\ and obj\ folders in the test solution.

4. Build and run the app.




## Results: Good

The breakpoint hits successfully on both Android and iOS.
Comment 16 Miguel de Icaza [MSFT] 2017-03-05 16:10:07 UTC
Do we have this on track for an SR0 relase?
Comment 17 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-05 21:06:58 UTC
I'm not sure if Comment 16 was intended more for the XamarinVS engineers, or perhaps for anyone on the Xamarin team.  In case it might at least partially answer the question, this bug _is_ included in the "List of Cycle9-tagged bugs sorted by number of non-Xamarin users on CC" that I emailed out to the team.  It is accordingly also included on the Cycle 9 SR 0 tracking board.
Comment 18 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-05 22:35:04 UTC
In case it might also be helpful to have a bit more of the engineers' flavor on the tracking, I think I can fill in a bit of that for them too.

The candidate change described in Comment 13 is set up to be merged for inclusion in Cycle 9 SR 0 [1], but it has not been merged quite yet.

[1] Non-public XamarinVS/cycle9 pull request #7036

(Also for the other appropriate branches:
Non-public XamarinVS/d15-1 pull request #7035
Non-public XamarinVS/master pull request #7034)
Comment 19 Daniel Cazzulino 2017-03-08 11:04:06 UTC
Fixed in matster and d15.1
Comment 20 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-08 16:37:42 UTC
## Additional status details for users watching this bug

Comment 19 is describing that the pull requests mentioned in Comment 18 have now been merged into the source the development branches.  New builds that include those changes will be published to the updater channels in the near future.  (The latest Alpha version Xamarin 4.3.1.2 (486e68a) from yesterday does not include the change quite yet.)
Comment 22 Jose Gallardo 2017-03-13 15:53:30 UTC
*** Bug 53281 has been marked as a duplicate of this bug. ***
Comment 24 Mike 2017-03-21 17:34:08 UTC
Any idea when the VS2017 installer (Stable or Preview) will receive this fix? This bug fix is critical for me right now.
Comment 25 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-21 17:40:50 UTC
> Any idea when the VS2017 installer (Stable or Preview) will receive this
> fix?

Tentatively within the next week or so.  Note that you can manually apply the fix by following Comment 13 and Comment 14 in the mean time if desired.
Comment 26 Mike 2017-03-22 08:23:17 UTC
I already tried that but unfortunately it didn't work. There's also a slight difference in my case, I'm using class libraries targeting .NET Standard instead of PCL's. I'm not sure if that'll make a difference using the fix you've mentioned.
Comment 27 Mike 2017-03-22 10:12:47 UTC
I've done some more troubleshooting. At first I only built it for Android, but didn't try building it to iOS. When I tried building it to my Mac, which has Visual Studio for Mac Preview installed, my debugger seems to work perfectly in Visual Studio 2017. (With the fix mentioned at Comment 13 and Comment 14 still in place, I didn't test it just yet without it.)

Not the ideal situation since building on my Mac and deploying it to the iOS simulator takes longer than building on my PC and deploying it to my Android Emulator, but I can live with that in the meantime.

Am I correct that this is because the newest Visual Studio for Mac Preview has a newer version of Xamarin which fixes this issue?
Comment 28 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-22 18:24:53 UTC
For Comment 26 and Comment 27, I double-checked the workaround on the test case from Comment 0 in Visual Studio 2017, and it worked as desired (results below).  If the original test case from Comment 0 behaves correctly for you, but your full project still shows an issue even after manually deleting the bin\ and obj\ folders, then please file a new bug report (optionally marked private), including as many details about your project as possible (ideally a minimized test case that demonstrates the issue) [1].  Thanks!

[1] https://developer.xamarin.com/guides/cross-platform/troubleshooting/questions/howto-file-bug/




## Test of Comment 13 using Visual Studio 2017 on the original environment from Comment 0



### First, I confirmed the bug using the steps to replicate from Comment 0: Confirmed

As expected, Visual Studio 2017 skips over the breakpoint when the change for the .targets file is not yet in place.



### Apply the workaround

1. Manually apply the change from Comment 13 in:

> C:\Program Files (x86)\MSBuild\15.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Microsoft.Common.Targets\ImportAfter\Xamarin.Common.targets
2. Quit Visual Studio.

3. Manually delete all of the bin\ and obj\ folders in the test solution.

4. Build and run the app.



### New Results: Good

The breakpoint hits successfully on both Android and iOS.



### Testing environment info (brief)

#### Testing device

LG Optimus L9, Android 4.1.2 (API 16)

#### Windows computer

XamarinVS 4.3.0.784 (73f58d6)

Microsoft Visual Studio Enterprise 2017
Version 15.0.26304.0 D15REL
Microsoft .NET Framework
Version 4.6.01586

Windows 10 (64-bit) Version 1607 (OS Build 14393.321)
US English locale, US eastern time zone