Bug 45540 - The IL that mcs generates for `async` methods apparently prevents the debugger from breaking at the top of the stack trace for unhandled exceptions in those methods
Summary: The IL that mcs generates for `async` methods apparently prevents the debugge...
Status: RESOLVED DUPLICATE of bug 51684
Alias: None
Product: Runtime
Classification: Mono
Component: Debugger ()
Version: 4.6.0 (C8)
Hardware: PC Mac OS
: --- major
Target Milestone: ---
Assignee: Brendan Zagaeski (Xamarin Team, assistant)
Depends on:
Reported: 2016-10-17 08:22 UTC by Brendan Zagaeski (Xamarin Team, assistant)
Modified: 2017-01-21 04:49 UTC (History)
3 users (show)

Tags: papercut
Is this bug a regression?: No
Last known good build:

Screencast of the "bad" behavior in Xamarin Studio master + Mono master (1.41 MB, application/x-shockwave-flash)
2016-10-21 18:57 UTC, Brendan Zagaeski (Xamarin Team, assistant)

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Brendan Zagaeski (Xamarin Team, assistant) 2016-10-17 08:22:44 UTC
Mono soft debugger does not break at top of exception stack trace for unhandled exceptions in `async` methods

## Motivation and background

The example provided below might look a bit strange because `async` methods are normally awaited and called either from event handlers or from application life cycle method overrides that allow the `async` keyword.  In fact, I had originally been using an event handler in the console sample project to more closely mimic that normal usage pattern, but then I noticed that I could replicate the same behavior even without the event handler, so I decided to stick with that simpler usage pattern for further investigation.

The original observation that sparked this investigation is that unhandled exceptions in async methods in Xamarin.iOS and Xamarin.Android apps don't break where expected in Xamarin Studio or Visual Studio.  In those "real-world" us cases, the parent async method for all of the async calls is indeed a life cycle method override or an event handler as normal.

## Partial workaround

Set the debugger to break on all thrown `System.Exception` exceptions via "Debug > Exceptions" in VS 2013, "Debug > Windows > Exception Settings" in VS 2015, or "Run > New Exception Catchpoint" in Xamarin Studio.

The Mono debugger will then break at the desired location: Program.cs, line 20.

This is only a partial workaround because the debugger will also break on the _handled_ exception in the sample code.  The ideal behavior would be for the debugger to ignore that exception in this particular case.

## Steps to replicate

1. Compile the following program, for example by running `mcs -debug Program.cs`.

using System;

namespace Program
    class MainClass
        public static void Main(string[] args)
            try {
                throw new Exception("Ignore this handled exception");
            catch {
        public static async void ThrowAnExceptionAsync()
            throw new Exception("An Exception");

2. Run the program with the Mono soft debugger attached.  For example, you can launch it via "Run > Debug Application" in Xamarin Studio, or you can use `sdb` [1] on the command line by doing something like:

### In one Terminal window

mono --debugger-agent=transport=dt_socket,server=y,address= Program.exe

### In a second Terminal window

sdb-dev "connect 55000"

[1] https://github.com/mono/sdb

## Results

The debugger breaks at `System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()` rather than at the line in the source file where the exception was thrown.

In contrast, the `StackTrace` property of the Exception stored in the local variable `$exception` at this stack frame _does_ reference the desired line of code:

>  at Program.MainClass+<ThrowAnExceptionAsync>c__async0.MoveNext () [0x00018] in /Users/macuser/Desktop/Program.cs:20 

### Example of the Call Stack from Xamarin Studio

> System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/runtime/exceptionservices/exceptionservicescommon.cs:143
> System.Runtime.CompilerServices.AsyncMethodBuilderCore.AnonymousMethod__1(System.Runtime.ExceptionServices.ExceptionDispatchInfo state) in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1034
> System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(System.Threading.QueueUserWorkItemCallback state) in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1304
> System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Threading.QueueUserWorkItemCallback state, bool preserveSyncCtx) in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:957
> System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Threading.QueueUserWorkItemCallback state, bool preserveSyncCtx) in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:904
> System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1281
> System.Threading.ThreadPoolWorkQueue.Dispatch() in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:854
> System.Threading._ThreadPoolWaitCallback.PerformWaitCallback() in /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1209

### Example results in sdb

> Trapped unhandled exception of type 'System.Exception'
> #0 [0x0000000C] System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
> at /private/tmp/source-mono-4.6.0-c8sr0/bockbuild-mono-4.6.0-branch-c8sr0
> /profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/referencesource
> /mscorlib/system/runtime/exceptionservices/exceptionservicescommon.cs:143
> (no source)
>     ldarg.0 

## Expected results (in .NET on Windows or in .NET Core in Visual Studio Code on Mac)

### Example of the Call Stack from Visual Studio

> Program.exe!Program.MainClass.ThrowAnExceptionAsync() Line 20	C#
> Program.exe!Program.MainClass.Main(string[] args) Line 14	C#

### Example of the Call Stack with .NET Core in Visual Studio Code on Mac

> Program.MainClass.ThrowAnExceptionAsync() Program.cs 20
> Program.MainClass.Main(string[] args) Program.cs 14
> [External Code] Unknown Source 0

## Additional observations with .NET on Windows

### It looks like the behavior of MDbg.exe [2] might be more similar to the Mono debugger?

[2] https://msdn.microsoft.com/en-us/library/ms229861(v=vs.110).aspx

> [p#:1, t#:no active thread] mdbg> g
> STOP: Unhandled Exception thrown
> Exception=System.Exception
>         s_EDILock=System.Object
>         _className=<null>
>         _exceptionMethod=<null>
>         _exceptionMethodString=<null>
>         _message="An Exception"
>         _data=<null>
>         _innerException=<null>
>         _helpURL=<null>
>         _stackTrace=array [384]
>         _watsonBuckets=array [5616]
>         _stackTraceString=<null>
>         _remoteStackTraceString=<null>
>         _remoteStackIndex=0
>         _dynamicMethods=<null>
>         _HResult=-2146233088
>         _source=<null>
>         _xptrs=0
>         _xcode=-532462766
>         _ipForWatsonBuckets=0
>         _safeSerializationManager=System.Runtime.Serialization.SafeSerializationManager
> This is unhandled exception, continuing will end the process
> IP: 0 @ System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_1 - MAPPING_APPROXIMATE
> [p#:1, t#:2] mdbg> w
> Thread [#:2]
> *0. System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_1 (source line information unavailable)
>  1. System.Threading.ExecutionContext.RunInternal (source line information unavailable)
>  2. System.Threading.ExecutionContext.Run (source line information unavailable)
>  3. System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem (source line informa
> tion unavailable)
>  4. System.Threading.ThreadPoolWorkQueue.Dispatch (source line information unavailable)
>     [Internal Frame, 'U-->M']

## Additional version info (brief)

Xamarin Studio (fa52f026)
Mono 4.6.1 (mono-4.6.0-branch-c8sr0/abb06f1)
sdb 1.5.6132.38284 (b8b3296)

Mac OS 10.11.6
Comment 1 Brendan Zagaeski (Xamarin Team, assistant) 2016-10-17 08:45:27 UTC
Huh!  It looks like this issue as described so far is actually due to how async methods are handled by the Mono _compiler_.

## GOOD: Program.exe compiled by `csc` on Windows, running under Xamarin Studio on Mac

Xamarin Studio successfully breaks at the desired location.  The Call Stack pad shows:

> Program.MainClass.ThrowAnExceptionAsync() in c:\Users\Windows User\Desktop\Program.cs:20
> Program.MainClass.Main(string[] args) in c:\Users\Windows User\Desktop\Program.cs:14

(I generated the `.mdb` file by running `pdb2mdb`.)

## BAD: Program.exe compiled by `mcs` on Mac, running in .NET on Windows, attached to Visual Studio

Just as in the bad cases when running under Mono, the Call Stack window shows the "bad" call stack:

> mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ThrowAsync.AnonymousMethod__6_1(object state)
> mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
> mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
> mscorlib.dll!System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
> mscorlib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()

... and the `$exception` local variable does include the desired location (or at least something closer to it):

>  at Program.MainClass.<ThrowAnExceptionAsync>c__async0.MoveNext()
Comment 2 Marek Safar 2016-10-20 12:57:27 UTC
I don't think this is mcs issue. Exactly same code works when debuged as project in XS but it fails when debugger via Debug Application.
Comment 3 Brendan Zagaeski (Xamarin Team, assistant) 2016-10-20 17:51:08 UTC
Reassigning back to the compiler just for a quick confirmation on Comment 2:

> Exactly same code works when debuged as project in XS

Just to make sure we don't miss any subtlety on this somewhat subtle bug, that is not one of the results that I reported in my testing in Comment 0 or Comment 1.  I also double-checked the behavior just now with Xamarin Studio (8937e51ff879e4062e660a70d13463510a2664c4) debugging straight from the project in XS, and I got the same results as in Comment 0.

Was the result reported in Comment 2 perhaps observed with a build of Xamarin Studio from master or similar?

Comment 4 Marek Safar 2016-10-20 20:46:40 UTC
I used XS from master 6.2 and debugging from project works as expected. I didn't try to downgrade but I believe that worked for me for some time.
Comment 5 Brendan Zagaeski (Xamarin Team, assistant) 2016-10-21 18:57:03 UTC
Created attachment 18171 [details]
Screencast of the "bad" behavior in Xamarin Studio master + Mono master

Hmm.  I can't seem to get the "good" behavior in Xamarin Studio master [1] + Mono master [2].  I get the same results with those versions as in Comment 0.

I've attached a screencast showing the steps I followed and the resulting call stack that appears in Call Stack Pad when the debugger breaks on the unhandled exception.  The call stack is all "external code": it does not mention `MainClass` at all.

> [1] monodevelop/master 1ca70ccbd00c26d6b6439b85b8b3ec11143e68d5 (2016/10/19 12:18:31 UTC)
> [2] mono/master        57d111577faf5ad1f5f69685ff551d884ae3b760 (2016/10/18 18:54:37 UTC)

Comment 6 Marek Safar 2016-11-11 17:17:35 UTC
I think I know what you tried and I can reproduce the issue. However, when trying exactly same scenario with VS I get same behaviour.

If you enable .Framework source code stepping in VS you'll break in mscorlib source code (instead of your code). If not VS shows mscorlib.pdb not loaded page (instead of your code).
Comment 7 Brendan Zagaeski (Xamarin Team, assistant) 2016-11-11 19:39:16 UTC
I will reopen this bug and assign it to myself for the moment.  My concern is that on the surface Comment 6 sounds like it's just repeating the results described in Comment 1: the application compiled with Mono shows the bad behavior on Windows.  The thing I'm worried about is that the app compiled on Windows with .NET works _as desired_ on Mac, and I'm not sure Comment 6 fully explains that.

I will experiment more with these 2 scenarios and the comment about "If you enable .Framework source code stepping" from Comment 6 to see what I can find out about whether Comment 6 does indeed explain why the .NET-compiled binary works as desired on Mac.
Comment 8 Brendan Zagaeski (Xamarin Team, assistant) 2017-01-21 04:49:50 UTC
After much head-scratching trying to figure out why I could not replicate my results from Comment 1 anymore today, my best theory is that I must have left out the `async` keyword for the Windows .exe.  I must have removed it in an earlier test to make sure I understood the "good" results on both Windows and Mac, and then (very unfortunately) I must have forgotten to add it back.

In any case, I found some better clues about this bug today and am now comfortable that there is indeed _no_ compiler issue.  I have filed a new bug to provide a clean starting point that focuses on the new clues for the Mono runtime + debugger: Bug 51684.  I am marking this bug as a duplicate accordingly.

*** This bug has been marked as a duplicate of bug 51684 ***