Bug 56246 - Tracking bug for upstream Roslyn issue 17934: "error MSB6006: "csc.exe" exited with code 1" due to KeyNotFoundException during `Microsoft.Cci.FullMetadataWriter.GetMethodDefinitionHandle()` when compiling projects that include `async partial` methods
Summary: Tracking bug for upstream Roslyn issue 17934: "error MSB6006: "csc.exe" exite...
Alias: None
Product: Compilers
Classification: Mono
Component: C# (show other bugs)
Version: 5.0.0 (2017-02)
Hardware: PC Mac OS
: --- critical
Target Milestone: 15.3
Assignee: Bugzilla
: 55736 56181 56214 56217 56226 56236 56451 56583 (view as bug list)
Depends on:
Blocks: 55283
  Show dependency tree
Reported: 2017-05-11 20:06 UTC by Brendan Zagaeski (Xamarin Team, assistant)
Modified: 2017-08-02 11:11 UTC (History)
22 users (show)

Tags: 15.2KI xamexttriage
Is this bug a regression?: ---
Last known good build:

Attached IDE logs (11.41 KB, application/zip)
2017-08-02 11:11 UTC, Aman Dharwal

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Brendan Zagaeski (Xamarin Team, assistant) 2017-05-11 20:06:58 UTC
Tracking bug for upstream Roslyn issue 17934: "error MSB6006: "csc.exe" exited with code 1" due to KeyNotFoundException during `Microsoft.Cci.FullMetadataWriter.GetMethodDefinitionHandle()` when compiling projects that include `async partial` methods

This is a public tracking bug to track the inclusion of the fix for https://github.com/dotnet/roslyn/issues/17934 into the Mono framework for use with Xamarin.

## Partial temporary workaround

Add the following `PropertyGroup` element to the bottom of the .csproj file for your .app project just before the closing `</Project>` tag:


NOTE: This switches the compiler back to using the old default `mcs` compiler, and it will therefore generate the old style .mdb debugging symbol files rather than the new portable .pdb debugging symbol files.  The old style .mdb debugging symbols might not behave 100% as expected with the Xamarin 15.2 release.

## Regression status: new feature bug in the newly introduced `csc` executable

> BAD: Mono (2017-02/9667aa6)           ("15.2")
> BAD: Mono 4.8.1     (mono-4.8.0-branch/22a39d7) ("15.1" + security update)

Note that Mono 4.8.1 was set to use `mcs` by default, so users would be unlikely to hit the issue.  But Mono 4.8.1 did include the problematic `csc.exe` Rosyln executable [1].

[1] /Library/Frameworks/Mono.framework/Versions/4.8.1/lib/mono/msbuild/15.0/bin/Roslyn/csc.exe

## Steps followed to test

1. Download the "AzureTodoMac" sample from https://github.com/xamarin/mac-samples.

2. Open the projet in Visual Studio for Mac to restore the NuGet packages.

3. Attempt to build the project on the command line:

msbuild /v:diagnostic /t:Build /p:Configuration=Debug AzureTodoMac.sln

### Alternate steps to test

1. Copy the following code into test.cs:

public partial class TestClass {
    partial void X ();
    async partial void X () {}

2. Attempt to compile the code with `/debug:portable`:

csc test.cs /t:library /debug:portable

## BAD Results with Mono 5.0 (2017-02/9667aa6), when using the new Roslyn `csc.exe` compiler

The build fails due to an error in `csc.exe`.

### Error from the end of the build output

/Library/Frameworks/Mono.framework/Versions/5.0.0/lib/mono/msbuild/15.0/bin/Roslyn/Microsoft.CSharp.Core.targets(71,5): error MSB6006: "csc.exe" exited with code 1.

### Top of the exception stack trace

>   System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary. (TaskId:31)
>     at System.Collections.Generic.Dictionary`2[TKey,TValue].get_Item (TKey key) [0x0001e] in <b697cffb61b74023aa9c587e6c49beb3>:0  (TaskId:31)
>     at Microsoft.Cci.FullMetadataWriter+DefinitionIndex`1[T].get_Item (T item) [0x00000] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.Cci.FullMetadataWriter.GetMethodDefinitionHandle (Microsoft.Cci.IMethodDefinition def) [0x00007] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.Cci.MetadataWriter.SerializeMethodDebugInfo (Microsoft.Cci.IMethodBody bodyOpt, System.Int32 methodRid, System.Reflection.Metadata.StandaloneSignatureHandle localSignatureHandleOpt, System.Reflection.Metadata.LocalVariableHandle& lastLocalVariableHandle, System.Reflection.Metadata.LocalConstantHandle& lastLocalConstantHandle) [0x0024d] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.Cci.MetadataWriter.SerializeMethodBodies (System.Reflection.Metadata.BlobBuilder ilBuilder, Microsoft.Cci.PdbWriter nativePdbWriterOpt, System.Reflection.Metadata.Blob& mvidStringFixup) [0x000d7] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.Cci.MetadataWriter.BuildMetadataAndIL (Microsoft.Cci.PdbWriter nativePdbWriterOpt, System.Reflection.Metadata.BlobBuilder ilBuilder, System.Reflection.Metadata.BlobBuilder mappedFieldDataBuilder, System.Reflection.Metadata.BlobBuilder managedResourceDataBuilder, System.Reflection.Metadata.Blob& mvidFixup, System.Reflection.Metadata.Blob& mvidStringFixup) [0x00032] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.Cci.PeWriter.WritePeToStream (Microsoft.CodeAnalysis.Emit.EmitContext context, Microsoft.CodeAnalysis.CommonMessageProvider messageProvider, System.Func`1[TResult] getPeStream, System.Func`1[TResult] getPortablePdbStreamOpt, Microsoft.Cci.PdbWriter nativePdbWriterOpt, System.String pdbPathOpt, System.Boolean allowMissingMethodBodies, System.Boolean isDeterministic, System.Threading.CancellationToken cancellationToken) [0x0004c] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.CodeAnalysis.Compilation.SerializeToPeStream (Microsoft.CodeAnalysis.Emit.CommonPEModuleBuilder moduleBeingBuilt, Microsoft.CodeAnalysis.Compilation+EmitStreamProvider peStreamProvider, Microsoft.CodeAnalysis.Compilation+EmitStreamProvider pdbStreamProvider, System.Func`1[TResult] testSymWriterFactory, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, System.Boolean metadataOnly, System.Threading.CancellationToken cancellationToken) [0x00120] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.CodeAnalysis.CommonCompiler.RunCore (System.IO.TextWriter consoleOutput, Microsoft.CodeAnalysis.ErrorLogger errorLogger, System.Threading.CancellationToken cancellationToken) [0x00475] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)
>     at Microsoft.CodeAnalysis.CommonCompiler.Run (System.IO.TextWriter consoleOutput, System.Threading.CancellationToken cancellationToken) [0x00035] in <12d27e16f5684917a6cbffa8f9cc6eae>:0  (TaskId:31)

## Testing environment info (brief)

Mono (2017-02/9667aa6) (64-bit)

Xcode 8.3 (12169), Build 8E162

macOS 10.12.4
US English locale, US Eastern time zone
Comment 1 Brendan Zagaeski (Xamarin Team, assistant) 2017-05-11 20:09:24 UTC
*** Bug 56181 has been marked as a duplicate of this bug. ***
Comment 2 Brendan Zagaeski (Xamarin Team, assistant) 2017-05-11 20:13:48 UTC
*** Bug 56226 has been marked as a duplicate of this bug. ***
Comment 3 Brendan Zagaeski (Xamarin Team, assistant) 2017-05-11 20:14:29 UTC
*** Bug 56236 has been marked as a duplicate of this bug. ***
Comment 4 Brendan Zagaeski (Xamarin Team, assistant) 2017-05-11 20:24:17 UTC
## Additional verification testing: upstream Roslyn fix _not_ yet included in today's Alpha build of Mono 5.2

> BAD: Mono (2017-04/4a0006f) includes CSC version
> BAD: Mono (2017-02/9667aa6) includes CSC version
Comment 7 Neal 2017-05-12 12:31:38 UTC
This fix works when building on a Mac but our CI (Bamboo) which uses MSBuild on Windows is now broken. What can we do?

"D:\Builds\APDL-AID-JOB1\apdl-ios\APDL.iOS\APDL.iOS.csproj" (Rebuild target) (8:4) ->
C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.CSharp.Core.targets(67,5): error MSB6004: The specified task executable location "C:\Program Files (x86)\MSBuild\14.0\bin\mcs.exe" is invalid. [D:\Builds\APDL-AID-JOB1\apdl-ios\APDL.iOS\APDL.iOS.csproj]
Comment 8 Neal 2017-05-12 12:53:21 UTC
I switched the workaround to this and it works for me now:

<PropertyGroup Condition=" '$(OS)' == 'Unix' ">
Comment 9 Brendan Zagaeski (Xamarin Team, assistant) 2017-05-15 18:39:04 UTC
*** Bug 55736 has been marked as a duplicate of this bug. ***
Comment 10 Brendan Zagaeski (Xamarin Team, assistant) 2017-05-15 20:22:18 UTC
*** Bug 56451 has been marked as a duplicate of this bug. ***
Comment 11 Brendan Zagaeski (Xamarin Team, assistant) 2017-05-15 20:38:45 UTC
## Additional information about the workaround

> The old style .mdb debugging symbols might not behave 100% as expected
> with the Xamarin 15.2 release.

For users attempting the workaround on Mac, it is also recommended to _disable_ "Visual Studio > Preferences > Build with MSBuild instead of xbuild" and then close and restart Visual Studio for Mac.

(The new `msbuild` default in Visual Studio for Mac does not copy the .mdb files for user assemblies into the "bin" folder or application package as expected, so the debugger will not be able to hit breakpoints or break at the expected source locations for exceptions.  Switching to `xbuild` allows the build process to copy the .mdb files as expected.)
Comment 12 Rolf Bjarne Kvinge [MSFT] 2017-05-16 09:07:51 UTC
Alternative workaround: disable debug symbols (see https://github.com/xamarin/ios-samples/commit/794ee89456df0cb490bed4312d96afa572db4546 for an example). This will obviously prevent debugging from working as well.
Comment 13 Rolf Bjarne Kvinge [MSFT] 2017-05-18 11:00:27 UTC
*** Bug 56583 has been marked as a duplicate of this bug. ***
Comment 14 Marek Safar 2017-05-26 16:37:55 UTC
*** Bug 56214 has been marked as a duplicate of this bug. ***
Comment 15 Marius Ungureanu 2017-05-30 02:53:53 UTC
*** Bug 56217 has been marked as a duplicate of this bug. ***
Comment 16 Brendan Zagaeski (Xamarin Team, assistant) 2017-06-08 02:55:34 UTC
## Alternate possible temporary workaround

Install pre-release version 2.3.0-beta1 or higher of the "Microsoft.Net.Compilers" NuGet package [1] into the affected projects (or simply all of the projects in the solution).  This NuGet package provides a new version of the Roslyn C# compiler that includes the fix for the upstream bug.

Note that this approach might not be effective for all scenarios, but it did resolve the issue for several test scenarios that I tried, including the original AzureTodoMac scenario from Comment 0.

[1] https://www.nuget.org/packages/Microsoft.Net.Compilers/
Comment 17 John Cachat 2017-06-19 15:53:56 UTC
I can confirm that adding a reference to the pre-release "Microsoft.Net.Compilers" package worked for me.  Currently, the latest is 2.3.0-beta2.
Comment 18 Marek Safar 2017-06-20 18:16:09 UTC
It has been fixed few weeks ago in 15.3
Comment 19 Neal 2017-06-20 18:20:33 UTC
Your use of these 15.x versions is beyond confusing. They don't correspond to any version observed in VS for Mac for example. So I have no idea by looking at what's on my Mac as to whether I have a fix or not based on your version cycles.
Comment 20 Brendan Zagaeski (Xamarin Team, assistant) 2017-06-20 18:36:38 UTC
> It has been fixed few weeks ago in 15.3
To be more precise, the Roslyn dependency in the Mono 2017-04 branch (https://github.com/mono/mono/tree/2017-04) has now been updated to a version that includes the upstream fix for the Roslyn issue (https://github.com/dotnet/roslyn/issues/17934).

This bug is currently marked as Resolved Fixed against the 15.3 target milestone for verification testing by the Xamarin team against that target milestone.

> these 15.x versions
Note that the verification testing will by default include specific build numbers for the product versions that were used during verification testing.

See also the following 2 pages to track information about the correspondence of 15.x target milestones to product version numbers:
Comment 24 Aman Dharwal 2017-08-02 11:11:19 UTC
Created attachment 23970 [details]
Attached IDE logs

This issue does exist on d15-4

## Verified on Environment : 


## Command line Build logs


## Screencast link :