Bug 2752 - Code generated with -O=shared crashes when an AppDomain is unloaded
Summary: Code generated with -O=shared crashes when an AppDomain is unloaded
Alias: None
Product: Runtime
Classification: Mono
Component: JIT ()
Version: unspecified
Hardware: PC All
: --- normal
Target Milestone: ---
Assignee: Bugzilla
Depends on:
Reported: 2012-01-05 16:07 UTC by Michael Mudge
Modified: 2012-02-01 01:33 UTC (History)
5 users (show)

Is this bug a regression?: ---
Last known good build:

Windows.Forms application that starts an instance of itself in a new AppDomain. (3.12 KB, text/plain)
2012-01-09 16:29 UTC, Eric Samsel
Crashes mono with -O=shared with latest from git. (4.16 KB, text/plain)
2012-01-16 15:07 UTC, Eric Samsel
Update to previous test case using ISponsor (5.00 KB, text/plain)
2012-01-19 10:41 UTC, Eric Samsel

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Michael Mudge 2012-01-05 16:07:42 UTC
The following code crashes Mono 2.10.2, somewhere near AppDomain Unload, if Mono is started with -O=shared.

using System;

class Program
    static void Main(string[] args)
        AppDomain testDomain = AppDomain.CreateDomain("TestDomain");
        Program remoteObject = (Program)testDomain.CreateInstanceAndUnwrap(typeof(Program).Assembly.FullName, typeof(Program).FullName);



    private void SayHello()
        Console.WriteLine("Hello mom!");
Comment 1 Michael Mudge 2012-01-05 16:29:33 UTC
In one particular crash case, the cause was a failed assert mono_delegate_free_ftnptr():

ji = mono_jit_info_table_find (mono_domain_get (), mono_get_addr_from_ftnptr (ptr));
g_assert (ji);
Comment 2 Zoltan Varga 2012-01-05 17:49:52 UTC
This is already fixed in HEAD by 274c89197243eb0368102368e4b22ee4beb68241.
Comment 3 Michael Mudge 2012-01-06 09:58:21 UTC
I applied the patch from 274c89197243eb0368102368e4b22ee4beb68241, but the symptom still appears the same.  I'm not sure if that commit was before or after 2.10.8, but the bug still exists in 2.10.8.  I'm not set up to test it against HEAD in a reasonable amount of time.
Comment 4 Zoltan Varga 2012-01-06 13:26:31 UTC
I tested with the current mono 2.10 branch, which is mostly equal to 2.10.8. Before that patch, the testcase crashed. After the patch, it worked.
Comment 5 Zoltan Varga 2012-01-06 16:38:07 UTC
You might want to try 47b55a2fa87715cc58433289a91541780615a304 as well.
Comment 6 Michael Mudge 2012-01-07 17:25:45 UTC
The second patch seems to have improved something, allowing execution to get further.  I'm running a piece of code that looks like this:

static Thread foo;

public void DoSomething()
  if(foo == null) foo = Thread.CurrentThread;
  if(foo != Thread.CurrentThread)
    throw new SomethingBad();

This code executes in the root domain from the entry thread, then later executes in a new AppDomain on its own thread.  SomethingBad gets thrown from the new AppDomain's thread.

If this isn't enough info, I'll see if I can work up a small but complete program that provokes the issue.
Comment 7 Zoltan Varga 2012-01-07 21:21:09 UTC
A complete testcase would help.
Comment 8 Eric Samsel 2012-01-09 16:29:07 UTC
Created attachment 1143 [details]
Windows.Forms application that starts an instance of itself in a new AppDomain.

This program works successfully on linux using X Windows. However, when -O=shared is added to the command line, it crashes.
Comment 9 Michael Mudge 2012-01-10 11:17:17 UTC
It also crashes on our embedded device, which uses a managed implementation of Windows Forms... There are two P/Invokes in there.  It seems that something being shared by both appdomains is freed when the non-root domain is unloaded.  Several machines, including Linux and our embedded device are failing the assert in mono_delegate_free_ftnptr().
Comment 10 Zoltan Varga 2012-01-10 16:54:58 UTC
Try applying 28a571c02201d76ae17838c244d428e0518bf3ed.
Comment 11 Eric Samsel 2012-01-11 14:48:15 UTC
Applying patch 28a571c02201d76ae17838c244d428e0518bf3ed fixed the previous test case and improved the application on our device, but we're now crashing at a different spot. Here is a new test case that crashes on Linux. All 3 patches have been applied.

using System;
using System.IO;

namespace atest
    class Program
        static void Main(string[] args)
            while (true)
                AppDomainSetup setup = new AppDomainSetup();
                AppDomain scanDomain = AppDomain.CreateDomain("ScanDomain", AppDomain.CurrentDomain.Evidence, setup);

$ mono -O=shared ConsoleApp.exe
* Assertion at metadata.c:960, condition `idx < t->rows' not met


  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) object.__icall_wrapper_mono_ldtoken_wrapper (intptr,intptr,intptr) <0xffffffff>
  at TypeInfo__TypeMetadata..ctor () <0x0001f>
  at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <0xffffffff>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.MonoCMethod.InternalInvoke (System.Reflection.MonoCMethod,object,object[],System.Exception&) <0xffffffff>
  at System.Reflection.MonoCMethod.Invoke (object,System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo) <0x00204>
  at System.Reflection.MonoCMethod.Invoke (System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo) <0x0003d>
  at System.Reflection.ConstructorInfo.Invoke (object[]) <0x00053>
  at System.Activator.CreateInstance (System.Type,bool) <0x0022a>
  at System.Activator.CreateInstance (System.Type) <0x00016>
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.CreateMemberTypeMetadata (System.Type) <0x00080>
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.GetObjectData (object,System.Runtime.Serialization.Formatters.Binary.TypeMetadata&,object&) <0x00809>
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteObject (System.IO.BinaryWriter,long,object) <0x00065>
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteObjectInstance (System.IO.BinaryWriter,object,bool) <0x00192>
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteQueuedObjects (System.IO.BinaryWriter) <0x0003f>
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteObjectGraph (System.IO.BinaryWriter,object,System.Runtime.Remoting.Messaging.Header[]) <0x00070>
  at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize (System.IO.Stream,object,System.Runtime.Remoting.Messaging.Header[]) <0x002a9>
  at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize (System.IO.Stream,object) <0x00020>
  at System.Runtime.Remoting.Channels.CADSerializer.SerializeObject (object) <0x000cb>
  at System.AppDomain.GetMarshalledDomainObjRef () <0x0006d>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_object__this__ (object,intptr,intptr,intptr) <0xffffffff>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) <0xffffffff>
  at System.AppDomain.InvokeInDomain (System.AppDomain,System.Reflection.MethodInfo,object,object[]) <0x0009c>
  at System.Runtime.Remoting.RemotingServices.GetDomainProxy (System.AppDomain) <0x0004f>
  at System.AppDomain.CreateDomain (string,System.Security.Policy.Evidence,System.AppDomainSetup) <0x001b4>
  at atest.Program.Main (string[]) <0x000b0>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void_object (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

        mono() [0x496631]
        /lib/libpthread.so.0(+0xf8f0) [0x7f8d3b3278f0]
        /lib/libc.so.6(gsignal+0x35) [0x7f8d3afc8a75]
        /lib/libc.so.6(abort+0x180) [0x7f8d3afcc5c0]
        mono() [0x5dddcb]
        mono() [0x5dde77]
        mono(mono_metadata_decode_row+0xd9) [0x55ff29]
        mono(mono_class_from_typeref+0x68) [0x51e8c8]
        mono(mono_class_get_full+0x14d) [0x51d82d]
        mono() [0x5233c9]
        mono(mono_ldtoken+0xbf) [0x52446f]
        mono() [0x462432]

Debug info from gdb:

Mono support loaded.
[Thread debugging using libthread_db enabled]
[New Thread 0x7f8d39715700 (LWP 17645)]
[New Thread 0x7f8d3a759700 (LWP 17644)]
0x00007f8d3b32748d in waitpid () from /lib/libpthread.so.0
  3 Thread 0x7f8d3a759700 (LWP 17644)  0x00007f8d3b32385c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  2 Thread 0x7f8d39715700 (LWP 17645)  0x00007f8d3b325b50 in sem_wait ()
   from /lib/libpthread.so.0
* 1 Thread 0x7f8d3bdd0740 (LWP 17643)  0x00007f8d3b32748d in waitpid ()
   from /lib/libpthread.so.0

Thread 3 (Thread 0x7f8d3a759700 (LWP 17644)):
#0  0x00007f8d3b32385c in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#1  0x00000000005f3925 in GC_wait_marker () at pthread_support.c:1868
#2  0x00000000005e49a1 in GC_help_marker (my_mark_no=21) at mark.c:1116
#3  0x00000000005f2548 in GC_mark_thread (id=0x0) at pthread_support.c:552
#4  0x00007f8d3b31e9ca in start_thread () from /lib/libpthread.so.0
#5  0x00007f8d3b07b70d in clone () from /lib/libc.so.6
#6  0x0000000000000000 in ?? ()

Thread 2 (Thread 0x7f8d39715700 (LWP 17645)):
#0  0x00007f8d3b325b50 in sem_wait () from /lib/libpthread.so.0
#1  0x00000000005c8c78 in mono_sem_wait (sem=0x9154e0, alertable=1)
    at mono-semaphore.c:113
#2  0x000000000051414b in finalizer_thread (unused=<value optimized out>)
    at gc.c:1073
#3  0x0000000000528f40 in start_wrapper_internal (data=<value optimized out>)
    at threads.c:571
#4  start_wrapper (data=<value optimized out>) at threads.c:619
#5  0x00000000005ba1d3 in thread_start_routine (args=0x16db5a0)
    at wthreads.c:286
#6  0x00000000005ca499 in inner_start_thread (arg=0x17214e0)
    at mono-threads-posix.c:49
#7  0x00000000005f3592 in GC_start_routine (arg=0x7f8d3bc27fc0)
    at pthread_support.c:1473
#8  0x00007f8d3b31e9ca in start_thread () from /lib/libpthread.so.0
#9  0x00007f8d3b07b70d in clone () from /lib/libc.so.6
#10 0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7f8d3bdd0740 (LWP 17643)):
#0  0x00007f8d3b32748d in waitpid () from /lib/libpthread.so.0
#1  0x0000000000496754 in mono_handle_native_sigsegv (
    signal=<value optimized out>, ctx=<value optimized out>)
    at mini-exceptions.c:2241
#2  <signal handler called>
#3  0x00007f8d3afc8a75 in raise () from /lib/libc.so.6
#4  0x00007f8d3afcc5c0 in abort () from /lib/libc.so.6
#5  0x00000000005dddcb in monoeg_g_logv (log_domain=0x61a65c "", log_level=
    G_LOG_LEVEL_ERROR, format=<value optimized out>,
    args=<value optimized out>) at goutput.c:136
#6  0x00000000005dde77 in monoeg_assertion_message (format=
    0x6 <Address 0x6 out of bounds>) at goutput.c:156
#7  0x000000000055ff29 in mono_metadata_decode_row (t=<value optimized out>,
    idx=<value optimized out>, res=<value optimized out>,
    res_size=<value optimized out>) at metadata.c:960
#8  0x000000000051e8c8 in mono_class_from_typeref (image=0x7f8d393eb000,
    type_token=16777217) at class.c:99
#9  0x000000000051d82d in mono_class_get_full (image=0x44eb, type_token=17643,
    context=0x6) at class.c:6799
#10 0x00000000005233c9 in mono_type_get_full (image=0x44eb, type_token=17643,
    context=0x6) at class.c:6847
#11 0x000000000052446f in mono_ldtoken (image=0x7f8d393eb000, token=16777217,
    handle_class=0x7fff8a1cb0a8, context=0x0) at class.c:7909
#12 0x0000000000462432 in mono_ldtoken_wrapper (image=0x44eb, token=17643,
    context=0xffffffffffffffff) at jit-icalls.c:801
#13 0x000000004195f440 in ?? ()
#14 0x00007fff8a1cb490 in ?? ()
#15 0x0000000000000000 in ?? ()

Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.

Comment 12 Zoltan Varga 2012-01-11 15:26:12 UTC
I'm not sure -O=shared and appdomain unloading can ever correctly work together, since shared methods could depend on assemblies which are unloaded when an appdomain is unloaded.
Comment 13 Zoltan Varga 2012-01-11 15:27:50 UTC
It might be possible to restrict the set of shared methods to a safe subset, like only methods from mscorlib etc.
Comment 14 Zoltan Varga 2012-01-15 20:36:22 UTC
Can this be considered fixed ?
Comment 15 Michael Mudge 2012-01-16 13:21:39 UTC
The behavior has changed a lot as we applied patches, but the bug is still there.  Eric got the latest from git, which includes all of the patches suggested above, and is still able to provoke the issue on a Linux box.

The attached code provokes the issue, provided:
 - The AppDomain is unloaded at the end of the function that loaded it. (this needs to be added to the example code)
 - The UI is used to load an AppDomain after one has been unloaded.  It may take more than one load/unload cycle.
Comment 16 Eric Samsel 2012-01-16 15:07:33 UTC
Created attachment 1206 [details]
Crashes mono with -O=shared with latest from git.

Run the application with mono -O=shared. Click Duplicate in App Domain 0 form. Click Exit in App Domain 1 form. Click Duplicate in App Domain 0 form. It should crash at this point.
Comment 17 Zoltan Varga 2012-01-16 17:17:39 UTC
Could you try again with HEAD ?
Comment 18 Eric Samsel 2012-01-17 14:26:02 UTC
Commit 6ad31b5157c0f7fe96c83975baf0878334f81c48 fixes the application in the
most recent attachment.

This also allows us to get further in our production application.
Unfortunately, we still crash after idling a while in the child app domain
after we have loaded/unloaded application domains a few times.

It still looks delegate related, very similar to the last patch. The assertion
and stacktrace are below.

* Assertion at object.c:6051, condition `method' not met


  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) object.__icall_wrapper_mono_delegate_ctor
(object,object,intptr) <0xffffffff>
  at System.Runtime.Remoting.Lifetime.Lease.CheckNextSponsor () <0x00142>
  at System.Runtime.Remoting.Lifetime.Lease.UpdateState () <0x00130>
  at (wrapper remoting-invoke-with-check)
System.Runtime.Remoting.Lifetime.Lease.UpdateState () <0xffffffff>
  at System.Runtime.Remoting.Lifetime.LeaseManager.ManageLeases (object)
  at System.Threading.Timer/Scheduler.TimerCB (object) <0x001a7>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object
(object,intptr,intptr,intptr) <0xffffffff>

I don't have a test case yet but can probably come up with one if the problem
isn't obvious from this stacktrace.
Comment 19 Zoltan Varga 2012-01-17 21:27:04 UTC
A testcase would help.
Comment 20 Eric Samsel 2012-01-19 10:41:01 UTC
Created attachment 1233 [details]
Update to previous test case using ISponsor

Updated the previous test case to make the form be a sponsor of the object created in the child app domain. 

Run the program with mono -O=shared. Click Duplicate in the first form. After 20 seconds you should see "Renewing lease from AppDomain 0" as console output. When it's time for the next renewal, the application will crash with the assert and stacktrace I previously posted.
Comment 21 Zoltan Varga 2012-01-19 20:36:13 UTC
That should be fixed now.
Comment 22 Zoltan Varga 2012-01-28 12:01:58 UTC
Can this be considered fixed ?
Comment 23 Zoltan Varga 2012-02-01 01:33:58 UTC
Marking fixed for now.