Bug 15695

Summary: sgen hangs when creating threads with pthread_create
Product: [Mono] Runtime Reporter: Marian Salaj <salajm>
Component: GCAssignee: Bugzilla <bugzilla>
Status: RESOLVED FIXED    
Severity: normal CC: kumpera, mark, mono-bugs+mono, mono-bugs+runtime
Priority: ---    
Version: 3.2.x   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Tags: Is this bug a regression?: ---
Last known good build:

Description Marian Salaj 2013-10-25 09:56:10 UTC
When threads are created with function pthread_create and managed code is executed on them sgen will hang when trying to stop the world. I have tested this on git head version and 3.2.3, I have used following test code:

[DllImport("libpthread.so.0", SetLastError = false, EntryPoint = "pthread_create", CallingConvention = CallingConvention.Cdecl)]

private static extern int pthread_create(out ulong thread, IntPtr attr, [MarshalAs(UnmanagedType.FunctionPtr)] Delegate d, IntPtr arg);

static void TestThread()
{
  Console.WriteLine("Hello!");
  GC.Collect();
}

static void Main(string[] args)
{
   ulong thread_t = 0;

   while(true)
   {
      int ret = pthread_create(out thread_t, IntPtr.Zero, new ThreadStart(TestThread), IntPtr.Zero);
      GC.Collect();
   }
}
Comment 1 Marian Salaj 2013-10-28 12:26:10 UTC
I have debugged why the sgen deadlocks. Version 3.2.3 deadlocks for another reason then git head version and I think this commit should fix it:
https://github.com/mono/mono/commit/a1d20a2d53dbc15f5f334ba5889382ecd223b0bf

But this commit also introduces another bug, where two threads deadlocks:

thread 1 is performing garbage collection:

1. mono_gc_collect -> LOCK_GC   <- gc lock is acquired
2. sgen_stop_world -> acquire_gc_locks -> mono_thread_info_suspend_lock <- it waits here for lock which is hold by thread 2


thread 2 is terminating:

1. unregister_thread -> mono_thread_info_suspend_lock 	<- lock is acquired
2. unregister_thread -> threads_callbacks.thread_unregister -> sgen_thread_unregister -> mono_thread_detach -> thread_cleanup -> mono_free_static_data -> mono_gc_free_fixed -> mono_gc_deregister_root -> LOCK_GC    <- it waits here for gc lock which is hold by thread 1
Comment 2 Marian Salaj 2013-11-06 05:30:02 UTC
I see that the sgen deadlock has been fixed by commit cf8c83129debd5a5f34ec9c1a821a31c6de7866c, but the test code is still not working, now it is crashing on assertion:
* Assertion at mono-threads.c:242, condition `info' not met

[Thread debugging using libthread_db enabled]
[New Thread 0x7febd5f63700 (LWP 18511)]
[New Thread 0x7febd6764700 (LWP 18510)]
[New Thread 0x7febd6f69700 (LWP 18509)]
[New Thread 0x7febd776a700 (LWP 18507)]
[New Thread 0x7febd7f6b700 (LWP 18506)]
[New Thread 0x7fec05b49700 (LWP 18420)]
0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
  Id   Target Id         Frame 
  7    Thread 0x7fec05b49700 (LWP 18420) "mono" 0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
  6    Thread 0x7febd7f6b700 (LWP 18506) "mono" 0x00007fec08584bdd in waitpid () from /lib/libpthread.so.0
  5    Thread 0x7febd776a700 (LWP 18507) "mono" 0x00007fec085833c0 in sem_wait () from /lib/libpthread.so.0
  4    Thread 0x7febd6f69700 (LWP 18509) "mono" 0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
  3    Thread 0x7febd6764700 (LWP 18510) "mono" 0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
  2    Thread 0x7febd5f63700 (LWP 18511) "mono" 0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
* 1    Thread 0x7fec0902f780 (LWP 18419) "mono" 0x00007fec08246564 in sigsuspend () from /lib/libc.so.6

Thread 7 (Thread 0x7fec05b49700 (LWP 18420)):
#0  0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005cbbc4 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7fec05b48900) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007fec085833be in sem_wait () from /lib/libpthread.so.0
#5  0x0000000000628658 in mono_sem_wait (sem=0x962780, alertable=1) at mono-semaphore.c:119
#6  0x00000000005a6085 in finalizer_thread (unused=<optimized out>) at gc.c:1073
#7  0x0000000000582f17 in start_wrapper_internal (data=<optimized out>) at threads.c:609
#8  start_wrapper (data=0x1217a40) at threads.c:654
#9  0x000000000061c573 in thread_start_routine (args=0x11c6730) at wthreads.c:294
#10 0x000000000062d4f0 in inner_start_thread (arg=0x1217e20) at mono-threads-posix.c:49
#11 0x00007fec0857c8ca in start_thread () from /lib/libpthread.so.0
#12 0x00007fec082e3b6d in clone () from /lib/libc.so.6
#13 0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7febd7f6b700 (LWP 18506)):
#0  0x00007fec08584bdd in waitpid () from /lib/libpthread.so.0
#1  0x00000000004a57d7 in mono_handle_native_sigsegv (signal=<optimized out>, ctx=<optimized out>) at mini-exceptions.c:2361
#2  0x0000000000417e9f in mono_sigsegv_signal_handler (_dummy=11, info=0x7fec08e52f70, context=0x7fec08e52e40) at mini.c:6671
#3  <signal handler called>
#4  get_hazardous_pointer (pp=0x158, hp=0x7fec07987240, hazard_index=0) at hazard-pointer.c:180
#5  0x00000000005a3f07 in mono_jit_info_table_find_internal (domain=0x0, addr=0x7fec082461b5 "H=", try_aot=1) at domain.c:397
#6  0x0000000000507955 in sigabrt_signal_handler (_dummy=6, info=0x7febd7f6a830, context=0x7febd7f6a700) at mini-posix.c:202
#7  <signal handler called>
#8  0x00007fec082461b5 in raise () from /lib/libc.so.6
#9  0x00007fec08248fc0 in abort () from /lib/libc.so.6
#10 0x000000000063186d in monoeg_g_logv (log_domain=0x64e27c "", log_level=G_LOG_LEVEL_ERROR, format=<optimized out>, args=<optimized out>) at goutput.c:175
#11 0x0000000000631917 in monoeg_assertion_message (format=0x6 <Address 0x6 out of bounds>) at goutput.c:195
#12 0x000000000062cd18 in mono_thread_info_current () at mono-threads.c:242
#13 0x00000000005cc739 in mono_gc_set_current_thread_appdomain (domain=0x47f3) at sgen-gc.c:1728
#14 0x0000000000580ccd in thread_cleanup (thread=0x7fec0797fc90) at threads.c:412
#15 0x0000000000583062 in mono_thread_detach (thread=<optimized out>) at threads.c:1063
#16 0x000000000062c850 in unregister_thread (arg=<optimized out>) at mono-threads.c:187
#17 0x00007fec0857bee9 in __nptl_deallocate_tsd () from /lib/libpthread.so.0
#18 0x00007fec0857c8d8 in start_thread () from /lib/libpthread.so.0
#19 0x00007fec082e3b6d in clone () from /lib/libc.so.6
#20 0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7febd776a700 (LWP 18507)):
#0  0x00007fec085833c0 in sem_wait () from /lib/libpthread.so.0
#1  0x0000000000628658 in mono_sem_wait (sem=0x962be0, alertable=0) at mono-semaphore.c:119
#2  0x00000000005cbcf2 in sgen_wait_for_suspend_ack (count=6) at sgen-os-posix.c:165
#3  0x00000000005cbe08 in sgen_thread_handshake (suspend=<optimized out>) at sgen-os-posix.c:199
#4  0x00000000005f9a33 in sgen_stop_world (generation=1) at sgen-stw.c:219
#5  0x00000000005d5ac8 in sgen_perform_collection (requested_size=0, generation_to_collect=1, reason=0x701781 "user request", wait_to_finish=1) at sgen-gc.c:3444
#6  0x00000000005d5fe8 in mono_gc_collect (generation=1) at sgen-gc.c:4604
#7  0x000000004004444a in ?? ()
#8  0x000000000125e960 in ?? ()
#9  0x00007fec05c88ff0 in System.IO.SynchronizedWriter:WriteLine (this=Cannot access memory at address 0xffffffffffffffd0
) at <unknown>:668
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 4 (Thread 0x7febd6f69700 (LWP 18509)):
#0  0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005cbbc4 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7febd6f688c0) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007fec08583df5 in __lll_unlock_wake () from /lib/libpthread.so.0
#5  0x00007fec085805d4 in _L_unlock_548 () from /lib/libpthread.so.0
#6  0x00007fec08580504 in __pthread_mutex_unlock_usercnt () from /lib/libpthread.so.0
#7  0x000000004004444a in ?? ()
#8  0x0000000001226b20 in ?? ()
#9  0x00007fec05c88ff0 in System.IO.SynchronizedWriter:WriteLine (this=Cannot access memory at address 0xffffffffffffffd0
) at <unknown>:668
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 3 (Thread 0x7febd6764700 (LWP 18510)):
#0  0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005cbbc4 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7febd67638c0) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007fec08583c72 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007fec0857f179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007fec0857ef9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005d5fcd in mono_gc_collect (generation=1) at sgen-gc.c:4601
#8  0x000000004004444a in ?? ()
#9  0x0000000001226e30 in ?? ()
#10 0x00007fec05c88ff0 in System.IO.SynchronizedWriter:WriteLine (this=Cannot access memory at address 0xffffffffffffffd0
) at <unknown>:668
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 2 (Thread 0x7febd5f63700 (LWP 18511)):
#0  0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005cbbc4 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7febd5f628c0) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007fec08583c72 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007fec0857f179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007fec0857ef9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005ed4f9 in mono_gc_alloc_mature (vtable=vtable("System.Threading.InternalThread")) at sgen-alloc.c:611
#8  0x00000000005847ef in mono_thread_attach_full (domain=0x11d3420, force_attach=0) at threads.c:979
#9  0x000000000041ade8 in mono_jit_thread_attach (domain=0x11d3420) at mini.c:2737
#10 0x00000000400441e4 in ?? ()
#11 0x0000000000000003 in ?? ()
#12 0x00007fec0857c8ca in start_thread () from /lib/libpthread.so.0
#13 0x00007fec082e3b6d in clone () from /lib/libc.so.6
#14 0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7fec0902f780 (LWP 18419)):
#0  0x00007fec08246564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005cbbc4 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7fff1f2ed0c0) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007fec08583c74 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007fec0857f179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007fec0857ef9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005d5fcd in mono_gc_collect (generation=1) at sgen-gc.c:4601
#8  0x000000004004444a in ?? ()
#9  0x0000000001214390 in ?? ()
#10 0x00000000400440f0 in ?? ()
#11 0x0000000001214390 in ?? ()
#12 0x0000000000000000 in ?? ()

=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

Aborted
Comment 3 Rodrigo Kumpera 2013-11-13 16:16:23 UTC
An additional fix in there was done in cf8c83129debd5a5f34ec9c1a821a31c6de7866c, does it fixes for you?
Comment 4 Marian Salaj 2013-11-14 05:13:40 UTC
Rodrigo please read my second comment, this answers your question. The commit did fix deadlocking, but test code now crashes on a assertion (see stacktrace in second comment). Please fix it.
Comment 5 Marian Salaj 2014-01-08 08:58:59 UTC
Please Rodrigo fix the assertion at mono-threads.c:242 already (see my second comment). Is should be a easy fix, I assume that the assertion needs to be removed.
Comment 6 Rodrigo Kumpera 2014-01-08 10:33:32 UTC
Marian, it does look like a bug in your end, that assertion should be be in the above code path.

The detach code doesn't unregister a thread until after it has finished cleaning up, which means the failsafe of the list search should work regardless.

What's your test system spec?
Does it have a custom libpthread?
Do you have native code that tries to access the mono runtime from a TLS destructor?
Comment 7 Marian Salaj 2014-01-08 11:42:10 UTC
What's your test system spec?
Debian 6, Debian 3.2.46-1~bpo60+1 x86_64 GNU/Linux

Does it have a custom libpthread?
Don't know

Do you have native code that tries to access the mono runtime from a TLS
destructor?
NO, I have provided you with simple testcase, please test it.
Comment 8 Marian Salaj 2014-01-08 11:44:27 UTC
Here is the test case again:

using System;
using System.Threading;
using System.Runtime.InteropServices;

namespace Test
{
class Program
{
[DllImport("libpthread.so.0", SetLastError = false, EntryPoint =
"pthread_create", CallingConvention = CallingConvention.Cdecl)]

private static extern int pthread_create(out ulong thread, IntPtr attr,
[MarshalAs(UnmanagedType.FunctionPtr)] Delegate d, IntPtr arg);

static void TestThread()
{
  Console.WriteLine("Hello!");
  GC.Collect();
}

static void Main(string[] args)
{
   ulong thread_t = 0;

   while(true)
   {
      int ret = pthread_create(out thread_t, IntPtr.Zero, new ThreadStart(TestThread), IntPtr.Zero);
      GC.Collect();
   }
}
}
}
Comment 9 Rodrigo Kumpera 2014-01-08 11:57:15 UTC
Oh, I see the problem now.

The above won't work, externally created threads must explicitly attach and detach from the runtime.

Why do you need to do it like this? Just use System.Threading.Thread.
Comment 10 Marian Salaj 2014-01-08 13:16:18 UTC
I don't understand, the thread is attached to runtime when it enters managed code in function TestThread, or not? Please explain. I have simply commented the assert and it is working. Also with boehm this code is running fine.
Comment 11 Rodrigo Kumpera 2014-01-08 13:24:38 UTC
The problem is very simple, the thread must detach itself manually, that's what the runtime expects it to do.
Comment 12 Marian Salaj 2014-01-08 13:41:47 UTC
So how to do it? What does it mean : thread must detach itself manually?
I need to call managed code on thread created by unmanaged code with pthread_create.
Comment 13 Rodrigo Kumpera 2014-01-08 16:16:36 UTC
I can see that there might be a problem in there. The fix is not to remove the assert, that will just result in a crash further down the road.
Comment 14 Rodrigo Kumpera 2014-02-12 15:43:15 UTC
This is how mono embedding works,threads must explicitly detach themselves. 
We don't plan on changing that.
Comment 15 Marian Salaj 2014-02-14 13:22:03 UTC
I DON'T do mono embedding - I'm not embedding the runtime into a C application. I'm just doing p/invoke. The thread is just spawned by unmanaged code and then C# code is called in a callback. Why are you telling me the thread must be detached explicitly when I'm seeing in the code that the thread is detached implicitly? How you even looked at the stack trace? From what I can tell a thread is ending and runtime tries to implicitly detach the thread, but at the same time garbage collection is performed and the the app crashes on assertion. This is clearly a sgen bug!
Comment 16 Rodrigo Kumpera 2014-02-14 14:01:45 UTC
Try master, we did some fixing in this area that might help you.
Comment 17 Marian Salaj 2014-03-03 03:32:00 UTC
I have tried master, test code is still crashing on the same assert. 

* Assertion at mono-threads.c:242, condition `info' not met


Native stacktrace:

        /home/user/mono/mono-git-new/bin/mono() [0x4a5245]
        /home/user/mono/mono-git-new/bin/mono() [0x4177bf]
        /lib/libpthread.so.0(+0xeff0) [0x7f5fe214aff0]
        /home/user/mono/mono-git-new/bin/mono() [0x629417]
        /home/user/mono/mono-git-new/bin/mono() [0x5a25a7]
        /home/user/mono/mono-git-new/bin/mono() [0x5062a5]
        /lib/libpthread.so.0(+0xeff0) [0x7f5fe214aff0]
        /lib/libc.so.6(gsignal+0x35) [0x7f5fe1e0c1b5]
        /lib/libc.so.6(abort+0x180) [0x7f5fe1e0efc0]
        /home/user/mono/mono-git-new/bin/mono() [0x63010d]
        /home/user/mono/mono-git-new/bin/mono() [0x6301b7]
        /home/user/mono/mono-git-new/bin/mono() [0x62b5b8]
        /home/user/mono/mono-git-new/bin/mono() [0x5cab09]
        /home/user/mono/mono-git-new/bin/mono() [0x57f44d]
        /home/user/mono/mono-git-new/bin/mono(mono_thread_detach+0x12) [0x5817e2]
        /home/user/mono/mono-git-new/bin/mono() [0x62b0f0]
        /lib/libpthread.so.0(+0x5ee9) [0x7f5fe2141ee9]
        /lib/libpthread.so.0(+0x68d8) [0x7f5fe21428d8]
        /lib/libc.so.6(clone+0x6d) [0x7f5fe1ea9b6d]

Debug info from gdb:

Mono support loaded.
[Thread debugging using libthread_db enabled]
[New Thread 0x7f59270bb700 (LWP 16702)]
[New Thread 0x7f59278bc700 (LWP 16701)]
[New Thread 0x7f59280c5700 (LWP 16700)]
[New Thread 0x7f59290c7700 (LWP 16698)]
[New Thread 0x7f59298c8700 (LWP 16697)]
[New Thread 0x7f592a0c9700 (LWP 16696)]
[New Thread 0x7f5fdf734700 (LWP 13130)]
0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
  Id   Target Id         Frame 
  8    Thread 0x7f5fdf734700 (LWP 13130) "mono" 0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
  7    Thread 0x7f592a0c9700 (LWP 16696) "mono" 0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
  6    Thread 0x7f59298c8700 (LWP 16697) "mono" 0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
  5    Thread 0x7f59290c7700 (LWP 16698) "mono" 0x00007f5fe21493c0 in sem_wait () from /lib/libpthread.so.0
  4    Thread 0x7f59280c5700 (LWP 16700) "mono" 0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
  3    Thread 0x7f59278bc700 (LWP 16701) "mono" 0x00007f5fe214abdd in waitpid () from /lib/libpthread.so.0
  2    Thread 0x7f59270bb700 (LWP 16702) "mono" 0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
* 1    Thread 0x7f5fe2bf5780 (LWP 13129) "mono" 0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6

Thread 8 (Thread 0x7f5fdf734700 (LWP 13130)):
#0  0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005c9f74 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7f5fdf733880) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007f5fe2149c72 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007f5fe2145179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007f5fe2144f9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005cc5ba in mono_gc_invoke_finalizers () at sgen-gc.c:3850
#8  0x00000000005a4471 in finalizer_thread (unused=<optimized out>) at gc.c:1102
#9  0x0000000000581697 in start_wrapper_internal (data=<optimized out>) at threads.c:609
#10 start_wrapper (data=0x2250a70) at threads.c:654
#11 0x000000000061ae13 in thread_start_routine (args=0x21fe730) at wthreads.c:294
#12 0x000000000062bd90 in inner_start_thread (arg=0x2250e50) at mono-threads-posix.c:49
#13 0x00007f5fe21428ca in start_thread () from /lib/libpthread.so.0
#14 0x00007f5fe1ea9b6d in clone () from /lib/libc.so.6
#15 0x0000000000000000 in ?? ()


Thread 7 (Thread 0x7f592a0c9700 (LWP 16696)):
#0  0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005c9f74 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7f592a0c88c0) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007f5fe2149c72 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007f5fe2145179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007f5fe2144f9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005ebb61 in mono_gc_alloc_mature (vtable=vtable("System.Threading.InternalThread")) at sgen-alloc.c:611
#8  0x0000000000582f5f in mono_thread_attach_full (domain=0x220b420, force_attach=0) at threads.c:978
#9  0x000000000041aa08 in mono_jit_thread_attach (domain=0x220b420) at mini.c:2744
#10 0x0000000040d62064 in ?? ()
#11 0x0000000000000003 in ?? ()
#12 0x00007f5fe21428ca in start_thread () from /lib/libpthread.so.0
#13 0x00007f5fe1ea9b6d in clone () from /lib/libc.so.6
#14 0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7f59298c8700 (LWP 16697)):
#0  0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005c9f74 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7f59298c7880) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007f5fe2149c72 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007f5fe2145179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007f5fe2144f9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005ebb61 in mono_gc_alloc_mature (vtable=vtable("System.Threading.Thread")) at sgen-alloc.c:611
#8  0x0000000000580c56 in new_thread_with_internal (domain=0x974300, internal=0x80) at threads.c:495
#9  0x000000000058303b in mono_thread_attach_full (domain=0x220b420, force_attach=0) at threads.c:1015
#10 0x000000000041aa08 in mono_jit_thread_attach (domain=0x220b420) at mini.c:2744
#11 0x0000000040d62064 in ?? ()
#12 0x0000000000000003 in ?? ()
#13 0x00007f5fe21428ca in start_thread () from /lib/libpthread.so.0
#14 0x00007f5fe1ea9b6d in clone () from /lib/libc.so.6
#15 0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7f59290c7700 (LWP 16698)):
#0  0x00007f5fe21493c0 in sem_wait () from /lib/libpthread.so.0
#1  0x0000000000626ef8 in mono_sem_wait (sem=0x9617c0, alertable=0) at mono-semaphore.c:119
#2  0x00000000005ca0a2 in sgen_wait_for_suspend_ack (count=7) at sgen-os-posix.c:165
#3  0x00000000005ca1b8 in sgen_thread_handshake (suspend=<optimized out>) at sgen-os-posix.c:199
#4  0x00000000005f80c3 in sgen_stop_world (generation=1) at sgen-stw.c:219
#5  0x00000000005d3e98 in sgen_perform_collection (requested_size=0, generation_to_collect=1, reason=0x700421 "user request", wait_to_finish=1) at sgen-gc.c:3463
#6  0x00000000005d43e8 in mono_gc_collect (generation=1) at sgen-gc.c:4623
#7  0x0000000040d6231d in ?? ()
#8  0x0000000002293760 in ?? ()
#9  0x00007f5fe1530010 in ?? ()
#10 0x0000000000000000 in ?? ()

Thread 4 (Thread 0x7f59280c5700 (LWP 16700)):
#0  0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005c9f74 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7f59280c48c0) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007f5fe2149c72 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007f5fe2145179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007f5fe2144f9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005d43cd in mono_gc_collect (generation=1) at sgen-gc.c:4620
#8  0x0000000040d6231d in ?? ()
#9  0x00007f5d880098f0 in ?? ()
#10 0x00007f5fe1530010 in ?? ()
#11 0x0000000000000000 in ?? ()

Thread 3 (Thread 0x7f59278bc700 (LWP 16701)):
#0  0x00007f5fe214abdd in waitpid () from /lib/libpthread.so.0
#1  0x00000000004a52d8 in mono_handle_native_sigsegv (signal=<optimized out>, ctx=<optimized out>) at mini-exceptions.c:2299
#2  0x00000000004177bf in mono_sigsegv_signal_handler (_dummy=11, info=0x7f5fe2a1bf70, context=0x7f5fe2a1be40) at mini.c:6735
#3  <signal handler called>
#4  get_hazardous_pointer (pp=0x158, hp=0x7f5fe154d300, hazard_index=0) at hazard-pointer.c:180
#5  0x00000000005a25a7 in mono_jit_info_table_find_internal (domain=0x0, addr=0x7f5fe1e0c1b5 "H=", try_aot=1) at domain.c:397
#6  0x00000000005062a5 in sigabrt_signal_handler (_dummy=6, info=0x7f59278bb830, context=0x7f59278bb700) at mini-posix.c:202
#7  <signal handler called>
#8  0x00007f5fe1e0c1b5 in raise () from /lib/libc.so.6
#9  0x00007f5fe1e0efc0 in abort () from /lib/libc.so.6
#10 0x000000000063010d in monoeg_g_logv (log_domain=0x64cb7c "", log_level=G_LOG_LEVEL_ERROR, format=<optimized out>, args=<optimized out>) at goutput.c:175
#11 0x00000000006301b7 in monoeg_assertion_message (format=0x6 <Address 0x6 out of bounds>) at goutput.c:195
#12 0x000000000062b5b8 in mono_thread_info_current () at mono-threads.c:242
#13 0x00000000005cab09 in mono_gc_set_current_thread_appdomain (domain=0x3349) at sgen-gc.c:1729
#14 0x000000000057f44d in thread_cleanup (thread=0x7f5fe1544dd0) at threads.c:412
#15 0x00000000005817e2 in mono_thread_detach (thread=<optimized out>) at threads.c:1062
#16 0x000000000062b0f0 in unregister_thread (arg=<optimized out>) at mono-threads.c:187
#17 0x00007f5fe2141ee9 in __nptl_deallocate_tsd () from /lib/libpthread.so.0
#18 0x00007f5fe21428d8 in start_thread () from /lib/libpthread.so.0
#19 0x00007f5fe1ea9b6d in clone () from /lib/libc.so.6
#20 0x0000000000000000 in ?? ()

Thread 2 (Thread 0x7f59270bb700 (LWP 16702)):
#0  0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005c9f74 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7f59270ba880) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007f5fe2149c72 in __lll_lock_wait () from /lib/libpthread.so.0
#5  0x00007f5fe2145179 in _L_lock_953 () from /lib/libpthread.so.0
#6  0x00007f5fe2144f9b in pthread_mutex_lock () from /lib/libpthread.so.0
#7  0x00000000005cc499 in mono_gc_register_root_inner (start=0x7f5fe1545048 "0OT\341_\177", size=128, descr=0x0, root_type=-1) at sgen-gc.c:3924
#8  0x0000000000583132 in mono_thread_attach_full (domain=0x220b420, force_attach=0) at threads.c:1001
#9  0x000000000041aa08 in mono_jit_thread_attach (domain=0x220b420) at mini.c:2744
#10 0x0000000040d62064 in ?? ()
#11 0x0000000000000003 in ?? ()
#12 0x00007f5fe21428ca in start_thread () from /lib/libpthread.so.0
#13 0x00007f5fe1ea9b6d in clone () from /lib/libc.so.6
#14 0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7f5fe2bf5780 (LWP 13129)):
#0  0x00007f5fe1e0c564 in sigsuspend () from /lib/libc.so.6
#1  0x00000000005c9f74 in suspend_thread (context=<optimized out>, info=<optimized out>) at sgen-os-posix.c:113
#2  suspend_handler (sig=<optimized out>, siginfo=<optimized out>, context=0x7fff9fa08ec0) at sgen-os-posix.c:131
#3  <signal handler called>
#4  0x00007f5fe1ea65b7 in mprotect () from /lib/libc.so.6
#5  0x00007f5fe214359e in pthread_create@@GLIBC_2.2.5 () from /lib/libpthread.so.0
#6  0x0000000040d61f60 in ?? ()
#7  0x000000000224d400 in ?? ()
#8  0x0000000000000068 in ?? ()
#9  0x00007f5fd8000da0 in ?? ()
#10 0x00007fff9fa094a0 in ?? ()
#11 0x00007fff9fa093c0 in ?? ()
#12 0x000000000220b420 in ?? ()
#13 0x0000000000000000 in ?? ()

=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

Aborted
Comment 18 Mark Probst 2014-03-14 18:38:40 UTC
Fixed in 71ad74dc11c5fa4bc8c178a5457d2cab732fdb01.