Bug 31932

Summary: Regression: Stack Overflow with native P/Invoke Callback
Product: [Mono] Runtime Reporter: marc hoffman <mh>
Component: InteropAssignee: Bugzilla <bugzilla>
Status: RESOLVED FIXED    
Severity: normal CC: mono-bugs+mono, mono-bugs+runtime, saper, vargaz
Priority: ---    
Version: 4.0.0   
Target Milestone: ---   
Hardware: Macintosh   
OS: Mac OS   
Tags: Is this bug a regression?: ---
Last known good build:

Description marc hoffman 2015-07-14 12:55:19 UTC
The following code will Stack-Overflow on Mono 4.x master (revision 9b0b44) on Mac OS X 10.10.4. Custom-built Mono, 64bit. it works fine in Mono 3.

using System;
using System.Runtime.InteropServices;

namespace Mono4PInvokeCrashtest
{
	static class Program
	{
		[DllImport("/usr/lib/system/libsystem_pthread.dylib", CallingConvention = CallingConvention.Cdecl)]
		extern void sched_yield(CallbackType x);
		
		[UnmanagedFunctionPointer(CallingConvention.Cdecl, CharSet = CharSet.Ansi)]	 
		public delegate bool CallbackType(string fn, [Out] StringBuilder outfn);

		public static Int32 Main(string[] args)
		{
			Console.WriteLine("The magic happens here.");
			sched_yield( (a,b) => { return true; });
			return 0;
		}
	}
}

Output:

Ribbons:Debug mh $ ~/Code/Frameworks/mono4___master_14Jun15/bin/mono Mono4PInvokeCrashtest.exe
The magic happens here.
Stack overflow in unmanaged: IP: 0x7fff8cc946f3, fault addr: 0x7fff588cbff0
Stack overflow in unmanaged: IP: 0x7fff8cc9582c, fault addr: 0x7fff588c9ff8
Stack overflow in unmanaged: IP: 0x7fff8cc946d1, fault addr: 0x7fff588c8ff4
Stack overflow in unmanaged: IP: 0x7fff8cc94716, fault addr: 0x7fff588c7fa8
Stack overflow in unmanaged: IP: 0x7fff8cc94716, fault addr: 0x7fff588c6fe8
Stack overflow in unmanaged: IP: 0x7fff8cc95834, fault addr: 0x7fff588c5ff8
Stack overflow in unmanaged: IP: 0x7fff8cc946f3, fault addr: 0x7fff588c4ff0
Stack overflow: IP: 0x7fff8cc9582c, fault addr: 0x7fff588c2ff8
Stacktrace:
  at <unknown> <0xffffffff>
^C

a more complete call stack (from a more complex in-production test case) can be found at http://pastebin.com/UQKzqXWU
Comment 1 Marcin Cieślak 2015-07-14 14:46:56 UTC
I got the same running on FreeBSD (just changed the library to /usr/lib/pthread.so plus added missing "static" and "using System.Text;":

(gdb) run
Starting program: /usr/local/bin/mono --debug Mono4PInvokeCrashtest.exe
[New LWP 100668]
[New Thread 802006400 (LWP 100668/mono-sgen)]
The magic happens here.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 802006400 (LWP 100668/mono-sgen)]
0x0000000801552b07 in sbrk () from /lib/libc.so.7
(gdb) bt
#0  0x0000000801552b07 in sbrk () from /lib/libc.so.7
#1  0x0000000801552842 in sbrk () from /lib/libc.so.7
#2  0x000000080154e02f in sbrk () from /lib/libc.so.7
#3  0x000000080154dbbf in sbrk () from /lib/libc.so.7
#4  0x0000000801538f94 in syscall () from /lib/libc.so.7
#5  0x00000008015584f5 in calloc () from /lib/libc.so.7
#6  0x00000000006396ea in monoeg_malloc0 (x=24) at gmem.c:84
#7  0x0000000000640523 in monoeg_g_list_alloc () at glist.c:35
#8  0x00000000006405a9 in new_node (prev=0x8022d4e20, data=0x8021c90f8, next=0x0) at glist.c:41
#9  0x00000000006406af in monoeg_g_list_append (list=0x80225dde0, data=0x8021c90f8) at glist.c:87
#10 0x000000000056b461 in mono_mb_add_local (mb=0x8021a06e0, type=0x802000000) at method-builder.c:276
#11 0x000000000054e74b in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1684
#12 0x000000000054e7f8 in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1992
#13 0x000000000054e7f8 in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1992
#14 0x000000000054e7f8 in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1992
#15 0x000000000054e7f8 in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1992
#16 0x000000000054e7f8 in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1992


(gdb) frame 12
#12 0x000000000054e7f8 in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1992
1992            emit_struct_conv_full (mb, klass, to_object, -1);
(gdb) print klass->name
$4 = 0x803ab440e "StringBuilder"
(gdb) frame 13
#13 0x000000000054e7f8 in emit_struct_conv_full (mb=0x8021a06e0, klass=0x8021d56a8, to_object=0, string_encoding=4294967295) at marshal.c:1992
1992            emit_struct_conv_full (mb, klass, to_object, -1);
(gdb) print klass->name
$5 = 0x803ab440e "StringBuilder"
Comment 2 Marcin Cieślak 2015-07-14 15:50:09 UTC
It looks like it just loops trying to serialize System.Text.SystemBuilder m_ChunkPrevious field, because it is also System.Text.SystemBuilder....
Comment 3 Zoltan Varga 2015-07-23 22:34:40 UTC
Previous mono versions didn't crash, but marshalling of stringbuilders with an [out] attribute in pinvoke callbacks was never implemented.
Comment 4 Zoltan Varga 2015-07-23 23:09:41 UTC
Fixed in mono master e1101a5eb1910b2cc72290c5bce5e855e88c8f90. We still don't do any marshalling in this case, but we don't crash either.