Bug 3380 - Segfault when getting NSObject for NSDocumentController.OpenDocument/MakeDocument return value
Summary: Segfault when getting NSObject for NSDocumentController.OpenDocument/MakeDocu...
Alias: None
Product: MonoMac
Classification: Desktop
Component: Bindings ()
Version: unspecified
Hardware: PC Mac OS
: --- normal
Target Milestone: ---
Assignee: Bugzilla
Depends on:
Reported: 2012-02-10 12:00 UTC by Jérémie Laval
Modified: 2012-08-21 16:35 UTC (History)
3 users (show)

Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Jérémie Laval 2012-02-10 12:00:07 UTC
When using the OpenDocument or MakeDocument methods that return an id, the runtime segfault when trying to transform the pointer back to a NSObject subclass in .NET land. More precisely the segfault occurs in native when we send the [class] message to the id that we got back to find out its type. This (atm) only happens with the OpenDocument/MakeDocument calls.

For context, this is inside the macdoc sample application (https://github.com/garuma/monomac/blob/macdoc/samples/macdoc). The call is made from a Apple event handler (defined in AppDelegate.cs) and it uses a custom NSDocumentController instance although this instance doesn't overload the Make/Open methods.

Stacktrace is given below:


  at (wrapper managed-to-native) MonoMac.ObjCRuntime.Messaging.intptr_objc_msgSend (intptr,intptr) <0xffffffff>
  at MonoMac.ObjCRuntime.Runtime.GetNSObject (intptr) <0x000f7>
  at MonoMac.AppKit.NSDocumentController.MakeDocument (MonoMac.Foundation.NSUrl,string,MonoMac.Foundation.NSError&) <0x00157>
  at macdoc.MonodocDocumentController.OpenDocument (MonoMac.Foundation.NSUrl,bool,MonoMac.Foundation.NSError&) <0x000ee>
  at macdoc.AppDelegate.HandleGetURLEvent (MonoMac.Foundation.NSAppleEventDescriptor,MonoMac.Foundation.NSAppleEventDescriptor) <0x00064>
  at (wrapper dynamic-method) object.[macdoc.AppDelegate.Void HandleGetURLEvent(MonoMac.Foundation.NSAppleEventDescriptor, MonoMac.Foundation.NSAppleEventDescriptor)] (MonoMac.Foundation.NSObject,MonoMac.ObjCRuntime.Selector,MonoMac.Foundation.NSAppleEventDescriptor,MonoMac.Foundation.NSAppleEventDescriptor) <0x00033>
  at (wrapper native-to-managed) object.[macdoc.AppDelegate.Void HandleGetURLEvent(MonoMac.Foundation.NSAppleEventDescriptor, MonoMac.Foundation.NSAppleEventDescriptor)] (MonoMac.Foundation.NSObject,MonoMac.ObjCRuntime.Selector,MonoMac.Foundation.NSAppleEventDescriptor,MonoMac.Foundation.NSAppleEventDescriptor) <0xffffffff>
  at (wrapper managed-to-native) MonoMac.AppKit.NSApplication.NSApplicationMain (int,string[]) <0xffffffff>
  at MonoMac.AppKit.NSApplication.Main (string[]) <0x00017>
  at macdoc.MainClass.Main (string[]) <0x00017>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void_object (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

	0   macdoc                              0x00094efc mono_handle_native_sigsegv + 284
	1   macdoc                              0x00004fe8 mono_sigsegv_signal_handler + 248
	2   libsystem_c.dylib                   0x9afe159b _sigtramp + 43
	3   ???                                 0xffffffff 0x0 + 4294967295
	4   ???                                 0x02ccc4e8 0x0 + 46974184
	5   ???                                 0x04b7eaf0 0x0 + 79162096
	6   ???                                 0x04b7df97 0x0 + 79159191
	7   ???                                 0x04b7de95 0x0 + 79158933
	8   ???                                 0x01f50f7c 0x0 + 32837500
	9   ???                                 0x02cc789c 0x0 + 46954652
	10  CoreFoundation                      0x95b10548 -[NSObject performSelector:withObject:withObject:] + 72
	11  Foundation                          0x9bb8921a __-[NSAppleEventManager setEventHandler:andSelector:forEventClass:andEventID:]_block_invoke_1 + 121
	12  Foundation                          0x9bb87fcd -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] + 476
	13  Foundation                          0x9bb87da1 _NSAppleEventManagerGenericHandler + 234
	14  AE                                  0x9bb1b045 _Z20aeDispatchAppleEventPK6AEDescPS_mPh + 202
	15  AE                                  0x9bb04b67 _ZL25dispatchEventAndSendReplyPK6AEDescPS_ + 43
	16  AE                                  0x9bb04a54 aeProcessAppleEvent + 253
	17  HIToolbox                           0x94ca5fea AEProcessAppleEvent + 103
	18  AppKit                              0x93caa2af _DPSNextEvent + 1301
	19  AppKit                              0x93ca98ab -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113
	20  AppKit                              0x93ca5c22 -[NSApplication run] + 911
	21  AppKit                              0x93f3a18a NSApplicationMain + 1054
	22  ???                                 0x02cc8278 0x0 + 46957176
	23  ???                                 0x02cc81c0 0x0 + 46956992
	24  ???                                 0x004abda0 0x0 + 4898208
	25  ???                                 0x004abe37 0x0 + 4898359
	26  macdoc                              0x0000d282 mono_jit_runtime_invoke + 722
	27  macdoc                              0x001a436a mono_runtime_invoke + 170
	28  macdoc                              0x001a6f01 mono_runtime_exec_main + 705
	29  macdoc                              0x001a6111 mono_runtime_run_main + 929
	30  macdoc                              0x00069995 mono_jit_exec + 149
	31  macdoc                              0x0006bf13 mono_main + 9587
	32  macdoc                              0x00002299 main + 441
	33  macdoc                              0x000020a6 start + 54

Debug info from gdb:

Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
Comment 1 Jérémie Laval 2012-02-10 12:20:30 UTC
interesting, the crash also happens in Call_OpenDocument (given below) with NSError processing (also when sending class message in GetNSObject) because the code seems to detect there is an error (i.e. outErrorValue isn't IntPtr.Zero) although there doesn't seem to be any (removing the three last lines make the document open normally) 

public void Call_OpenDocument (NSUrl absoluteUrl, bool displayDocument, out NSError outError)
	if (absoluteUrl == null)
		throw new ArgumentNullException ("absoluteUrl");
	IntPtr outErrorPtr = Marshal.AllocHGlobal(4);
	Marshal.WriteInt32(outErrorPtr, 0);

	MonoMac.ObjCRuntime.Messaging.IntPtr_objc_msgSend_IntPtr_bool_IntPtr (controller.Handle, selOpenDocumentWithContentsOfURLDisplayError_, absoluteUrl.Handle, displayDocument, outErrorPtr);

	IntPtr outErrorValue = Marshal.ReadIntPtr(outErrorPtr);
	outError = outErrorValue != IntPtr.Zero ? (NSError)Runtime.GetNSObject(outErrorValue) : null;
Comment 2 Martin Baulig 2012-08-21 11:17:06 UTC
After a long and frustrating debugging session, I finally found and fixed the problem :-)

The bug was actually completely unrelated to OpenDocument(), but happened somewhere else: we were incorrectly marshaling 'out' parameters which resulted in returning random garbage when returning from native->managed.

The call to openDocumentWithContentsOfURL:display:error: actually worked without problems (since it's managed->native), but Cocoa's implementation calls NSDocument's readFromURL:ofType:error:, thus going native->managed.  Due to that marshaling bug, we returned random garbage as 'error' back to Cocoa, but when the function returns success, Cocoa doesn't do anything with the 'error' var except simply passing it up to its caller.

So when openDocumentWithContentsOfURL:display:error: returns, the 'error' var will contain that random junk.
Comment 4 Jérémie Laval 2012-08-21 16:35:05 UTC