Bug 25095 - Struct w/nested structs as return value from P/Invoke is incorrectly marshalled on x86_64/OSX
Summary: Struct w/nested structs as return value from P/Invoke is incorrectly marshall...
Alias: None
Product: Runtime
Classification: Mono
Component: Interop ()
Version: unspecified
Hardware: PC Mac OS
: --- normal
Target Milestone: ---
Assignee: Bugzilla
Depends on:
Reported: 2014-12-05 06:05 UTC by Rolf Bjarne Kvinge [MSFT]
Modified: 2014-12-06 06:18 UTC (History)
3 users (show)

Is this bug a regression?: ---
Last known good build:

testcase.zip (1.17 KB, application/zip)
2014-12-05 06:05 UTC, Rolf Bjarne Kvinge [MSFT]

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Rolf Bjarne Kvinge [MSFT] 2014-12-05 06:05:03 UTC
Created attachment 8974 [details]

* Download & extract provided test case.
* Fixup Makefile ('test' target) to use a 64-bit capable mono.
* 'make test'

  0xbb113311 0x553311
  0 0 0 0

  0x7fff5626c618 0xbb113311
  0 0 5.712068E-33 1.401298E-45
  **** FAILED **** 

The problem is that the nested structure {{float,float,float},float} is marshalled differently from a flat structure {float,float,float,float} - Mono thinks the native function requires a pointer to the memory for the struct as the first argument, when the native function will return the struct in xmm0/xmm1. This effectively shifts the rest of the arguments by one register.
Comment 1 Zoltan Varga 2014-12-06 06:18:28 UTC
Fixed in mono master 2a8f86f22388107e1c978d28179832b71295afac.