Bug 25095 - Struct w/nested structs as return value from P/Invoke is incorrectly marshalled on x86_64/OSX
Summary: Struct w/nested structs as return value from P/Invoke is incorrectly marshall...
Status: RESOLVED FIXED
Alias: None
Product: Runtime
Classification: Mono
Component: Interop (show other bugs)
Version: unspecified
Hardware: PC Mac OS
: --- normal
Target Milestone: ---
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2014-12-05 06:05 UTC by Rolf Bjarne Kvinge [MSFT]
Modified: 2014-12-06 06:18 UTC (History)
3 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
testcase.zip (1.17 KB, application/zip)
2014-12-05 06:05 UTC, Rolf Bjarne Kvinge [MSFT]
Details

Description Rolf Bjarne Kvinge [MSFT] 2014-12-05 06:05:03 UTC
Created attachment 8974 [details]
testcase.zip

Repro:
* Download & extract provided test case.
* Fixup Makefile ('test' target) to use a 64-bit capable mono.
* 'make test'

Result:
  Flat:
  0xbb113311 0x553311
  0 0 0 0

  Nested:
  0x7fff5626c618 0xbb113311
  0 0 5.712068E-33 1.401298E-45
  **** FAILED **** 

The problem is that the nested structure {{float,float,float},float} is marshalled differently from a flat structure {float,float,float,float} - Mono thinks the native function requires a pointer to the memory for the struct as the first argument, when the native function will return the struct in xmm0/xmm1. This effectively shifts the rest of the arguments by one register.
Comment 1 Zoltan Varga 2014-12-06 06:18:28 UTC
Fixed in mono master 2a8f86f22388107e1c978d28179832b71295afac.

Note You need to log in before you can comment on or make changes to this bug.