Bug 55443 - SO_REUSEADDR does not produce expected results after Mono 4.2.1
Summary: SO_REUSEADDR does not produce expected results after Mono 4.2.1
Status: NEW
Alias: None
Product: Runtime
Classification: Mono
Component: io-layer (show other bugs)
Version: 4.8.0 (C9)
Hardware: All Linux
: --- normal
Target Milestone: ---
Assignee: Ludovic Henry
URL:
Depends on:
Blocks:
 
Reported: 2017-04-21 04:51 UTC by H3V2
Modified: 2017-04-24 13:46 UTC (History)
4 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Program which demonstrates the reported issue (1.08 KB, text/plain)
2017-04-21 04:51 UTC, H3V2
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report for Bug 55443 on GitHub or Developer Community if you have new information to add and do not yet see a matching new report.

If the latest results still closely match this report, you can use the original description:

  • Export the original title and description: GitHub Markdown or Developer Community HTML
  • Copy the title and description into the new report. Adjust them to be up-to-date if needed.
  • Add your new information.

In special cases on GitHub you might also want the comments: GitHub Markdown with public comments

Related Links:
Status:
NEW

Description H3V2 2017-04-21 04:51:21 UTC
Created attachment 21672 [details]
Program which demonstrates the reported issue

Observed behaviour: When a socket is double bound to the same endpoint, an AddressAlreadyInUse exception is thrown
Expected behaviour: An exception should not be thrown

I originally posted this as a question on Stack Overflow, but further investigation leads me to believe that what I'm experiencing is a bug..

http://stackoverflow.com/questions/43526650/so-reuseaddr-does-not-produce-expected-results-after-mono-4-2-1


Reading back through the source code and issues I can see there has been a lot of confusion about SO_REUSEADDR, SO_REUSEPORT and ExclusiveAddressUse which I believe has led to this bug.

My issue is that using Mono on Linux after version 4.2.1, I am unable to Bind() a Socket to a LocalEndPoint if there is already a socket bound and listening to that same endpoint.

I've attached a sample program which demonstrates this phenomenon, and to save time i'll briefly re-cap the function of each of the moving parts involved.

  SO_REUSEADDR (0x0004) = allows binding to an address which is in a TIME_WAIT state.
  SO_REUSEPORT (0x0200) = allows multiple processes to bind to the same address provided all of them use the SO_REUSEPORT option.

Normally applications cannot bind to an address and port belonging to a recently closed socket that is in a TIME_WAIT state. The SO_REUSEADDR flag can be used to override this behaviour.

The SO_REUSEPORT flag is (or was) not universally available on all operating systems.

On Windows when a user issues the SO_REUSEADDR ioctl, it automatically sets the SO_REUSEPORT flag on the socket which has given rise to threads like this https://github.com/mono/mono/pull/4104, this https://github.com/dotnet/corefx/pull/11509 and commits like this https://github.com/mono/mono/commit/9d3d44013f5ca5311404737bb384b76654d414ff, that last one broke things.

The situation was further clouded by the Windows-only ExclusiveAddressUse ioctl, and this control code became conflated as a fix for the TIME_WAIT issue, for the corefx team anyway.

I've managed to identify the problem. If you compile my attached test program and run it using strace you can see the setsockopt system calls which get made by each version of Mono.

$ mcs socketTest.cs
$ strace -o socketTest.log mono socketTest.exe
$ grep setsockopt socket.log

Under Mono 4.2.1
================

  setsockopt(3, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEPORT, [1], 4) = 0
  setsockopt(5, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(5, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(5, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(5, SOL_SOCKET, SO_REUSEPORT, [1], 4) = 0

Under Mono 4.2.2
================

  setsockopt(3, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(5, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(5, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(5, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0

Under Mono 4.8.1
================

  setsockopt(3, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(3, SOL_TCP, TCP_NODELAY, [0], 4) = 0
  setsockopt(4, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(4, SOL_TCP, TCP_NODELAY, [0], 4) = 0
  setsockopt(5, SOL_IP, IP_MTU_DISCOVER, [0], 4) = 0
  setsockopt(5, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(5, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(5, SOL_TCP, TCP_NODELAY, [0], 4) = 0


As you can see, the system calls to setsockopts() are quite different depending on which version of Mono we're executing under given the same C#. The breaking change was introduced between version 4.2.1 and 4.2.2.

In version 4.2.1 when the SocketOptionName.ReuseAddress flag is passed to SetSocketOption() it results in system calls which correctly set BOTH the SO_REUSEADDR and SO_REUSEPORT flags, emulating the behaviour of Windows.

In later versions, only the SO_REUSEADDR is set. Unfortunately there appears to be no way to independently set the SO_REUSEPORT flag using managed code, which has the effect of disabling this functionality.

Notice that in /mono/metadata/w32socket-unix.c at line #862 the code still exists to handle this situation correctly https://github.com/mono/mono/blob/17aa73d0ae216529b59d5aa3d0ee68007bb035c4/mono/metadata/w32socket-unix.c#L862

I'm not familiar enough with the Mono project to understand why this isn't used, but it seems instead that the method ves_icall_System_Net_Sockets_Socket_SetSocketOption_internal() in w32socket.c is responsible for setting socket options and failing to set the SO_REUSEPORT flag.

I think the possible fixes are either:

  1. Restore earlier functionality. Copy Windows and set SO_REUSEPORT in conjunction with SO_REUSEADDR.
  2. Provide a means for the SO_REUSEPORT flag to be set.

I've been unable to track down the exact commit which broke this functionality, but it would be good to get this resolved.
Comment 1 H3V2 2017-04-24 13:46:12 UTC
As a work-around, call setsockopt() with the SO_REUSEPORT option on an existing socket.

...

public static bool IsMono => Type.GetType("Mono.Runtime") != null;
public static int SOL_SOCKET = 1;
public static int SO_REUSEPORT = 15;

[DllImport("libc", SetLastError = true)]
public static extern int setsockopt(int sockfd, int level, int option, ref int value, int len);

...

var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

...

if (IsMono)
{
  var enable = 1;
  if (Libc.setsockopt(socket.Handle.ToInt32(), Libc.SOL_SOCKET, Libc.SO_REUSEPORT, ref enable, sizeof(int)) < 0)
  {
    throw new Exception(Marshal.GetLastWin32Error());
  }
}