The following code will cause an unhandled exception after about 30 seconds. It seems that the connection attempt continues even after the socket is closed. I've compared this vs. standard Microsoft .NET implementation and the standard implementation doesn't throw an error.
using (var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
//Simulate Socket Async Connection
var e = new SocketAsyncEventArgs();
e.RemoteEndPoint = new DnsEndPoint(host, port);
//Simulate aborting connection before fully connected
//Useful if you want to implement a timeout