Bug 58824 - Large File Upload
Summary: Large File Upload
Status: RESOLVED NOT_REPRODUCIBLE
Alias: None
Product: Class Libraries
Classification: Mono
Component: System (show other bugs)
Version: master
Hardware: Other Linux
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2017-08-16 19:56 UTC by Paul Mackintosh
Modified: 2017-10-13 14:49 UTC (History)
5 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
execution log (1.05 MB, text/plain)
2017-08-16 21:21 UTC, Paul Mackintosh
Details
code sample (2.32 KB, text/plain)
2017-08-16 21:24 UTC, Paul Mackintosh
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED NOT_REPRODUCIBLE

Description Paul Mackintosh 2017-08-16 19:56:48 UTC
When uploading an MultipartFormDataContent using HttpClient.PostAsync to an https endpoint and where our parts consist of a StringContent (30 chars) followed by a StreamContent (Header 'application/zip') we see the Exception below.  This occurs when our StreamContent length exceeds ~28 MB.  Smaller file sizes succeed.  Running the same code on Windows from Visual Studio succeeds with a 120 MB stream, the same is true when running the same code in dotnetcore on the same VM as the mono application runs and fails:


System.Net.Http.HttpRequestException: An error occurred while sending the request 
---> System.IO.IOException: Error writing request 
---> System.Net.Sockets.SocketException: The socket has been shut down#011  
at System.Net.WebConnection.EndWrite (System.Net.HttpWebRequest request, System.Boolean throwOnError, System.IAsyncResult result) [0x000ad] in <5641e4edad4f4464ba58c620a7b8ea48>:0 #011
at System.Net.WebConnectionStream.WriteAsyncCB (System.IAsyncResult r) [0x00013] in <5641e4edad4f4464ba58c620a7b8ea48>:0 #011   
--- End of inner exception stack trace ---#011
at System.Net.WebConnectionStream.EndWrite (System.IAsyncResult r) [0x000d9] in <5641e4edad4f4464ba58c620a7b8ea48>:0 #011  
at System.IO.Stream.<BeginEndWriteAsync>m__8 (System.IO.Stream stream, System.IAsyncResult asyncResult) [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at (wrapper delegate-invoke) System.Func`3[System.IO.Stream,System.IAsyncResult,System.Threading.Tasks.VoidTaskResult]:invoke_TResult_T1_T2 (System.IO.Stream,System.IAsyncResult)#011  at System.Threading.Tasks.TaskFactory`1+FromAsyncTrimPromise`1[TResult,TInstance].Complete (TInstance thisRef, System.Func`3[T1,T2,TResult] endMethod, System.IAsyncResult asyncResult, System.Boolean requiresSynchronization) [0x00002] in <f712f98eb8e445c8918edaf595bbe465>:0 #011
--- End of stack trace from previous location where exception was thrown ---#011  

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <f712f98eb8e445c8918edaf595bbe465>:0 #011
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Net.Http.MultipartContent+<SerializeToStreamAsync>c__async0.MoveNext () [0x0026e] in <cbcaef35d3364f8bb9ae7449eac280e9>:0 #011
--- End of stack trace from previous location where exception was thrown ---#011  

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Net.Http.HttpClientHandler+<SendAsync>c__async0.MoveNext () [0x00363] in <cbcaef35d3364f8bb9ae7449eac280e9>:0 #011   
--- End of inner exception stack trace ---#011  
at System.Net.Http.HttpClientHandler+<SendAsync>c__async0.MoveNext () [0x0049a] in <cbcaef35d3364f8bb9ae7449eac280e9>:0 #011
--- End of stack trace from previous location where exception was thrown ---#011  

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 #011  
at System.Net.Http.HttpClient+<SendAsyncWorker>c__async0.MoveNext () [0x000f3] in <cbcaef35d3364f8bb9ae7449eac280e9>:0
Comment 1 Paul Mackintosh 2017-08-16 19:57:54 UTC
Forgot to add, we have the same issue on version 5
Comment 2 Ludovic Henry 2017-08-16 20:24:13 UTC
Could you please attach a log of the execution when passing the `MONO_LOG_LEVEL=debug MONO_LOG_MASK=io-layer` environment variables? Could you also please provide a reproduction case that we can easily run on our end? Thank you.
Comment 3 Paul Mackintosh 2017-08-16 21:21:48 UTC
Created attachment 24253 [details]
execution log

Log attached.
Comment 4 Paul Mackintosh 2017-08-16 21:24:11 UTC
Created attachment 24254 [details]
code sample

Attaching the code I'm running, unfortunately I'm unable to share the input output endpoints with you directly
Comment 5 Paul Mackintosh 2017-08-17 16:41:28 UTC
We asked the 3rd party we're sending our files to to give us an HTTP endpoint rather than HTTPS so that we could examine the TCP stuff more easily.

When we tested against HTTP the issue went away, so it points to an issue with SSL.
Comment 6 Ludovic Henry 2017-08-17 17:04:06 UTC
Thanks for the attached log and reproduction case, as well as the fact that it reproduces with SSL only. Can you reproduce if you are posting the string and the file to any HTTPS endpoint, like for example a local or remote (over the internet) apache or nginx server (or any other HTTPS server) that doesn't do anything with the content?
Comment 7 Paul Mackintosh 2017-08-17 17:11:58 UTC
I tried this a couple of days ago and no, the issue doesn't occur.

I set an HTTPS endpoint up on a Windows VM running on the same machine and I was able to POST the file and get a response.
Comment 8 Paul Mackintosh 2017-08-17 17:30:58 UTC
I've just tried the request with curl from the Ubuntu VM where I've been running the sample I sent you.

The curl request succeeds.  

I sent the same zipped content stream that failed in mono, i.e. I used curl to send the stream created by CreateZipStream in the sample code.
Comment 9 Paul Mackintosh 2017-08-17 17:35:10 UTC
Observation from our 3rd party:

"Yes, I see it. Appears in our logs as "Processing of multipart/form-data request failed. Unexpected EOF read on the socket". So it does appear that the data/request is being truncated/cut off and not by our side. I see the whole request took 14.251s and we returned a 400 bad request code. (We do have an idle timeout of 15s but that didn't come into play here (would have been a different code)). So yes, looks like your client is cutting out."
Comment 10 Rodrigo Kumpera 2017-10-11 18:00:24 UTC
Thanks for the test case, we'll take a look at it soon.
Comment 11 Marek Safar 2017-10-13 14:49:09 UTC
I cannot reproduce it with Mono 5.4 and using following entry points in your sample

private const string _inputFileEndpoint = @"http://mirror.filearena.net/pub/speed/SpeedTest_64MB.dat";
private const string _destinationEndpoint = @"https://posttestserver.com/post.php";