Bug 23792

Summary: SOAP response incorrectly truncated at chunk boundary on Android using httpClient to send SOAP over SSL
Product: [Mono] Class Libraries Reporter: Valentin Polushkin <valentin>
Component: System.WebAssignee: Miguel de Icaza [MSFT] <miguel>
Status: RESOLVED FIXED    
Severity: normal CC: adrian.murphy, chrisntr, gmarrot, jim_hanmer, martin.baulig, mono-bugs+mono, pj.beaman
Priority: High    
Version: unspecified   
Target Milestone: Untriaged   
Hardware: PC   
OS: Mac OS   
Tags: Is this bug a regression?: ---
Last known good build:
Attachments: SOAPErrorDemo.zip
FiddlerTraceSuccessfulChunks.txt
FiddlerTraceFailedChunks.txt
error.txt
Much simpler, self contained, command line version of the error

Description Valentin Polushkin 2014-10-13 16:09:57 UTC
Created attachment 8395 [details]
SOAPErrorDemo.zip

This is possibly related to bug 20583 which I raised earlier in the year. 

In summary I have the following issue:
- Using Xamarin Android to call a Portable Class Library
- The PCL uses httpClient to POST a SOAP message to a web service and receive and process the response
- The web service is configured to send a chunked response
- When calling the service over HTTP in the clear, it works perfectly

Problem: when calling the same service over HTTPS / SSL a response is received, but intermittently the response content is incorrectly truncated at a chunk boundary, causing an XML parsing error.

NB There is one subtle complexity that may or may not be relevant – the SSL certificate is self-generated, so I have used System.Net.ServicePointManager.ServerCertificateValidationCallback to tell the app to trust the certificate.

Repro
I have attached a demo project (SOAPErrorDemo.zip) that illustrates the issue (you will need to restore the NuGet packages, which should happen automatically if you have VS setup to do so).

In the solution are 2 projects:
SOAPErrorDemo – the PCL project making the web service call.
SOAPErrorDemo.Droid – a Xamarin Android project that uses the PCL to call the web service.
There are radio buttons to toggle between:
SSL and non-SSL
A web service call that works (label: ‘This will work’) and one that intermittently fails (to repro the issue) (label: ’This may fail')
A TextView acts as a console to output the results (in some cases via events raised from the PCL)
If you run the code with ‘SSL’ selected and ‘This may fail’ selected, you will see that when using SSL on Android, this line:

var response = await  httpClient.SendAsync(message, HttpCompletionOption.ResponseContentRead);

will intermittently succeed. However on (usually) the second or third call it will timeout after 60s, then the line

xml = XDocument.Load(contentStream);

will fail with the attached error (error.txt). The error is caused because the XML in the response is truncated at the HTTP chunk boundary (4072 bytes).

To prove this further I have used Fiddler to show that the error is occurring at a chunk boundary. Note that other calls that retrieve chunked content succeed and retrieve multiple 4072 byte chunks successfully through to the end of the response stream.
The Fiddler trace for the failed response is in the attached file FiddlerTraceFailedChunks.txt
The Fiddler trace for the successful chunked response is in the attached file FiddlerTraceSuccessfulChunks.txt
Could you please investigate and if possible provide a solution or workaround? This is critical to the success of our app due to ship on 07 November so is rather urgent for us.

Dev setup
 
Note that we have not upgraded to Xamarin 3.8 because of this issue with TFS.

Microsoft Visual Studio Ultimate 2013
Version 12.0.30723.00 Update 3
Microsoft .NET Framework
Version 4.5.51641

Installed Version: Ultimate

NuGet Package Manager   2.8.50313.46

Xamarin   3.6.262.0 (d3cf238e3845e930e312b6ec9b4c6c5437c33067)

Xamarin.Android   4.16.0.17 (b5dc5ce91305e19de51d71a1122c109719c4bc34)
Comment 1 Valentin Polushkin 2014-10-13 16:10:47 UTC
Created attachment 8396 [details]
FiddlerTraceSuccessfulChunks.txt
Comment 2 Valentin Polushkin 2014-10-13 16:11:05 UTC
Created attachment 8397 [details]
FiddlerTraceFailedChunks.txt
Comment 3 Valentin Polushkin 2014-10-13 16:11:31 UTC
Created attachment 8398 [details]
error.txt
Comment 4 Adrian Murphy 2014-11-17 18:18:03 UTC
Miguel - Assigning to you for your comments, as per our discussion.
Comment 5 Miguel de Icaza [MSFT] 2015-01-21 13:12:49 UTC
The certificate on the server is invalid, these are the exceptions that I am getting:

mac$ mono --debug --trace=N:Xamarin *exe
calling with useSsl=True and longWait=True
Using SSL

About to send request

[0xb0541000:] EXCEPTION handling: System.Security.Cryptography.CryptographicException: Store Root doesn't exists.
[0xb0541000:] EXCEPTION handling: Mono.Security.Protocol.Tls.TlsException: Invalid certificate received from server. Error code: 0x5
[0xb0541000:] EXCEPTION handling: Mono.Security.Protocol.Tls.TlsException: Invalid certificate received from server. Error code: 0x5
[0xb0541000:] EXCEPTION handling: System.IO.IOException: The authentication or decryption has failed.
[0xb0541000:] EXCEPTION handling: System.IO.IOException: The authentication or decryption has failed.
[0xb0541000:] EXCEPTION handling: System.IO.IOException: The authentication or decryption has failed.
[0xb0541000:] EXCEPTION handling: System.IO.IOException: The authentication or decryption has failed.
[0xb0541000:] EXCEPTION handling: System.Net.WebException: Error: SendFailure (Error writing headers)
[0xb0541000:] EXCEPTION handling: System.Net.WebException: Error: SendFailure (Error writing headers)
[0xb0541000:] EXCEPTION handling: System.Net.WebException: Error: SendFailure (Error writing headers)
[0xb0541000:] EXCEPTION handling: System.Net.WebException: Error: SendFailure (Error writing headers)
[0xb0541000:] EXCEPTION handling: System.Net.WebException: Error: SendFailure (Error writing headers)
Comment 6 Miguel de Icaza [MSFT] 2015-01-21 13:14:07 UTC
Created attachment 9438 [details]
Much simpler, self contained, command line version of the error

This is a much simpler version of the problem, runs with desktop Mono.

Use:

make
make run
make curl
Comment 7 Martin Baulig 2015-05-19 22:53:55 UTC
Fixed in mono/master commit
https://github.com/mono/mono/commit/754395be235ee67df917421d3f86e3ff2458ee81
Comment 8 Gaëtan MARROT 2015-08-06 11:41:09 UTC
Hi,

We are currently experimenting similar issues on our Xamarin applications.
Is this fixed already applied on current Mono versions (we are using the last Xamarin.Android and Xamarin.iOS versions) ?

Thanks for your response.