Bug 58411 - System.Security.Cryptography.CryptographicException: Store root doesn't exist
Summary: System.Security.Cryptography.CryptographicException: Store root doesn't exist
Status: RESOLVED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: BCL Class Libraries (show other bugs)
Version: XI 10.99 (xcode9)
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: 15.4
Assignee: Rolf Bjarne Kvinge [MSFT]
URL:
: 59914 59942 59960 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-07-26 11:42 UTC by koenhendriks89
Modified: 2017-11-07 15:52 UTC (History)
17 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments

Description koenhendriks89 2017-07-26 11:42:15 UTC
Hi fellow Dev's,

On iOS 11 Beta 4 I get a System.Security.Cryptography.CryptographicException: Store root doesn't exist
When I try to build a certificate chain using the following command.

newChain.Build (new X509Certificate2 (cert));

The root certifcate should be there, since I add it using:
newChain.ChainPolicy.ExtraStore.Add (root);

on the line before.

Does anyone know a solution to this error?

Btw, I'm using

Xamarin.iOS 10.99.0.93
Xamarin.Mac 3.99.0.93
Visual Studio for Mac 7.1.0.1267
Mono 5.2.0.209
Thanks!

Koen
Comment 1 Sebastien Pouliot 2017-07-26 13:33:52 UTC
@Koen this is a tricky (in general) and, specifically for iOS (tvOS and watchOS) not friendly API, largely due to sandboxing.

- the real iOS certificate stores are not accessible like the Windows ones [1]. Native API only respect the OS stores (with the OS, not mono, logic);

- mono has an (incomplete [2]) X509 implementation that mimics the .NET semantics. The later works well on desktop mono where additional tools (e.g. certsync) can be used to create and update the stores. This is not possible on iOS.

If `newChain` was created not to use the system store (which does not exists and cannot be created) then you _might_ be able to workaround the issue inside your application (e.g. create the missing store directory) but you'll still be required to keep it in sync (e.g. intermediate CA).


[1] Depending on your need you should be able to use `Security.SecTrust` and `Security.SecPolicy` to get similar results, even Apple API don't provide a lot of details compared to .NET. This will give you results that match the OS and works across iOS, tvOS, watchOS and macOS.

[2] Part of the trickiness is that different version of OS (e.g. Windows) gives different results (the verification rules evolved over time).
Comment 2 koenhendriks89 2017-07-27 07:12:54 UTC
@Sebastien: I forgot to mention that the code above does work correctly on production apps running on iOS 10 and earlier.

We're checking all our apps to make them compatible with iOS 11. A part of them fail on iOS 11 (crash) and after looking into it we diagnosed that apps using the managed stack for the http client crash on this error.

When apps DO NOT use the callback to check the certificates, they also fail with a root certificate error missing. Changing the project settings to use the native stack resolves this. The problem is we use a provided root certificate to do certificate pinning, but as we cannot build a chain anymore this is not an option anymore. Other apps do certificate pinning on the leaf certificate, and that is also not a problem because the chain is not built to check for a match.

It looks like there is a generic problem with the Managed stack in combination with iOS 11. And especially because existing apps no longer work, this looks like a critical problem for Xamarin. There are probably many apps that still use the managed stack.
Comment 3 koenhendriks89 2017-07-27 08:51:55 UTC
The issue occurs when I use 
MonoFramework-MDK-5.2.0.209.macos10.xamarin.universal
as well as the weekly build:
MonoFramework-MDK-5.5.0.507.macos10.xamarin.universal
Comment 4 Sebastien Pouliot 2017-07-27 13:18:08 UTC
So you have code that works on iOS 10 (and below) and fails (with the mentioned exception) in iOS 11 beta ?

Are you using the same binary .app generated by Xamarin.iOS 10.99.0.93 on both iOS 10 and 11 ?

Or did you use a different XI version for your result in iOS 10 ?

   In that case which version of XI was used (and working on iOS 10) ?


Also please attach a test case so we can investigate. The original description did not mention HttpClient nor certificate pinning.



Notes:

a. It's not clear from the description if issue is related _only_ to an iOS (Apple) version (10 vs 11);

b. the system mono version, comment #3, should not have any effect.
Comment 5 koenhendriks89 2017-07-27 14:45:36 UTC
Yes we were using the same binary on iOS 10.3.2 and iOS 11 Beta 4.

I've uploaded a test project here:

https://github.com/pineapple216/CertificateTest

This project successfully does 2 requests (one using the .Net httpclient and one using the native iOS httpclient) on iOS 10.

But gives a Trust Failure using the .Net httpclient on iOS 11.
I think there's something wrong with root certificates on iOS 11.
Comment 6 koenhendriks89 2017-08-11 07:39:06 UTC
I've tried the test project today with:
- Xamarin iOS 10.99.2.12
- Xamarin iOS 10.99.3.19

On iOS 10.3.3 and iOS 11 Developer Beta 5.

And in both cases I get the same error.

The stacktrace can be seen below:

{System.Net.WebException: Error: TrustFailure (CertificateUnknown) ---> Mono.Security.Interface.TlsException: CertificateUnknown
  at Mono.AppleTls.AppleTlsContext.EvaluateTrust () [0x000a5] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.AppleTls/AppleTlsContext.cs:268 
  at Mono.AppleTls.AppleTlsContext.RequirePeerTrust () [0x00008] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.AppleTls/AppleTlsContext.cs:217 
  at Mono.AppleTls.AppleTlsContext.ProcessHandshake () [0x00046] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.AppleTls/AppleTlsContext.cs:193 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncProtocolRequest asyncRequest, Mono.Net.Security.AsyncOperationStatus status) [0x0002a] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:594 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (Mono.Net.Security.AsyncOperationStatus status) [0x0006b] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:272 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation () [0x0000d] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:218 
  at Mono.Net.Security.AsyncProtocolRequest.StartOperation () [0x0003c] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:204 
  at Mono.Net.Security.AsyncProtocolRequest.StartOperation (Mono.Net.Security.AsyncOperation operation) [0x00024] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:189 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Net.LazyAsyncResult lazyResult) [0x00057] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:216 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/referencesource/mscorlib/system/runtime/exceptionservices/exceptionservicescommon.cs:151 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Net.LazyAsyncResult lazyResult) [0x0006c] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:218 
  at Mono.Net.Security.MobileAuthenticatedStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) [0x0000c] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:126 
  at Mono.Net.Security.Private.MonoSslStreamWrapper.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/MonoSslStreamWrapper.cs:75 
  at Mono.Net.Security.MonoTlsStream.CreateStream (System.Byte[] buffer) [0x0007b] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.99.3.19/src/mono/mcs/class/System/Mono.Net.Security/MonoTlsStream.cs:116 
  at System.Net.WebConnection.CreateStream (System.Net.HttpWebRequest request) [0x00073] in /Library/

Hopefully this helps you!

Koen
Comment 7 ranson 2017-08-14 07:26:42 UTC
I have exactly the same problem.

In my case, it started to occur after the Mono TLS Removal, that has been replaced by the newer AppleTLS provider (Xamarin.iOS.10.8)

I still observe the problem with Xamarin.iOS.10.10 and Xamarin.iOS.10.12, on iOS 10.3.3.

In my case, I use an HttpWebRequest instead of an HttpClient. Also, the server certificate and the server certificate authority are signed with SHA1.

On the client-side, I verify the server certificate and have the same exception when I try to build the certification chain: Store root doesn't exist.

This verification works if the application is built with any Xamarin.iOS realease before the Mono TLS Removal, whatever the iOS version.   

If I comment the certification chain validation, then the httpwebrequest timed-out...
Comment 8 ranson 2017-08-17 12:08:52 UTC
I have done additional tests:

ServicePointManager.ServerCertificateValidationCallback += (x1, x2, x3, x4) =>
{
    //var x509_2 = (X509Certificate2)x2; // fails since the 10.8 release
    var x509_2 = new  X509Certificate2(x2);
    ...

Before the 10.8 release, x2 used to be an X509Certificate2 instance, now it is an X509Certificate instance.
This previous fix removes the Trust Failure error.

If the remote service is configured to require a Client Certificate, then the request hangs.

If the remote service doesn't require Client Certificate, then the request works.

HttpWebRequest with Client Certificates used to work before the 10.8 release.

By the way, I found no way to support Client Certificates with the current HttpClient implementation.
Comment 9 koenhendriks89 2017-08-21 09:49:45 UTC
I put a link to a test project in comment 5 earlier.
Please look into this issue.
It's not fixed in Xamarin.iOS 10.99.4.10.
Comment 10 koenhendriks89 2017-08-21 13:26:11 UTC
Today we've tested our test project again.
We found out that an installed configuration profile was causing the above mentioned Mono.Security.Interface.TlsException: CertificateUnknown error.

When using the managed stack (httpclient).
When we used NSUrlSession stack, everything works well.

The mentioned profile installs the root certificate that is causing the problem, but it's hashes are the same as the hashes of the native iOS root certificate.

It seems that the mono stack has no access to the trust store of iOS, when we install a custom root certifcate (even if it's the same as the globally known one).

In iOS 10 this isn't a problem at all.

FYI, we build the app using
Mono 5.2.0.215 and Xamarin iOS 10.99.4.10
Comment 11 Timothy Risi 2017-08-22 17:57:14 UTC
Using the repro project above, it sounds like the error should occur for the Managed version?  I tested using XI 10.99.4.11 on a device running iOS 11 Beta 7 and am not seeing any error
Comment 12 koenhendriks89 2017-08-22 18:00:46 UTC
@Timothy Risi, it should indeed.
When we as mentioned above install a custom root certificate, the error does occur.

It seems that when we a custom certificate, the managed version suddenly can't access the iOS certificate store anymore.
Comment 13 Sebastien Pouliot 2017-08-22 18:51:48 UTC
@ranson your problem (at least the client side certificates) seems different, please file a new bug report and include a test case. Thanks!


@koenhendriks89 The code paths are different between the implementations. However in both cases it ends up to iOS (tvOS, watchOS or macOS) to decide if it trust or not a certificate. 

> the managed version suddenly can't access the iOS certificate store

This does not requires access to the certificates (and there's no concept of stores). You simply ask if the certificate being used is trusted (SecTrustEvaluate) and the OS gives back its decision.

> In iOS 10 this isn't a problem at all.

There is also no version-specific code path our code.

However different versions of iOS can differ, e.g. SHA1 certificates are deprecated in iOS 11

> https://support.apple.com/en-us/HT207459

You might be running into this or other iOS version specific changes - a lot of older features were removed in the past years to increase TLS security.
Comment 14 ranson 2017-09-02 13:14:40 UTC
@Sebastien: I have filed a bug report: 59192 with a repro.
Comment 15 koenhendriks89 2017-09-19 12:24:37 UTC
@Sebastien Pouliot:

We already knew this and our whole certificate chain uses SHA-256 certificates.

The issue is also reproducible by doing a GetAsync to https://www.nytimes.com.

You'll get a trust failure if install  the Comodo RSA Certification Authority Certificate manually.
Which can be found here:

https://support.comodo.com/index.php?/Knowledgebase/Article/View/969/108/root-comodo-rsa-certification-authority-sha-2
Comment 16 koenhendriks89 2017-09-25 06:33:48 UTC
@Sebastien Pouliot, @Timothy Risi: Are you guys still working on the issue, or should we create a new ticket?
Comment 17 Vincent Dondain [MSFT] 2017-10-03 18:34:34 UTC
*** Bug 59914 has been marked as a duplicate of this bug. ***
Comment 18 Vincent Dondain [MSFT] 2017-10-04 15:16:30 UTC
*** Bug 59960 has been marked as a duplicate of this bug. ***
Comment 19 Hugo Logmans 2017-10-05 06:32:34 UTC
FOUND THE BUG:

https://github.com/xamarin/xamarin-macios/blob/d1361d963e86796eaa864fda0c16449b35a037d1/src/Security/Tls/AppleCertificateHelper.cs

Line 95. The result SecTrustResult.Proceed is also a valid answer, and is used in the case of the custom installed root.
Comment 20 Rolf Bjarne Kvinge [MSFT] 2017-10-05 09:52:03 UTC
Yeah, I found the same thing last night too.

I'll prepare a patch.
Comment 21 Rolf Bjarne Kvinge [MSFT] 2017-10-05 10:28:35 UTC
mono/master: https://github.com/mono/mono/pull/5703
xamarin-macios/master: https://github.com/xamarin/xamarin-macios/pull/2844
Comment 22 Jeremy Sinclair 2017-10-05 13:35:49 UTC
I've had this EXACT issue (managed profile with root cert pulled down on device via MDM causing TrustError). 

ModernHttpClient seemed to work for me, using the NativeHttpClientHandler that uses NSUrlSession stack underneath. The awkward thing I've been seeing (without using ModernHttpClient) is that I've encountered the TrustError even when selecting NSUrlSession as the HttpClient implementation within my project settings. 

I can reproduce this if I am passing in an HttpClientHandler to an HttpClient object. As soon as I rip out the HttpClientHandler and use the default HttpClient ctor, the connection goes through. All I'm doing is setting the AutomaticDecompression property in the handler.
Comment 23 GouriKumari 2017-10-05 20:42:30 UTC
This issue is fixed with fix provided in comment #21. Please see the verification step, https://bugzilla.xamarin.com/show_bug.cgi?id=59914#c3. Marking this bug as resolved fixed. 

## Test Env:
XI 11.2.0.10
iPhone6S iOS 11
Comment 24 Rolf Bjarne Kvinge [MSFT] 2017-10-06 05:49:13 UTC
If everything goes according to plan, this will be included in d15-4.

PRs for tracking purposes:

mono/2017-06: https://github.com/mono/mono/pull/5719
mono/2017-08: https://github.com/mono/mono/pull/5718
mono/2017-10: https://github.com/mono/mono/pull/5717
mono/d15-4-2017-04: https://github.com/mono/mono/pull/5712

xamarin-macios/d15-4-xi: https://github.com/xamarin/xamarin-macios/pull/2850
xamarin-macios/xcode9.1: https://github.com/xamarin/xamarin-macios/pull/2851 and https://github.com/xamarin/xamarin-macios/pull/2847

still pending: xamarin-macios/d15-5 and xamarin-macios/master (mono/2017-06's PR has to be merged first)
Comment 26 Rolf Bjarne Kvinge [MSFT] 2017-10-06 10:42:29 UTC
Last PRs:

xamarin-macios/d15-5: https://github.com/xamarin/xamarin-macios/pull/2855
xamarin-macios/master: https://github.com/xamarin/xamarin-macios/pull/2856
Comment 27 Hugo Logmans 2017-10-10 12:13:52 UTC
@Jeremy, when adding a parameter to the HttpClient constructor, you override the project setting which handler you use. With new  HttpClient(new HttpClientHandler()) you setup a managed stack, ignoring whatever you set in the project settings.
Comment 28 User20984 2017-10-16 18:35:33 UTC
Is there something I can do as a workaround for this, or is there a beta with this patch in it or anything?
Comment 29 Rolf Bjarne Kvinge [MSFT] 2017-10-17 09:20:52 UTC
@Kyle, this fix is included in the current stable release of Xamarin.iOS.
Comment 30 Plac3hold3r+github 2017-11-07 15:52:06 UTC
*** Bug 59942 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.