Bug 28694 - Crash in cert chain validation
Summary: Crash in cert chain validation
Status: NEW
Alias: None
Product: Class Libraries
Classification: Mono
Component: System (show other bugs)
Version: 3.12.0
Hardware: PC Mac OS
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2015-04-02 11:58 UTC by David Fowler
Modified: 2015-12-04 04:09 UTC (History)
5 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments

Description David Fowler 2015-04-02 11:58:29 UTC
info: Exec
info:   program: .nuget/nuget.exe
info:   commandline: install "CoreCLR" -OutputDirectory "packages" -pre -nocache
info:   workingdir: /home/parallels/dev/git/dnx
ERROR building certificate chain: System.NullReferenceException: Object reference not set to an instance of an object
  at System.Security.Cryptography.X509Certificates.X509Chain.get_Roots () [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Chain.get_CertificateCollection () [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Chain.FindParent (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Chain.BuildChainFrom (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Chain.Build (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00000] in <filename unknown>:0 
  at System.Net.ServicePointManager+ChainValidationHelper.ValidateChain (Mono.Security.X509.X509CertificateCollection certs) [0x00000] in <filename unknown>:0 
Please, report this problem to the Mono team
ERROR processing certificate: System.NullReferenceException: Object reference not set to an instance of an object
  at System.Net.ServicePointManager+ChainValidationHelper.CheckCertificateUsage (System.Security.Cryptography.X509Certificates.X509Certificate2 cert) [0x00000] in <filename unknown>:0 
Please, report this problem to the Mono team
Comment 1 Miguel de Icaza [MSFT] 2015-04-10 22:20:33 UTC
David, how can I reproduce this issue?

Please provide instructions, the above does not tell me much.

Where do I run nuget from?
Comment 2 David Fowler 2015-04-11 04:15:11 UTC
Run build.sh on this repository:

https://github.com/aspnet/dnx

It should reproduce the failure at some point. To avoid using the cache, you can:

rm -rf ~/.local/share/dnu/cache/
./build.sh
Comment 3 Miguel de Icaza [MSFT] 2015-04-16 12:35:11 UTC
Bummer, still not able to reproduce this issue on Mac with 3.12

David are you doing this on Mac or Linux?   

I have now been trying with Mono/master and Mono/mono-4.0.0 on Ubuntu/64 with no luck.
Comment 4 David Fowler 2015-04-17 19:40:11 UTC
On ubuntu with 3.12.1 it repros all of the time.
Comment 5 Miguel de Icaza [MSFT] 2015-04-17 20:27:59 UTC
David, can you share your Ubuntu configuration?

I am running 14.04, with 3.12.1, mono-4.0 and mono-master, and in all cases, I could not reproduce it.   I have the following script in a loop running for the last 2 days:

while true; do rm -rf ~/.local/share/dnu/cache/; ./build.sh; done >& logs

And have not seen those errors (I saw others, every few hours Roslyn aborts with some parser error), on some temporary file that is deleted by the time of the failure.
Comment 6 Miguel de Icaza [MSFT] 2015-04-17 20:29:16 UTC
Info that I would love:

* Ubuntu version
* Host version (if virtualized, VM, what kind of VM, VM configuration, plus #CPUs, memory, swap size)
* Source of your Mono installation (official packages, unofficial packages, compiled on your own, 32 or 64 bits, flags displayed by mono --version)
Comment 7 David Fowler 2015-04-18 18:31:34 UTC
> Ubuntu Version
14.04

Host:
MacBook Pro (Retina, 13-inch, Early 2013)
3 GHz Intel Core i7
8 GB 1600 MHz DDR3
Intel HD Graphics 4000 1024 MB
OSX Yosemite

VM
Parallels, 
CPUs: 2
Memory: 1GB


parallels@ubuntu:~/Desktop/Parallels Shared Folders/Home/dev/git/ProjectK$ mono --version
Mono JIT compiler version 3.12.1 (tarball Fri Mar  6 19:12:47 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	LLVM:          supported, not enabled.
	GC:            sgen

From: 

http://www.mono-project.com/docs/getting-started/install/linux/#debian-ubuntu-and-derivatives
Comment 8 David Fowler 2015-04-18 21:48:59 UTC
Maybe we just need a better repro. I ran into it again:

Restoring packages for /home/parallels/dev/git/Home/samples/HelloWeb/project.json
  GET https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Kestrel'.
  GET https://nuget.org/api/v2/FindPackagesById()?Id='Kestrel'.
  GET https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Diagnostics'.
  GET https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Diagnostics'.
  GET https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Hosting'.
  GET https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Hosting'.
  GET https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.IIS'.
  GET https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.IIS'.
  GET https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.WebListener'.
  GET https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.WebListener'.
  GET https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.StaticFiles'.
  GET https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.StaticFiles'.
ERROR building certificate chain: System.NullReferenceException: Object reference not set to an instance of an object
  at System.String.FormatHelper (System.Text.StringBuilder result, IFormatProvider provider, System.String format, System.Object[] args) [0x00000] in <filename unknown>:0 
  at System.String.Format (IFormatProvider provider, System.String format, System.Object[] args) [0x00000] in <filename unknown>:0 
  at System.String.Format (System.String format, System.Object[] args) [0x00000] in <filename unknown>:0 
  at Locale.GetText (System.String fmt, System.Object[] args) [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Certificate2Collection.Find (X509FindType findType, System.Object findValue, Boolean validOnly) [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Chain.FindParent (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Chain.BuildChainFrom (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00000] in <filename unknown>:0 
  at System.Security.Cryptography.X509Certificates.X509Chain.Build (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00000] in <filename unknown>:0 
  at System.Net.ServicePointManager+ChainValidationHelper.ValidateChain (Mono.Security.X509.X509CertificateCollection certs) [0x00000] in <filename unknown>:0 
Please, report this problem to the Mono team
ERROR processing certificate: System.Security.Cryptography.CryptographicException: Certificate instance is empty.
  at System.Security.Cryptography.X509Certificates.X509Certificate2.get_Version () [0x00000] in <filename unknown>:0 
  at System.Net.ServicePointManager+ChainValidationHelper.CheckCertificateUsage (System.Security.Cryptography.X509Certificates.X509Certificate2 cert) [0x00000] in <filename unknown>:0 
Please, report this problem to the Mono team
Warning: FindPackagesById: Microsoft.AspNet.Diagnostics
  Error: SendFailure (Error writing headers)
  GET https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Diagnostics'.
  OK https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Diagnostics' 2813ms
  OK https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Hosting' 2860ms
  OK https://nuget.org/api/v2/FindPackagesById()?Id='Kestrel' 2924ms
  OK https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.IIS' 8518ms
  OK https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.StaticFiles' 9018ms
  OK https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Kestrel' 9598ms
  OK https://nuget.org/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.WebListener' 10020ms
  OK https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Hosting' 10523ms
  OK https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.IIS' 11029ms
  OK https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.StaticFiles' 11525ms
  OK https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Diagnostics' 15771ms
  OK https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?Id='Microsoft.AspNet.Server.WebListener' 18029ms
Comment 9 James Humphries 2015-10-13 08:00:40 UTC
I seem to be coming across this issue too when creating lots of https requests.

Here is the stacktrace I have got: 

ERROR building certificate chain: System.NullReferenceException: Object reference not set to an instance of an object
  at System.Security.Cryptography.X509Certificates.X509Certificate2Collection.AddRange (System.Security.Cryptography.X509Certificates.X509Certificate2Collection certificates) [0x00011] in /tmp/buildd/mono-3.12.1/mcs/class/System/System.Security.Cryptography.X509Certificates/X509Certificate2Collection.cs:101 
  at System.Security.Cryptography.X509Certificates.X509Chain.get_Roots () [0x00035] in /tmp/buildd/mono-3.12.1/mcs/class/System/System.Security.Cryptography.X509Certificates/X509Chain.cs:264 
  at System.Security.Cryptography.X509Certificates.X509Chain.BuildChainFrom (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00055] in /tmp/buildd/mono-3.12.1/mcs/class/System/System.Security.Cryptography.X509Certificates/X509Chain.cs:379 
  at System.Security.Cryptography.X509Certificates.X509Chain.Build (System.Security.Cryptography.X509Certificates.X509Certificate2 certificate) [0x00017] in /tmp/buildd/mono-3.12.1/mcs/class/System/System.Security.Cryptography.X509Certificates/X509Chain.cs:119 
  at System.Net.ServicePointManager+ChainValidationHelper.ValidateChain (Mono.Security.X509.X509CertificateCollection certs) [0x0009f] in /tmp/buildd/mono-3.12.1/mcs/class/System/System.Net/ServicePointManager.cs:457 
Please, report this problem to the Mono team
ERROR processing certificate: System.Security.Cryptography.CryptographicException: Certificate instance is empty.
  at System.Security.Cryptography.X509Certificates.X509Certificate2.get_Version () [0x00016] in /tmp/buildd/mono-3.12.1/mcs/class/System/System.Security.Cryptography.X509Certificates/X509Certificate2.cs:332 
  at System.Net.ServicePointManager+ChainValidationHelper.CheckCertificateUsage (System.Security.Cryptography.X509Certificates.X509Certificate2 cert) [0x00000] in /tmp/buildd/mono-3.12.1/mcs/class/System/System.Net/ServicePointManager.cs:618 
Please, report this problem to the Mono team
Comment 10 Miguel de Icaza [MSFT] 2015-10-13 08:10:02 UTC
James, we would love to have a test case for this.

Also, we would love if you used something newer than 3.12, which is nine months old.
Comment 11 Miguel de Icaza [MSFT] 2015-10-13 08:10:09 UTC
James, we would love to have a test case for this.

Also, we would love if you used something newer than 3.12, which is nine months old.
Comment 12 James Humphries 2015-10-13 08:16:45 UTC
I'm working on reproducing this now reliably without our huge application. I can't reproduce it on osx, however inside my docker image I can reproduce this (or a sigsegv error) in mono 3.12, 4.0.4 and 4.2

James
Comment 15 jamie_venning 2015-12-04 04:09:41 UTC
If you're getting segfaults on a web application, I would suggest looking at https://bugzilla.xamarin.com/show_bug.cgi?id=29212. Worked for me. (Upgrade your kernel)

Note You need to log in before you can comment on or make changes to this bug.