Bug 23242 - Null reference exception occurs after the call to Int.ToString() from multiple threads
Summary: Null reference exception occurs after the call to Int.ToString() from multipl...
Status: VERIFIED FIXED
Alias: None
Product: Installers
Classification: Mono
Component: General (show other bugs)
Version: unspecified
Hardware: PC All
: --- normal
Target Milestone: 3.12.0
Assignee: Bugzilla
URL:
: 4629 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-09-22 09:09 UTC by Mike
Modified: 2015-01-07 11:13 UTC (History)
10 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Source code for issue reproduction (693 bytes, application/octet-stream)
2014-09-22 09:10 UTC, Mike
Details
Debug executable reproducing the issue (4.00 KB, application/octet-stream)
2014-09-22 10:26 UTC, Mike
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
VERIFIED FIXED

Description Mike 2014-09-22 09:09:34 UTC
The mono runtime is compiled from recent 'master' branch on github -i.e. Mono JIT compiler version 3.10
The system is
Linux 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
running on  Intel(R) Xeon(R) CPU E5-2680 with 32 hw threads support

In order to reproduce the issue you need to compile the attached source and execute it
/opt/mono/bin/mono-sgen --server ./getset.exe

In my case the app instantly crashes with the following stack trace:

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.Int16.ToString () [0x00000] in <filename unknown>:0 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0 

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.Int16.ToString () [0x00000] in <filename unknown>:0 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0 
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.Int16.ToString () [0x00000] in <filename unknown>:0 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0 
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.Int16.ToString () [0x00000] in <filename unknown>:0 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0 

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.Int16.ToString () [0x00000] in <filename unknown>:0 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
Comment 1 Mike 2014-09-22 09:10:12 UTC
Created attachment 8143 [details]
Source code for issue reproduction
Comment 2 Mike 2014-09-22 10:05:11 UTC
Same issue on Mono JIT compiler version 3.10.1 (master/5f9c74f Mon Sep 22 14:03:21 UTC 2014)
(latest master from github)
running on 4-core vm
Linux ubuntu 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Comment 3 Mike 2014-09-22 10:26:38 UTC
Created attachment 8145 [details]
Debug executable reproducing the issue

Maybe the problem is related to IL code generation
This binary is build on
os x 10.10 (14A361p) (beta 3)
with mono runtime
Mono JIT compiler version 3.8.0 ((no/45d0ba1 Tue Aug 26 20:33:43 EDT 2014)
Comment 4 Rodrigo Kumpera 2014-09-22 10:49:36 UTC
Can you run it with --debug?
Comment 5 Mike 2014-09-22 11:17:02 UTC
Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00007] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:554 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00032] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:1019 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00026] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:945 
  at System.Int16.ToString () [0x00000] in /root/mono_off/mcs/class/corlib/System/Int16.cs:228 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00016] in /root/mono_off/mcs/class/corlib/System.Threading/Thread.cs:691 
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00007] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:554 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00032] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:1019 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00026] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:945 
  at System.Int16.ToString () [0x00000] in /root/mono_off/mcs/class/corlib/System/Int16.cs:228 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00016] in /root/mono_off/mcs/class/corlib/System.Threading/Thread.cs:691 
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00007] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:554 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00032] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:1019 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00026] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:945 
  at System.Int16.ToString () [0x00000] in /root/mono_off/mcs/class/corlib/System/Int16.cs:228 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00016] in /root/mono_off/mcs/class/corlib/System.Threading/Thread.cs:691 
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
  at System.NumberFormatter.ResetCharBuf (Int32 size) [0x00007] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:554 
  at System.NumberFormatter.FastIntegerToString (Int32 value, IFormatProvider fp) [0x00032] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:1019 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00026] in /root/mono_off/mcs/class/corlib/System/NumberFormatter.cs:945 
  at System.Int16.ToString () [0x00000] in /root/mono_off/mcs/class/corlib/System/Int16.cs:228 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00016] in /root/mono_off/mcs/class/corlib/System.Threading/Thread.cs:691
Comment 6 Mike 2014-09-22 11:17:39 UTC
Same issue while running with --debug on Mono JIT compiler version 3.8.0 ((detached/e451fb2 Mon Sep 22 15:09:03 UTC 2014)
Comment 7 Mike 2014-09-23 10:54:01 UTC
I've provided the requested info.
Comment 8 Mike 2014-09-23 12:32:23 UTC
Well, I've found a workaround for this issue - the mono runtime option --debug=mdb-optimizations
effectively 'disables' the crash.
Comment 9 Zoltan Varga 2014-09-23 19:15:41 UTC
Try running with -O=-aot, that might work around the problem.
Comment 10 Mike 2014-09-24 05:46:33 UTC
In that case a different exception occurs in one of 20 runs of getset.exe
 at System.Globalization.TextInfo..ctor (System.Globalization.CultureInfo ci, Int32 lcid, System.Void* data, Boolean read_only) [0x00000] in <filename unknown>:0 
  at System.Globalization.CultureInfo.CreateTextInfo (Boolean readOnly) [0x00000] in <filename unknown>:0 
  at System.Globalization.CultureInfo.ConstructInvariant (Boolean read_only) [0x00000] in <filename unknown>:0 

Unhandled Exception:
System.TypeInitializationException: An exception was thrown by the type initializer for System.Globalization.CultureInfo
  at System.Threading.Thread.get_CurrentCulture () [0x00000] in <filename unknown>:0 
  at System.NumberFormatter..ctor (System.Threading.Thread current) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.GetInstance (IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.NumberFormatter.NumberToString (Int32 value, IFormatProvider fp) [0x00000] in <filename unknown>:0 
  at System.Int16.ToString () [0x00000] in <filename unknown>:0 
  at getset.MainClass+<Main>c__AnonStorey0.<>m__0 () [0x00000] in <filename unknown>:0 
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
Comment 11 Mike 2014-09-24 07:37:50 UTC
I've reproduced the issue on OS X
Xamarin Studio Version 5.5 (build 198)
Runtime:
	Mono 3.10.0 ((detached/ac51002)
	GTK+ 2.24.23 (Raleigh theme)

Build Information
Release ID: 505000198
Git revision: 7495942eb76d6b80c460ddd61f2b94cba1a97fa2
Build date: 2014-09-18 09:50:12-04
Xamarin addins: c571b625445d60f2c8b189b309a6ffc87386caed

Operating System
Mac OS X 10.10.0 14.0.0 Darwin Kernel Version 14.0.0
    Mon Sep  8 05:27:41 PDT 2014
    root:xnu-2782.1.96~5/RELEASE_X86_64 x86_64
Comment 13 Marek Safar 2014-09-26 05:07:54 UTC
I can reproduce the issue too and as Zoltan suggested -O=-aot workarounds the issue
Comment 14 Mike 2014-09-26 16:28:58 UTC
1. It doesn't workaround the issue - if you run the executable in a loop - using the simple bash loop construct you will see that the exception still occurs even with -O=-aot
Well, it occurs for example in 31 iteration after 30 successful runs - but it still not a workaround.
The static constructor of EmptyArray class solves the problem 100%
2. AOT is really not an option for our real application because it uses emit
Comment 15 Marek Safar 2014-09-27 03:02:17 UTC
I ran it for 10 minutes without single error with -0=-aot on

ono JIT compiler version 3.10.0 (mono-3.10.0-branch/491d1f5 Wed 17 Sep 2014 11:23:27 CEST)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           normal
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  x86
	Disabled:      none
	Misc:          softdebug
	LLVM:          supported, not enabled.
	GC:            sgen

static constructor for EmptyArray is wrong as we would have to fix all cases like this for any AOT-ed code including user code.
Comment 16 Zoltan Varga 2014-09-30 15:34:33 UTC
This is caused by the handling of got slots of type MONO_PATCH_INFO_SFLDA in the aot runtime.  Only the first thread which initializes the got slot waits for the type initializer to finish, the others don't.
Comment 17 Rolf Bjarne Kvinge [MSFT] 2014-11-17 10:32:22 UTC
I can repro this easily (50% of the time) with the initial test case.

> mono --version
Mono JIT compiler version 3.10.0 ((detached/633e444 Thu Oct  2 22:07:37 EDT 2014)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           normal
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  x86
	Disabled:      none
	Misc:          softdebug 
	LLVM:          yes(3.4svn-mono-(detached/e656cac)
	GC:            sgen

I can reproduce this with desktop mono, and there are users on the forums with released iOS apps running into it as well (http://forums.xamarin.com/discussion/comment/88543).
Comment 18 Zoltan Varga 2014-11-17 21:46:14 UTC
Fixed in mono master df8abf4920062fc93211ba2c1f65b77def5d9b1c and mono-3.12.0-branch f07e7d085010ef549df91611aaab029c244422d3. Thanks for the testcase.
Comment 21 Zoltan Varga 2014-12-09 07:57:09 UTC
It was a random failure, so it might not be reproducible to everyone. Doing 
mono --aot <path to mscorlib.dll> before running the test might make it more likely for the problem to occur.
Comment 22 Mohit Kheterpal 2014-12-09 08:21:17 UTC
As per comment 21, this issue is not easy to reproduce.

Hence, I am closing this issue by marking it as Verified.
Comment 23 Zoltan Varga 2015-01-07 11:13:29 UTC
*** Bug 4629 has been marked as a duplicate of this bug. ***