Bug 37822 - Repeated array allocation fails with certain code paths
Summary: Repeated array allocation fails with certain code paths
Status: NEEDINFO
Alias: None
Product: Runtime
Classification: Mono
Component: GC (show other bugs)
Version: 4.2.0 (C6)
Hardware: PC Windows
: --- normal
Target Milestone: ---
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2016-01-20 00:28 UTC by secondary.programmer
Modified: 2017-09-06 13:53 UTC (History)
5 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
source code to exhaust memory. (1.75 KB, text/plain)
2016-01-20 00:28 UTC, secondary.programmer
Details
stdout for crash (86 bytes, text/plain)
2016-02-12 19:37 UTC, secondary.programmer
Details
stderror for crash (4.50 KB, text/plain)
2016-02-12 19:38 UTC, secondary.programmer
Details
stdout for crash on Mono 4.6.1 (995 bytes, text/plain)
2016-11-12 16:55 UTC, secondary.programmer
Details
stderr for crash on Mono 4.6.1 (569 bytes, text/plain)
2016-11-12 16:56 UTC, secondary.programmer
Details
stderr for crash on Mono 4.6.1 with GC_DEBUG=2 (66.51 KB, text/plain)
2016-11-12 17:07 UTC, secondary.programmer
Details

Description secondary.programmer 2016-01-20 00:28:28 UTC
Created attachment 14635 [details]
source code to exhaust memory.

The attached code throws:

OutOfMemoryException
[ERROR] FATAL UNHANDLED EXCEPTION: System.OutOfMemoryException: Out of memory
  at (wrapper managed-to-native) System.Object:__icall_wrapper_mono_gc_alloc_vector (intptr,intptr,intptr)
  at (wrapper alloc) System.Object:AllocVector (intptr,intptr)
  at Microsoft.FSharp.Collections.ArrayModule.ZeroCreate[T] (Int32 count) <0x3c214e8 + 0x0001f> in <filename unknown>:0
  at Program.crash () <0x3c21328 + 0x00097> in <filename unknown>:0
  at Program.main (System.String[] _arg1) <0x3c20f70 + 0x0005b> in <filename unknown>:0

Under mono 
Mono JIT compiler version 4.2.1 (Visual Studio built mono)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           normal
        SIGSEGV:       normal
        Notification:  Thread + polling
        Architecture:  x86
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

With default MS.net it runs to completion.

Code is compiled with Visual Studios 2015 Community Edition, and the only difference between code that throws an exception versus code that does not throw an exception is a secondary array reference versus a mutable type

Code: (use the crash argument to crash the program, and dontcrash to not crash the program)

let dontcrash () =
  let mutable prev : byte[] = Array.zeroCreate 250000000
  let mutable arr : byte[] = Array.zeroCreate 0

  for i in 0..255 do
    System.Console.WriteLine(i)
    do arr <- Array.zeroCreate (250000000 + i)
    if i % 2 = 0 then
      for j = 0 to arr.Length - 1 do arr.[j] <- byte j
    else
      for j = 0 to prev.Length - 1 do arr.[j] <- prev.[j]
      for j = prev.Length - 1 to arr.Length - 1 do arr.[j] <- byte j
    prev <- arr


  0 // return an integer exit code

let crash () =
  let prev : byte[] ref = ref <| Array.zeroCreate 250000000
  let mutable arr : byte[] = Array.zeroCreate 0

  for i in 0..255 do
    System.Console.WriteLine(i)
    let prev' = !prev
    do arr <- Array.zeroCreate (250000000 + i)
    if i % 2 = 0 then
      for j = 0 to arr.Length - 1 do arr.[j] <- byte j
    else
      for j = 0 to prev'.Length - 1 do arr.[j] <- prev'.[j]
      for j = prev'.Length - 1 to arr.Length - 1 do arr.[j] <- byte j
    prev := arr


  0 // return an integer exit code

[<EntryPoint>]
let main = 
  function
  | [|"crash"|] -> crash()
  | [|"dontcrash"|] -> dontcrash()
  | _ -> printfn "options : crash | dontcrash" ; 1
Comment 1 secondary.programmer 2016-02-05 01:13:36 UTC
In 4.2.2.30 both code paths now crash with an OutOfMemoryException.
Comment 2 Andi McClure 2016-02-08 21:07:52 UTC
I built the project using fsc 4.0 as distributed with Xamarin Studio. I then tested with 4.3.0 on x86, and current mono master (4.3.2-ish) on x86_64, which were the two versions of mono I had handy. Both versions ran to completion with the "crash" argument.
Comment 3 Andi McClure 2016-02-08 21:35:47 UTC
Tested with a locally built 4.2.2.30; also succeeded. Note, all my tests are on mac; I think Windows has a lower memory limit for 32-bit processes. I did notice memory usage was a lot higher on 4.2.2.30 than 4.3.x.

Secondary.programmer: I'm so far not able to reproduce this. Before I try to reproduce your exact setup, could you run this test with the environment variable MONO_GC_DEBUG set to 2, and send me the output of the program with that flag set? Setting the flag should result in many extra messages in stdout such as "Start major collection" and "Heap size:". You can assign the bug back to me after that. Thanks
Comment 4 secondary.programmer 2016-02-12 19:37:04 UTC
Created attachment 15026 [details]
stdout for crash

stdout for 
mono CrashMono.exe crash 1>stdout 2>stderror

while 
set MONO_GC_DEBUG=2
Comment 5 secondary.programmer 2016-02-12 19:38:27 UTC
Created attachment 15027 [details]
stderror for crash

produced via the following commands
set MONO_GC_DEBUG=2
mono CrashMono.exe crash 1>stdout 2>stderror
Comment 6 Andi McClure 2016-02-25 20:38:15 UTC
secondary.programmer, i have this on my radar, but-- we have just released a Mono version 4.4. Is it possible you could rerun your tests with 4.4? I do see different memory behavior in our 4.4 branch.
Comment 7 Vlad Brezae 2016-11-08 01:22:28 UTC
The only thing I can reproduce about this bug is an increased memory consumption of 250MB for the crash case, which I assume is leaving the system with not enough memory.

This difference between the tests comes from the fact that the first large array remains pinned on the stack in the crash test (even though the object is logically dead the GC can do nothing about it since there is a reference on the stack). Objects allocated inside a function are likely to remain alive all the way to the function end since references to those objects can exist in stack slots of that function. In general you should not rely on an object's immediate death after its usage stops, especially with large objects, which are expected to be long lived.

Let me know if you have any questions.
Comment 8 secondary.programmer 2016-11-12 16:55:50 UTC
Created attachment 18440 [details]
stdout for crash on Mono 4.6.1
Comment 9 secondary.programmer 2016-11-12 16:56:18 UTC
Created attachment 18441 [details]
stderr for crash on Mono 4.6.1
Comment 10 secondary.programmer 2016-11-12 17:07:39 UTC
Created attachment 18442 [details]
stderr for crash on Mono 4.6.1 with GC_DEBUG=2

So with mono 4.6.1 (32-bit) the program now crashes with a null reference exception.  The program does not crash under native windows .net or dotnet core.

I have mitigated these problems within my code base by refactoring several libraries to compact data structures or reuse arrays to prevent stressing the GC.

Note You need to log in before you can comment on or make changes to this bug.