Bug 57962

Summary: Mono 5.0 regression with IronPython 2.7.7
Product: [Mono] Class Libraries Reporter: Alex Earl <slide.o.mix>
Component: System.CoreAssignee: Alexander Köplinger [MSFT] <alkpli>
Status: IN_PROGRESS ---    
Severity: major CC: alkpli, bernhard.urban, github.com, ludovic, masafa, mono-bugs+mono, mono-bugs+runtime, slide.o.mix
Priority: ---    
Version: 5.4 (2017-06)   
Target Milestone: Future Release   
Hardware: PC   
OS: All   
Tags: bugpool Is this bug a regression?: Yes
Last known good build:
Attachments: Log from user

Description Alex Earl 2017-07-05 22:39:20 UTC
Created attachment 23314 [details]
Log from user

1) Download https://github.com/IronLanguages/main/releases/download/ipy-2.7.7/IronPython-2.7.7-win.zip
2) unzip
3) mono --trace ./ipy.exe

Expected (and behavior under 4.8): an IronPython REPL will start
Actual: stackoverflow/core dump

The exact same binaries will start a REPL on 4.8, but have a stackoverflow from some weird recursive call. This is a regression in mono 5.0.

Also, see https://github.com/IronLanguages/ironpython2/issues/37 for the issue reported by a user.
Comment 1 Ludovic Henry 2017-07-05 23:56:22 UTC
Hello,

I can reproduce with the information you provided. The change in Mono is we switched the System.Linq.Expressions implementation from ReferenceSource to CoreFX.

Thank you,
Ludovic
Comment 2 Bernhard Urban 2017-09-14 19:51:28 UTC
From a conversation with Alex:

> So, for this issue, the interesting thing is if I BUILD the code using
> the newer mono version, I don't see the issue, it's only if I run a
> version of IPy that was built with 4.8 that the issue arises

is that something supposed to work? Also:

> It works fine to build something on 5.x and then run under 4.8

So the other way around works.
Comment 3 Bernhard Urban 2017-09-19 08:04:03 UTC
from a conversation with Marek:

> Marek Safar [23:04]:
> @lewurm interesting, this is the first time we have this kind of issue
> it’s supported unless it’s a bug which in this case would probably have to mcs bug or some really obscure bcl hole
Comment 4 Marek Safar 2017-10-02 14:25:32 UTC
I can still reproduce it with Mono 5.4. Running bundled ipy (which is what we build and distribute) fails for me

$ ipy --version
Unhandled exception:
Traceback (most recent call last):
TypeError: Value cannot be null.
Parameter name: method
Comment 5 Alex Earl 2017-10-02 15:56:01 UTC
Does mono build the latest ipy? The repository changed to IronLanguages/ironpython2 instead of IronLanguages/main, so just want to make sure that you are grabbing the correct code for distribution.
Comment 6 Marek Safar 2017-10-02 16:41:47 UTC
We are building it using our script at https://github.com/mono/mono/blob/master/packaging/MacSDK/ironlangs.py
Comment 7 Alex Earl 2017-10-02 16:46:36 UTC
That looks like an incredibly old version to me. IronRuby is no longer supported at all and there is no longer an IronPython.Mono.sln. Is this only distributed on macOS?
Comment 8 Alexander Köplinger [MSFT] 2018-01-09 12:23:50 UTC
@Alex Earl: yes we bundle pretty ancient versions and we should update them at some point. They are only distributed on macOS.

I'll take a look at the issue, an error still happens with Mono 5.11 as mentioned in Comment 4, even with the downloaded IronPython-2.7.7-win.zip
Comment 9 Alexander Köplinger [MSFT] 2018-01-09 15:37:54 UTC
Preliminary update:

First of all the originally reported stackoverflow/runtime crash seems to be fixed, I can't repro it anymore with Mono 5.4

Going further, in the bundled ipy wrapper script we do the following:

> export IRONPYTHONPATH=/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
> exec /Library/Frameworks/Mono.framework/Versions/5.4.1/bin/mono /Library/Frameworks/Mono.framework/Versions/5.4.1/lib/ironpython/ipy.exe "$@"

This was done by https://github.com/mono/bockbuild/commit/f4c0f248ed6b7ef1b0c905cba69fad28aec1797a so we get access to the "os" etc modules.

If I remove the IRONPYTHONPATH env var and just run "mono /path/to/ipy.exe" then it starts the REPL normally.

I then remembered that we switched the main "mono" binary from 32bit to 64bit in Mono 5.2. Indeed if I run "mono32 ipy.exe" _with_ the IRONPYTHONPATH env var set then it starts the REPL normally, but still prints an error:

> Traceback (most recent call last):
>   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 62, in <module>
>   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 49, in <module>
>   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 17, in <module>
>   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/warnings.py", line 8, in <module>
> SystemError: Object reference not set to an instance of an objectIronPython 3.0 (3.0.0.0) on .NET 4.0.30319.42000

My current (unverified) assumption is that it tries to load some OS library and falls over the 32/64bit trap. A quick test using ipy64.exe resulted in the initial error from Comment 4.

Next I tried using the binaries in IronPython-2.7.7-win.zip from Comment 1. I see the same behavior there ("works" with mono32), except that IRONPYTHONPATH env var isn't set but it loads modules from the Lib/ subfolder. If I rename the folder then the REPL boots with the python traceback from before.

I assume it is the same underlying issue in both IronPython-2.7.7 and the old one we bundle in Mono. I'll try building IronPython now and step through the code :)
Comment 10 Alex Earl 2018-01-09 16:07:37 UTC
Make sure you use the correct repo, https://github.com/IronLanguages/ironpython2.
Comment 11 Alexander Köplinger [MSFT] 2018-01-09 16:11:36 UTC
@Alex Earl: I'm using ipy-2.7-maint branch in https://github.com/IronLanguages/main since that seems to be where https://github.com/IronLanguages/main/releases/download/ipy-2.7.7/IronPython-2.7.7-win.zip was produced from. Do you think this would be fixed in current https://github.com/IronLanguages/ironpython2 code?
Comment 12 Alex Earl 2018-01-09 16:31:09 UTC
No, not necessarily, we still see some issues on the latest code, but this specific issue was about the 2.7.7 release that someone was trying to run, so the branch you specify is probably correct.