Bug 2098 - Mono JVM 4x slower than Java / C++ and "unsafe" code not showing perf improvement
Summary: Mono JVM 4x slower than Java / C++ and "unsafe" code not showing perf improve...
Alias: None
Product: Runtime
Classification: Mono
Component: JIT ()
Version: unspecified
Hardware: All All
: --- normal
Target Milestone: ---
Assignee: Bugzilla
Depends on:
Reported: 2011-11-20 07:53 UTC by Jonathan Shore
Modified: 2017-07-11 21:20 UTC (History)
4 users (show)

Is this bug a regression?: ---
Last known good build:

project with c#, C++, and java code (32.47 KB, application/x-gzip)
2011-11-20 07:53 UTC, Jonathan Shore

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Jonathan Shore 2011-11-20 07:53:36 UTC
Created attachment 912 [details]
project with c#, C++, and java code

Description of Problem:
A simple calculation on an array is 4x slower than Java, MS .NET, or C++.  I have tried this as a safe and unsafe implementation with little difference.   Have tested on OSX Lion, Linux (Ubuntu 11.10), and evaluated on windows (Just C++ and MS .NET).

The MS .NET unsafe version is close to C++ speeds and 2x as fast as the unsafe version on the MS VM.  The Mono versions of safe and unsafe are only marginally different and still 3-4x slower in performance.

The OLS code is simple, so hopefully easy to diagnose.   Can strip out the matrix tests if you like.

Steps to reproduce the problem:
1. Code project included with csharp, cpp, and java directories
2. Compile to optimised 
3. Observe the timing of the OLS performance (C++ vs Java vs Mono C#)
4. Can also look at MS .NET if one likes for comparison

Actual Results:

Mono implementation has 4x slower runtimes.  The unsafe and safe versions both gave similar runtimes, with the unsafe a bit faster.

Expected Results:

Safe will probably be about 2x slower than java/c++ and unsafe should be only marginally slower than C# / java if MS .NET is used as a comparison.

How often does this happen? 

Additional Information:

See attached project
Comment 1 Jonathan Shore 2011-11-20 08:10:46 UTC
Here are the safe and unsafe runtimes of the OLS test on linux (mono 2.10..6) with llvm (similar results without):

safe ols: elapsed: 00:00:23.717
unsafe ols: elapsed: 00:00:21.930

The C++ version:

OLS: elapsed: 00:06:544
Comment 2 Zoltan Varga 2011-11-20 08:55:40 UTC
Your mono version is probably not compiled with llvm enabled. You can check using:
mono --stats --llvm <app> | grep LLVM

Here are the results on my machine:
normal mono JIT: safe ols: 1414797.29949947, elapsed: 00:00:28.0607354
mono LLVM: safe ols: 1414797.29949947, elapsed: 00:00:11.9290159
g++ -O3: OLS: 1414797.299499, elapsed: 00:07:207
Comment 3 Jonathan Shore 2011-11-20 10:25:00 UTC
Thanks, you are correct.  I thought my build included LLVM, but the build script seems to have excluded the option.  Will check the performance once the build completes. Expect it will be as fast as what you reported.
Comment 4 Jonathan Shore 2011-11-20 15:00:33 UTC
I ran this with LLVM and obtained much better results.  Consider this closed.   I am not sure why the unsafe implementation is no faster (and in fact slower) than the safe version though.
Comment 5 Thomas Goldstein 2011-11-20 18:08:03 UTC
Isn't this bug still valid?
LLVM is one thing, but don't we want to optimize the default JIT?

Unless there are plans to make LLVM the default, and get rid of the current default JIT.
Comment 6 Jonathan Shore 2011-11-20 18:46:16 UTC
Woulld like to see either mono JIT reach LLVM level of performance or LLVM become fully supported / default.

Also, why is the unsafe code slower than the safe code?   Is there some additional overhead with pinning memory? or perhaps unsafe not passed through the same optimisation paths?
Comment 7 Zoltan Varga 2011-11-20 20:02:08 UTC
The LLVM backend was developed for the express purpose of achieving native or close to native performance without having to do a ton of work on the current JIT.
This means that we do not plan on doing major JIT work in the near future.
LLVM is fully supported, it is just a bit hard to build the two together, but our released packages usually had LLVM support.
Comment 8 Jonathan Shore 2011-11-20 21:21:07 UTC
Thanks.  Can we get make LLVM a default in the builds now.  I've had a very hard time building mono with LLVM manually for linux, though have succeeded.   If made a default, builds for distributions will likely contain LLVM support such that application developers will not need to do their own builds.

I have not been able to do a successful build of mono on OSX, much less with LLVM.   Having problems with libiconv.   If we are expected to do our own builds for LLVM would be nice to get a recipe for building on various platforms.  I have been able to build on linux however.
Comment 9 Rodrigo Kumpera 2017-07-11 21:20:00 UTC
Mono now ships with LLVM on OSX.