Created attachment 22096 [details]
Users are reporting much slower performance in apps that are using the Xamarin.Forms RelativeLayout simply after updating to Xamarin.Android 7.3. This is happening on all recent versions of Xamarin.Forms, and in one report a user is also experiencing this when using Prism with Unity.
I've attached a Xamarin.Forms Android project that navigates to a page that uses a RelativeLayout. I used a Stopwatch to measure how many milliseconds it took to navigate to the test page.
The time difference after updating to Xamarin.Android 7.3 is visibly noticeable when using the app. I ran the tests below on Visual Studio for Mac and used the Xamarin.Forms 184.108.40.206 nightly with XamlC enabled. The issue can also be seen with Xamarin.Forms 220.127.116.11 and 2.3.5-pre2.
Full debug outputs: https://gist.github.com/jimmgarrido/c34e5909985e9ff999055deb6d4a214e
# Steps to Reproduce
1. Run the attached Android project
2. Press "Go to Test Page"
# Test Results
## Android 18.104.22.168
## Mono 22.214.171.124
## Android 126.96.36.199
## Mono 4.8.1
Obvious question: what changed between XA 7.2 and 7.3?
Last URL is private, and is a whopping 119460 line diff, most of which is due to file renames and "reference assembly API descriptions":
> $ git diff d15-1..d15-2 --stat
> tools/api-diff/reference/Mono.Android.xml | 49433 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------
> 113 files changed, 66423 insertions(+), 15545 deletions(-)
The other change between XA 7.2 and 7.3 is mono, bumping from mono 4.8 to "mono 5.0" (mono/2017-02), which is *yuuge*:
Among the other changes between mono 4.8 and 2017-02 is that System.Linq.Expressions now comes from referencesource, which is implicated in the Xamarin Forums thread linked in Comment #0.
An "interesting" avenue of investigation would be to take xamarin-android and "bump" it to use mono-4.8.0-branch instead of mono/2017-02, and see if the performance issue remains.
Yeah, my money would be on mono, too. We don't have RelativeLayout at all, but see a performance degradation that is so bad that the app is unusable.
* Splash screen shows app quickly (Android activity)
* Then the screen turns black, wait of way over a minute (could be several) while the XF app is loading.
* When debugging, didn't see a single choke point, everything seems to take ages (before I attempt to even load the screen, such as IOC setups)
I started a thread on this on the Xamarin Android forum back when preview 1 was out. It might be of interest: https://forums.xamarin.com/discussion/94505/15-2-preview-1-performance-degradation
As I see it, the reason is a massive slowdown on compilation of expression trees.
Doing "poor man profilling" I found that the time was spend in:
I'm not so sure about Mono 5, since switching to the 4.8 runtime in Preferences in Xamarin.Studio does not alleviate the problem. Maybe others can confirm this hold true for them as well?
So I downloaded Jimmy Garrido sample to subject it to a bit of poor man's profilling, and I can confirm that I see the same stack traces:
1. Run the attached Android project
2. Press "Go to Test Page"
3. Break the debugger
4. Take note of the stack trace (you need to expand the hidden external part)
6. Repeat from 3 until Test page shows on screen.
@JonP I forked the xamarin-android repo and "bumped" Mono back to 4.8.0: https://github.com/jimmgarrido/xamarin-android/tree/downgrade-mono
I used my local version of X.A with the project already attached and the performance issue was still noticeable. There did not appear to be a performance difference between the publicly released X.A 7.3 and my local X.A 7.3 built with Mono 4.8.
I tested with X.A 188.8.131.52 as well which is the first build in the 15.2 release and this version _also_ experiences the issue. Hopefully this info will help narrow down what change may be causing this.
Also, I tried different Xamarin.Forms versions to verify this is not an issue on their end. All the most recent versions experienced this performance issue when using X.A 7.3:
Same with me.
Rolling back to 184.108.40.206 fixes the issue.
Any news? This basically breaks Android development for everybody.
I faced the same issue. Any updates?
As @JonP pointed out to me, I did not correctly downgrade Mono as mentioned in comment 5.
Here is a new branch that should have Mono downgraded this time: https://github.com/jimmgarrido/xamarin-android/tree/downgrade_mono
This new branch was created off the "d15-2" branch and I had to make additional changes for it to build successfully with Mono 4.8.0.
### Test Results
Testing with this version of X.A 7.3 built with Mono 4.8.0 _did_ appear to resolve the issue. The time to navigate to the test page in the repro project matched the times seen when testing with X.A 7.2. I am convinced now that this is an issue in the Mono source and not X.A.
@JimmyGariod & @JonP
Is it possible to "correctly" downgrade Mono to 4.8 without downgrading Xamarin.Android to 7.2 currently?
@Kasper: *Most* of Xamarin.Android doesn't "care" about which Mono version it's running on, and thus most of it could continue using Mono 4.8.
Unfortunately, "most" is not "all"; the primary change between Mono 4.8 and 5.0 is support for Portable PDB files, which spreads to various places (app packaging, deployment, on-device app startup...)
With Portable PDB now being "a thing," and a near certainty that more and more NuGet packages will be using and distributing Portable PDB files going forward, that's a *large* feature to be missing, unless you don't care about debugging at all. This largely means that Xamarin.Android *itself* reverting to Mono 4.8 is not considered to be viable.
Which is to say, if you needed/wanted to, you should be able to take @Jimmy's xamarin-android/downgrade_mono branch and use that to build your app. It just won't be "pretty".
We will have the class library team look into optimizing this code path.
Created attachment 22213 [details]
minimal repro project
I am attaching a new, minimal Android (non-Forms) project that reproduces the issue by executing an expression tree.
# Steps to Reproduce
1. Run the attached project on an Android device
2. Press "Click Me"
3. The debug output will show the milliseconds elapsed using a Stopwatch
# Test Results
## X.A 7.2 / Mono 4.8
- 210 ms
- 20 ms
- 20 ms
- 21 ms
- 20 ms
## X.A 7.3 / Mono 5.0
- 1627 ms
- 1412 ms
- 1421 ms
- 1416 ms
- 1421 ms
## X.A 7.3 / Mono 4.8
- 333 ms
- 29 ms
- 26 ms
- 30 ms
- 28 ms
*** Bug 56123 has been marked as a duplicate of this bug. ***
Apart from the current performance issue with Mono 5.0: Do you also have an idea about the 50% performance degradation you observed in X.A 7.3 / Mono 4.8 compared to 7.2?
I quickly tried the sample code from #c12 in a console app and I'm seeing better numbers in Mono 220.127.116.11 compared to 5.0.
How much better?
(I increased the counter to 20000)
OSX mono64 4.8.1: 1467ms
OSX mono64 18.104.22.168: 1547ms
OSX mono64 22.214.171.124: 1283ms
Linux mono64 126.96.36.1994: 1323ms
Linux mono64 188.8.131.52: 21254ms
Linux mono64 184.108.40.2062: 20625ms
Interestingly I don't see the huge performance degradation on OSX Mono 5.0, but it *is* there on Linux and is not yet fixed in later builds.
The Linux measurements were done using a Docker container on the same MacBook.
Created attachment 22217 [details]
The results below are from the console app I'm attaching. I don't have a Linux setup to compare with, but I am seeing a marginal difference on macOS as mentioned in comment 17.
## macOS Mono 5.0
Run 1: 1850 ms
Run 2: 1812 ms
Run 3: 1853 ms
Run 4: 1836 ms
Run 5: 1861 ms
## macOS Mono 4.8
Run 1: 1742 ms
Run 2: 1766 ms
Run 3: 1760 ms
Run 4: 1751 ms
Run 5: 1769 ms
*** Bug 56490 has been marked as a duplicate of this bug. ***
I can reproduce the slowdown, but only on linux, osx performance is about the same.
I tried to figure out since when this bug exists, but the mono-snapshot packages are only available since March 1... The code must have been introduced in February, January or December.
Seriously, that is a major fuckup. How did nobody notice this?
Nice! Now we need a hotfix for xamarin.android and VS 2017.2.xamarin.android
Please push change immediately to Xamarin.Android on Visual Studio 2017, both to the 15.3 preview and to the 15.2 production release.
> Please push change immediately
To provide a little context on timing, integrating the change into Xamarin.Android in Visual Studio 2017 will take a little bit more time. There is a tentative round of fixes for Visual Studio 2017 due out this week, but since the candidate change from Comment 22 is new as of today, that change will have to catch the next bug fix release, possibly within the next week or so.
The Xamarin team will keep this bug up-to-date with information about the release location of the fix as it becomes available. Depending on the testing results of the candidate change from Comment 22 with internal development builds, it might also be possible to provide temporary packages for Visual Studio 2017 until the next full Visual Studio 2017 update is available.
Any chance that hotfixes will be aligned with bug #56275? I'm currently stuck in developer hell between restarting VS all the time, and then waiting for ages for the app to actually do something. If those get fixed independently, I'd hate to revert one bugfix with another.
The current "stable" version of Visual Studio 2017 is basically useless for Android App development without this fix. If you can't integrate it in the next VS update, please provide a hotfix .vsix on releases.xamarin.com.please
> Any chance that hotfixes will be aligned with bug #56275
It is certainly possible, but it will be necessary to wait for a little while longer to see what further information about Bug 56275 unfolds over the next few days. In particular, there is not yet a precisely defined candidate fix for Bug 56275.
You can imagine that nobody wants to wait a little longer. I was rather thinking along the lines of a second hotfix containing the first one ;)
Brendan, I appreciate there may be difficulties but this is an absolute breaking change which renders all Android development impossible and Visual Studio 2017 is not conducive to downgrading. A temporary package would be fine - the main point is to please make it possible to actually build usable Android applications independent of other bugs that, while painful, do not actually cause a complete down situation.
I can't imagine any stable release of Xamarin has ever been so fundamentally broken.
To be clear, _if_ the candidate change from Comment 22 does indeed help resolve the issue in local testing once it has been integrated into development builds (as soon as possible), it will be made available as soon as possible to users in whatever way is possible. In Comment 25, I wanted to make it clear that the overall Visual Studio 2017 release process has a certain cadence, and I wanted to give some pre-warning that the _exact_ new build of Visual Studio 2017 that is tentatively planned to be made available this week will _not_ yet include the fix.
I wouldn't classify any of this information in particular as "difficulties." It is aimed at presenting facts of how a change in a product can be propagated into various kinds of builds in the hopes that that might be helpful for general context.
Hope that helps!
Brendan, thank you for the clarification on why this fix may take a bit longer to find its way into Visual Studio itself. As you suggest, I will monitor this bug for release information on whatever fix becomes available.
*** Bug 56441 has been marked as a duplicate of this bug. ***
Following are the step I followed to verify this bug
1. Open Visual Studio Enterprise 2017 for MAC
2. Run the project which was attached in the bug on an Android device
3. Press "Click Me"
4. Captured the debug output
Observed the following
The milliseconds elapsed showed
For Mono 220.127.116.11 / Xamarin.Android Version 18.104.22.168
Milliseconds elapsed: 2653
Milliseconds elapsed: 2640
Milliseconds elapsed: 2653
Milliseconds elapsed: 2639
Milliseconds elapsed: 2671
Milliseconds elapsed: 2644
Milliseconds elapsed: 2660
For Mono 22.214.171.124 / Xamarin.Android Version 126.96.36.199
Milliseconds elapsed: 209
Milliseconds elapsed: 28
Milliseconds elapsed: 29
Milliseconds elapsed: 22
Milliseconds elapsed: 19
Milliseconds elapsed: 21
Milliseconds elapsed: 19
Observe the clear difference between output of Xamarin.Android Version 188.8.131.52 and 184.108.40.206
Please let me know if anything else is required
## Note to users watching this bug
For a quick clarification, Comment 34 is verification attempt based on an in-progress development build "Xamarin.Android 220.127.116.11". That build has not yet been published in a public location, but that experimental build or another experimental build that includes that fix will be provided as soon as possible for additional user verification of the candidate change.
Please do let us know when the desired build is ready so that the team here can verify the same on our end.
Agreed - please let us know when we can get this fix because this is negatively impacting our release which *was* scheduled for today but now has moved to next week.
## Status update
The candidate change from Comment 22 has now been published to the Stable updater channel for Mac only for the moment  to provide the earliest possible availability of the candidate fix on Mac while the corresponding Windows builds are being finalized.
As mentioned on , the corresponding Windows builds are in progress for finalization. The tentative goal will be to have the Windows versions available within no more than a couple business days. I will keep the status up-to-date as the builds and integration move forward in case there are any unexpected hiccups, and I will advise if builds suitable for distribution become available sooner (for example, before they are published via the updaters).
Great job, works perfectly!
Thank you very much!
Thank you for the update. Awaiting the Windows fix as per your comment 38.
Thanks for the fix. Would it be possible to update Xamarin.Android to 18.104.22.168 in the Alpha channel as well, so that we can still experiment with the live player?
For me, this bug caused my app to suddenly take about 3 Minutes just to start. It's very easy to spot this issue without doing any performance profiling. Therefore I'm quite surprised this bug made it into a stable release build.
While it's easy the spot the bug, the effect is detrimental to my productivity. It took me a few hours to realize it's not my code's fault and it will hurt my productivity even further until a fix is finally available.
May I suggest that you also add an integration test to your test suite that monitors performance of compiled code inside an emulator? It would probably have avoided distributing a bug of such magnitude.
Looking forward to the Windows fix! Has halted our production deployments.
Could I also propose along with Stable, Alpha, Beta channels perhaps a Previous channel which could take us easily back to the previous build which would allow developers to rapidly diagnose if a problem relates to their own code or the new update, it would also crucially allow us to revert so we can continue production.
Feature requests should be handled in UserVoice. Somebody already posted a request for that on https://xamarin.uservoice.com/forums/144858-xamarin-platform-suggestions/suggestions/19263328-allow-installing-older-versions-of-xamarin-in-visu
## Bookkeeping note
Comment 46 and Comment 47 are not relevant to users, so I have marked them as non-public to keep the public-facing bug report focused on the information that is relevant to users watching this bug.
It's been a few hours since this bug was set to RESOLVED / VERIFIED. It's also been over 2 business days (Brendan Zagaeski's original estimation of when this will be resolved). When can we expect the windows build of the hotfix?
This ticket says Status Verified Fixed, but I don't see this fix available from the updates section in Visual Studio 2017 (windows).
This is has had a major impact on our team and our schedule - can someone please advise how we can install this fix on windows?
15 days since this issue was reported. A lot of us anxiously awaiting that Windows fix or a way to revert a version. Please advise ETA.
## Visual Studio 2017
The fix for this issue has been added to the next upcoming Visual Studio 2017 update in the Visual Studio Installer. The most direct workaround until that release would be to install Visual Studio 2017 version 15.0 (see https://www.visualstudio.com/vs/older-downloads/) to get back to Xamarin.VisualStudio 4.3.0 with Xamarin.Android 7.1.0.
My understanding is that the timeline for the next Visual Studio 2017 update is as soon as possible, but unfortunately I in Xamarin support don't have more insight about the updated timeline than that.
One potential way to apply the fix to Visual Studio 2017 in the mean time is the following (with a warning that I have only experimented with this briefly on one Android 6.0 armeabi-v7a device using the particular test project from Comment 0).
1. Extract the Visual Studio 2015 22.214.171.1245 installer (from "Download links" on ) in a cmd.exe command prompt. (You can optionally customize the TARGETDIR.)
msiexec /a Xamarin.VisualStudio_126.96.36.1995.msi /qb TARGETDIR=C:\Temp\Xamarin
2. Move the following directory from your existing Visual Studio 2017 installation to a backup location:
By default the full expanded path for this (for example in a Developer Command Prompt) will be one of the following, depending on Visual Studio edition:
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\Xamarin\Android\lib
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Xamarin\Android\lib
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Xamarin\Android\lib
3. Now move the corresponding "MSBuild\Xamarin\Android\lib" directory from the extracted installer into "%VSINSTALLDIR%\MSBuild\Xamarin\Android\lib" as a replacement.
4. Follow steps 2 and 3 once more to replace:
... with the "Reference Assemblies\Microsoft\Framework\MonoAndroid" directory from the extracted installer.
5. Similarly, move all of the "*DebugRuntime*" files out of "%VSINSTALLDIR%\MSBuild\Xamarin\Android" to a backup location, and move all of "*DebugRuntime*" files from the extracted installer's "MSBuild\Xamarin\Android" directory in as replacements.
6. Close and reopen Visual Studio, and clean, rebuild, and redeploy the Android app project.
>  https://kb.xamarin.com/customer/portal/articles/1699777-older-downloads
## Visual Studio 2015
For Visual Studio 2015 users, an installer is available on the Xamarin updater channels that includes the fix for this bug, or it is also possible to downgrade to Xamarin.VisualStudio 4.4.0 by following the steps on .
That is a great problem. The only solution that i can find for the future is move all my xamarin development to a mac as i believe the update rollouts it is handled my xamarin. My boss understands by our clients not as we are obligated by contracts to push in beta channels new versions. We are entering 3rd week, ready to run of the cliff.
*** Bug 56548 has been marked as a duplicate of this bug. ***
Can you update this thread when the next Preview version of VS 2017 comes out? I'm using 15.3 (26510.0) now, and I'm assuming it will be the next after this.
## Status update for Visual Studio 2017
The candidate fix for this bug report (from Comment 22) is now included in today's Visual Studio 2017 version 15.2 (26430.12) update (https://www.visualstudio.com/news/releasenotes/vs2017-relnotes
(2017-05-30) Visual Studio 2017 version 15.2 (26430.12)
Includes Xamarin.VisualStudio IDE extension version: 188.8.131.526 (1be4f0c)
If any user still hits a similar behavior consistently after updating to Visual Studio 2017 version 15.2 (26430.12) or higher (or Xamarin.VisualStudio 184.108.40.2065 or higher in Visual Studio 2015 and below), please create a new bug report. Be sure to mention any details for the remaining problematic scenario that might differ from the original scenario described in this bug report in Comment 0. Thanks in advance!
## Status update for Visual Studio 2017 Preview
> update this thread when the next Preview version of VS 2017 comes out
As an additional status clarification, the candidate fix from Comment 22 is now included in the Visual Studio 2017 version 15.2 (26430.12) from the _release_ channel, but it is not yet included in the current Visual Studio 2017 _Preview_ version 15.3 (26510.00 - Preview) from May 11. By default it will indeed be included in the next Visual Studio 2017 Preview version of 15.3. I will plan to update the bug once more when a new preview version with the fix is published.
Trying out the latest 15.2, and the app is usable again. However: Is it just me, or is the performance still clearly below what we had with 15.1? I'm still getting a black screen (for a much shorter period though) while loading my app, and overall startup time (navigation and retrieving data) is clearly slower.
Philipp, I will be testing soon on the new VS 15.2. As Brendan stated, we should create a new bug report if we see any exhibiting behavior of performance problems once more.
Philip Summi, preview is 15.3, not 15.2.
I'm not happy with this new concept of installation introduced in Visual Studio 2017, having Preview as separate installation. It requires additional disk space, and if I have limited disk space (e.g. SSD disk), we need to uninstall VS completely, then install Preview, and if we are not satisfied, perform rollback using the same steps which requires much more time then before.
For me, the previous option to be able to update Xamarin (and even swich channel to beta or alpha) using Visual Studio was much better update option.
Since cross-platform development tools itself require much more attention than desktop, or web development tools, updates should be more often than slow VS updates, and this is the reason why we should have the option to update Xamarin separately of Visual Studio.
I have just installed VS2017 15.2 (26430.12) and after cleaning and rebuilding my project I have found it to be significantly faster that it was before the update (previous start-up time was around 30+ seconds whereas now it is less than 10, opening a new form is near instant whereas it was taking up to 10 seconds the first time - though near instant for subsequent instances of the same form).
I am not however using RelativeLayout though I am using both Prism and Unity.
> concept of installation
Note that this bug report is focused on the specific technical problem described in Comment 0. For broader discussions about installation features, UserVoice is one place to surface information to the Xamarin team.
*** Bug 56562 has been marked as a duplicate of this bug. ***