Bug 11907 - Editing Localizable.string causing big processor spike
Summary: Editing Localizable.string causing big processor spike
Alias: None
Product: Xamarin Studio
Classification: Desktop
Component: Text Editor ()
Version: 4.1
Hardware: Macintosh Mac OS
: High normal
Target Milestone: ---
Assignee: Mike Krüger
Depends on:
Reported: 2013-04-23 11:37 UTC by Paul Johnson
Modified: 2013-07-17 04:47 UTC (History)
2 users (show)

Tags: gtk
Is this bug a regression?: ---
Last known good build:

string file (50.05 KB, application/octet-stream)
2013-04-23 11:37 UTC, Paul Johnson

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Paul Johnson 2013-04-23 11:37:14 UTC
Created attachment 3856 [details]
string file

I have smallish Localizable.string file (attached). When I edit using Xam Studio 4.1, it is incredibly slow on adding new strings to it (looking at the Activity Monitor application, it is showing 130-180% processor use) or editing existing strings.

Editing with Xcode or a bog standard text editor is not giving any sort of issue.

Using a Mac Mini, 8Gb memory, OS X.8.3, Xam Studio 4.1 and Xam.iOS
Comment 1 Mike Krüger 2013-05-03 04:11:46 UTC
Seems that pango has a problem with lines that consists of many white spaces. Here it is line 143

it's a line consisting of many <tab><4spaces> pairs. I get trace dumps as followed:

"GUI Thread"  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) Pango.Layout.pango_layout_get_size (intptr,int&,int&) <IL 0x00025, 0xffffffff>
  at Pango.Layout.GetSize (int&,int&) <IL 0x00008, 0x0001b>
  at Mono.TextEditor.TextViewMargin.CreateLinePartLayout (Mono.TextEditor.Highlighting.ISyntaxMode,Mono.TextEditor.DocumentLine,int,int,int,int,int) [0x00665] in /Users/mkrueger/work/monodevelop/main/src/core/Mono.Texteditor/Mono.TextEditor/Gui/TextViewMargin.cs:1019
  at Mono.TextEditor.TextViewMargin.CreateLinePartLayout (Mono.TextEditor.Highlighting.ISyntaxMode,Mono.TextEditor.DocumentLine,int,int,int,int) [0x00000] in /Users/mkrueger/work/monodevelop/main/src/core/Mono.Texteditor/Mono.TextEditor/Gui/TextViewMargin.cs:1290
  at Mono.TextEditor.TextViewMargin.GetLayout (Mono.TextEditor.DocumentLine) [0x00041] in /Users/mkrueger/work/monodevelop/main/src/core/Mono.Texteditor/Mono.TextEditor/Gui/TextViewMargin.cs:1390
  at Mono.TextEditor.TextArea.RenderMargins (Cairo.Context,Cairo.Context,Cairo.Rectangle) [0x000fc] in /Users/mkrueger/work/monodevelop/main/src/core/Mono.Texteditor/Mono.TextEditor/Gui/TextArea.cs:1727
  at Mono.TextEditor.TextArea.OnExposeEvent (Gdk.EventExpose) [0x000f0] in /Users/mkrueger/work/monodevelop/main/src/core/Mono.Texteditor/Mono.TextEditor/Gui/TextArea.cs:1807
  at Gtk.Widget.exposeevent_cb (intptr,intptr) <IL 0x00014, 0x00061>
  at (wrapper native-to-managed) Gtk.Widget.exposeevent_cb (intptr,intptr) <IL 0x00024, 0xffffffff>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) Gtk.Application.gtk_main () <IL 0x00022, 0xffffffff>
  at Gtk.Application.Run () <IL 0x00000, 0x0000b>
  at MonoDevelop.Ide.IdeApp.Run () [0x00000] in /Users/mkrueger/work/monodevelop/main/src/core/MonoDevelop.Ide/MonoDevelop.Ide/Ide.cs:397
  at MonoDevelop.Ide.IdeStartup.Run (MonoDevelop.Ide.MonoDevelopOptions) [0x00763] in /Users/mkrueger/work/monodevelop/main/src/core/MonoDevelop.Ide/MonoDevelop.Ide/IdeStartup.cs:287
  at MonoDevelop.Ide.IdeStartup.Main (string[]) [0x00083] in /Users/mkrueger/work/monodevelop/main/src/core/MonoDevelop.Ide/MonoDevelop.Ide/IdeStartup.cs:534
  at MonoDevelop.Startup.MonoDevelopMain.Main (string[]) [0x00000] in /Users/mkrueger/work/monodevelop/main/src/core/MonoDevelop.Startup/MonoDevelop.Startup/MonoDevelopMain.cs:16
  at (wrapper runtime-invoke) <Module>.runtime_invoke_int_object (object,intptr,intptr,intptr) <IL 0x0005c, 0xffffffff>
Comment 2 Mikayla Hutchinson [MSFT] 2013-05-03 19:28:35 UTC
Kris, any ideas about improving the pango perf on long lines?
Comment 3 Kristian Rietveld (inactive) 2013-06-01 08:32:13 UTC
I have first looked if the problems also occur in testtext, which is a simple wrapper around GtkTextView. This sample indeeds exhibits slowness when editing long strings. Every time a string is updated, GtkTextView frees the PangoLayout and a new one is created, which means that the line of text has to be re-itemized.

It is not a surprise that pango_layout_check_lines() shows up high in the profiles. A large part of the time is spent in the CoreText part of the shaping process. I get the impression that because the string consists out of alternating tab characters and spaces, small sequences of spaces are fed to the CoreText shaping engine. This engine is then invoked over and over again, which is expensive. Tabs are handled with a different code path within Pango.

A quick test with all spaces replaced with tabs shows that there is no decrease in performance for such lines. If all tabs are placed with spaces, then the spaces are not rendered (likely because these are trailing spaces?).

If I now look at MonoDevelop, the string handling is clearly slower than testtext. I will investigate why this is and see if we can improve the MonoDevelop text editor backend to attain the same performance as testtext. Compared with what I see in MonoDevelop, the performance in testtext could be acceptable (but still would be nice to improve).
Comment 4 Kristian Rietveld (inactive) 2013-06-01 11:14:17 UTC
Okay, that was not a fully fair comparison. I have also tested testtext with wrapping set to FALSE (so it behaves like MonoDevelop), and testtext then shows performance similar to MonoDevelop for the tabs/spaces line. In this case with wrapping disabled, the time profile is a bit different. Approx. 50% of the time is spent in the line_width() function in Pango. This function contains the following comment:

  /* Compute the width of the line currently - inefficient, but easier
   * than keeping the current width of the line up to date everywhere

line_width() is only called by shape_tab(), which is the special case for shaping tabs.

When I test a string of approx. the same length, but containing normal characters and no tabs, the performance is noticeably better.

I would propose we start by trying to find a small optimization to counter the time spent in line_width() specifically for strings of tabs and spaces. The comment suggests that fixing this for the general case will not be easy. Furthermore, I do have to say that such long strings of alternating tabs and spaces are quite uncommon and it is possible that upstream would be against adding a lot of complexity for improving this. First, we have to see if we can find a simple patch to improve this at all.
Comment 5 Kristian Rietveld (inactive) 2013-06-02 06:19:58 UTC
The hotspots are clearly line_width() and also reorder_runs_recurse(). My impression is that these functions become hot spots due to the large number of items/runs in the test string. The large number of items/runs are likely caused because the tab characters seem to forcibly start new items/runs after each tab character.

Next week, I will spend some more time on understanding the internals of the code, so that I can judge whether it is feasible to easily optimize this.
Comment 6 Kristian Rietveld (inactive) 2013-06-16 09:29:05 UTC
We have found three hot spots in the Pango code. Patches have been written and submitted upstream for review in GNOME bug 702389.

I have named the giving string file "long-line-test.txt". When I edit *only* this file in Xamarin Studio with the patches applied, the performance is noticeably improved. It is still not very snappy, however. Further addressing this might be hard due to the way Pango is designed internally -- we will engage in discussions with the upstream maintainer.
Comment 7 Kristian Rietveld (inactive) 2013-06-16 09:34:46 UTC
There is one but: if next to "long-line-test.txt" I open another file that is syntax highlighted (say Main.cs) and switch back to "long-line-test.txt", editing the string becomes very slow again, even with the Pango patches applied.

I have debugged this and this problem seems to be caused by the syntax highlighting mechanism in Xamarin Studio. Before Main.cs is opened, only 1 PangoAttribute is set for the entire long line. After Main.cs is opened, 27757 PangoAttributes are set, and this is seriously hurting performance.

It looks like this is due to how chunks are created/handled within the text editor code. In AddRealChunk (Mono.TextEditor.Highlighting/SyntaxMode.cs), the following code fragment handles the length extension of an existing chunk:

if (endChunk.Style.Equals (chunk.Style) && chunk.SpanStack.Equals (endChunk.SpanStack)) {
	endChunk.Length += chunk.Length;

It turns out that after opening Main.cs, the body of this if-statement is no longer executed. This is caused by chunk.SpanStack and endChunk.SpanStack no longer being equal. If the above if-statement is replaced with:

if (endChunk.Style.Equals (chunk.Style) /* && chunk.SpanStack.Equals (endChunk.SpanStack)*/) {

Also this performance problem disappears.

I have not investigated further, because I am not familiar with the text editor internals. Maybe a text editor developer can further investigate this?
Comment 8 Mikayla Hutchinson [MSFT] 2013-06-17 18:21:49 UTC
Comment 9 Mike Krüger 2013-06-18 01:34:04 UTC
MHutch: I'll look at it next monodevelop friday.
Comment 10 Mike Krüger 2013-06-21 02:31:40 UTC
fixed - should make highlighting much faster :)
Comment 11 Kristian Rietveld (inactive) 2013-06-23 03:53:25 UTC
Nice -- indeed seems the highlighting performance issue is fixed.

Should we leave this bug open for the Pango patches to land upstream or handle that in another bug?
Comment 12 Mike Krüger 2013-06-24 02:14:55 UTC
It's fixed - pango patches land in sooner or later.
Comment 13 Kristian Rietveld (inactive) 2013-07-17 04:47:10 UTC
The first patch made it's way into Pango master after some slight modification. We are waiting for upstream review of the remaining two patches.