Bug 2405 - Special characters break the editor
Summary: Special characters break the editor
Alias: None
Product: Xamarin Studio
Classification: Desktop
Component: Text Editor ()
Version: Trunk
Hardware: PC Mac OS
: Highest major
Target Milestone: ---
Assignee: Mikayla Hutchinson [MSFT]
Depends on:
Reported: 2011-12-07 20:33 UTC by Alan McGovern
Modified: 2012-02-19 15:45 UTC (History)
6 users (show)

Tags: gtk
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Alan McGovern 2011-12-07 20:33:57 UTC
I happened to use a special character (TM) [Option-2] to have a TM superscript in a string in a normal .cs file.

When I did this the text editing on that line is totally wonky. It's like the cursor is offset to the wrong character and there's a mix of overwriting and new characters inserted as you type.

Perhaps I'm doing something wrong by trying to encode a special character in a normal .CS file for a string in my code.

However, if this is illegal, it would be helpful to have the special insert-keystrokes disabled for those characters, or at least a warning to appear.

Or, maybe I can have a TM in my string as a native character and Mono will compile it.

MonoDevelop: 2.9.1
Runtime: Mono 2.10.7 (tarball Mon Dec 5 20:41:19 EST 2011)
Operating System: Mac OSX (Unix
Distributor: Xamarin
Comment 2 Alan McGovern 2011-12-07 20:35:13 UTC
This is likely just an issue where we don't calculate the right offset when we have multibyte characters in the line of code.
Comment 3 Mike Krüger 2011-12-09 06:35:04 UTC
Works for me using 2.10.6:


However 2.10.7 makes things break:


I suppose it has to do with the gtk# / pango installed with 2.10.7.  

@Mhutch: I've no idea who to assign this issue - it's something in pango I suppose. Monodevelop doesn't calculate anytihing on c# level - all is done by pango and 2.10.7 makes things break - somehow -. Can you pass it to the right one ?
Comment 4 Cédric Belin 2011-12-14 12:45:39 UTC
Same issue for me (using Mac OS X Lion 10.7.2, Mono 2.10.7 and MonoDevelop 2.8.5) : I'm French, so a lot of text is using multi-bytes characters (like é, è, à, ç, etc...).
It's a huge problem: it makes MonoDevelop editor completely unusable for non-English languages.
Comment 5 Alan McGovern 2011-12-14 12:51:45 UTC
Cedric, for now simply downgrade Mono to 2.10.6 to avoid the issue. It was a regression in the font support provided by Mono itself.
Comment 6 Cédric Belin 2011-12-14 13:15:11 UTC
@Alan: Already done! But thanks for the advice :o)
Comment 7 Mikayla Hutchinson [MSFT] 2012-01-03 14:16:06 UTC
This is an issue with the new Pango 1.29.4 that we shipped in the Mono 2.10.7 beta. Certain characters/fonts seem to trigger Pango into RTL behaviour. It's reproducible with GTK entries. For the latest Mono beta, we reverted to the old Pango temporarily.
Comment 8 Kristian Rietveld (inactive) 2012-01-08 10:09:25 UTC
Interesting -- quick testing with a default GtkTextView and default font does not seem to reveal problems.  I will work on distilling what MonoDevelop is doing into a small test case so I have something to debug.  Possibly, the fixed-width font is the problem.
Comment 9 Mikayla Hutchinson [MSFT] 2012-01-09 13:30:15 UTC
Kris, I can reproduce this in a GtkEntry with the tm char. In that case it also makes text selection change direction and makes a weak caret appear, so it looks like the font is somehow triggering RTL behaviour.
Comment 10 Kristian Rietveld (inactive) 2012-01-15 10:04:12 UTC
I now know why the problems were not revealed, I assumed I was testing with all patches attached, but in fact the zero-width patch I did was missing.

As I kind of expected, this zero-width patch is introducing these problems.  I hope to work on a fix tonight.
Comment 11 Kristian Rietveld (inactive) 2012-01-17 05:31:53 UTC
The problem is indeed in the patch in bug 664125.  Brief analysis of the problem:

Multibyte UTF-8 characters were not dealt with correctly, this could be seen by broken cursor movement in e.g. GtkEntry when strings containing such characters were edited.  In the old patch, we used the indices obtained from the CTRun as offset parameter to set_glyph().  This is wrong, because the indices in the CTRun are relative to character positions, while the offset parameter expects byte positions in the original UTF-8 string.  This has been corrected by translating from character position to UTF8 byte position.

Please find an updated patch that corrects this problem in bug 664125. Direct link to the patch:


There still seems to be a problem with cursor movement when editing a string containing zero-width spaces.  I will look into that next -- but for now the most critical problems with special characters should have been fixed.
Comment 12 Mikayla Hutchinson [MSFT] 2012-01-19 18:54:20 UTC
I have verified that the patch works and updated the patchset in bockbuild. Alex, could we include the new Pango the next time we do a build with the new GTK+?
Comment 13 Jeffrey Stedfast 2012-02-15 15:35:22 UTC
fixed in Mono 2.10.9
Comment 14 Kristian Rietveld (inactive) 2012-02-19 15:45:00 UTC
From comment 11:
> There still seems to be a problem with cursor movement when editing a string
> containing zero-width spaces.  I will look into that next -- but for now the
> most critical problems with special characters should have been fixed.

Upstream bug 664125 now has a new patch which fixes cursor movement when editing a  string containing zero-width spaces and in general makes the shaping engine more robust.