Bug 30709 - System.Text.Encoding.GetEncoding (string) - internal bug adding key to a hash table
Summary: System.Text.Encoding.GetEncoding (string) - internal bug adding key to a hash...
Alias: None
Product: Class Libraries
Classification: Mono
Component: mscorlib ()
Version: unspecified
Hardware: PC Other
: --- normal
Target Milestone: Untriaged
Assignee: Marek Safar
Depends on:
Reported: 2015-06-03 06:57 UTC by Jeffrey Stedfast
Modified: 2016-04-16 08:33 UTC (History)
4 users (show)

Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Jeffrey Stedfast 2015-06-03 06:57:29 UTC
2015-06-02 17:29:59.264 NachoClientiOS[45579:916454] Error:21:: Exception : System.ArgumentException: Item has already been added. Key in dictionary: '1256'  Key being added: '1256'
  at System.Collections.Hashtable.Insert (System.Object key, System.Object nvalue, Boolean add) [0x001d7] in /Users/builder/data/lanes/1503/6481535e/source/mono/external/referencesource/mscorlib/system/collections/hashtable.cs:979 
  at System.Collections.Hashtable.Add (System.Object key, System.Object value) [0x00000] in /Users/builder/data/lanes/1503/6481535e/source/mono/external/referencesource/mscorlib/system/collections/hashtable.cs:444 
  at System.Text.Encoding.GetEncoding (Int32 codepage) [0x00218] in /Users/builder/data/lanes/1503/6481535e/source/mono/external/referencesource/mscorlib/system/text/encoding.cs:528 
  at System.Text.Encoding.GetEncoding (System.String name) [0x00000] in /Users/builder/data/lanes/1503/6481535e/source/mono/external/referencesource/mscorlib/system/text/encoding.cs:651 
  at MimeKit.IO.Filters.CharsetFilter..ctor (System.String sourceEncodingName, System.String targetEncodingName) [0x00002] in /Users/tje/Projects/MimeKit/MimeKit/IO/Filters/CharsetFilter.cs:70 
  at NachoCore.IMAP.ImapSyncCommand.getTextFromStream (System.IO.Stream stream, MimeKit.ContentType type, ContentEncoding enc) [0x00033] in /Users/tje/Projects/NachoClientX/NachoClient.Android/NachoCore/BackEnd/IMAP/Commands/ImapSyncCommand.cs:454 

CharsetFilter.cs:70 is here:

        public CharsetFilter (string sourceEncodingName, string targetEncodingName)
—>            : this (Encoding.GetEncoding (sourceEncodingName), Encoding.GetEncoding (targetEncodingName))
Comment 1 Jeffrey Stedfast 2015-06-03 07:13:05 UTC
Looking through encoding.cs, I suspect the problem might be that when codepage = 1256, it ends up calling:

encodings.Add (1256, null);

Later, if GetEncoding() is called again with a 1256, it finds that encodings[codepage] has a null value, so proceeds to try and get an encoding for it and then eventually ends up at encodings.Add(codepage, result) which blows up because 1256 was already added, just with a null value.
Comment 2 Marek Safar 2015-06-03 11:54:21 UTC
@jeff, how can I reproduce it? master code is quite different and your stack trace shows 

GetEncoding (System.String name)

but you suggested it's caused by encoding with code.
Comment 3 Jeffrey Stedfast 2015-06-03 12:16:36 UTC
From the stack trace, it looks like Encoding.GetEncoding(string) calls GetEncoding(int).

I'm assuming a test case like this would produce the error:

Encoding.GetEncoding ("cp1256");
Encoding.GetEncoding ("cp1256");

Unfortunately, in a standard console app, that works and in an iOS project, it throws NotSupportedException rather than hitting the bug.

So this might already be fixed. I'm not entirely sure.
Comment 4 Jeffrey Stedfast 2015-06-03 12:23:03 UTC
Correction: it throws NotSupportedException unless I include the various I18N assemblies in the iOS app.
Comment 5 Marek Safar 2015-06-04 10:58:54 UTC
NotSupported is still fine (expected).

Please reopen if you have repro failing in master
Comment 6 Jeffrey Stedfast 2015-08-01 22:29:21 UTC
Nat is running into this with his app as well. He's getting it for codepages 936 (gb2312) and 51949 (euc-kr). Happening in the exact same spot (encoding.cs:528) using the latest 8.10 stable release of Xamarin.iOS.

Something wonky is going on.

Reopening since a diff between the mono used by cycle5 XI and git master of mono has no changes to encoding.cs except 1-line change on line ~713 (the Environment.GetResourceStringEncodingName change)
Comment 7 Marek Safar 2015-08-02 05:15:30 UTC
How can I reproduce it?
Comment 8 Marek Safar 2015-08-03 04:03:46 UTC
btw, for correct diff you need to diff reference sources as mono code is no longer used
Comment 9 Jeffrey Stedfast 2015-08-03 10:57:25 UTC

Did you have any luck finding the messages that caused this?

FWIW, you should be able to X-out the actual content of the message, all we should need are the headers (the bug is probably in the Subject or To/From/Cc headers).

The charsets that caused this were gb2312 (Chinese) and euc-kr (Korean), so that may help you narrow it down a bit.
Comment 10 Jeffrey Stedfast 2015-08-03 10:58:40 UTC

Yea, I diffed the referencesource encoding.cs.
Comment 11 Mirco Bauer 2016-01-16 12:57:57 UTC
This issue was also reported against Smuxi with Mono 4.0.5, see https://smuxi.im/issues/show/1082
Comment 12 Marek Safar 2016-04-16 08:33:48 UTC
I believe this is fixed in Mono 4.4 (we use reference sources here). Please reopen if you can still repro