Bug 17615

Summary: Unexpected parsing with Lithuanian culture
Product: [Mono] Class Libraries Reporter: Matt Jones <matt>
Component: mscorlibAssignee: Marek Safar <masafa>
Status: RESOLVED FIXED    
Severity: normal CC: alek, mohitk, mono-bugs+monotouch, mono-bugs+mono, sebastien
Priority: ---    
Version: 3.2.x   
Target Milestone: Untriaged   
Hardware: Macintosh   
OS: Mac OS   
Tags: Is this bug a regression?: ---
Last known good build:
Attachments: Sample Unit Tests for Failed Decimal Parsing

Description Matt Jones 2014-02-05 10:30:44 UTC
The Lithuanian number parser seems broken. Consider the following:

 var st="5,5";
 var litcult=CultureInfo.CreateSpecificCulture("lt");
 var styles=NumberStyles.AllowDecimalPoint | NumberStyles.AllowLeadingSign;
 decimal val=0;

 var res=decimal.TryParse(st,styles,litcult,out val);

The parse fails in MonoTouch but works under Windows.
Comment 1 Sebastien Pouliot 2014-02-05 10:57:11 UTC
This looks like an issue inside the Mono BCL (or more likely the localization tables). I get the same results with a console app.
Comment 2 Matt Jones 2014-02-05 11:51:25 UTC
Thought it might be but only had a chance to try it in MT. We think it's not always been this way, but even so we couldn't tell you which version it reverted in.
Comment 3 Matt Jones 2014-02-05 12:41:05 UTC
Additionally there appear to be similar problems with the following cultures:

- Tajik (tg-Cyrl-TJ)
- Persian (fa-IR)
- Kirghiz (ky-KG)
Comment 4 Marek Safar 2014-03-07 11:56:31 UTC
Fixed in master
Comment 6 Matt Jones 2015-03-27 11:00:19 UTC
These still fail:

- Tajik (tg-Cyrl-TJ)
- Persian (fa-IR)
- Kirghiz (ky-KG)

This is the test code:

		[Test]
		public void TestStringToDecimalConversion()
		{
			String[] testStrings = new String[]{"5.5","5|5","5500.55","5500|55","-55.5","N55|5","monkey"};
			bool[] shouldPass = new bool[]{true,true,true,true,true,true,false};
			decimal[] resultValues = new decimal[]{5.5m,5.5m,5500.55m,5500.55m,-55.5m,-55.5m,1337m};
			int index=0;
			var cults = CultureInfo.GetCultures (CultureTypes.AllCultures);
			var test1 = new List<FailedCulture> ();
			var test2 = new List<FailedCulture> ();
			foreach (String testString in testStrings)
			{
				foreach (CultureInfo aCulture in cults)
				{
					if (aCulture.IsNeutralCulture)
						continue;
					decimal val = 1337;
					bool didParse = false;
					String tString = testString.Replace ("|", aCulture.NumberFormat.CurrencyDecimalSeparator);
					String tString2 = tString.Replace ("N", aCulture.NumberFormat.NegativeSign);
					didParse = ParseUtils.UserInputDecimalTryParse (tString2, out val, aCulture);
					//Assert.True (didParse==shouldPass[index], "Failed to parse user input string to decimal with culture " + aCulture.EnglishName + " [" + tString2 + "]");
					if (didParse != shouldPass [index])
						test1.Add (new FailedCulture (){ info = aCulture, source=testString,input = tString2 });
					if (didParse)
					{
						//Assert.True (val == resultValues [index], "Failed to parse user input string to decimal CORRECTLY with culture " + aCulture.EnglishName + " [" + tString2 + "]");
						if (val != resultValues [index])
							test2.Add (new FailedCulture (){ info = aCulture, source=testString,input = tString2 });
					}
				}
				index++;
			}

			Assert.True (test1.Count == 0, "Failed to parse user input string to decimal for the following cultures: {0}", FailedCulture.AllToString (test1));
			Assert.True (test2.Count == 0, "Failed to parse user input string to decimal CORRECTLY for the following cultures: {0}", FailedCulture.AllToString (test2));
		}


////////////////

and these are the failures:

 [Tajik (Cyrillic, Tajikistan)(tg-Cyrl-TJ)_;_ _ts:5|5_userinput:5;5] 
 [Persian (Iran)(fa-IR)_/_,_ts:5|5_userinput:5/5] 
 [Uzbek (Latin, Uzbekistan)(uz-Latn-UZ)_,_ _ts:5|5_userinput:5,5] 
 [Uzbek (Cyrillic, Uzbekistan)(uz-Cyrl-UZ)_,_ _ts:5|5_userinput:5,5] 
 [Tajik (Cyrillic, Tajikistan)(tg-Cyrl-TJ)_;_ _ts:5500|55_userinput:5500;55] 
 [Persian (Iran)(fa-IR)_/_,_ts:5500|55_userinput:5500/55] 
 [Uzbek (Latin, Uzbekistan)(uz-Latn-UZ)_,_ _ts:5500|55_userinput:5500,55] 
 [Uzbek (Cyrillic, Uzbekistan)(uz-Cyrl-UZ)_,_ _ts:5500|55_userinput:5500,55] 
 [Tajik (Cyrillic, Tajikistan)(tg-Cyrl-TJ)_;_ _ts:N55|5_userinput:-55;5] 
 [Persian (Iran)(fa-IR)_/_,_ts:N55|5_userinput:‎−55/5] 
 [Uzbek (Latin, Uzbekistan)(uz-Latn-UZ)_,_ _ts:N55|5_userinput:-55,5] 
 [Uzbek (Cyrillic, Uzbekistan)(uz-Cyrl-UZ)_,_ _ts:N55|5_userinput:-55,5]
Comment 7 Marek Safar 2015-03-31 04:47:38 UTC
It should be fixed in master but I could not verify it as your sample code is incomplete
Comment 8 Alek Slater 2015-03-31 04:54:42 UTC
Really? Do you mind pasting the commit so I can take a look? As far as I can tell, what you fixed was the lithuanian culture, but not all the other ones that suffer from the same bug.
Comment 9 Marek Safar 2015-03-31 04:56:59 UTC
Commit 54c079ad011f1c751bdc1f319a33da47af7315f6
Comment 10 Matt Jones 2015-03-31 05:00:47 UTC
try adding this to my test code:

public class ParseUtils
{
		static public bool UserInputDecimalTryParse(string val,out decimal output,CultureInfo currentUserCultureInfo)
		{
			NumberStyles numStyles = NumberStyles.AllowDecimalPoint | NumberStyles.AllowLeadingSign;
			bool res= decimal.TryParse (val,numStyles,cult,out output);
			if (!res)
			{
				res=decimal.TryParse (val,numStyles,currentUserCultureInfo,out output);
			}
			return res;
		}

		static public bool UserInputDecimalTryParse(string val, out decimal output)
		{
			return UserInputDecimalTryParse (val, out output,CultureInfo.CurrentCulture);
		}

}
Comment 11 Alek Slater 2015-03-31 05:08:36 UTC
I'll upload a working example, hang on.
Comment 12 Alek Slater 2015-03-31 05:09:34 UTC
Created attachment 10574 [details]
Sample Unit Tests for Failed Decimal Parsing
Comment 13 Alek Slater 2015-03-31 13:14:05 UTC
Marek, I'm a bit confused sorry, the commit you pasted above is this one:

https://github.com/mono/mono/commit/54c079ad011f1c751bdc1f319a33da47af7315f6

That sorts out the remaining broken locales? 
- Tajik (tg-Cyrl-TJ)
- Persian (fa-IR)
- Kirghiz (ky-KG) 

I've attached a complete solution with a unit test that's currently failing, you can verify your fix if you use that, hope that helps!
Comment 14 Marek Safar 2015-04-01 08:23:45 UTC
Yes, the latest master fails with same error as .net.

 [Marathi (India)(mr-IN)_`_,_ts:5|5_userinput:5`5] 
 [Marathi (India)(mr-IN)_`_,_ts:5500|55_userinput:5500`55] 
 [Marathi (India)(mr-IN)_`_,_ts:N55|5_userinput:-55`5]
Comment 15 Matt Jones 2015-04-01 08:27:31 UTC
Is there something I'm missing about the Status fields on this system? Because this bug definitely isn't resolved or fixed.
Comment 16 Matt Jones 2015-11-26 08:02:07 UTC
Thanks for getting it into a release guys, we have been celebrating in Lithuanian national dress all week (photos to follow.)

Fortunately the last Marathi speaker left last year so hopefully we can sweep the last bug under the carpet for another 8 months or so.