Bug 59596 - Xamarin iOS not get the correct CurrentCultureInfo
Summary: Xamarin iOS not get the correct CurrentCultureInfo
Status: RESOLVED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: BCL Class Libraries (show other bugs)
Version: XI 11.0 (xcode9)
Hardware: Macintosh Mac OS
: Normal major
Target Milestone: 15.7
Assignee: Manuel de la Peña [MSFT]
URL:
: 59604 60697 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-09-20 08:39 UTC by Javi Campaña
Modified: 2018-03-26 10:24 UTC (History)
19 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Test app. (641.37 KB, application/zip)
2017-09-20 11:22 UTC, Manuel de la Peña [MSFT]
Details
Workaround Test App (53.90 KB, application/zip)
2017-10-02 17:13 UTC, Manuel de la Peña [MSFT]
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Javi Campaña 2017-09-20 08:39:10 UTC
The var currentCulture = CultureInfo.CurrentCulture; return always en-US, but my iPhone device have es-ES for current language.
It's works correct in iOS 10, I see this bug when update the iPhone to iOS 11.
Comment 1 Manuel de la Peña [MSFT] 2017-09-20 11:22:25 UTC
Created attachment 24826 [details]
Test app.

I can confirm the bug with current master, Xcode9-GM. running the app and clicking the button will print the culture info. If the sim changes language, and the app is re-ran the result is not correct. Current environment information:

=== Visual Studio Community 2017 for Mac (Preview) ===

Version Preview - Internal Dogfood (7.2 build 576)
Installation UUID: 8d12e55e-3489-463f-ac52-8cb4573c5a81
Runtime:
	Mono 5.4.0.193 (2017-06/5066d707acf) (64-bit)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 504000193

=== NuGet ===

Version: 4.3.0.4199

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	1.1.2
	1.0.5
	1.0.0
SDK: /usr/local/share/dotnet/sdk/1.0.4/Sdks
SDK Versions:
	1.0.4
	1.0.0-preview2-003121
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.4.0/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.5.5
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Xamarin.Android ===

Version: 7.4.0.21 (Visual Studio Community)
Android SDK: /Users/mandel/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		4.0.3 (API level 15)
		4.3   (API level 18)
		4.4   (API level 19)
		5.0   (API level 21)
		6.0   (API level 23)
		7.1   (API level 25)

SDK Tools Version: 25.2.5
SDK Platform Tools Version: 25.0.4
SDK Build Tools Version: 25.0.1

Java SDK: /usr
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Xamarin Inspector ===

Version: 1.2.2
Hash: b71b035
Branch: d15-1
Build date: Fri, 21 Apr 2017 17:57:12 GMT

=== Apple Developer Tools ===

Xcode 9.0 (13247)
Build 9A235

=== Xamarin.iOS ===

Version: 11.3.0.6 (Visual Studio Community)
Hash: 1690ccbc
Branch: master
Build date: 2017-09-18 17:19:33+0200

=== Xamarin.Mac ===

Version: 4.1.0.7 (Visual Studio Community)

=== Build Information ===

Release ID: 702000576
Git revision: e3f9469d94ad4932e083ecaa6f998865f15fa452
Build date: 2017-09-05 04:52:47-04
Xamarin addins: 088cef5814d09f9af2866d860110075e14f24435
Build lane: monodevelop-lion-dogfood-vNext

=== Operating System ===

Mac OS X 10.12.6
Darwin 16.7.0 Darwin Kernel Version 16.7.0
    Thu Jun 15 17:36:27 PDT 2017
    root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
Comment 2 Paul DiPietro [MSFT] 2017-09-25 21:17:40 UTC
*** Bug 59604 has been marked as a duplicate of this bug. ***
Comment 3 Manuel de la Peña [MSFT] 2017-09-26 14:27:22 UTC
Hello,

This is a regression introduced by Apple. CFLocaleCopyCurrent(), used in the iOS code, will return the value of the application's CFBundleDevelopmentRegion Info.plist key if all of the following conditions are true:

* CFBundleDevelopmentRegion is present in the Info.plist
* The CFBundleDevelopmentRegion language is in the list of preferred languages on the iOS device, but isn't the first one
* There are no localized resources (i.e. no .lproj directory) in the app for the first preferred locale


This differs from iOS 10 where the presence of the CFBundleDevelopmentRegion key had no effect. 

Note that if the CFBundleDevelopmentRegion key is not present at all, CFLocaleCopyCurrent() always returns the first preferred locale as it did in iOS 10. I have a PR with the fix, which means that we should not set the CFBundleDevelopmentRegion on applications by default, so that users do not see this issue.

You can reproduce this behaviour, the one found in Xamarin the following way:

Assume that:

Set English as the "development language" on the Mac (resulting in CFBundleDevelopmentRegion = English)
The iOS device region/language is set to Spanish/Spain
The iOS device preferred languages is set to Spanish/Spain, English/US
Create a new application in Xcode and add some code to log the identifier of a CFLocale object obtained via CFLocaleCopyCurrent(). Deploy and run the application on an iOS device.

PR: https://github.com/xamarin/xamarin-macios/pull/2779

The above PR might not land, since this change was to be discussed.
Comment 4 Manuel de la Peña [MSFT] 2017-09-26 14:29:48 UTC
@sebastien I rdar might be in order.
Comment 5 Rolf Bjarne Kvinge [MSFT] 2017-09-27 09:00:16 UTC
Someone already filed a radar: http://www.openradar.me/34581827 (looks like it comes from here: https://github.com/lionheart/openradar-mirror/issues/18442)
Comment 6 Manuel de la Peña [MSFT] 2017-09-27 17:58:47 UTC
@rolf looking if Xcode9.1 fixed the issue.
Comment 7 Manuel de la Peña [MSFT] 2017-09-28 10:31:53 UTC
*** Bug 59358 has been marked as a duplicate of this bug. ***
Comment 8 Manuel de la Peña [MSFT] 2017-10-02 17:12:51 UTC
@Javi as per my last comment in the PR (copy paste here):

After some tests with Xcode 9 (no to be confused with 9.1)

    * If we set the CFBundleDevelopmentRegion to empty -> Mono native code returns en-us (get_current_locale_name), Apple native code returns en_US (tested with the phone set to es-US)
    * If we set the CFBundleDevelopmentRegion missing -> native code returns en-us (get_current_locale_name), Apple native code returns en_US (tested with the phone set to es-US)
    * If the user sets the languages to contain a list with the supported languages AND NO language pack (Xcode style) -> native code returns es-us (get_current_locale_name), Apple native code returns es_US (tested with the phone set to es-US)
    * If the user sets the languages to an empty list -> native code returns en-us (get_current_locale_name), Apple native code returns en_US (tested with the phone set to es-US)
So after the tests, I believe that the correct way to fix the issue would be:

    1.Do not add CFBundleDevelopmentRegion default value at all. If it is not there, we will always get en-US which is the current default value we use.
    2. Show CFBundleDevelopmentRegion to en-US as default in the IDE plist.
    3. Tell users to use CFBundleLocalizations with all the languages supported. That will return the correct value EVEN when we do not have a language pack and will default to the value set in CFBundleDevelopmentRegion by the user.

Following what I said, I'm happy to give you a nice workaround to unblock you.

Add:

<key>CFBundleDevelopmentRegion</key>
<string>es</string>

To your plist to set the default language to Spanish so that if the language of the phone is not supported, you will show the Spanish version.

Add:

<key>CFBundleLocalizations</key>
    <array>
        <string>en</string>
        <string>es</string>
    </array>

To list ALL the languages you support. There is no need for you to add a locale package ( no .lproj ) since you are probably using .Net resource files. With the localisations listed you will get the correct value or the development one if you do not support a language. The you can use the normal .net localisation pattern. I'll upload the test with the final plist added for you to test.
Comment 9 Manuel de la Peña [MSFT] 2017-10-02 17:13:50 UTC
Created attachment 25054 [details]
Workaround Test App

App with the explained plist to use to workaround this issue.
Comment 10 Rolf Bjarne Kvinge [MSFT] 2017-10-03 11:07:23 UTC
I'm not sure that the workaround is suitable.

The problem is that it won't get the right formatting data.

"CultureInfo.CurrentCulture" is documented to be: "The CultureInfo object that is returned by this property and its associated objects determine the default format for dates, times, numbers, currency values, the sorting order of text, casing conventions, and string comparisons." [1]

Xamarin.iOS ships its own tables of locale data, and we use the name of the current culture to look up that data.

Say you have the following scenario:

* An app that uses and display currency amounts is developed in en_UK. CFBundleDevelopmentRegion is set to en_UK.
* App is translated into en_UK and es_ES, and sets CFBundleLocalizations to en_UK, es_ES.
* Now a Danish user finds the app and runs it. She speaks English, so she's fine with the English UI. Her locale is 'da', which is not in CFBundleLocalizations, which means CultureInfo.CurrentCulture will return 'en_UK', Xamarin.iOS will look up the currency format accordingly and all her currency amounts show up in British pounds, not Danish crowns.

@Manuel, can you test this scenario and see if my analysis is correct?

[1] https://msdn.microsoft.com/en-us/library/system.globalization.cultureinfo.currentculture(v=vs.110).aspx
Comment 11 Rolf Bjarne Kvinge [MSFT] 2017-10-03 11:16:29 UTC
BTW there's a Q&A answer from Apple about this: https://developer.apple.com/library/content/qa/qa1828/_index.html
Comment 12 Manuel de la Peña [MSFT] 2017-10-03 11:58:25 UTC
@rolf I believe it is even more complicated, we can have language that are not supported by the translation and a different region for the phone. I would be a good example, I've got my region set to Spain yet my default language is english. Looking at the different scenarios we have (and the results with the workaround):

1. Language and region match (en-US) and are present in the languages list:
    * CultureInfo.CurrentCulture is en-US.
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $
    * RegionInfo.CurrentRegion.IsMetric false
    * NSLocale.CurrentLocale.LocaleIdentifier is en_US
    * NSLocale.CurrentLocale.CurrencySymbol is $
    * NSLocale.CurrentLocale.MeasurementSystem is U.S.
2. Language and region match YET we change temperature to Celsius (en-US metric):
    * CultureInfo.CurrentCulture is en-US.
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $
    * RegionInfo.CurrentRegion.IsMetric false
    * NSLocale.CurrentLocale.LocaleIdentifier is en_US
    * NSLocale.CurrentLocale.CurrencySymbol is $
    * NSLocale.CurrentLocale.MeasurementSystem is U.S.
3. Language and region DO NOT match (en-SP) and are present in the languages list:
    * CultureInfo.CurrentCulture is en-US. (XAMARIN.IOS BUG wrong REGION)
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $ (XAMARIN.IOS BUG wrong REGION)
    * RegionInfo.CurrentRegion.IsMetric false (XAMARIN.IOS BUG wrong REGION)
    * NSLocale.CurrentLocale.LocaleIdentifier is en_ES 
    * NSLocale.CurrentLocale.CurrencySymbol is €
    * NSLocale.CurrentLocale.MeasurementSystem is Metric
4. Language and region match (fr-FR) and are NOT present in the languages list, in this case we DO expect to get en as the language, since is the default of the app AND the correct regions (is @rolf danish example):
    * CultureInfo.CurrentCulture is en-US. (XAMARIN.IOS BUG, wrong REGION)
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $ (XAMARIN.IOS BUG wrong REGION)
    * RegionInfo.CurrentRegion.IsMetric false (XAMARIN.IOS BUG wrong REGION)
    * NSLocale.CurrentLocale.LocaleIdentifier is en_FR
    * NSLocale.CurrentLocale.CurrencySymbol is €
    * NSLocale.CurrentLocale.MeasurementSystem is Metric

So in summary, the work around will fix the language issues BUT we have a different bug related to the regions, which are NOT correctly retrieved in mono. So, we are providing the wrong language-culture combinations and it is not due to the Apple change because NSLocale does the right thing when we have added the supported languages.
Comment 13 Paul Morris 2017-10-03 13:41:21 UTC
First of all thank you for the quick response and for the workaround from Manuel. It looks like the workaround will tide us over until this is fixed the right way. From a QA perspective can we get some assurance that a test (or tests) will accompany the fix and be run before public releases of Xamarin iOS to catch this sort of regression in the future?
Comment 14 Rolf Bjarne Kvinge [MSFT] 2017-10-03 18:03:01 UTC
@Paul, yes, we'll add tests, but the problem is really that we never know what Apple will do, so we can't realistically cover everything.

@Manuel, what happened in case #4 for Xamarin.iOS and iOS 10.3?
Comment 15 Paul Morris 2017-10-03 19:50:44 UTC
@Rolf I hear you. It just seemed to me that in this case even a shallow check would have shown up the problem. But I may be missing something.
Comment 16 Rolf Bjarne Kvinge [MSFT] 2017-10-05 15:52:09 UTC
I'm bumping this to d15-6, because time is running out to get any fixes in d15-5, and a workaround is available.
Comment 17 Manuel de la Peña [MSFT] 2017-10-11 09:59:00 UTC
Agreed on moving fwd the fix. I need to talk with runtime since this is a more bel related bug. We have to allow CultureInfo to have a language and region that is not present in the default values.
Comment 18 Manuel de la Peña [MSFT] 2017-11-06 12:43:50 UTC
Re tested the code and the region in the iOS devices works as expected.
Comment 19 Rolf Bjarne Kvinge [MSFT] 2017-11-06 13:38:09 UTC
App developers using Xcode runs into this too: https://jaanus.com/ios-11-changes-localized-date-handling/
Comment 20 Steve Pearson 2017-11-15 15:04:04 UTC
Potential workaround by adding to AppDelegate

            try
            {
                // Get the user's preferred language
                var language = NSLocale.PreferredLanguages[0];
                // Extract the language code
                var languageDic = NSLocale.ComponentsFromLocaleIdentifier(language);
                var languageCode = languageDic["kCFLocaleLanguageCodeKey"].ToString();
                // Force .NET culture to that of the device
                var culture = new System.Globalization.CultureInfo(languageCode);
                System.Globalization.CultureInfo.CurrentCulture = culture;
                System.Globalization.CultureInfo.CurrentUICulture = culture;
            } 
            catch (Exception)
            {
                Console.WriteLine("Failed to set culture from iOS device culture");
            }

Should set the culture based on the language of the device, appears to work ok but very early testing.
Comment 21 Tylerian 2017-11-15 15:42:54 UTC
*** Bug 60697 has been marked as a duplicate of this bug. ***
Comment 22 softlion 2017-11-30 16:05:53 UTC
Still not fixed in 15.4.5
Comment 23 softlion 2017-11-30 16:06:35 UTC
Workaround:

		    try
		    {
		        CultureInfo.CurrentCulture = CultureInfo.CurrentUICulture = new CultureInfo(culture);
		        Console.WriteLine($"mvx:Culture set to {culture}, was {startCulture}");
		    }
		    catch (Exception)
		    {
		        Console.WriteLine($"mvx:Failed to set culture");
		    }
Comment 24 Manuel de la Peña [MSFT] 2017-12-01 16:52:30 UTC
This is a small update. We have removed the default development region from being added behind the scenes to be more clear about what is happening: https://github.com/xamarin/xamarin-macios/pull/2779

We will add the key in the template for the users to edit.
Comment 25 Vincent Dondain [MSFT] 2017-12-05 19:42:16 UTC
Hi,

I created a VSTS user story (https://devdiv.visualstudio.com/DevDiv/_workitems/edit/533599) to track the IDE work as well as a documentation issue (https://github.com/xamarin/documentation/issues/2745) to document Apple's changes around this.
Comment 26 Marcin Wieczorek 2018-02-15 15:24:09 UTC
I was struggling with it for a bit, and I solved issue (in my app) by configuring properly info.plist. 
Before change in my info.plist I had something like
<key>CFBundleDevelopmentRegion</key>
<string>pl</string>
<key>CFBundleLocalizations</key>
<array>
<string>pl</string>
</array>

Now I changed it to someghing like:

<key>CFBundleDevelopmentRegion</key>
<string>en_US</string>
<key>CFBundleLocalizations</key>
<array>
<string>cs_CZ</string>
<string>da_DK</string>
<string>de_DE</string>
<string>en_US</string>
<string>nl_NL</string>
<string>pl_PL</string>
<string>pt_BR</string>
<string>pt_PT</string>
<string>ru_RU</string>
<string>sv_SE</string>
<string>zh_CN</string>
</array>

When I did it, localization started to work as expected.
Comment 27 softlion 2018-02-15 15:57:55 UTC
+1 Marcin. Sorry i forgot to report the solution.
Comment 28 Manuel de la Peña [MSFT] 2018-02-19 12:15:36 UTC
Comment 8 gave the solution with that, I wish we could somehow pin comments here :/

"
Following what I said, I'm happy to give you a nice workaround to unblock you.

Add:

<key>CFBundleDevelopmentRegion</key>
<string>es</string>

To your plist to set the default language to Spanish so that if the language of the phone is not supported, you will show the Spanish version.

Add:

<key>CFBundleLocalizations</key>
    <array>
        <string>en</string>
        <string>es</string>
    </array>
"
Comment 29 softlion 2018-02-19 12:40:07 UTC
Yes, but the phrase is not clear.

You have to add all supported languages in CFBundleLocalizations, otherwise iOS will set the app's default language to the 1st one (ie: en).
Comment 30 Manuel de la Peña [MSFT] 2018-03-26 10:24:40 UTC
Closing this bug since we landed the small change and the documentation was updated to reflect the changes:

https://docs.microsoft.com/en-us/xamarin/ios/app-fundamentals/localization/#specifying-default-and-supported-languages-in-