Bug 59596 - Xamarin iOS not get the correct CurrentCultureInfo
Summary: Xamarin iOS not get the correct CurrentCultureInfo
Status: IN_PROGRESS
Alias: None
Product: iOS
Classification: Xamarin
Component: BCL Class Libraries (show other bugs)
Version: XI 11.0 (xcode9)
Hardware: Macintosh Mac OS
: Normal major
Target Milestone: 15.6
Assignee: Manuel de la Peña
URL:
: 59604 60697 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-09-20 08:39 UTC by Javi Campaña
Modified: 2017-12-16 22:55 UTC (History)
16 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Test app. (641.37 KB, application/zip)
2017-09-20 11:22 UTC, Manuel de la Peña
Details
Workaround Test App (53.90 KB, application/zip)
2017-10-02 17:13 UTC, Manuel de la Peña
Details

Description Javi Campaña 2017-09-20 08:39:10 UTC
The var currentCulture = CultureInfo.CurrentCulture; return always en-US, but my iPhone device have es-ES for current language.
It's works correct in iOS 10, I see this bug when update the iPhone to iOS 11.
Comment 1 Manuel de la Peña 2017-09-20 11:22:25 UTC
Created attachment 24826 [details]
Test app.

I can confirm the bug with current master, Xcode9-GM. running the app and clicking the button will print the culture info. If the sim changes language, and the app is re-ran the result is not correct. Current environment information:

=== Visual Studio Community 2017 for Mac (Preview) ===

Version Preview - Internal Dogfood (7.2 build 576)
Installation UUID: 8d12e55e-3489-463f-ac52-8cb4573c5a81
Runtime:
	Mono 5.4.0.193 (2017-06/5066d707acf) (64-bit)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 504000193

=== NuGet ===

Version: 4.3.0.4199

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	1.1.2
	1.0.5
	1.0.0
SDK: /usr/local/share/dotnet/sdk/1.0.4/Sdks
SDK Versions:
	1.0.4
	1.0.0-preview2-003121
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.4.0/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.5.5
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Xamarin.Android ===

Version: 7.4.0.21 (Visual Studio Community)
Android SDK: /Users/mandel/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		4.0.3 (API level 15)
		4.3   (API level 18)
		4.4   (API level 19)
		5.0   (API level 21)
		6.0   (API level 23)
		7.1   (API level 25)

SDK Tools Version: 25.2.5
SDK Platform Tools Version: 25.0.4
SDK Build Tools Version: 25.0.1

Java SDK: /usr
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Xamarin Inspector ===

Version: 1.2.2
Hash: b71b035
Branch: d15-1
Build date: Fri, 21 Apr 2017 17:57:12 GMT

=== Apple Developer Tools ===

Xcode 9.0 (13247)
Build 9A235

=== Xamarin.iOS ===

Version: 11.3.0.6 (Visual Studio Community)
Hash: 1690ccbc
Branch: master
Build date: 2017-09-18 17:19:33+0200

=== Xamarin.Mac ===

Version: 4.1.0.7 (Visual Studio Community)

=== Build Information ===

Release ID: 702000576
Git revision: e3f9469d94ad4932e083ecaa6f998865f15fa452
Build date: 2017-09-05 04:52:47-04
Xamarin addins: 088cef5814d09f9af2866d860110075e14f24435
Build lane: monodevelop-lion-dogfood-vNext

=== Operating System ===

Mac OS X 10.12.6
Darwin 16.7.0 Darwin Kernel Version 16.7.0
    Thu Jun 15 17:36:27 PDT 2017
    root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
Comment 2 Paul DiPietro [MSFT] 2017-09-25 21:17:40 UTC
*** Bug 59604 has been marked as a duplicate of this bug. ***
Comment 3 Manuel de la Peña 2017-09-26 14:27:22 UTC
Hello,

This is a regression introduced by Apple. CFLocaleCopyCurrent(), used in the iOS code, will return the value of the application's CFBundleDevelopmentRegion Info.plist key if all of the following conditions are true:

* CFBundleDevelopmentRegion is present in the Info.plist
* The CFBundleDevelopmentRegion language is in the list of preferred languages on the iOS device, but isn't the first one
* There are no localized resources (i.e. no .lproj directory) in the app for the first preferred locale


This differs from iOS 10 where the presence of the CFBundleDevelopmentRegion key had no effect. 

Note that if the CFBundleDevelopmentRegion key is not present at all, CFLocaleCopyCurrent() always returns the first preferred locale as it did in iOS 10. I have a PR with the fix, which means that we should not set the CFBundleDevelopmentRegion on applications by default, so that users do not see this issue.

You can reproduce this behaviour, the one found in Xamarin the following way:

Assume that:

Set English as the "development language" on the Mac (resulting in CFBundleDevelopmentRegion = English)
The iOS device region/language is set to Spanish/Spain
The iOS device preferred languages is set to Spanish/Spain, English/US
Create a new application in Xcode and add some code to log the identifier of a CFLocale object obtained via CFLocaleCopyCurrent(). Deploy and run the application on an iOS device.

PR: https://github.com/xamarin/xamarin-macios/pull/2779

The above PR might not land, since this change was to be discussed.
Comment 4 Manuel de la Peña 2017-09-26 14:29:48 UTC
@sebastien I rdar might be in order.
Comment 5 Rolf Bjarne Kvinge [MSFT] 2017-09-27 09:00:16 UTC
Someone already filed a radar: http://www.openradar.me/34581827 (looks like it comes from here: https://github.com/lionheart/openradar-mirror/issues/18442)
Comment 6 Manuel de la Peña 2017-09-27 17:58:47 UTC
@rolf looking if Xcode9.1 fixed the issue.
Comment 7 Manuel de la Peña 2017-09-28 10:31:53 UTC
*** Bug 59358 has been marked as a duplicate of this bug. ***
Comment 8 Manuel de la Peña 2017-10-02 17:12:51 UTC
@Javi as per my last comment in the PR (copy paste here):

After some tests with Xcode 9 (no to be confused with 9.1)

    * If we set the CFBundleDevelopmentRegion to empty -> Mono native code returns en-us (get_current_locale_name), Apple native code returns en_US (tested with the phone set to es-US)
    * If we set the CFBundleDevelopmentRegion missing -> native code returns en-us (get_current_locale_name), Apple native code returns en_US (tested with the phone set to es-US)
    * If the user sets the languages to contain a list with the supported languages AND NO language pack (Xcode style) -> native code returns es-us (get_current_locale_name), Apple native code returns es_US (tested with the phone set to es-US)
    * If the user sets the languages to an empty list -> native code returns en-us (get_current_locale_name), Apple native code returns en_US (tested with the phone set to es-US)
So after the tests, I believe that the correct way to fix the issue would be:

    1.Do not add CFBundleDevelopmentRegion default value at all. If it is not there, we will always get en-US which is the current default value we use.
    2. Show CFBundleDevelopmentRegion to en-US as default in the IDE plist.
    3. Tell users to use CFBundleLocalizations with all the languages supported. That will return the correct value EVEN when we do not have a language pack and will default to the value set in CFBundleDevelopmentRegion by the user.

Following what I said, I'm happy to give you a nice workaround to unblock you.

Add:

<key>CFBundleDevelopmentRegion</key>
<string>es</string>

To your plist to set the default language to Spanish so that if the language of the phone is not supported, you will show the Spanish version.

Add:

<key>CFBundleLocalizations</key>
    <array>
        <string>en</string>
        <string>es</string>
    </array>

To list ALL the languages you support. There is no need for you to add a locale package ( no .lproj ) since you are probably using .Net resource files. With the localisations listed you will get the correct value or the development one if you do not support a language. The you can use the normal .net localisation pattern. I'll upload the test with the final plist added for you to test.
Comment 9 Manuel de la Peña 2017-10-02 17:13:50 UTC
Created attachment 25054 [details]
Workaround Test App

App with the explained plist to use to workaround this issue.
Comment 10 Rolf Bjarne Kvinge [MSFT] 2017-10-03 11:07:23 UTC
I'm not sure that the workaround is suitable.

The problem is that it won't get the right formatting data.

"CultureInfo.CurrentCulture" is documented to be: "The CultureInfo object that is returned by this property and its associated objects determine the default format for dates, times, numbers, currency values, the sorting order of text, casing conventions, and string comparisons." [1]

Xamarin.iOS ships its own tables of locale data, and we use the name of the current culture to look up that data.

Say you have the following scenario:

* An app that uses and display currency amounts is developed in en_UK. CFBundleDevelopmentRegion is set to en_UK.
* App is translated into en_UK and es_ES, and sets CFBundleLocalizations to en_UK, es_ES.
* Now a Danish user finds the app and runs it. She speaks English, so she's fine with the English UI. Her locale is 'da', which is not in CFBundleLocalizations, which means CultureInfo.CurrentCulture will return 'en_UK', Xamarin.iOS will look up the currency format accordingly and all her currency amounts show up in British pounds, not Danish crowns.

@Manuel, can you test this scenario and see if my analysis is correct?

[1] https://msdn.microsoft.com/en-us/library/system.globalization.cultureinfo.currentculture(v=vs.110).aspx
Comment 11 Rolf Bjarne Kvinge [MSFT] 2017-10-03 11:16:29 UTC
BTW there's a Q&A answer from Apple about this: https://developer.apple.com/library/content/qa/qa1828/_index.html
Comment 12 Manuel de la Peña 2017-10-03 11:58:25 UTC
@rolf I believe it is even more complicated, we can have language that are not supported by the translation and a different region for the phone. I would be a good example, I've got my region set to Spain yet my default language is english. Looking at the different scenarios we have (and the results with the workaround):

1. Language and region match (en-US) and are present in the languages list:
    * CultureInfo.CurrentCulture is en-US.
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $
    * RegionInfo.CurrentRegion.IsMetric false
    * NSLocale.CurrentLocale.LocaleIdentifier is en_US
    * NSLocale.CurrentLocale.CurrencySymbol is $
    * NSLocale.CurrentLocale.MeasurementSystem is U.S.
2. Language and region match YET we change temperature to Celsius (en-US metric):
    * CultureInfo.CurrentCulture is en-US.
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $
    * RegionInfo.CurrentRegion.IsMetric false
    * NSLocale.CurrentLocale.LocaleIdentifier is en_US
    * NSLocale.CurrentLocale.CurrencySymbol is $
    * NSLocale.CurrentLocale.MeasurementSystem is U.S.
3. Language and region DO NOT match (en-SP) and are present in the languages list:
    * CultureInfo.CurrentCulture is en-US. (XAMARIN.IOS BUG wrong REGION)
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $ (XAMARIN.IOS BUG wrong REGION)
    * RegionInfo.CurrentRegion.IsMetric false (XAMARIN.IOS BUG wrong REGION)
    * NSLocale.CurrentLocale.LocaleIdentifier is en_ES 
    * NSLocale.CurrentLocale.CurrencySymbol is €
    * NSLocale.CurrentLocale.MeasurementSystem is Metric
4. Language and region match (fr-FR) and are NOT present in the languages list, in this case we DO expect to get en as the language, since is the default of the app AND the correct regions (is @rolf danish example):
    * CultureInfo.CurrentCulture is en-US. (XAMARIN.IOS BUG, wrong REGION)
    * CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol is $ (XAMARIN.IOS BUG wrong REGION)
    * RegionInfo.CurrentRegion.IsMetric false (XAMARIN.IOS BUG wrong REGION)
    * NSLocale.CurrentLocale.LocaleIdentifier is en_FR
    * NSLocale.CurrentLocale.CurrencySymbol is €
    * NSLocale.CurrentLocale.MeasurementSystem is Metric

So in summary, the work around will fix the language issues BUT we have a different bug related to the regions, which are NOT correctly retrieved in mono. So, we are providing the wrong language-culture combinations and it is not due to the Apple change because NSLocale does the right thing when we have added the supported languages.
Comment 13 Paul Morris 2017-10-03 13:41:21 UTC
First of all thank you for the quick response and for the workaround from Manuel. It looks like the workaround will tide us over until this is fixed the right way. From a QA perspective can we get some assurance that a test (or tests) will accompany the fix and be run before public releases of Xamarin iOS to catch this sort of regression in the future?
Comment 14 Rolf Bjarne Kvinge [MSFT] 2017-10-03 18:03:01 UTC
@Paul, yes, we'll add tests, but the problem is really that we never know what Apple will do, so we can't realistically cover everything.

@Manuel, what happened in case #4 for Xamarin.iOS and iOS 10.3?
Comment 15 Paul Morris 2017-10-03 19:50:44 UTC
@Rolf I hear you. It just seemed to me that in this case even a shallow check would have shown up the problem. But I may be missing something.
Comment 16 Rolf Bjarne Kvinge [MSFT] 2017-10-05 15:52:09 UTC
I'm bumping this to d15-6, because time is running out to get any fixes in d15-5, and a workaround is available.
Comment 17 Manuel de la Peña 2017-10-11 09:59:00 UTC
Agreed on moving fwd the fix. I need to talk with runtime since this is a more bel related bug. We have to allow CultureInfo to have a language and region that is not present in the default values.
Comment 18 Manuel de la Peña 2017-11-06 12:43:50 UTC
Re tested the code and the region in the iOS devices works as expected.
Comment 19 Rolf Bjarne Kvinge [MSFT] 2017-11-06 13:38:09 UTC
App developers using Xcode runs into this too: https://jaanus.com/ios-11-changes-localized-date-handling/
Comment 20 Steve Pearson 2017-11-15 15:04:04 UTC
Potential workaround by adding to AppDelegate

            try
            {
                // Get the user's preferred language
                var language = NSLocale.PreferredLanguages[0];
                // Extract the language code
                var languageDic = NSLocale.ComponentsFromLocaleIdentifier(language);
                var languageCode = languageDic["kCFLocaleLanguageCodeKey"].ToString();
                // Force .NET culture to that of the device
                var culture = new System.Globalization.CultureInfo(languageCode);
                System.Globalization.CultureInfo.CurrentCulture = culture;
                System.Globalization.CultureInfo.CurrentUICulture = culture;
            } 
            catch (Exception)
            {
                Console.WriteLine("Failed to set culture from iOS device culture");
            }

Should set the culture based on the language of the device, appears to work ok but very early testing.
Comment 21 Tylerian 2017-11-15 15:42:54 UTC
*** Bug 60697 has been marked as a duplicate of this bug. ***
Comment 22 softlion 2017-11-30 16:05:53 UTC
Still not fixed in 15.4.5
Comment 23 softlion 2017-11-30 16:06:35 UTC
Workaround:

		    try
		    {
		        CultureInfo.CurrentCulture = CultureInfo.CurrentUICulture = new CultureInfo(culture);
		        Console.WriteLine($"mvx:Culture set to {culture}, was {startCulture}");
		    }
		    catch (Exception)
		    {
		        Console.WriteLine($"mvx:Failed to set culture");
		    }
Comment 24 Manuel de la Peña 2017-12-01 16:52:30 UTC
This is a small update. We have removed the default development region from being added behind the scenes to be more clear about what is happening: https://github.com/xamarin/xamarin-macios/pull/2779

We will add the key in the template for the users to edit.
Comment 25 Vincent Dondain [MSFT] 2017-12-05 19:42:16 UTC
Hi,

I created a VSTS user story (https://devdiv.visualstudio.com/DevDiv/_workitems/edit/533599) to track the IDE work as well as a documentation issue (https://github.com/xamarin/documentation/issues/2745) to document Apple's changes around this.

Note You need to log in before you can comment on or make changes to this bug.