Bug 5037 - No support for .Net style resources
Summary: No support for .Net style resources
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: MSBuild (show other bugs)
Version: 4.1.x
Hardware: PC Windows
: Highest normal
Target Milestone: 4.8.x (post async)
Assignee: dean.ellis
URL:
Depends on:
Blocks:
 
Reported: 2012-05-11 17:53 UTC by Jonathan Pobst
Modified: 2013-08-28 03:58 UTC (History)
15 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Scratch.I18n.zip (17.02 KB, application/zip)
2013-03-21 14:12 UTC, Jonathan Pryor
Details
I18nResourcesBug.zip (24.15 KB, application/zip)
2013-04-12 11:31 UTC, Jonathan Pryor
Details
Screenshot / .target and apk how it does not work (2.03 MB, application/octet-stream)
2013-04-25 13:16 UTC, Stefan Schoeb
Details

Description Jonathan Pobst 2012-05-11 17:53:35 UTC
.Net style resources work by creating satellite assemblies in directories with the language code for a name:

\bin
  - MyDll.dll
  \fr-FR
    - MyDll.resources.dll
  \fr-CA
    - MyDll.resources.dll

These assemblies need to be added the .apk, however we do not support nested directories for assemblies in the .apk, so they will have to be renamed.

This will require support in our targets, as well as the ResourceManager code that finds the appropriate assembly on disk.
Comment 4 Matthew Leibowitz 2012-08-26 10:36:27 UTC
What about creating a build step automatically creates the required folder structure that Android wants (for UI xml files)?
We don't have to use the ResourceManager to get the stings out of those files, we could use a new, cross-platform version.
This will also allow a MonoTouch build step that converts the resx to whatever format iOS likes. And then those guys could also use the cross-platform version of ResourceManager.

Or am I missing something important?
Comment 5 Jonathan Pryor 2012-08-26 10:45:50 UTC
> What about creating a build step automatically creates the required folder> 
> structure that Android wants (for UI xml files)?

While this would be nice as an "extra", the "normal" .NET Resources mechanism should also be properly supported so that existing code can be easily ported and reused.
Comment 6 Matthew Leibowitz 2012-08-26 14:44:15 UTC
I looked at this issue, and I think that there are two solutions:

Firstly, the tool (or whatever does the copying out of the apk) that copies the assemblies to the actual device should, to work best, keep and copy sub-directories. Why can't the tool do this? If all the assemblies have to be in the root, all that needs to be done is to add an extra, generated, file that simply maps the particular assembly to its correct location.
In the case of the resource files:

Before packaging:
.\Debug\en-GB\ClassLib.resources.dll
.\Debug\es-ES\ClassLib.resources.dll
.\Debug\ClassLib.dll

After packaging:
.\__SUB__en-GB__ClassLib.resources.dll
.\__SUB__en-GB__ClassLib.resources.dll
.\ClassLib.dll
.\__SUB_FOLDER_MAPPING.txt

and the contents of __SUB_FOLDER_MAPPING.txt:
__SUB__en-GB__ClassLib.resources.dll => en-GB\ClassLib.resources.dll
__SUB__es-ES__ClassLib.resources.dll => es-ES\ClassLib.resources.dll

Then when the app is unpackaged, all should work... Or am I being naïve?

If the above is not going to work, then there is the code-based solution:

On line 461 of corlib\System.Reflection\Assembly.cs, this exists:
Path.Combine (culture.Name, aname.Name + ".dll")
https://github.com/mono/mono/blob/master/mcs/class/corlib/System.Reflection/Assembly.cs#L461

This is where the problem happens. Now, in order not to break the existing functionality of Assembly, I copied the function to ResourceManager and fixed it there. ie: No longer calling MainAssembly.GetSatelliteAssembly(...), but rather this.GetSatelliteAssembly(MainAss, ...).
I am not sure if this is going to work either. Let me give an example (let me know if my logic is incorrect even though I know this is probably the most insane thing any developer can do): 
Let's say that I manually include some assemblies as a raw resource. 
I then copy them to a folder on my device. 
These extra assemblies contain a library and some extra, localized resource dlls.
I also ensure that I keep my sub-folder structure as the compiler outputs.
I then try and load it using a bunch of strings :(. 
Assuming all goes well, I create an instance of a class that tries to get its own localized resources.
It will now fail to get the correct resource as it will not be able to find that resource as we are using a different path than the default GetSatelliteAssembly method.
Or is this not possible so we don't even need to consider this?

What are your comments?
Comment 12 Jonathan Pryor 2013-03-21 14:12:43 UTC
Created attachment 3674 [details]
Scratch.I18n.zip

The fundamental problem here isn't so much that Xamarin.Android doesn't support resource assemblies -- though that is the case -- it's that mkbundle doesn't support multiple resource assemblies.

Consider Scratch.I18n.zip, which is a Console version of the original project with two changes: it includes localizations for 3 cultures (fallback, fr-CA, de-DE) and it uses the ResourceManager.GetString(string, CultureInfo) overload to use all cultures.

So let's use it with mkbundle:

    $ unzip Scratch.I18n.zip
    $ cd Scratch.I18n
    $ xbuild
    $ AS="as -arch i386" \
      CC="cc -m32" \
      PKG_CONFIG_PATH=/Library/Frameworks/Mono.framework/Libraries/pkgconfig \
      mkbundle --deps --keeptemp -o i18n bin/Debug/*.dll bin/Debug/*/*.dll

This is with Mono 3.0.6 on OS X.

The result of the mkbundle command is that it doesn't build:

> OS is: Darwin
> Sources: 5 Auto-dependencies: True
>    embedding: /Users/jon/Development/Projects/Scratch.I18n/Scratch.I18n/bin/Debug/MyResources.dll
>    embedding: /Library/Frameworks/Mono.framework/Versions/3.0.6/lib/mono/4.5/mscorlib.dll
>    embedding: /Users/jon/Development/Projects/Scratch.I18n/Scratch.I18n/bin/Debug/de-DE/MyResources.resources.dll
>    embedding: /Users/jon/Development/Projects/Scratch.I18n/Scratch.I18n/bin/Debug/de-DE/Scratch.I18n.resources.dll
>    embedding: /Users/jon/Development/Projects/Scratch.I18n/Scratch.I18n/bin/Debug/fr-CA/MyResources.resources.dll
>    embedding: /Users/jon/Development/Projects/Scratch.I18n/Scratch.I18n/bin/Debug/fr-CA/Scratch.I18n.resources.dll
> Compiling:
> as -arch i386 -o temp.o temp.s 
> temp.s:2949153:FATAL:Symbol _assembly_data_MyResources_resources_dll already defined.
> [Fail]

The cause for this is that mkbundle assumes that filenames are unique, which is not the case once resource assemblies are introduced, as we have _two_ MyResources.resources.dll assemblies, resulting in "double duplication."

This is arguably a mkbundle bug, which can be fixed by e.g. encoding the culture into the name. However, that brings us to the next deficiency: mono_register_bundled_assemblies() and the associated bundle framework only deals with filenames, not path names.

In order for Xamarin.Android to properly support resource assemblies, mkbundle needs to properly support resource assemblies (as they both use mono_register_bundled_assemblies() and the associated code).
Comment 13 Jonathan Pryor 2013-03-25 17:58:54 UTC
Fixed in monodroid/5cedcf01.
Comment 14 Jonathan Pryor 2013-04-09 13:38:08 UTC
Backported to 4.6 branch in monodroid/bs1/9dec4386.
Comment 15 Jonathan Pryor 2013-04-12 11:31:53 UTC
Created attachment 3808 [details]
I18nResourcesBug.zip

Test case. You an change your system locale to fr-CA to see fr-CA messages, or you can leave your system locale in place. The bottommost message (after running) will explicitly be fr-CA.
Comment 16 Stefan Schoeb 2013-04-22 05:48:23 UTC
Checked this bug using the given TestCase:
It does not work.

It does always show everywhere the english version:
I'm using Xamarin Studio 4.0.3 Build 13

Mono Android: tried the newest Stable(4.6.4) and Beta (4.7.x)

I tried running:
Release or Debug without Fast Build/ Shared runtime...

Additionally I disabled the linker.
What I see, the assemblies are included in the APK. Which is actually ok. So it may be a problem with the TestCase.

But as soon as I enabled the linker (Link SDK only) the localized resources are gone from the apk.

Let me know when you need any additional informations on this.
Comment 17 Jonathan Pryor 2013-04-22 22:51:40 UTC
I wasn't able to reproduce, and then I was, in line with the 6 stages of debugging: https://twitter.com/the_webhamster/status/256954058131267584

Now that I can repro the failure, I can fix it (though I'm wondering why this ever appeared to work for me in the first place; more proof I'm simply losing my mind.)

Applying the following patch to 4.6.4 Xamarin.Android.Common.targets _should_ fix things:

https://gist.github.com/jonpryor/a5767e5c1d6b4c456a95
Comment 18 NiWa 2013-04-23 03:12:35 UTC
Hi Jonathan,

for me it works in release mode and debug mode *without* fast deploy.

Greets
Comment 19 Stefan Schoeb 2013-04-23 03:13:25 UTC
With or without linker enabled?
Comment 20 NiWa 2013-04-23 03:22:46 UTC
Both "Sdk Assemblies only" and "Sdk and User Assemblies" works in Debug-Mode.
Comment 21 Jonathan Pryor 2013-04-23 10:13:56 UTC
In Debug mode linking is disabled, so that's not particularly surprising.

@NiWa: Does it work with or without the patch in Comment #17?
Comment 22 NiWa 2013-04-23 10:15:40 UTC
Hi Jon, it works with the patch.
Comment 23 Stefan Schoeb 2013-04-23 10:42:29 UTC
Actually TestCase does not work as I expect. 

When setting the language manually to france it works:
Thread.CurrentThread.CurrentUICulture = new CultureInfo("fr-CA");

Then every string is shown as a french one. When not using this line, I can do whatever I want, it keeps being english :S

But actually the resource-dll's are always included in the package which is the point of this bug. This seems to work fine.
Comment 24 Jonathan Pryor 2013-04-23 11:00:34 UTC
> When not using this line, I can do whatever I want, it keeps being english :S

I'm not sure what behavior you expect.

During _process startup_, our Java bootstrap code determines the locale that Android is set for via java.util.Locale (see obj\$(Configuration)\android\src\mono\MonoPackageManager.java).

	Locale locale       = Locale.getDefault ();
	String language     = locale.getLanguage () + "-" + locale.getCountry ();

The `language` value is used to set the $LANG environment variable, which provides the default language settings for the process, including System.Resources.

For example, if I install I18nResourcesBug onto my Android device with English as the default Android locale, English is the detected default so I get intermixed English and French in I18nResourcesBug output. (The intermixing due to the explicit use of French.)

If I then set my Android device locale to French (Settings > Personal > Language & input > Language), then LANG will be set to fr-FR, which won't match fr-CA, and thus the default will _still_ be intermixed English (default via fallback) and French. This is inline with proper System.Resources fallback semantics.

If I then rename strings.fr-CA.resx to strings.fr-FR.resx and rebuild, while maintaining my Android device's locale as French, _then_ all my displayed strings are French, not intermixed English + French.

This is as I would expect.

NOTE: Language is detected AT PROCESS STARTUP. If you launch the app, change Android's locale, then return to your app (which presumably hasn't exited yet), it will display the locale from when it was launched, NOT the just-set locale. This is a known architectural limitation (at least until I figure out how to get Android to notify me of locale changes so that I can notify the BCL...).
Comment 25 Stefan Schoeb 2013-04-25 13:16:54 UTC
Created attachment 3876 [details]
Screenshot / .target and apk how it does not work

Attached a screenshot how it looks in my emulator.
I've set the local to de-DE and now I expect to see an english and a french text. But both keep staying in english.

I did a fresh installation/run of the app after changing to german-> the process has started up in german.

It is a release build with Link SDK enabled.

In the zip my patched .target file as well as the .apk out of the release-folder.

In the .apk we can see, that the french-resources are included. So where is the problem?
Comment 26 Jonathan Pryor 2013-04-25 22:51:04 UTC
> I've set the local to de-DE and now I expect to see an english and a french
> text. But both keep staying in english.

This is correct behavior.

The app does not have de-DE translations [0]. Consequently, System.Resources will use the fallback/default localization, which is the English translation, which is what you see for the [Default locale] output.

If you want to see German, you'll need to add a strings.de-DE.resx file (or rename strings.fr-CA.resx to strings.de-DE.resx) and redeploy.

Related: the Main.axml file should be updated to use a <ScrollView/> so that you can actually see all of the <TextView/>s. As is, you can only see the first few, with no way to see the ones at the end without making your display physically larger.

Because of this you can't see the explicit uses of fr-CA strings, as they're after the "[Default locale]" strings.

[0] You can verify this by `unzip`ing the .apk, which does contain fr-CA resources, but not de-DE:

$ unzip -l I18nResourcesBug.I18nResourcesBug.apk 
Archive:  I18nResourcesBug.I18nResourcesBug.apk
  Length     Date   Time    Name
 --------    ----   ----    ----
...
     3584  04-25-13 18:56   assemblies/fr-CA/I18nResourcesBug.resources.dll
     3072  04-23-13 22:12   assemblies/fr-CA/MyResources.resources.dll
Comment 27 Stefan Schoeb 2013-04-26 02:09:21 UTC
Oh man... how embarrassing is that :-(

Yes the explicit french TextView was simply not visible -> I was searching for this.. 

At least I can now confirm, that your patch works like a charm! 
Great!

Thanks and sorry for the circumstances.
Comment 28 Tim Dawson 2013-05-14 09:51:20 UTC
Is this fix available in the stable channel yet?
Comment 29 Jonathan Pryor 2013-05-14 11:00:31 UTC
> Is this fix available in the stable channel yet?

No, a complete fix is not yet in the stable channel. 4.6.4 + the patch in Comment #17 should let things work.

HOWEVER, it won't work with Fast Deployment and Xamarin Studio 4.0.3. (I'm not sure which Xamarin Studio release has the appropriate Fast Deployment fix.) Command-line installation (`msbuild /t:Install`) and Visual Studio fast deployment should work, as will disabling fast deployment.
Comment 30 Tim Dawson 2013-05-14 11:39:48 UTC
I clicked on the link in comment #17 and have absolutely no idea what to do with that file.

I am happy to await the next stable release, I assume the fix will be included.

Thanks!
Comment 31 Pierce Boggan [MSFT] 2013-06-19 19:30:21 UTC
Using Jonathan's test case, I was able to confirm this issue *is* still present. Prashant was also able to reproduce the issue.

Version Informatoin Below:
Microsoft Visual Studio Premium 2012
Version 11.0.50727.1 RTMREL
Microsoft .NET Framework
Version 4.5.50709

Installed Version: Premium

Team Explorer for Visual Studio 2012   04941-004-0043007-02094
Microsoft Team Explorer for Visual Studio 2012

Visual Basic 2012   04941-004-0043007-02094
Microsoft Visual Basic 2012

Visual C# 2012   04941-004-0043007-02094
Microsoft Visual C# 2012

Visual C++ 2012   04941-004-0043007-02094
Microsoft Visual C++ 2012

Visual F# 2012   04941-004-0043007-02094
Microsoft Visual F# 2012

Visual Studio 2012 Code Analysis Spell Checker   04941-004-0043007-02094
Microsoft® Visual Studio® 2012 Code Analysis Spell Checker

Portions of International CorrectSpell™ spelling correction system © 1993 by Lernout & Hauspie Speech Products N.V. All rights reserved.

The American Heritage® Dictionary of the English Language, Third Edition Copyright © 1992 Houghton Mifflin Company. Electronic version licensed from Lernout & Hauspie Speech Products N.V. All rights reserved.

NuGet Package Manager   2.0.30625.9003
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

PreEmptive Analytics Visualizer   1.0
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

Xamarin.Android   4.7.10024 (c27a9cd9)
Visual Studio plugin to enable development for Xamarin.Android.

Xamarin.iOS   1.1.200 (7d63692c)
Visual Studio extension to enable development for Xamarin.iOS
Comment 33 Tim Dawson 2013-06-20 04:39:45 UTC
It was my support request that caused this bug to be reopened. I await with interest to see what is wrong; it's been a long time since this functionality was supposed to be enabled.
Comment 34 Jonathan Pryor 2013-06-20 23:03:48 UTC
Works for me...and fails for me.

WORKS:

<voice style="Carman>Screw VS, I'm going home!</voice>

Open the Developer Command Prompt for Visual Studio 20XX, then enter:

    # uninstall app from target, then:
    > cd Path\I18nResourcesBug\I18nResourcesBug
    > msbuild /t:Install I18nResourcesBug.csproj

For good measure, locate `adb.exe` in the Android SDK, and run:

>	> adb shell ls -l /data/data/I18nResourcesBug.I18nResourcesBug/files/.__override__
>	-rw-rw-rw- shell    shell        6656 2013-06-20 22:58 I18nResourcesBug.dll
>	-rw-rw-rw- shell    shell         885 2013-06-20 22:58 I18nResourcesBug.dll.mdb
>	-rw-rw-rw- shell    shell        4096 2013-06-20 22:58 MyResources.dll
>	-rw-rw-rw- shell    shell         313 2013-06-20 22:58 MyResources.dll.mdb
>	drwxrwxr-x shell    shell             2013-06-20 22:58 fr-CA

Note the fr-CA directory.

FAILS:

Run normally from Visual Studio (Debug, etc.), and it fails as stated in Comment #31. This is because the fr-CA directory is missing.

Interestingly, after installing from Visual Studio, if you install using MSBuild (see WORKS, above), the fr-CA directory is created, and when the app is restarted it displays correctly.
Comment 36 Tim Dawson 2013-06-24 12:00:45 UTC
When will this be fixed, please? From what our customers are telling us in our latest version, localization IS partially working in that the appropriate .resx file has been chosen according to the device settings, but the user cannot switch language using the standard .net means of calling ResourceManager.GetString with an explicit CultureInfo or setting the static Culture property of the resources codegen class.
Comment 37 Jonathan Pryor 2013-06-24 14:23:30 UTC
@Tim:

> the user cannot switch language using the standard .net means of calling
> ResourceManager.GetString with an explicit CultureInfo

Does I18nResourcesBug.zip (Attachment #3808 [details]) work for you (though you may need to install via command line, as mentioned in Comment #34)? It uses ResourceManager.GetString(string, CultureInfo) within I18nResourcesBug/I18nResourcesBug/Activity1.cs:

	var c = CultureInfo.GetCultureInfo ("fr-CA");
	text = FindViewById<TextView>(Resource.Id.myText_fr);
	text.Text = _manager.GetString ("MY_STRING", c);
Comment 39 Tim Dawson 2013-06-25 05:34:33 UTC
I confirm that works for me, but only when I install via the command line. The problem appears to be with the Visual Studio debug deployment, which of course we use for testing.
Comment 43 Stephen Shaw 2013-07-02 14:47:40 UTC
Looks like this might be a problem with fast deployment. If I switch to Release it works, and if I use debug, but with fast deployment disable then it works. Console or XS.

I want to say that msbuild by default builds Release and that's probably why it works on the console, but not in VS assuming VS is set to debug. xbuild on the other hand I think defaults to debug?


In LogCat, I get this ->

07-02 11:27:08.913: A/MonoDroid(9219): No assemblies found in '/data/data/I18nResourcesBug.I18nResourcesBug/files/.__override__' or '/storage/emulated/0/Android/data/I18nResourcesBug.I18nResourcesBug/files/.__override__'. Assuming this is part of Fast Deployment. Exiting...
Comment 45 András Tóth 2013-08-27 05:20:49 UTC
I have downloaded Xamarin for Android 4.8.1 and it works well on desktop Windows, but on Android I am still getting only the default language. In other words, setting culture in any ways does not force the ResourceManager to use other resx other than the default.
Here is what we tried:
- setting Thread.CurrentThread.Current(UI)Culture = new CultureInfo(_otherLanguage_)
- switching Locale in the simulator and restarting application
- directly define the Culture of the resource: MyResource.Culture = new CultureInfo(...)

The Build Action for all of the resource files are "Embedded Resource".

Have we missed something, or is this fix not in the current release?
Comment 46 András Tóth 2013-08-27 05:30:58 UTC
Someone mentioned it is working with Release build. We can confirm that. We tried in Debug but it always showed the default file. 
Let us know then, if it works in Debug mode.
Comment 47 Jonathan Pryor 2013-08-27 11:30:49 UTC
András Tóth: Which IDE are you using, Visual Studio or Xamarin Studio?

"Fast deployment" (Debug deployment) requires Xamarin Studio 4.0.11 or later and Xamarin.Android 4.8.2 or later, while Visual Studio should work for Xamarin.Android 4.8.2 and later.

Note You need to log in before you can comment on or make changes to this bug.