Bug 54356 - 'No data is available for encoding 737' when creating ZipArchive with device language set to Greek
Summary: 'No data is available for encoding 737' when creating ZipArchive with device ...
Alias: None
Product: iOS
Classification: Xamarin
Component: General ()
Version: XI 10.3 (iOS 10.2)
Hardware: Other Other
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
Depends on: 54361
  Show dependency tree
Reported: 2017-03-31 19:47 UTC by Paul Morris
Modified: 2017-06-06 16:14 UTC (History)
4 users (show)

Is this bug a regression?: ---
Last known good build:

Imprecise test case that fails even when the linker is OFF (10.85 KB, application/zip)
2017-03-31 23:33 UTC, Brendan Zagaeski (Xamarin Team, assistant)

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Paul Morris 2017-03-31 19:47:52 UTC
On an iOS device with Greek as the device language, when attempting to new up a ZipArchive with or without specifying an encoding in the constructor, I get this exception:

"System.TypeInitializationException: The type initializer for 'SharpCompress.Common.ArchiveEncoding' threw an exception. ---> System.NotSupportedException: No data is available for encoding 737."

This causes a catastrophic failure for users of our app in Greece. They must change their device language to another language such as English to continue using the app.

All the Internationalization options are checked in the build configuration: cjk, mideast, other, rare, west

WORKAROUND: If linker is set to 'Do Not Link' it works. So something is being stripped at compile time. This workaround is not a practical option for shipping the app. Is there another workaround that will allow us to Link Framework SDKs?
Comment 1 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-31 21:23:13 UTC
## Bookkeeping note to the Xamarin Team

I dug into this kind of issue once before on Bug 37465, and I have some ideas for the reporter of the bug to try (as well as some info about the status of importing the CoreFx implementation of ZipArchive), so I'm working on a little initial response for this report.  I'll aim to post that reply shortly.
Comment 2 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-31 23:33:50 UTC
Created attachment 21120 [details]
Imprecise test case that fails even when the linker is OFF

## The symptoms in Comment 0 are probably already fixed indirectly in the current "15.2" source code branch of Xamarin.iOS

Caveat: These 2 results are for the _attached test case_.  The behavior of this test case is at least a bit different from the exact symptoms as reported in Comment 0: it hits the error _regardless of the linker setting_.  I'll say a bit more on that later, but long story short I suspect that the un-QA'd, experimental-quality 10.9.0 build linked below would be one (somewhat indirect) way to stop the problem described in Comment 0.

> GOOD: Xamarin.iOS (d15-2: 0719ced1) (Latest CI development build for the v-next "15.2" release from [1])
> BAD:  Xamarin.iOS (d15-1: a04678c2) (Latest CI development build for the upcoming, currently in Beta, "15.1" release from [2])
[1] From https://jenkins.mono-project.com/view/Xamarin.MaciOS/job/xamarin-macios-builds-d15-2/2/PublishUrls/
[2] From https://jenkins.mono-project.com/view/Xamarin.MaciOS/job/xamarin-macios-builds-d15-1/57/PublishUrls/

### Explanation

Xamarin.iOS 10.9 uses Mono's 2017-02 branch, and in that branch, `ZipArchive` has now been switched to the CoreFx implementation [3].  As described in Bug 37465, Comment 2, this means that `ZipArchive` no longer calls the problematic `System.Text.Encoding.GetEncoding()` method at all.

[3] https://github.com/mono/mono/tree/2017-02/mcs/class/System.IO.Compression

(Note: In Xamarin.iOS 10.9, `System.Text.Encoding.GetEncoding(737)` still produces the "No data is available for encoding 737" message, so the change in behavior for `ZipArchive` is because the implementation of `ZipArchive` has changed, and not because Mono has added support for codepage 737.)

## Alternate possible workaround

The attached test case explicitly sets the `CurrentCulture` to Greek to simplify testing on simulators or devices that aren't set to the Greek language.  This approach can potentially be used "in reverse" as a workaround for the problem reported in Comment 0.  That is, it might be acceptable to do:

CultureInfo originalCulture = Thread.CurrentThread.CurrentCulture;
Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US");
// Put the first call to `new ZipArchive()` here
Thread.CurrentThread.CurrentCulture = originalCulture;

This is effective, at least in the attached test case (when the simulator is set to Greek language).  Note that it is only necessary to add these lines around the _first_ call to `new ZipArchive()` because there is a static constructor for the internal `ArchiveEncoding` class that caches the initial `CurrentCulture` for all subsequent calls to `new ZipArchive()`.

I'm not sure if there might be some performance or rendering considerations of switching the CurrentCulture back and forth like this, but I think it might be alright.

## Possible approaches for further investigation of the _linker_ issue in particular

### One idea: Work on getting a test case attached onto this bug report that demonstrates the issue

As mentioned earlier, my first attempt at replicating this behavior (in the attached test case) is not 100% accurate.

The problem with this initial attempt is that it _always_ hits the error, regardless of the linker setting.

### Another idea: Iteratively narrow down the linker settings needed to avoid the issue

Iterating on the linker settings is the next step I think the Xamarin team would try once a test case had been attached on the bug.  But the reporter of the bug can also definitely try those steps locally in case that might be quicker than trying to provide a shareable test case.

The rough outline for the process would be:

A. First narrow down the particular assembly that needs to be skipped.  You can do this by testing each of the SDK assemblies by adding `--linksip` arguments under "Project options > Additional mtouch arguments".

(See "Skipping Assemblies" on https://developer.xamarin.com/guides/ios/advanced_topics/linker/#Skipping_Assemblies)

B. Next, you can narrow down the individual types from the assembly that need to be skipped by using a custom linker configuration.

(See https://developer.xamarin.com/guides/cross-platform/advanced/custom_linking/)

One approach is to start with a `fullname` wildcard for each namespace in the assembly and then work down.  For example, you could start with something like the following if step (A) revealed that System.IO.Compression was the problematic assembly:

        <assembly fullname="System.IO.Compression">
                <type fullname="SharpCompress*" />
                <type fullname="System*" />
Comment 3 Paul Morris 2017-04-03 13:09:45 UTC
I thank Brendan Zagaeski for his prompt response.

I tried the workaround tagged as "## Alternate possible workaround" wherein the culture is temporarily changed to en-US and a ZipArchive new-ed up before switching the culture back. I placed this code in the start-up method of AppDelegate. This did _not_ prevent the error when new ZipArchive was later called. That surprised me because the ArchiveEncoding ctor is static.

I am going to move on to linker settings. I am perplexed as to why "Don't Link" prevents this error right now since Brendan Zagaeski indicates that there simply is no support for code page 737. But I have to move forward with linker settings since that appears to be the most promising short-term solution. I need to ship an update this week to fix the app for our users in Greece.
Comment 4 Paul Morris 2017-04-03 19:41:25 UTC
Added a linker configuration file like this:

    <assembly fullname="mscorlib">
	<type fullname="System*" />

If we had more time we could narrow this down more than 'System*' but I have to move on to other work. 

For the record I tried System.IO*, System.Text* and System.Globalization* together but those weren't enough.
Comment 5 Brendan Zagaeski (Xamarin Team, assistant) 2017-04-03 20:38:49 UTC
Glad to hear the custom linker definition is successful as an intermediate workaround for now.  And interesting to know that something is apparently changing in the "codepage" lookup after linking.  I'll keep an eye out for reports of similar symptoms in the future.

> ... temporarily changed to en-US and a ZipArchive new-ed up before
> switching the culture back...
> placed this code in the start-up method of AppDelegate. This did _not_
> prevent the error when new ZipArchive was later called. That surprised
> me because the ArchiveEncoding ctor is static.
In case it might be helpful, one possible explanation for this behavior is if the first new-ing up of `ZipArchive` was opening an empty archive or creating a new archive.  I forgot to mention that `ArchiveEncoding` only gets called during `new ZipArchive()` if the archive stream that is passed in contains at least 1 zip entry.
Comment 6 Paul Morris 2017-04-03 21:08:57 UTC
Ha! Wouldn't you know that I new-ed it up with an empty MemoryStream! I should have realized the code would recognize that as invalid and return early. 

You mention this is likely fixed in 15.2 due to the CoreFx API so we look forward to moving to that in the near future and not having to skip linking any of mscorlib.

We have gotten good responses from the Xamarin team when reporting bugs. Please pass along our appreciation.
Comment 7 Timothy Risi 2017-05-11 23:05:36 UTC
Hi Paul,

Now that 15.2 has released into the stable branch, could you check this again and see if you still run into the issue without using the workaround?
Comment 8 Paul Morris 2017-06-06 15:40:32 UTC
Hi Timothy,

Just tested now after removing the exceptions in linker.xml and it looks good. Thanks again!

Comment 9 Timothy Risi 2017-06-06 16:14:34 UTC
Glad it works now :)