Bug 56834 - Can't write resource Multidex warnings with VS for Mac
Summary: Can't write resource Multidex warnings with VS for Mac
Status: CONFIRMED
Alias: None
Product: Android
Classification: Xamarin
Component: General (show other bugs)
Version: 7.3 (15.2)
Hardware: Macintosh Mac OS
: --- major
Target Milestone: 15.6
Assignee: Jonathan Pryor
URL:
Depends on:
Blocks:
 
Reported: 2017-05-25 11:58 UTC by Nate Cook
Modified: 2017-09-26 14:38 UTC (History)
7 users (show)

See Also:
Tags: 15.2I
Is this bug a regression?: ---
Last known good build:


Attachments
Sample project (31.37 KB, application/zip)
2017-05-25 11:58 UTC, Nate Cook
Details

Description Nate Cook 2017-05-25 11:58:17 UTC
# Steps to reproduce
Create a new native Android application
Enable Multidex (debug build configuration is fine)
Build

# Expected behavior
No warnings. (Xamarin Studio 6.3 produces no warnings)

# Actual behavior
CREATEMULTIDEXMAINDEXCLASSLIST: Warning: can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [android-support-multidex.jar:META-INF/MANIFEST.MF])

We are getting many of these warnings with a larger application. Here are the warnings we get with a larger application:

Target _CompileToDalvikWithDx:
    Creating directory "obj/Debug/proguard".
    /Users/ncook/Library/Developer/Xamarin/android-sdk-macosx/build-tools/25.0.3/mainDexClasses --output obj/Debug/multidex.keep '/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v5.0/mono.android.jar':'/Library/Frameworks/Xamarin.Android.framework/Versions/Current/lib/mandroid/android-support-multidex.jar':'obj/Debug/__library_projects__/UrlImageViewHelper/library_project_imports/urlimageviewhelper-1.0.4.jar':'obj/Debug/__library_projects__/mp4parser/library_project_imports/__reference__aspectjrt-1.7.3.jar':'obj/Debug/__library_projects__/mp4parser/library_project_imports/isoparser-1.0.5.4.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.Animated.Vector.Drawable/23.4.0.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.CustomTabs/23.4.0.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.Design/23.4.0.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v4/23.4.0.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v4/23.4.0.0/embedded/libs/internal_impl-23.4.0.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v7.AppCompat/23.4.0.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v7.RecyclerView/23.4.0.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.Vector.Drawable/23.4.0.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Analytics/8.4.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Base/8.4.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Basement/8.4.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Gcm/8.4.0/embedded/classes.jar':'/Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Measurement/8.4.0/embedded/classes.jar':'obj/Debug/android/bin/classes' 
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [android-support-multidex.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [urlimageviewhelper-1.0.4.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [__reference__aspectjrt-1.7.3.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [isoparser-1.0.5.4.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [.readme] (Duplicate zip entry [classes.jar:.readme])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [.readme] (Duplicate zip entry [classes.jar:.readme])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [classes.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [classes.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [classes.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [classes.jar:META-INF/MANIFEST.MF])
    CREATEMULTIDEXMAINDEXCLASSLIST : warning : can't write resource [META-INF/MANIFEST.MF] (Duplicate zip entry [classes.jar:META-INF/MANIFEST.MF])
    /usr/bin/java -Xmx2G -jar /Users/ncook/Library/Developer/Xamarin/android-sdk-macosx/build-tools/25.0.3/lib/dx.jar --no-strict --dex --multi-dex --main-dex-list=obj/Debug/multidex.keep --output=obj/Debug/android/bin obj/Debug/android/bin/classes /Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v5.0/mono.android.jar /Library/Frameworks/Xamarin.Android.framework/Versions/Current/lib/mandroid/android-support-multidex.jar obj/Debug/__library_projects__/UrlImageViewHelper/library_project_imports/urlimageviewhelper-1.0.4.jar obj/Debug/__library_projects__/mp4parser/library_project_imports/__reference__aspectjrt-1.7.3.jar obj/Debug/__library_projects__/mp4parser/library_project_imports/isoparser-1.0.5.4.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.Animated.Vector.Drawable/23.4.0.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.CustomTabs/23.4.0.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.Design/23.4.0.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v4/23.4.0.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v4/23.4.0.0/embedded/libs/internal_impl-23.4.0.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v7.AppCompat/23.4.0.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.v7.RecyclerView/23.4.0.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.Android.Support.Vector.Drawable/23.4.0.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Analytics/8.4.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Base/8.4.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Basement/8.4.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Gcm/8.4.0/embedded/classes.jar /Users/ncook/.local/share/Xamarin/Xamarin.GooglePlayServices.Measurement/8.4.0/embedded/classes.jar 
    Creating "obj/Debug/_dex_stamp" because "AlwaysCreate" was specified.
Done building target "_CompileToDalvikWithDx" in project "Xyz.Droid.csproj".

Others are reporting similar in this forum thread: 
https://forums.xamarin.com/discussion/55601/in-a-xamarin-android-project-how-can-i-remove-warnings-duplicate-zip-entry-classes-jar

# Supplemental info (logs, images, videos)
A sample project will be attached

# Test environment (full version information)
=== Visual Studio Professional 2017 for Mac ===

Version 7.0.1 (build 24)
Installation UUID: 3234d07d-f7a0-4180-b21d-d2c24ed872f6
Runtime:
	Mono 5.0.1.1 (2017-02/5077205) (64-bit)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 500010001

=== NuGet ===

Version: 4.0.0.2323

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
SDK: /usr/local/share/dotnet/sdk/1.0.0-preview2-003156/Sdks
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.5.4
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Xamarin.Android ===

Version: 7.3.1.1 (Visual Studio Professional)
Android SDK: /Users/ncook/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		4.4 (API level 19)
		5.0 (API level 21)
		5.1 (API level 22)
		6.0 (API level 23)
		7.0 (API level 24)
		7.1 (API level 25)

SDK Tools Version: 25.2.5
SDK Platform Tools Version: 25.0.6
SDK Build Tools Version: 25.0.3

Java SDK: /usr
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Apple Developer Tools ===

Xcode 8.3.2 (12175)
Build 8E2002

=== Xamarin.iOS ===

Version: 10.10.0.36 (Visual Studio Professional)
Hash: d2270eec
Branch: d15-2
Build date: 2017-05-22 16:30:53-0400

=== Xamarin.Mac ===

Version: 3.4.0.36 (Visual Studio Professional)

=== Xamarin Inspector ===

Version: 1.2.2
Hash: b71b035
Branch: d15-1
Build date: Fri, 21 Apr 2017 17:57:12 GMT

=== Build Information ===

Release ID: 700010024
Git revision: 7ab1ca2ced6f584e56b7a0d4d321d00775cd95c9
Build date: 2017-05-19 05:44:51-04
Xamarin addins: 08d17158f3365beee5e60f67999e607cce4b3f93
Build lane: monodevelop-lion-d15-2

=== Operating System ===

Mac OS X 10.12.5
Darwin 16.6.0 Darwin Kernel Version 16.6.0
    Fri Apr 14 16:21:16 PDT 2017
    root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64
Comment 1 Nate Cook 2017-05-25 11:58:37 UTC
Created attachment 22453 [details]
Sample project
Comment 2 Jon Douglas [MSFT] 2017-06-16 17:11:40 UTC
This happens in the CreateMultiDexMainDexList Task. Specifically, there are duplicates MANIFEST.MF being added in the call to proguard:

call "%proguard%" -injars %params% -dontwarn -forceprocessing  -outjars "%tmpJar%" -libraryjars "%shrinkedAndroidJar%" -dontoptimize -dontobfuscate -dontpreverify -include "%baserules%" 1>nul

Here's the original Task command:

C:\Android\android-sdk\build-tools\25.0.2\\mainDexClasses.bat --output obj\Debug\multidex.keep "'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\ReferenceAssemblies\Microsoft\Framework\MonoAndroid\v5.0\mono.android.jar';'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Xamarin\Android\android-support-multidex.jar';'obj\Debug\android\bin\classes'" 

There's a snippet from the ProGuard usage here:

https://www.guardsquare.com/en/proguard/manual/troubleshooting#duplicatezipentry

Your input jars contain multiple resource files with the same name. ProGuard continues copying the resource files as usual, skipping any files with previously used names. Once more, the warning may be an indication of some problem though, so it's advisable to remove the duplicates. A convenient way to do so is by specifying filters on the input jars. There is no option to switch off these warnings.

What this means is that the [META-INF/MANIFEST.MF] file inside these .jars are considered duplicate. Thus there needs to be a fix by excluding the META-INF/MANIFEST.MF file in each jar. Here's how it's done in Gradle:

        exclude "META-INF/MANIFEST.MF"
        exclude "META-INF/LICENSE.txt"
        exclude "META-INF/NOTICE.txt"

A couple ways I think this could be fixed:

1. https://www.guardsquare.com/en/proguard/manual/examples#resourcefiles

Specify the following in the default proguard.cfg:

-adaptresourcefilecontents **.properties,META-INF/MANIFEST.MF

2. https://www.guardsquare.com/en/proguard/manual/examples#manifestfiles

Ignore future MANIFEST.MF files with filters

-injars  in1.jar 
-injars  in2.jar(!META-INF/MANIFEST.MF) 
-injars  in3.jar(!META-INF/MANIFEST.MF) 
-outjars out.jar

Which will ignore any further manifest files.

Marking as CONFIRMED due to reproducing with https://bugzilla.xamarin.com/show_bug.cgi?id=56834#c1
Comment 3 Nate Cook 2017-06-27 18:06:52 UTC
@Jon Is there any workaround for this issue while we're waiting for this to be addressed? I have found tutorials on how to add a custom proguard file for use with Xamarin, but many of them seem out of date. Actually one of the more recent ones is your own here

http://www.jon-douglas.com/2016/11/22/xamarin-android-proguard/

but it seems mostly applicable to those wanting to shrink their APK size. We don't have proguard enabled; it seems that proguard is run on our behalf though because we multidex enabled. So I'm not sure how applicable the steps in the tutorial are to someone with an app using only multidex and not proguard.
Comment 4 Jon Douglas [MSFT] 2017-06-27 18:22:28 UTC
Multidex in turn runs a call to proguard. So technically you are still using proguard. I even listed this above in my comment showing the actual "call proguard" call from the CreateMultidexMainList task.

As for a workaround, unless there is a way to define these parameters via a msbuild property, there won't be an easy workaround without messing with the .targets. Honestly it's all just noise in logs that could probably be ignored to be quite honest until a fix is issued.

As for my blog, there is a difference between "Using proguard" and "Multidex using proguard". For example, using proguard would "Shrink" your application with whatever is unused, etc. However multidex using proguard is purely for an internal task which helps generate a main dex list.

If you were especially curious, open up the C:\Android\android-sdk\build-tools\25.0.2\\mainDexClasses.bat file and see what's happening :)

TLDR; When multidex needs to split up your classes to multiple dex files, it internally uses Proguard to determine which classes have to be kept in the main dex list.
Comment 5 Nate Cook 2017-06-27 18:33:59 UTC
Thanks for clearing that up. Since there is no workaround, is this something that can be fixed any time soon? It would be just noise in the logs if it were happening only with release builds (indeed--there are a bunch of other warnings showing up for release builds that I did not log bugzillas for), but this happens with Debug builds as well. That means the warnings show up while developing, distracting us from more important warnings.
Comment 6 alex 2017-07-06 02:03:26 UTC
Also interesting in sane workarounds for this problem.
Comment 7 alex 2017-07-06 02:03:37 UTC
Also interesting in sane workarounds for this problem.

Note You need to log in before you can comment on or make changes to this bug.