Bug 59036 - Multidex.keep file missing newlines
Summary: Multidex.keep file missing newlines
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: MSBuild (show other bugs)
Version: 7.4 (15.3)
Hardware: PC Mac OS
: High blocker
Target Milestone: 15.4
Assignee: dean.ellis
URL:
: 58940 59031 59369 (view as bug list)
Depends on: 59789
Blocks:
  Show dependency tree
 
Reported: 2017-08-25 09:13 UTC by nathan.lecoanet
Modified: 2017-10-05 10:32 UTC (History)
15 users (show)

See Also:
Tags:
Is this bug a regression?: Yes
Last known good build: Xamarin.Android 7.3 (d15-2)


Attachments
Reproduction (539.41 KB, application/x-zip-compressed)
2017-08-25 16:46 UTC, Jon Douglas [MSFT]
Details
Adjusted Test Case (for Comment 7) (77.42 KB, application/zip)
2017-09-06 03:14 UTC, Brendan Zagaeski (Xamarin Support)
Details

Description nathan.lecoanet 2017-08-25 09:13:51 UTC
The autogenerated multidex.keep file is missing new lines between each class so the end dex files are wrong and we are unable to launch the app on Android <5.

Steps to reproduce:
1. Create a Xamarin.Android app 
2. Enable multidex support
3. The generated multidex.keep file is incorrect


Example, from a completely new Android project:
https://gist.github.com/nathan-l/01050a888a42778969fbe4c6112b86c3

This is happening on Xamarin.Android 7.4.0

The multidex.keep file is properly generated on Xamarin.Android 7.1.1 



Fix:
Take the generated multidex file in obj/(Debug|Release)/multidex.keep.
Modify it to add a new line between each .class file.
Add the modified file in your android project with a build action of MultiDexMainDexList
Comment 1 Jon Douglas [MSFT] 2017-08-25 16:22:04 UTC
I can CONFIRM this issue on VS4Mac 7.1 and 7.2. This is also a problem on Windows as well using 15.3 as a baseline of testing.

The multidex.keep file that is generated does not include carriage returns and thus is an erroneous MultiDexMainDexListFile input parameter in the CompileToDalvik Task. However I believe the dx.jar command doesn't really care what's in that file so long as it's a file. If it did, it would throw an error saying the --main-dex-list is not formatted properly. I am guessing it silently runs as if it was properly formatted, yet doesn't apply the kept classes in the toolchain.

It seems that this was hinted at in the following bug report as well but I overlooked that detail initially to help the broader picture:

https://bugzilla.xamarin.com/show_bug.cgi?id=59031#c0

Thus it seems like either a bug in the CreateMultiDexMainDexClassList Task or build-tools.

I tested the following versions of build-tools first, in which 26.0.1, 26.0.0 and 25.0.3 all act and behave correctly. 

I then ran the command that the CreateMultiDexMainDexClassList Task uses inside a command line:

/usr/bin/java -Djava.ext.dirs=/Users/jonathandouglas/Library/Android/sdk/build-tools/26.0.1/lib com.android.multidex.MainDexListBuilder /var/folders/9n/5yc3jxkd4l39643n67qpslth0000gn/T/djvu9qw3.mmq.jar /Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v7.1/mono.android.jar:/Library/Frameworks/Xamarin.Android.framework/Versions/Current/lib/mandroid/android-support-multidex.jar:/Users/jonathandouglas/.local/share/Xamarin/Xamarin.Android.Support.v4/23.1.1.0/embedded/classes.jar:/Users/jonathandouglas/.local/share/Xamarin/Xamarin.Android.Support.v4/23.1.1.0/embedded/libs/internal_impl-23.1.1.jar:/Users/jonathandouglas/.local/share/Xamarin/Xamarin.Android.Support.v7.AppCompat/23.1.1.0/embedded/classes.jar:obj/Debug/android/bin/classes

This generates the following output(Notice correct returns):

android/support/v4/media/MediaMetadataCompat$RatingKey.class
android/support/v4/media/session/PlaybackStateCompatApi21.class
android/support/v4/media/MediaBrowserCompat$MediaItem.class
android/support/v4/view/ViewCompat$ScrollIndicators.class
android/support/v4/media/session/PlaybackStateCompat$1.class
android/support/v4/media/session/MediaSessionCompat$MediaSessionImplApi21.class
android/support/v4/view/ViewCompatLollipop.class
android/support/v4/media/MediaBrowserCompat$MediaItem$1.class
android/support/v4/widget/CursorAdapter.class
android/support/v4/media/MediaMetadataCompat$LongKey.class
android/support/v4/widget/SimpleCursorAdapter.class
android/support/v4/view/MenuCompat.class
android/support/v4/view/ViewCompat$JbMr1ViewCompatImpl.class
android/support/v4/media/session/MediaControllerCompat.class
android/support/v7/view/ActionMode$Callback.class
android/support/v4/view/ViewCompatHC.class
android/support/v4/view/AccessibilityDelegateCompat.class
android/support/v4/view/ViewCompat$HCViewCompatImpl.class
android/support/v4/media/VolumeProviderCompat.class
android/support/v4/view/ViewCompat$ICSViewCompatImpl.class
android/support/v4/app/Fragment.class
android/support/v4/media/VolumeProviderCompat$ControlType.class
android/support/v4/view/ViewCompat$EclairMr1ViewCompatImpl.class
android/support/v4/view/ViewPropertyAnimatorCompat.class
android/support/v4/widget/MaterialProgressDrawable$3.class
android/support/v4/view/ViewCompat$BaseViewCompatImpl.class
android/support/v4/media/session/MediaSessionCompat$MediaSessionImplBase.class
android/support/multidex/MultiDexExtractor.class
android/support/v7/appcompat/R$style.class
android/support/v4/view/ScrollingView.class
android/support/v4/content/PermissionChecker$PermissionResult.class
android/support/v4/view/ViewCompatJellybeanMr1.class
android/support/v7/app/ActionBar.class
android/support/v4/view/NestedScrollingChild.class
android/support/v4/view/ViewCompatICSMr1.class
android/support/v4/view/animation/LookupTableInterpolator.class
android/support/v7/view/menu/MenuItemImpl.class
android/support/v4/widget/MaterialProgressDrawable$2.class
android/support/v7/appcompat/R$styleable.class
android/support/v7/app/ActionBar$Tab.class
android/support/v4/app/FragmentManager.class
android/support/v4/view/ViewCompatGingerbread.class
android/support/multidex/MultiDexApplication.class
android/support/v4/view/ViewCompatKitKat.class
android/support/v4/media/MediaMetadataCompat.class
android/support/v4/widget/MaterialProgressDrawable.class
android/support/v7/widget/LinearLayoutCompat$DividerMode.class
android/support/v4/app/ActionBarDrawerToggle.class
android/support/v4/media/MediaBrowserCompat$ConnectionCallback.class
android/support/v4/widget/ResourceCursorAdapter.class
android/support/v4/media/MediaDescriptionCompat$Builder.class
android/support/v4/widget/MaterialProgressDrawable$1.class
android/support/v4/media/MediaMetadataCompat$Builder.class
android/support/v7/app/ActionBar$DisplayOptions.class
android/support/v4/media/RatingCompat$1.class
android/support/v4/view/ViewCompat$JBViewCompatImpl.class
android/support/v7/widget/LinearLayoutCompat$OrientationMode.class
android/support/v7/graphics/drawable/DrawerArrowDrawable$ArrowDirection.class
android/support/multidex/MultiDex.class
android/support/v4/media/MediaMetadataCompatApi21.class
android/support/v4/media/session/MediaSessionCompat$1.class
android/support/v4/media/RatingCompat.class
android/support/v4/view/accessibility/AccessibilityNodeInfoCompat.class
android/support/v4/app/FragmentTabHost.class
android/support/v4/media/session/MediaSessionCompat$SessionFlags.class
android/support/v4/view/ViewCompatJellybeanMr2.class
android/support/v4/view/ViewCompat$ICSMr1ViewCompatImpl.class
android/support/v4/util/ArrayMap.class
android/support/v4/app/NotificationCompat$Builder.class
android/support/v4/content/PermissionChecker.class
android/support/v4/widget/MaterialProgressDrawable$ProgressDrawableSize.class
android/support/v7/widget/ViewUtils.class
android/support/v4/media/session/MediaSessionCompat$Callback.class
android/support/v4/view/WindowInsetsCompat.class
android/support/v7/widget/LinearLayoutCompat.class
android/support/v4/media/MediaMetadataCompat$1.class
android/support/v4/media/MediaBrowserCompat.class
android/support/v4/media/MediaBrowserCompat$ItemCallback.class
android/support/v4/view/ViewCompat$LollipopViewCompatImpl.class
android/support/v4/media/session/PlaybackStateCompat$CustomAction.class
android/support/v4/media/MediaBrowserCompat$MediaItem$Flags.class
android/support/multidex/ZipUtil.class
android/support/v4/media/session/PlaybackStateCompat$Actions.class
android/support/v4/view/ViewCompat.class
android/support/multidex/MultiDexExtractor$1.class
android/support/v7/app/ActionBarActivity.class
android/support/v4/view/ViewCompatMarshmallow.class
android/support/v4/util/SimpleArrayMap.class
android/support/v7/widget/LinearLayoutCompat$LayoutParams.class
android/support/v4/media/session/MediaSessionCompat$Token.class
android/support/v7/graphics/drawable/DrawerArrowDrawable.class
android/support/v4/media/session/MediaSessionCompat$ResultReceiverWrapper.class
android/support/multidex/MultiDex$V4.class
android/support/v4/widget/SwipeRefreshLayout.class
android/support/v4/media/session/PlaybackStateCompatApi22.class
android/support/v4/view/OnApplyWindowInsetsListener.class
android/support/v4/media/MediaMetadataCompatApi21$Builder.class
android/support/v7/app/ActionBar$OnNavigationListener.class
android/support/v4/util/ContainerHelpers.class
android/support/multidex/MultiDex$V14.class
android/support/v4/view/ViewCompatBase.class
android/support/v7/appcompat/R$attr.class
android/support/v4/view/accessibility/AccessibilityNodeProviderCompat.class
android/support/v7/app/ActionBar$TabListener.class
android/support/v4/media/session/MediaSessionCompat$MediaSessionImpl.class
android/support/v7/appcompat/R.class
android/support/v4/view/ViewCompatJB.class
android/support/v7/app/ActionBar$LayoutParams.class
android/support/v7/widget/ActionMenuView$LayoutParams.class
android/support/v4/view/ViewCompatICS.class
android/support/v4/media/MediaDescriptionCompat.class
android/support/v4/media/session/MediaSessionCompat$QueueItem.class
android/support/v4/media/session/PlaybackStateCompat$Builder.class
android/support/v4/media/RatingCompat$StarStyle.class
android/support/v7/app/ActionBar$NavigationMode.class
android/support/v7/view/ActionMode.class
android/support/v4/media/MediaBrowserCompat$MediaBrowserImplBase.class
android/support/v4/media/VolumeProviderCompatApi21.class
android/support/v4/view/ViewCompat$ViewCompatImpl.class
android/support/v7/widget/TintTypedArray.class
android/support/v4/media/VolumeProviderCompat$1.class
android/support/v4/util/ArrayMap$1.class
android/support/v4/view/GravityCompat.class
android/support/v4/view/ViewCompat$KitKatViewCompatImpl.class
android/support/v4/media/session/MediaSessionCompat$OnActiveChangeListener.class
android/support/v4/media/RatingCompatApi21.class
android/support/v4/media/MediaMetadataCompat$BitmapKey.class
android/support/v4/view/animation/FastOutSlowInInterpolator.class
android/support/v4/graphics/drawable/DrawableCompat.class
android/support/v4/widget/MaterialProgressDrawable$Ring.class
android/support/v4/util/MapCollections.class
android/support/v4/widget/SlidingPaneLayout.class
android/support/multidex/ZipUtil$CentralDirectory.class
android/support/v4/view/ViewCompatEclairMr1.class
android/support/v4/view/ViewCompat$GBViewCompatImpl.class
android/support/v7/app/ActionBar$OnMenuVisibilityListener.class
android/support/v4/media/MediaBrowserCompat$1.class
android/support/v4/media/MediaMetadataCompat$TextKey.class
android/support/v4/app/AppOpsManagerCompat.class
android/support/v4/view/ViewCompat$JbMr2ViewCompatImpl.class
android/support/v4/media/VolumeProviderCompatApi21$Delegate.class
android/support/v4/view/ViewPager.class
android/support/v4/media/session/PlaybackStateCompat.class
android/support/v4/media/session/PlaybackStateCompat$State.class
android/support/multidex/MultiDex$V19.class
android/support/v4/media/VolumeProviderCompat$Callback.class
android/support/v4/media/session/MediaSessionCompat.class
android/support/v7/app/AppCompatActivity.class
android/support/v4/view/ViewCompat$MarshmallowViewCompatImpl.class
android/support/v4/media/MediaBrowserCompat$SubscriptionCallback.class
android/support/v4/media/RatingCompat$Style.class

So I am inclined to say something inside the Task is messing up the formatting before it's outputted to the obj/{Configuration}/multidex.keep file.

Thus I suspect a regression in this Task:

https://github.com/xamarin/xamarin-android/commits/master/src/Xamarin.Android.Build.Tasks/Tasks/CreateMultiDexMainDexClassList.cs

Changing priority to HIGH and setting as a BLOCKER as this blocks multidex functionality.
Comment 2 Jon Douglas [MSFT] 2017-08-25 16:45:58 UTC
I can additionally CONFIRM that in a sample test case, the custom application class is making the generated multidex.keep file.

2 line snippet:

classandroid/support/v4/view/accessibility/AccessibilityManagerCompat$AccessibilityStateChangeListenerCompat.classmd56fe3eb31817af4a890f7bce7e572be32/MyApplication.classandroid/support/v7/widget/ViewInfoStore$ProcessCallback.classandroid/support/v4/net/ConnectivityManagerCompatJellyBean.classandroid/support/v4/app/ServiceCompat$ServiceCompatImpl.

Which has:

md56fe3eb31817af4a890f7bce7e572be32/MyApplication.class

But because of the non carriage formatting, it is ignored and ends up in classes2.dex instead of the main dex list (classes.dex).
Comment 3 Jon Douglas [MSFT] 2017-08-25 16:46:36 UTC
Created attachment 24419 [details]
Reproduction

Reproduction project.
Comment 4 Jonathan Pryor 2017-08-25 18:56:32 UTC
Looks like this was introduced with:

https://github.com/xamarin/xamarin-android/pull/575
https://github.com/xamarin/xamarin-android/commit/6829b7d1becee2d979516bc0ac4513eac29ee865

The commit removes use of `mainDexClasses.bat` -- which didn't work well on Windows when dealing with paths containing spaces -- and replaces it with a "more direct" `java -jar proguard.jar ...` invocation.

The specific problem appears to be this line:

https://github.com/xamarin/xamarin-android/blob/814de17/src/Xamarin.Android.Build.Tasks/Tasks/CreateMultiDexMainDexClassList.cs#L110

`singleLine` will *never* contain a newline, so this `File.AppendAllText()` will never append a newline.
Comment 5 Jonathan Pryor 2017-08-25 19:19:36 UTC
PR: https://github.com/xamarin/xamarin-android/pull/790
Comment 7 Brendan Zagaeski (Xamarin Support) 2017-09-06 03:14:19 UTC
Created attachment 24585 [details]
Adjusted Test Case (for Comment 7)

## Regression status: Regression in Xamarin.Android 7.4 compared to 7.3

> BAD:  Xamarin.Android 7.4.0.21 (2851083) "Xamarin 15.3.2 Release"
> GOOD: Xamarin.Android 7.3.1.2  (9dbc4c5) "Xamarin 15.2.2 Release"



## Steps followed to test

1. Download the test case attached to this comment.  This is closely related to the test case from Comment 3, but it has been adjusted slightly to make it easy to test against older API devices.

2. Ensure that you have the Android 7.0 (API 24) SDK Platform installed.

3. Open the test case in Visual Studio for Mac to restore the NuGet packages.

4. Build the app and run it on an Android API 16, Google APIs, x86 emulator in the Debug or Release configuration.


### Testing notes

To get the on-device crash as described below, you must ensure that all of the Xamarin.Android.Support NuGet packages have been restored successfully and that "Project Options > Android Build > Enable Multi-Dex" remains enabled for the selected Debug or Release configuration.

This particular test app will not crash when running on higher API emulators such as the API 25 emulator due to the way the Android Support libraries work.  But the multidex.keep file will show the problem with missing newlines in both cases.




## BAD Results with Xamarin.Android 7.4


A. As described in the earlier comments, the generated multidex.keep file contains 0 newlines (line breaks).  For example testing with the `wc -l` command in a Terminal.app command prompt:

> $ wc -l AndroidApp1/obj/Debug/multidex.keep 
>        0 AndroidApp1/obj/Debug/multidex.keep

> $ wc -l AndroidApp1/obj/Release/multidex.keep 
>        0 AndroidApp1/obj/Release/multidex.keep

B. The app crashes on launch due to a ClassNotFoundException for the `MyApplication` class.  Here are two excerpts from the exception call stack to aid searchability of this bug report:

> E/AndroidRuntime( 2238): FATAL EXCEPTION: main
> E/AndroidRuntime( 2238): java.lang.RuntimeException: Unable to instantiate application md5c178831cd46fc53bebc42cf953f78ced.MyApplication: java.lang.ClassNotFoundException: md5c178831cd46fc53bebc42cf953f78ced.MyApplication
> E/AndroidRuntime( 2238): 	at android.app.LoadedApk.makeApplication(LoadedApk.java:501)

> E/AndroidRuntime( 2238): Caused by: java.lang.ClassNotFoundException: md5c178831cd46fc53bebc42cf953f78ced.MyApplication
> E/AndroidRuntime( 2238): 	at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:61)



## GOOD Results with Xamarin.Android 7.3

A. The generated multidex.keep file contains many newlines.  For example testing with the `wc -l`:

> $ wc -l AndroidApp1/obj/Release/multidex.keep 
>      676 AndroidApp1/obj/Release/multidex.keep

B. The app launches successfully and displays the "Hello World, Click Me!" button.




## Additional testing environment information (brief)

Mono MDK 5.2.0.215 (d15-3/da80840)
Visual Studio 2017 for Mac (7.1.0.1297) (9c52996)

Android SDK Tools          25.2.5
Android SDK Platform-tools 25.0.5
Android SDK Build-tools    25.0.3

Android SDK Platform 6.0 (API 23)
Android SDK Platform 7.0 (API 24)

Java JDK 8u101 (1.8.0_101), 64-bit

macOS 10.12.6


### Testing devices

Google API 16 x86 emulator ("system-images;android-16;google_apis;x86")
Comment 8 Brendan Zagaeski (Xamarin Support) 2017-09-06 03:31:29 UTC
*** Bug 59031 has been marked as a duplicate of this bug. ***
Comment 9 Brendan Zagaeski (Xamarin Support) 2017-09-06 03:39:12 UTC
*** Bug 58940 has been marked as a duplicate of this bug. ***
Comment 10 Brendan Zagaeski (Xamarin Support) 2017-09-06 03:46:45 UTC
## Tallying additional reports for bookkeeping

4 (probable) users: https://forums.xamarin.com/discussion/100532/ (2017-08-21 to 2017-09-01)
1 user: https://forums.xamarin.com/discussion/102617/ (2017-09-03)
Comment 11 Brendan Zagaeski (Xamarin Support) 2017-09-06 03:52:40 UTC
## Note to the Xamarin team

I am adjusting the target milestone to 15.4 and marking the bug as temporarily REOPENED to request that the fix be backported to 15.4 based on the information about regression status and number of affected users in Comment 7 - Comment 10.  Thanks in advance!
Comment 12 Brendan Zagaeski (Xamarin Support) 2017-09-06 17:57:25 UTC
## Tallying additional reports for bookkeeping

2 other users: Bug 47086, Comment 3 (2017-08-21) and Bug 47086, Comment 8 (2017-08-29)
Comment 13 Jon Douglas [MSFT] 2017-09-07 18:11:03 UTC
15.4 Backport:

https://github.com/xamarin/xamarin-android/pull/825
Comment 14 Jon Douglas [MSFT] 2017-09-09 15:11:32 UTC
*** Bug 59369 has been marked as a duplicate of this bug. ***
Comment 16 Jonathan Pryor 2017-09-11 14:06:16 UTC
Xamarin.Android 7.5.0.15 does not contain the fix for this bug.

You need Xamarin.Android 8.0.0.7 or later to get the fix for this bug.
Comment 17 Brendan Zagaeski (Xamarin Support) 2017-09-11 16:12:57 UTC
For users watching this bug, note that Comment 16 is a bookkeeping note for the Xamarin team.  The 8.0.x.x version mentioned in Comment 16 doesn't yet correspond to any published build.
Comment 18 Jake Louw 2017-09-12 15:50:52 UTC
ETA of Xamarin.Android 8.0.0.7 being released in alpha?
Comment 19 Jon Douglas [MSFT] 2017-09-12 15:58:47 UTC
(In reply to Jake Louw from comment #18)
> ETA of Xamarin.Android 8.0.0.7 being released in alpha?

Around 9/20 in 15.4 Preview 3. You can see https://www.visualstudio.com/en-us/productinfo/vs2017-release-rhythm for more details on the release process.
Comment 20 Jake Louw 2017-09-27 17:21:21 UTC
Any reason why this issue isn't immediately logged in the release notes:
https://developer.xamarin.com/releases/android/xamarin.android_8/xamarin.android_8.0/

Rather cumbersome not to get a complete list of bug-fixes on each alpha build. Forces me to update to alpha, check to see if the fix exists, and if not, reverting to stable. A rather time-wasting process...
Comment 21 Jon Douglas [MSFT] 2017-09-27 17:29:19 UTC
(In reply to Jake Louw from comment #20)
> Any reason why this issue isn't immediately logged in the release notes:
> https://developer.xamarin.com/releases/android/xamarin.android_8/xamarin.
> android_8.0/
> 
> Rather cumbersome not to get a complete list of bug-fixes on each alpha
> build. Forces me to update to alpha, check to see if the fix exists, and if
> not, reverting to stable. A rather time-wasting process...

Hi Jake,

I have made changes to the release notes that better reflect the current release. It is still under review to go out soon.
Comment 22 Gaurav Ganorkar 2017-10-05 10:32:41 UTC
Hello ,
I am not able to verify this fix  as I tried to launch the project given in the Comment 7(attachment 24585 [details]) on the AVD (API 16) but the application getting crashed as soon as launched.So have been blocked for verifying this bug


so this bug have been logged - https://bugzilla.xamarin.com/show_bug.cgi?id=59789

Verified on the build 
-XA: 8.0.0.33 (Visual Studio Enterprise)

Screencast link when ran via IDE
-https://www.screencast.com/t/D66Iie0tz

Screencast link ran via command line using `msbuild /t:_install` and `msbuild /t:_Run`
-https://www.screencast.com/t/uVOenD2Rym

Full build info
-https://gist.github.com/neerajdeshmukh/c2d4aa49b2b4bbb08f474a301ff7eee8

Note You need to log in before you can comment on or make changes to this bug.