Bug 30410 - iTunes Metadata keeps showing up in watch apps preventing submission
Summary: iTunes Metadata keeps showing up in watch apps preventing submission
Alias: None
Product: iOS
Classification: Xamarin
Component: MSBuild ()
Version: XI 8.10
Hardware: PC Mac OS
: --- normal
Target Milestone: 8.13 (C6 alpha)
Assignee: Bugzilla
Depends on: 30320
  Show dependency tree
Reported: 2015-05-25 11:42 UTC by Frank A. Krueger
Modified: 2015-06-10 15:16 UTC (History)
6 users (show)

Is this bug a regression?: ---
Last known good build:

Screenshot of rejection (67.04 KB, image/png)
2015-05-25 11:42 UTC, Frank A. Krueger

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Frank A. Krueger 2015-05-25 11:42:32 UTC
Created attachment 11316 [details]
Screenshot of rejection

Every time I want to submit an app I have to hack the Archive. You have bug #24412 marked resolved but I still get this error.

=== Xamarin Studio ===

Version 5.9.1 (build 3)
Installation UUID: fce13fdd-e8e3-48ef-99f1-4acbb06f0240
	Mono 4.0.0 ((detached/d136b79)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 400000143

=== Apple Developer Tools ===

Xcode 6.3.2 (7718)
Build 6D2105

=== Xamarin.iOS ===

Version: (Enterprise Edition)
Hash: 3c4e832
Branch: master
Build date: 2015-05-20 21:47:57-0400

=== Xamarin.Android ===

Version: (Enterprise Edition)
Android SDK: /Users/fak/Library/Developer/Xamarin/android-sdk-mac_x86
	Supported Android versions:
		2.3   (API level 10)
		4.0.3 (API level 15)
		4.3   (API level 18)
		4.4   (API level 19)
		5.0   (API level 21)
Java SDK: /usr
java version "1.8.0_20-ea"
Java(TM) SE Runtime Environment (build 1.8.0_20-ea-b23)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b22, mixed mode)

=== Xamarin Android Player ===

Version: Unknown version
Location: /Applications/Xamarin Android Player.app

=== Xamarin.Mac ===

Version: (Enterprise Edition)

=== Build Information ===

Release ID: 509010003
Git revision: aad75a6e7e48f18120ce41f47d0ff2c6216f49c3
Build date: 2015-05-08 12:46:18-04
Xamarin addins: 1246b3044cbb7f56a217334f8fc5489ef8eefe3f

=== Operating System ===

Mac OS X 10.10.3
Darwin muon.local 14.3.0 Darwin Kernel Version 14.3.0
    Mon Mar 23 11:59:05 PDT 2015
    root:xnu-2782.20.48~5/RELEASE_X86_64 x86_64
Comment 1 Rolf Bjarne Kvinge [MSFT] 2015-05-25 12:41:00 UTC
This is a dup of bug #29180, not #24412.

More information can also be found here: http://forums.xamarin.com/discussion/40388/disallowed-paths-itunesmetadata-plist-found-at-when-submitting-to-app-store/p1

*** This bug has been marked as a duplicate of bug 29180 ***
Comment 2 Frank A. Krueger 2015-05-25 13:10:05 UTC
That thread says that this should all be fixed now on Stable - and that bug is resolved. I'm confused because I got the error just this morning.
Comment 3 Brendan Zagaeski (Xamarin Team, assistant) 2015-05-25 16:50:35 UTC
To make a guess, if by chance you might be using Xcode to submit the app (rather than the "Sign and Distribute" button in Xamarin Studio followed by Application Loader), then you will still hit this error by default.

The explanation is essentially that Xamarin's `.xcarchive` format is not in fact 100% compatible with Apple's `.xcarchive` format (see Bug 29180, Comment 16). So if you wish to use Xcode to submit the app, you will need to manually remove the `iTunesMetadata.plist` file from the `.xcarchive` bundle before submitting the app.

From the forum thread:

>  If you wish to use Xcode to create the .ipa, you will need to apply the
>  "alternate workaround for Xamarin Studio".


It's also possible there are some Watch App project setups that somehow don't fit the expected bundle file location pattern that the new logic looks for when removing iTunesMetadata.plist from the `.ipa` before resigning the `.app` bundle. I believe the Xamarin QA team checked the WatchKitCatalog sample [1] for the proper behavior using the new fixes, but I can double-check on that and see if it misbehaves in any of the three "For ... builds" that I listed in the forum thread.

> [1] https://github.com/xamarin/monotouch-samples/tree/master/WatchKit/WatchKitCatalog
Comment 4 Frank A. Krueger 2015-05-31 12:12:06 UTC
Your description worries me.

Do you plan on fixing this bug?
Comment 5 Brendan Zagaeski (Xamarin Team, assistant) 2015-06-01 19:52:04 UTC
For this reply, I will assume comment 4 is asking about the first part of comment 3, which was about submitting the app via Xcode.

I'm just a member of the Xamarin Support Team, so I am not the perfect person to give the final word on whether allowing publishing to the app store from a `.xcarchive` via Xcode is something Xamarin should try to implement and regression test (a) for each stable release of Xamarin and (b) for each new release of Xcode.

But my personal assessment is that the comment I linked before (Bug 29180, Comment 16) mostly answers the question. To paraphrase that comment in the form of a question: is it worth the time and effort to attempt to more perfectly reverse engineer the `.xcarchive` format and the Xcode app submission logic?

Related follow-up questions:

- Is perfecting the reverse-engineered `.xcarchive` logic still worthwhile now that Xamarin Studio 5.9 aims to deprecate the Xcode app submission workflow in favor of its own built-in workflow for signing and distributing [1, 2]? Are there usage scenarios that cannot be handled by that new workflow? If so, would it be better to extend the new workflow rather than trying to more perfectly reverse-engineer the `.xcarchive` format?

- Given that Apple could (if they wanted) change the `.xcarchive` format in the future, is it still worth spending additional time reverse engineering the current version of the `.xcarchive` format?

> [1] http://developer.xamarin.com/releases/studio/xamarin.studio_5.9/xamarin.studio_5.9/#Publishing_Workflow

> [2] http://developer.xamarin.com/guides/ios/deployment,_testing,_and_metrics/app_distribution_overview/publishing_to_the_app_store/

(Be sure to select "Xamarin Studio" at the top right of the document to see the steps in Xamarin Studio.)

If there is enough user demand for perfectly reverse-engineering Xcode's app submission logic, perhaps it will need to be reconsidered. In theory a good place to poll the level of user demand would be on http://xamarin.uservoice.com/.
Comment 6 Brendan Zagaeski (Xamarin Team, assistant) 2015-06-01 20:33:18 UTC
In case you now go try the new "Sign and Distribute" workflow in Xamarin Studio, note that unfortunately (I have just learned) there is an unpleasant bug in the new workflow when creating iOS apps with extensions: Xamarin Studio does not provide an option to pick the provisioning profiles for each extension.

That lack of flexibility can lead to the following error when submitting the app via App Loader:

> ERROR ITMS-90161: "Invalid Provisioning Profile."

(That issue is being tracked in the non-public Bug 30320 that has just been marked as a dependency for this bug.)
Comment 7 t9mike 2015-06-09 22:39:54 UTC
As someone who just had to manually work around a *different* tool bug #30958 that caused me to need to manually edit the archive, I was very surprised I then had to also manually delete the iTunes Metadata file as well.

So please consider that from time to time customers may need to hack the archive because of unexpected bug, so ensure that the file is complaint otherwise.

But in general I found the feedback given by the double click archive method superior to Application Loader, and would prefer just to use it.
Comment 8 Frank A. Krueger 2015-06-10 11:51:46 UTC
@Brendan - I don't understand this "oh my gosh its so hard to reverse engineer argument"

You are generating a file that I didn't request. There is an option whether to include it in the project settings. I have chosen not to use it.

So why is the file being generated? It's a bug.

Xamarin's custom archive generator is scary to use - it resigns executables that were already signed - and usually screws that part up.

You see, I want archives to work because *I don't trust you* to reverse engineer anything. I would rather Apple do the signing and uploading.
Comment 9 Brendan Zagaeski (Xamarin Team, assistant) 2015-06-10 12:52:10 UTC
To reiterate, I'm just a member of the Support Team. My comments on this bug are purely a "thought experiment" expansion on the statements from the engineering team in Bug 29180.

I will mention this bug to Miguel and Jeff to see if they would like to offer additional insight from the engineering perspective.

> There is an option whether to include it in the project settings.

I assume you are referring to the "iTunes Metadata" field (for example, in Xamarin Studio 5.8, it's under "Project Options -> Build -> iOS IPA Options -> IPA Packaging Options [section] -> iTunes Metadata").

Unfortunately, that setting applies to IPA files only, and not archives. As I understand it, the real crux of the "reverse engineering" problem is the dual purpose use of `.xcarchive` files in Xcode after they are created by Xamarin Studio. That is, they can be used for either:

(a) Ad-Hoc IPAs, where you _do_ want the `iTunesMetadata.plist`.


(b) App Store submission, where you do _not_ want the `iTunesMetadata.plist`.

As stated in Bug 29180, Comment 14:

> FWIW, Xcode's Archive publishing logic somehow pulls the iTunesArtwork and
> iTunesMetadata.plist files from the project and not from anywhere in the
> archive afaict, which is why it can't possibly work for Xamarin projects in
> the same way.

And Bug 29180, Comment 16:

>  I never figured out exactly how they create .ipa's and pull in iTunesArtwork
>  and iTunesMetadata.plist files when making ad-hoc IPA's since Xcode's
>  archives do not include those files
Comment 10 Mikayla Hutchinson [MSFT] 2015-06-10 12:58:58 UTC
@Frank: I absolutely agree this is a bug, and we'll fix it. However, it's not a blocker - it's not one of the primary supported publishing workflow, and the supported workflows *do* work.

I'm sorry if we've done things that cause you not to trust our bundle signing, but it does use Apple's tools, and any problems should be caught by Apple's verification. If you find any blocking problems in these workflows please let us know and we will fix them as a matter of priority.

Let me explain the supported publishing workflows:

The first is direct IPA build. This is somewhat advanced and is mostly intended for CI etc). This involves setting up a custom solution+project configuration, setting the signing keys to the app store keys, and setting it to build an IPA.

The second is archive-and-publish. It uses the xcarchive format, because that allowed the "publish" step to be performed with Xcode in the past when we didn't have a publishing wizard.

Our goal with the publishing workflow is to:
* provide an publishing experience that's as fully integrated as possible
* simplify the default project build configurations
* allow publishing the exact same binary you've tested locally
* provide the same UX for iOS and Android
* provide a similar UX to Xcode

The reason the iTunes metadata is still included in the archive is that we may need it during publishing, and we haven't yet designed an alternative way to include this info in the archive. The build option is for building an IPA at build time, which doesn't affect the archive - the archive is created directly from the app bundle. We'll be looking at unifying the IPA creation code paths after we switch over fully to MSBuild and drop support for the old build engine.

If there are other ways you think we could improve it I'd love to hear them!
Comment 11 Mikayla Hutchinson [MSFT] 2015-06-10 13:30:20 UTC
@Frank I've spoken to Jeff about this and he says the fix has already been made and will be in the next major release.
Comment 12 Jeffrey Stedfast 2015-06-10 15:15:19 UTC
Hey guys,

When the iTunesMetadata.plist issue first showed up, I modified the MSBuild logic (which is part of Xamarin.iOS) to no longer copy the generated iTunesMetadata.plist file into the compiled app bundle directory (which gets archived as a whole). I also fixed XS's publishing workflow to remove the iTunesMetadata.plist and iTunesArtwork files from the archive during the publishing process (before re-signing the app bundle) if they are there (i.e. built with XI <= 8.10).

Unfortunately it was a fairly sizable change to make to the stable release since it changed behavior so the logic was only changed in master (Xamarin.iOS 8.12 and XS 5.10).
Comment 13 Jeffrey Stedfast 2015-06-10 15:16:53 UTC
Correction: XS's publishing logic was fixed in 5.9 to remove those files as part of its publishing workflow.