Bug 51298 - Codesign dylibs/frameworks in parallel
Summary: Codesign dylibs/frameworks in parallel
Alias: None
Product: iOS
Classification: Xamarin
Component: MSBuild ()
Version: master
Hardware: PC Mac OS
: High enhancement
Target Milestone: Future Cycle (TBD)
Assignee: Jeffrey Stedfast
Depends on:
Reported: 2017-01-09 11:18 UTC by Rolf Bjarne Kvinge [MSFT]
Modified: 2017-01-20 16:38 UTC (History)
2 users (show)

Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Rolf Bjarne Kvinge [MSFT] 2017-01-09 11:18:45 UTC
There can be quite a number of assemblies in some project, and since each assembly will be compiled to a separate dylib, we might end up having to sign a significant number of binaries, which can take a while.

Example: bug #44783, with 78 assemblies (and 78 dylibs to sign), it takes 47s to sign them all.

Suggestion: parallelize code signing.

Note: I haven't checked if it's actually faster to parallelize or not.
Comment 1 Jeffrey Stedfast 2017-01-09 21:30:06 UTC
This may need to be done by refactoring the Codesign task to take a list of items to sign rather than an individual item.

It's impossible (afaik) to get MSBuild to parallelize tasks.
Comment 2 Sebastien Pouliot 2017-01-13 17:17:33 UTC
Codesign can be quite slow. e.g. on my, not so new (nor fast), macbook air:

> 46757.074 ms  Codesign                                               18 calls
> 29319.475 ms  Codesign                                               16 calls

so more than 1 second per assembly/library being signed - and, like most of the time when we call Apple tools, there can be huge differences in times between executions.

e.g. replaying (via script) the last (16) signatures gave new (only 2 runs)

real	0m42.650s
user	0m26.865s
sys	0m7.679s

real	0m30.393s
user	0m20.410s
sys	0m5.718s

A quick C# version (where the integer is the `MaxDegreeOfParallelism`) gives me:

castor:xamarin-macios poupou$ time mono cs.exe 1

real	0m44.129s
user	0m27.326s
sys	0m7.312s

castor:xamarin-macios poupou$ time mono cs.exe 8

real	0m22.718s
user	0m38.952s
sys	0m7.426s

castor:xamarin-macios poupou$ time mono cs.exe 8

real	0m27.668s
user	0m45.770s
sys	0m9.537s

castor:xamarin-macios poupou$ time mono cs.exe 8

real	0m22.137s
user	0m39.698s
sys	0m6.772s

castor:xamarin-macios poupou$ time mono cs.exe 8

real	0m24.634s
user	0m38.959s
sys	0m8.008s

8 might be too high, with 4 it seems a bit better

castor:xamarin-macios poupou$ time mono cs.exe 4

real	0m16.080s
user	0m36.376s
sys	0m6.114s

castor:xamarin-macios poupou$ time mono cs.exe 4

real	0m20.611s
user	0m37.306s
sys	0m6.808s

castor:xamarin-macios poupou$ time mono cs.exe 4

real	0m18.601s
user	0m37.585s
sys	0m6.345s

In any case it's seems like a worthy opimization

Comment 3 Sebastien Pouliot 2017-01-13 19:26:38 UTC
While debugging something else in Xcode I got curious and checked how it sign.

Xcode uses `--timestamp=none` for debug (only?) builds, which removes the network access to a timestamp server.

Doing the same I get far more consistent times - which kind of make sense in retrospect :)

This optimization (for anything not released to AppStore/AdHoc) could be done to the existing task. In fact the "gain" I saw earlier is much lower when doing this, i.e. it seems the real gain was doing the network access in parallel, not the signature.

Still `--timestamp=none` cannot always be used, so those other cases would still benefit from being executed in parallel.
Comment 4 Jeffrey Stedfast 2017-01-16 00:56:00 UTC
@sebastien thanks for the info, this will be useful.

I've done a bit of research myself and noticed that codesign can take any number of path arguments to sign. I haven't yet timed to see if this would improve the performance, but I suspect it would even if only by reducing the start-up time of the codesign tool + processing time by xbuild spawning tasks and executing them 1-by-1.

My first step is going to be updating the targets to pass the full array of resources to the Codesign task and passing each resource path to a single invocation of the codesign tool.

After that, it'll be easier to implement parallelization.
Comment 5 Jeffrey Stedfast 2017-01-18 19:07:16 UTC
Phase 1 complete: https://github.com/xamarin/xamarin-macios/pull/1522
Comment 6 Jeffrey Stedfast 2017-01-19 17:42:49 UTC
Phase 2 (in progress as we work out the finer details): https://github.com/xamarin/xamarin-macios/pull/1528
Comment 7 Jeffrey Stedfast 2017-01-19 19:08:22 UTC
It's worth noting that I just did my own test and it looks like Xcode uses `--timestamp=none` for device builds (Product -> Build). They also seem to use `--timestamp=none` for "Product -> Archive" builds (and yes, they rebuild when Archiving which was not what I expected).

For the time being, we are only using --timestamp=none for (all) Simulator builds and for Device builds where MtouchDebug == true.

Since Debug builds are the most common, that will make the most impact and leaves Release builds alone in case our customers actually care to have it.

It's possible, however, that we could potentially just disable timestamps for *all* builds in MSBuild and only include the timestamp in the Publishing Workflow.

This would be more consistent with what Xcode seems to be doing (their Archived builds are not signed with a timestamp).
Comment 8 Jeffrey Stedfast 2017-01-20 16:38:55 UTC
The PR has now been merged.