Bug 29599 - mcs doesn't set compile timestamp in PE COFF file header
Summary: mcs doesn't set compile timestamp in PE COFF file header
Status: NEW
Alias: None
Product: Compilers
Classification: Mono
Component: C# (show other bugs)
Version: unspecified
Hardware: PC Linux
: --- enhancement
Target Milestone: ---
Assignee: Marek Safar
URL:
Depends on:
Blocks:
 
Reported: 2015-04-30 09:40 UTC by Arsen.Shnurkov
Modified: 2015-05-04 10:47 UTC (History)
3 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments

Description Arsen.Shnurkov 2015-04-30 09:40:16 UTC
Steps to reproduce:
1) compile "hello world" C# console program with mcs
2) look to results of
objdump -xf "helloworld.exe" | grep Date

Actual results:
Time/Date		Thu Jan  1 03:00:00 1970

Expected results:
Time/Date of binary creation

This prevents this code from working properly:
http://stackoverflow.com/questions/1600962/displaying-the-build-date

mono -V
Mono JIT compiler version 4.1.0 (branch-master/6a74b19 Mon Mar 23 01:18:53 MSK 2015)
Comment 1 Arsen.Shnurkov 2015-04-30 11:10:27 UTC
If I understand properly, mcs uses Reflection.Emit, and it's code is here:
https://github.com/mono/mono/blob/master/mono/metadata/reflection.c#L5741
code looks ok to me (get's current time and puts it into header).

I don't understood, why I don't see this after build (may be i look in different place?)
Comment 2 Arsen.Shnurkov 2015-04-30 12:01:35 UTC
I wrote bytes for 0x55417ECDul with ghex into helloworld's executable binary, and code from stackoverflow starts work.
Comment 3 Arsen.Shnurkov 2015-04-30 17:32:10 UTC
I patched line 5741 in the file to constant (0xFFFFFFFFul) instead calling time function, then I rebuilt mono, recompile application - but date is still 1970.  Bytes didn't get into file body.

Also I tested time function by creating test application on c language. Time function returns correct integer which different from zero, and really corresponds to current time.

Now I have following ideas:
1) maybe my guess about code path and place was wrong, and some other code is used for file creation;
2) code executes, but doesn't write structure into file (I didn't read that function up to full understanding);
3) it should be possible to debug mcs with monodevelop (and probably mscorlib with gdb?) and set breakpoint in place of assembly writing.
Comment 4 Alexander Köplinger 2015-04-30 20:50:49 UTC
This was a deliberate change to support reproducible builds, see the discussion in https://bugzilla.xamarin.com/show_bug.cgi?id=26842
Comment 5 Arsen.Shnurkov 2015-05-01 02:22:29 UTC
How to control this feature from command line?

So, do I understand properly, that:
1) the code from reflection.c is not used during compilation?
2) there should be command line switch /features:deterministic=false
https://github.com/dotnet/roslyn/commit/798942020da183159cec4d7ab0187116ba2b5313
3) monodevelop have no this settions in option dialogs
4) mcs has no description of this key in documentation?
Comment 6 Arsen.Shnurkov 2015-05-01 03:32:57 UTC
I don't see where ikvm's code is invoked.
I don't see where ikvm's set zero timestamp
https://github.com/mono/ikvm/blob/master/reflect/Writer/PEWriter.cs#L150

is there some article on how compilation with mcs works?
Comment 7 Arsen.Shnurkov 2015-05-01 04:49:44 UTC
About Jeroen’s IKVM.Reflection:

http://www.mono-project.com/docs/about-mono/releases/2.10.0/#new-c-compiler-backend

http://www.mono-project.com/docs/about-mono/releases/3.0.0/

C# compiler has now completed its migration from using System.Reflection.Emit as its code generation backend to use the IKVM.Reflection API.

gmcs, dmcs and smcs are now merely aliases to call the mcs compiler with the proper -sdk flag.

If you want to specify a specific SDK use the -sdk:PROFILE option.
predefined values are valid: 2, 4 (default) as well as any custom value.
(mcs will try to find Base Class Libraries in the mono  installed  location  PREFIX/lib/mono/<value>)
Comment 8 Jo Shields 2015-05-01 05:34:18 UTC
IKVM.Reflection is the code we use for emitting binaries. It's part of IKVM.NET, a Java runtime for .NET by Jeroen Frijters.

You are affected by this change: https://bugzilla.xamarin.com/show_bug.cgi?id=26842

The change allows builds to be verified and reproduced, by causing output to be deterministic between builds, which is an important security concern.

IKVM.Reflection *optionally* supports deterministic output (and the .NET spec says the date stamp is optional, so it is spec-compliant to set it to epoch 0) - see above bug.

mcs, our C# compiler, does NOT make this optional, and always produces deterministic output, as of a803d17038c0fcc8b40b12744801a87ceddb15ba

I don't see why we couldn't make deterministic output opt-out with a flag or environment variable, but nobody has done that work.
Comment 9 Arsen.Shnurkov 2015-05-01 08:40:45 UTC
Thanks, now i see
https://github.com/mono/mono/commit/a803d17038c0fcc8b40b12744801a87ceddb15ba
that i need to change
line 251 of file ikvm.cs
https://github.com/mono/mono/blob/master/mcs/mcs/ikvm.cs#L251
(need to remove "| UniverseOptions.DeterministicOutput" flag in my local sources).

I tested and this works for me.

UniverseOptions is enum from IKVM:
http://ikvm.cvs.sourceforge.net/viewvc/ikvm/ikvm/reflect/Universe.cs?view=markup

command line parsing is done at 
https://github.com/mono/mono/blob/master/mcs/mcs/settings.cs
(something like https://github.com/mono/mono/blob/master/mcs/mcs/settings.cs#L1130)
CommandLineParser returns CompilerSettings, settings are passed into CompilerContext,
and can be extracted with
var v = compiler.Settings.MyNewCustomOptionName;
Comment 10 Arsen.Shnurkov 2015-05-01 08:51:16 UTC
but what if i want to set specific nonzero date, not the current time?
this will give both - build date in exe and reproduceability of build...
Comment 11 Arsen.Shnurkov 2015-05-01 10:22:09 UTC
i think a bit, about requirements:
1) somewhere should be info about build dates of artifacts
2) artifacts should be verifiable after rebuilding

solution - move build time info into separate location (file with guid-timestamp pairs for example, or xml with build process description, timing and log).
Whose who need this feature will follow proposed standart and create that database. Database will be small, verified by system library, and will not affect trust to binaries (not more, then replacing digital signatures of binaries with signatures of another person)

It should be something like .mdb or .pdb but separate, to allow verification of debug information binaries.

This is a large work with following steps:
1) publish specification of format
2) create reference library to access such database from runtime
3) add support to build systems
4) add supportin into development tools (such as IDEs)

Before starting this work it should be explained, how it will help (why one should want to know the build date instead of last commit id? commit id's are not changing, and have a fixed date too).

As a shortcut it is possible to set timestamp from last commit date into PE COFF field - that will be nonzero and reproducible (mcs or separate utility have access to sources, and can check the presence of .git repository)
Comment 12 Arsen.Shnurkov 2015-05-01 10:32:51 UTC
echo `git log -1 --format=%at HEAD`

will give the date, which can be passed into mcs to embed into PE COFF header
Comment 13 Arsen.Shnurkov 2015-05-01 12:57:32 UTC
unfortunately, it is difficult to pass additional options (for mcs) with msbuild engine - see https://bugzilla.xamarin.com/show_bug.cgi?id=15510
Comment 14 Marek Safar 2015-05-04 10:42:08 UTC
PE header timestamp is optional and mcs no longer sets it. It's better to use .net specicif approach like AssemblyInformationalVersionAttribute instead of decoding PE header.
Comment 15 Arsen.Shnurkov 2015-05-04 10:47:20 UTC
Then mcs should have command line switch to embed that attribute to the resulting assembly.

Because if you add this attribute into source, it will cause problems with source control system (it will detect change to sources).

Note You need to log in before you can comment on or make changes to this bug.