Bug 57893 - mkbundle is not working on Windows
Summary: mkbundle is not working on Windows
Status: RESOLVED FIXED
Alias: None
Product: Tools
Classification: Mono
Component: mkbundle (show other bugs)
Version: 5.2 (2017-04)
Hardware: PC Windows
: --- normal
Target Milestone: Future Cycle (TBD)
Assignee: johan.lorensson
URL:
Depends on:
Blocks:
 
Reported: 2017-06-30 19:57 UTC by Arthur Khusnutdinov
Modified: 2017-10-24 15:20 UTC (History)
8 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
First test changes in mkbundle (2.80 KB, patch)
2017-07-01 08:10 UTC, Arthur Khusnutdinov
Details
Changed version 1 (40.65 KB, text/plain)
2017-07-01 08:11 UTC, Arthur Khusnutdinov
Details
Compilation output log with errors 1 (279.24 KB, text/plain)
2017-07-01 08:14 UTC, Arthur Khusnutdinov
Details
Compilation output log with errors 2 (46.49 KB, text/plain)
2017-07-01 08:14 UTC, Arthur Khusnutdinov
Details

Description Arthur Khusnutdinov 2017-06-30 19:57:47 UTC
Hello.

MKBUILD is not working in mono 5.2.

First problem in mkbundle is this https://github.com/mono/mono/blob/master/mcs/tools/mkbundle/mkbundle.cs#L954 code. Ie, it's trying to find file monoPath + @"\lib\monosgen-2.0-static.lib" that doesn't exists (I've filed bug https://bugzilla.xamarin.com/show_bug.cgi?id=57892 for issue, that msi hasn't static lib in it).

Second problem is that in MS VS 2015 stdlib is not in the C:\Program Files (x86)\Windows Kits\8.1** path, it was moved into the windows sdk kit. I have discussed this issue with https://github.com/akoeplinger Alexander Köplinger. 
That *.h file, it's there in C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt now.

Please, fix mkbundle for Windows 10, VS2015.

Thank you very much.
Comment 1 Arthur Khusnutdinov 2017-07-01 08:10:01 UTC
I made some changes in mkbundle.cs. Now it's successfully produced files with as GNU assembler, but stucked on linking.
Comment 2 Arthur Khusnutdinov 2017-07-01 08:10:33 UTC
Created attachment 23249 [details]
First test changes in mkbundle

First test changes in mkbundle
Comment 3 Arthur Khusnutdinov 2017-07-01 08:11:37 UTC
Created attachment 23250 [details]
Changed version 1
Comment 4 Arthur Khusnutdinov 2017-07-01 08:12:08 UTC
To start that I tried to use


set MONO=c:\MonoDist
set MONOPREFIX=c:\MonoDist
set PATH=%PATH%;c:\Mono\bin\;c:\cygwin64\bin;%PREFIX%;c:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\
set WINSDK=c:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\

set LIB=c:\Program Files (x86)\Windows Kits\10\Lib\10.0.15063.0\um\x64\;c:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.25017\lib\x64\;c:\Program Files (x86)\Windows Kits\10\Lib\10.0.15063.0\ucrt\x64\;c:\Program Files (x86)\Windows Kits\10\Lib\10.0.15063.0\um\x64\;c:\MonoDist\lib\;c:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\lib\amd64\

c:/MonoDist/lib/mono/4.5/mkbundle WindowsFormsApplication1.exe --deps -o eGuide -L "c:/Mono/lib/mono/4.5-api/" --static



cl.exe /MT /I "c:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\\shared" /I "c:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\\um" /I "c:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\\ucrt" /I "c:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\\Include\um" /I "c:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\\Include\shared" /I "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include" /I "c:\Mono\include\mono-2.0" /I "." "temp.c" "temp.obj" "temp2.obj" "c:\Mono\lib\libmono-static-sgen.lib" /link /NODEFAULTLIB /SUBSYSTEM:windows /ENTRY:mainCRTStartup kernel32.lib legacy_stdio_definitions.lib libcmtd.lib version.lib ComCtl32.Lib ComDlg32.Lib eglib.lib libgc.lib libgcmonosgen.lib libmonoruntime-sgen.lib libmonoutils.lib libtest.lib mono-2.0-dac-sgen.lib mono-2.0-sgen.lib MonoPosixHelper.lib Ws2_32.lib Mswsock.lib Psapi.lib shell32.lib OleAut32.lib ole32.lib winmm.lib user32.lib libvcruntime.lib advapi32.lib OLDNAMES.lib libucrt.lib /out:eGuide 

but it failed with errors:
Comment 5 Arthur Khusnutdinov 2017-07-01 08:14:04 UTC
Created attachment 23251 [details]
Compilation output log with errors 1
Comment 6 Arthur Khusnutdinov 2017-07-01 08:14:18 UTC
Created attachment 23252 [details]
Compilation output log with errors 2
Comment 7 Arthur Khusnutdinov 2017-07-06 18:19:53 UTC
Hello. 
Any news on this issue?
Comment 8 Arthur Khusnutdinov 2017-07-25 10:28:35 UTC
Any news on this issue?
Comment 9 Arthur Khusnutdinov 2017-08-06 16:55:43 UTC
Hello. Any news on this issue?
Comment 10 Arthur Khusnutdinov 2017-08-20 10:14:50 UTC
Hello.
Guys, is it really so complex to fix this issue?
Comment 11 Arthur Khusnutdinov 2017-08-20 10:22:15 UTC
Interesting moment - Mono is a product of Microsoft, that developed Windows, but this tool (mkbuild) works well with latest versions of the linux cc and mono, but doesn't work well on Windows with the latest version of the MS VS 2017 and Mono )))...
Comment 12 Arthur Khusnutdinov 2017-08-21 13:46:12 UTC
Oh, really, I am sorry, not mkbuild, but mkbundle :)))...
Comment 13 Arthur Khusnutdinov 2017-08-21 14:25:51 UTC
It will be nice, if this bug will be fixed on Windows 10 not only for MS VS 2015, but for MS VS 2017 too.

Thank you!
Comment 14 johan.lorensson 2017-08-24 17:26:05 UTC
I will put this on my todo list extending mkbundle to include support for newer Windows SDK versions together with pure VS2015/VS2017 toolchain support.
Comment 15 Arthur Khusnutdinov 2017-08-24 19:16:54 UTC
johan.lorensson@xamarin.com ,please, take a look at bug -
https://bugzilla.xamarin.com/show_bug.cgi?id=57892 - the build logs looks similar for this problem.

I've compiled in this case Mono static lib. After compilation it was about 7-8BM, like in case of the https://bugzilla.xamarin.com/show_bug.cgi?id=57892 . After small modification of the mkbundle (diff attached here) I was able to reach linking process and I got issues with unresolved symbols.

I think, both of these bug reports have the root issue.

Thank you!
Comment 16 Arthur Khusnutdinov 2017-08-24 22:57:06 UTC
I have updated https://bugzilla.xamarin.com/show_bug.cgi?id=57892 a lot - still trying to compile my app with embedded Mono.

Please, take a look at https://bugzilla.xamarin.com/show_bug.cgi?id=57892 again.

Thank you!
Comment 17 Arthur Khusnutdinov 2017-08-25 22:37:14 UTC
Johan, I kindly ask you to don't start your work on this in nearest future - I want to make changes by myself and to make git pull request for it.

Thank you for your support and info about variables in mono.props - it helped me well.

Best regards, Arthur.
Comment 18 johan.lorensson 2017-08-28 07:01:32 UTC
Hi Arthur,

Nice that you get your build working using static embedding! "Unfortunately" I already come quite far on fixing better toolchain support in mkbundle. It also includes additional support for pure VS builds making sure we can use Clang for VS, if available, extending support for Windows SDK's as well as 32 and 64 bit builds (currently not available at all in mkbundle) and more. Will your PR cover all above use cases or more inline with the proposed changes above?

//Johan
Comment 19 Arthur Khusnutdinov 2017-08-28 08:34:22 UTC
Hello.
Johan, let me see your changes, please. Can you make commit in master repo and send me URL for the commit?
I think, In any case your chansges will be more better, than mine - I don't have so much experience.
You are more experienced, than me ))).

Thank you again.
Comment 20 Arthur Khusnutdinov 2017-08-29 18:59:11 UTC
Johan, what is the status of this issue?
Is it in progress or paused/delayed?
I am waiting for your changes and still not working on mine similar changes, because I am expecting, that your changes will be better than mine :) .
Thank you very much for your time, work and your answers ;) !
Comment 21 johan.lorensson 2017-08-30 15:00:07 UTC
Hi Arthur,

I have pushed a commit to my mono fork that includes most of the changes done to better support VS toolchains in mkbundle on Windows:

https://github.com/lateralusX/mono/tree/lateralusX/mkbundle-windows-toolchains

You can look at the commit description and the changes to get an understanding of what's been done so far.

Next thing is to change the mono build to get a merged static lib installed by the setup, but I need to context switch a couple of days before I can continue on that work. Once that is done I will make a PR of all changes as one package towards mono master.

So right now you can try out the dynamic linking scenario using the changes from above fork. It would be good if you could make some testing on your system and get back with some results as well (picking VS and SDK installs is a little tricky).

I have successfully run using VS2017, VS2015 and VS2013, with both Windows 8.1 and various version of 10 SDK's. I have also tried both 32 and 64 bit mkbundle builds.

//Johan
Comment 22 johan.lorensson 2017-08-30 15:04:17 UTC
Hi Arthur,

One more thing, you can (as described in the commit) run using the different flavours of VS developer consoles to get the same environment when doing mkbundle's as normal VS C builds. This will inherit include and lib paths as well as Visual Studio and Windows SDK versions from the environment, bypassing all addition logic needed to auto detect all this information.
Comment 23 Arthur Khusnutdinov 2017-08-30 20:50:18 UTC
Whoa! 
Cool! 
Thank you!
Indeed, this is one of the huge commit messages.
I also thought about to run mkbundle in VS developer console, to get environment vars, such lib and include paths.
Nice, I'll dig into this in nearest 16 hours.
Comment 24 Hin-Tak Leung 2017-09-01 23:10:10 UTC
You should not use the *-api stub directory with mkbundle. For embedding, you need the real dll's. -L "c:/Mono/lib/mono/4.5-api/" is wrong.

At least on linux, mono 4.8.0, the stubs are in *-api, the real ones are symbolic links in /usr/lib/mono/4.5/ - both /usr/lib/mono/4.5 /usr/lib/mono/4.5-api/ exist, and they are different, for different purposes.
Comment 25 Arthur Khusnutdinov 2017-09-01 23:19:05 UTC
OK.
Did you wrong with issues?
In this issue there are another questions, not same https://bugzilla.xamarin.com/show_bug.cgi?id=58914 )))
Comment 26 Hin-Tak Leung 2017-09-01 23:32:41 UTC
Okay, while I am not upgrading my mono 4.8.0 on linux, I do have a mono 5.2.0 for mac.
You need to use the 4.5 directory, not any of the *-api ones. mono 5.2.0 ships :

2.0-api
4.0-api
4.5
4.5-api
4.5.1-api
4.5.2-api
4.6-api
4.6.1-api
4.6.2-api
4.7-api

None of the *-api ones are suitable for mkbundle's embedding. They are for compiling. For embedding, you need what's in the "4.5" directory .
Comment 27 Arthur Khusnutdinov 2017-09-01 23:57:09 UTC
Thank you very much!
Comment 28 Jens Eckervogt 2017-10-23 14:02:07 UTC
Hello @Johan nice patch of mono for VS 2015 and 2017.

What do I need requirements if I need Git "CORRECT", Visual Studio 2015 or Visual Studio 2017 "CORRECT", Windows SDK "CORRECT" and what do I need any tools? Pacman, Cygwin or TDM-GCC? If I don't need gcc, as and any than I use Visual Studio 2015 or 2017. Is it correct?

Thanks I will check your github server and I download it.
Comment 29 johan.lorensson 2017-10-23 14:34:16 UTC
Hi Jens,

The PR has been merged into public mono, so you can get fixes from there:

https://github.com/mono/mono/pull/5664

To build mono master (fixes currently not in the pre-build Windows downloads), do regular mono build from source on Windows (requires git + cygwin + VS2015 or VS2017). NOTE, this is only needed to build mono + BCL, once that is done and packaged you can use it on target machines just having a VS2015/VS2017 install (with Clang for VS codegen to remove gcc dependecy). More info around mono build on Windows can be found here:

http://www.mono-project.com/docs/compiling-mono/windows/

So to answer your question, once you have a working mono build from mono master, the target machines running mkbundle using build mono, should only need VS2015 and|or VS2017 (with Clang VS codegen option installed) + the WinSDK version you would like to use for your target build. The easiest way to build using the same settings as a selected VS install is to run mkbundle build from a VS development command prompt since it will have a good preset environment for all things needed.
Comment 30 Jens Eckervogt 2017-10-24 15:20:41 UTC
Hi @Johan, thanks for explanation!

PS: Please don't forget to edit / change mono-2.0.pc from MONO_INSTALL_PATH/lib/pkgconfig! Because I found problem with wrong path to WinSDK 8.1 to 10/include/<Version number>/um, shared and ucrt and path to VC/14.0/.../Host(arch) ( like Hostx86 or Hostx64 )
If you use ./configure --host=i686-w64-mingw32 = VCTOOL_PATH/Hostx86/bin/cl.exe
or x64 than x86_64-w64-mingw gives for VCTOOL_PATH/Hostx64/bin/cl.exe

Since I found many gcc or ld problems like export-dynamic and export-all-symbols conflicts

If we use latest gcc and ld version than we need remove arguments like -mno-cygwin or unusable parameters of gcc and ld.

Thanks I will try....

Note You need to log in before you can comment on or make changes to this bug.