Bug 21374 - encoderShouldEmitUTF8Identifier inconsistently set when using System.Diagnostics.Process
Summary: encoderShouldEmitUTF8Identifier inconsistently set when using System.Diagnost...
Alias: None
Product: Class Libraries
Classification: Mono
Component: System ()
Version: 4.0.0
Hardware: PC Linux
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
Depends on:
Reported: 2014-07-16 13:13 UTC by delcypher
Modified: 2018-03-13 11:07 UTC (History)
2 users (show)

Is this bug a regression?: ---
Last known good build:

Application demonstrating the issue. (5.17 KB, application/x-bzip)
2014-07-16 13:13 UTC, delcypher
New version of bug demonstration that works with mono 4.0.4 (5.31 KB, application/x-bzip)
2015-10-08 18:00 UTC, delcypher

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description delcypher 2014-07-16 13:13:42 UTC
Created attachment 7393 [details]
Application demonstrating the issue.

Steps to reproduce

1. Extract the attached application, load the Solution into Monodevelop and build the application

2. If the application is executed it seems the ``emitIdentifier`` is set to False for the UTF8Encoding attached to the Process.StandardInput. This results in the UTF8 Byte order mark not being emitted to Standard Input.

3. If the unit test is executed then it seems the ``emitIdentifier`` is set to True for the UTF8Encoding attached to the 
Process.StandardInput. This results in the UTF8 Byte order mark being emitted to Standard Input. I've deliberately written the code in my example to throw an exception if this happens. 

I believe the ``emitIdentifier`` field corresponds to the ``encoderShouldEmitUTF8Identifier`` constructor parameter to UTF8Encoding which is why I mention it in the Summary text.

Please note I am using Monodevelop (5.0.1) to

- build the application
- run the application
- run the unit test

Unfortunately it doesn't seem possible to run the unit test from the command line with

$ nunit-console4 test.csproj
Unhandled Exception:
System.IO.DirectoryNotFoundException: Directory "/home/dsl11/playground/test/test/bin\Debug" not found.

which is another bug...

What I expect to happen

``emitIdentifier`` should be set consistently on the UTF8Encoding attached to Process.StandardInput so that instances of the Process class behave consistently whether or not they are invoked from Main() or from a unit test.
Comment 1 delcypher 2015-10-08 18:00:58 UTC
Created attachment 13247 [details]
New version of bug demonstration that works with mono 4.0.4
Comment 2 delcypher 2015-10-08 18:11:05 UTC
The underlying implementation inside mono has changed so I've uploaded slightly modified code that demonstrates the same issue.

This issue recently bit me again because my workaround stopped working. The workaround was to basically write to the process's standard input using a different StreamWriter.

var streamWriter = new StreamWriter(TheProcess.StandardInput.BaseStream, new UTF8Encoding(false));

this no longer seems to work in Mono 4.0.4 because I observe the byte ordering mark being emitted to the underlying stream.

I took a look at Mono's internal implementation and I've found where the issues seems to be in System.Diagnostics.Process.Start_noshell().

The code of interest is

			if (startInfo.RedirectStandardInput == true) {
				MonoIO.Close (stdin_rd, out error);
				process.input_stream = new StreamWriter (new MonoSyncFileStream (stdin_wr, FileAccess.Write, true, 8192), Console.Out.Encoding);
				process.input_stream.AutoFlush = true;

The encoding for the input_stream is taken from ``Console.Out.Encoding`` and for some unknown reason, when running as normal executable ``Console.Out.Encoding.emitUTF8Identifier`` is false but when running as an NUnit test it is true.

I'm unsure why this difference exists and whether it is a bug in NUnit or in mono.

It seems if I do

Console.OutputEncoding = new UTF8Encoding(false);

before creating a process that this works around the issue.
Comment 3 Marek Safar 2016-04-16 09:17:30 UTC
Could you please re-test with Mono 4.4?
Comment 4 Marek Safar 2018-03-13 11:07:38 UTC
We have not received the requested information. If you are still experiencing this issue please provide all the requested information and reopen the bug report.

Thank you!