Bug 21374 - encoderShouldEmitUTF8Identifier inconsistently set when using System.Diagnostics.Process
Summary: encoderShouldEmitUTF8Identifier inconsistently set when using System.Diagnost...
Alias: None
Product: Class Libraries
Classification: Mono
Component: System (show other bugs)
Version: 4.0.0
Hardware: PC Linux
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
Depends on:
Reported: 2014-07-16 13:13 UTC by delcypher
Modified: 2016-04-16 09:17 UTC (History)
2 users (show)

See Also:
Is this bug a regression?: ---
Last known good build:

Application demonstrating the issue. (5.17 KB, application/x-bzip)
2014-07-16 13:13 UTC, delcypher
New version of bug demonstration that works with mono 4.0.4 (5.31 KB, application/x-bzip)
2015-10-08 18:00 UTC, delcypher

Description delcypher 2014-07-16 13:13:42 UTC
Created attachment 7393 [details]
Application demonstrating the issue.

Steps to reproduce

1. Extract the attached application, load the Solution into Monodevelop and build the application

2. If the application is executed it seems the ``emitIdentifier`` is set to False for the UTF8Encoding attached to the Process.StandardInput. This results in the UTF8 Byte order mark not being emitted to Standard Input.

3. If the unit test is executed then it seems the ``emitIdentifier`` is set to True for the UTF8Encoding attached to the 
Process.StandardInput. This results in the UTF8 Byte order mark being emitted to Standard Input. I've deliberately written the code in my example to throw an exception if this happens. 

I believe the ``emitIdentifier`` field corresponds to the ``encoderShouldEmitUTF8Identifier`` constructor parameter to UTF8Encoding which is why I mention it in the Summary text.

Please note I am using Monodevelop (5.0.1) to

- build the application
- run the application
- run the unit test

Unfortunately it doesn't seem possible to run the unit test from the command line with

$ nunit-console4 test.csproj
Unhandled Exception:
System.IO.DirectoryNotFoundException: Directory "/home/dsl11/playground/test/test/bin\Debug" not found.

which is another bug...

What I expect to happen

``emitIdentifier`` should be set consistently on the UTF8Encoding attached to Process.StandardInput so that instances of the Process class behave consistently whether or not they are invoked from Main() or from a unit test.
Comment 1 delcypher 2015-10-08 18:00:58 UTC
Created attachment 13247 [details]
New version of bug demonstration that works with mono 4.0.4
Comment 2 delcypher 2015-10-08 18:11:05 UTC
The underlying implementation inside mono has changed so I've uploaded slightly modified code that demonstrates the same issue.

This issue recently bit me again because my workaround stopped working. The workaround was to basically write to the process's standard input using a different StreamWriter.

var streamWriter = new StreamWriter(TheProcess.StandardInput.BaseStream, new UTF8Encoding(false));

this no longer seems to work in Mono 4.0.4 because I observe the byte ordering mark being emitted to the underlying stream.

I took a look at Mono's internal implementation and I've found where the issues seems to be in System.Diagnostics.Process.Start_noshell().

The code of interest is

			if (startInfo.RedirectStandardInput == true) {
				MonoIO.Close (stdin_rd, out error);
				process.input_stream = new StreamWriter (new MonoSyncFileStream (stdin_wr, FileAccess.Write, true, 8192), Console.Out.Encoding);
				process.input_stream.AutoFlush = true;

The encoding for the input_stream is taken from ``Console.Out.Encoding`` and for some unknown reason, when running as normal executable ``Console.Out.Encoding.emitUTF8Identifier`` is false but when running as an NUnit test it is true.

I'm unsure why this difference exists and whether it is a bug in NUnit or in mono.

It seems if I do

Console.OutputEncoding = new UTF8Encoding(false);

before creating a process that this works around the issue.
Comment 3 Marek Safar 2016-04-16 09:17:30 UTC
Could you please re-test with Mono 4.4?

Note You need to log in before you can comment on or make changes to this bug.