Bug 19667

Summary: System.Net.Http.HttpClient.GetStringAsync return strings with unicode BOM
Product: [Mono] Class Libraries Reporter: Frank A. Krueger <fak>
Component: SystemAssignee: Marek Safar <masafa>
Status: RESOLVED FIXED    
Severity: normal CC: masafa, mono-bugs+monotouch, mono-bugs+mono, ramc
Priority: ---    
Version: unspecified   
Target Milestone: Untriaged   
Hardware: Macintosh   
OS: Mac OS   
Tags: Is this bug a regression?: ---
Last known good build:

Description Frank A. Krueger 2014-05-10 20:26:15 UTC
Some servers return unicode documents that begin with a BOM. 

System.Net.Http.HttpClient.GetStringAsync is unfortunately returning strings with that BOM.

The problem seems to be with using Encoding.GetString in HttpContent.ReadAsString. If a StreamReader is used instead, then the BOM is properly ignored.

System.Net.WebClient also has this problem.

For instance, the following URL returns a string whose first character is 0xFEFF: http://www.blogtalkradio.com/billcosby.rss

var client = new WebClient ();
var xml = client.DownloadString("http://www.blogtalkradio.com/billcosby.rss");
xml[0] == 0xFEFF

I expect the first character to be '<'.
Comment 1 Marek Safar 2014-05-12 09:26:33 UTC
Fixed in master
Comment 2 Ram Chandra 2014-06-03 14:25:50 UTC
I have check this issue and when I write the following code I am getting the "xml" as output.

var client = new WebClient ();
var xml = client.DownloadString("http://www.blogtalkradio.com/billcosby.rss");

Screencast: http://www.screencast.com/t/0ACd54gJ

This issue has been fixed. Hence closing this issue.

Environment Info

=== Xamarin Studio ===

Version 5.1 (build 327)
Installation UUID: 449f40dd-b3f1-4028-9a6b-cca0d1a2307d
Runtime:
	Mono 3.4.0 ((no/c3fc3ba)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 304000204

=== Apple Developer Tools ===

Xcode 5.1.1 (5085)
Build 5B1008

=== Xamarin.iOS ===

Version: 7.2.99.420 (Enterprise Edition)
Hash: 5aa4bec
Branch: 
Build date: 2014-06-02 00:04:26-0400

=== Xamarin.Android ===

Version: 4.14.0 (Enterprise Edition)
Android SDK: /Users/360logicaxamarinmacmini/Desktop/android-sdk-macosx_Róbert_à
	Supported Android versions:
		1.6   (API level 4)
		2.1   (API level 7)
		2.2   (API level 8)
		2.3   (API level 10)
		3.1   (API level 12)
		3.2   (API level 13)
		4.0   (API level 14)
		4.0.3 (API level 15)
		4.1   (API level 16)
		4.2   (API level 17)
		4.3   (API level 18)
		4.4   (API level 19)
Java SDK: /usr
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)

=== Xamarin.Mac ===

Xamarin.Mac: 1.8.0.7

=== Build Information ===

Release ID: 501000327
Git revision: 9a4bf62f59ec39169e4e9b61c3816a03c8ac961f
Build date: 2014-06-03 06:01:06-04
Xamarin addins: b68a34ef2fc4c46b045dc38e26fb199bfe1b201d

=== Operating System ===

Mac OS X 10.8.4
Darwin 360Logicas-Mac-mini.local 12.4.0 Darwin Kernel Version 12.4.0
    Sun Mar 10 18:01:10 PDT 2013
    root:xnu-2050.24.6~1/RELEASE_X86_64 x86_64