Bug 19667 - System.Net.Http.HttpClient.GetStringAsync return strings with unicode BOM
Summary: System.Net.Http.HttpClient.GetStringAsync return strings with unicode BOM
Alias: None
Product: Class Libraries
Classification: Mono
Component: System ()
Version: unspecified
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: Untriaged
Assignee: Marek Safar
Depends on:
Reported: 2014-05-10 20:26 UTC by Frank A. Krueger
Modified: 2014-06-03 14:25 UTC (History)
4 users (show)

Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description Frank A. Krueger 2014-05-10 20:26:15 UTC
Some servers return unicode documents that begin with a BOM. 

System.Net.Http.HttpClient.GetStringAsync is unfortunately returning strings with that BOM.

The problem seems to be with using Encoding.GetString in HttpContent.ReadAsString. If a StreamReader is used instead, then the BOM is properly ignored.

System.Net.WebClient also has this problem.

For instance, the following URL returns a string whose first character is 0xFEFF: http://www.blogtalkradio.com/billcosby.rss

var client = new WebClient ();
var xml = client.DownloadString("http://www.blogtalkradio.com/billcosby.rss");
xml[0] == 0xFEFF

I expect the first character to be '<'.
Comment 1 Marek Safar 2014-05-12 09:26:33 UTC
Fixed in master
Comment 2 Ram Chandra 2014-06-03 14:25:50 UTC
I have check this issue and when I write the following code I am getting the "xml" as output.

var client = new WebClient ();
var xml = client.DownloadString("http://www.blogtalkradio.com/billcosby.rss");

Screencast: http://www.screencast.com/t/0ACd54gJ

This issue has been fixed. Hence closing this issue.

Environment Info

=== Xamarin Studio ===

Version 5.1 (build 327)
Installation UUID: 449f40dd-b3f1-4028-9a6b-cca0d1a2307d
	Mono 3.4.0 ((no/c3fc3ba)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 304000204

=== Apple Developer Tools ===

Xcode 5.1.1 (5085)
Build 5B1008

=== Xamarin.iOS ===

Version: (Enterprise Edition)
Hash: 5aa4bec
Build date: 2014-06-02 00:04:26-0400

=== Xamarin.Android ===

Version: 4.14.0 (Enterprise Edition)
Android SDK: /Users/360logicaxamarinmacmini/Desktop/android-sdk-macosx_Róbert_à
	Supported Android versions:
		1.6   (API level 4)
		2.1   (API level 7)
		2.2   (API level 8)
		2.3   (API level 10)
		3.1   (API level 12)
		3.2   (API level 13)
		4.0   (API level 14)
		4.0.3 (API level 15)
		4.1   (API level 16)
		4.2   (API level 17)
		4.3   (API level 18)
		4.4   (API level 19)
Java SDK: /usr
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)

=== Xamarin.Mac ===


=== Build Information ===

Release ID: 501000327
Git revision: 9a4bf62f59ec39169e4e9b61c3816a03c8ac961f
Build date: 2014-06-03 06:01:06-04
Xamarin addins: b68a34ef2fc4c46b045dc38e26fb199bfe1b201d

=== Operating System ===

Mac OS X 10.8.4
Darwin 360Logicas-Mac-mini.local 12.4.0 Darwin Kernel Version 12.4.0
    Sun Mar 10 18:01:10 PDT 2013
    root:xnu-2050.24.6~1/RELEASE_X86_64 x86_64