Bug 52112 - The user defined web proxy is not used by the HttpClientHandler
Summary: The user defined web proxy is not used by the HttpClientHandler
Alias: None
Product: Xamarin.Mac
Classification: Desktop
Component: Library (Xamarin.Mac.dll) (show other bugs)
Version: Master
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: Future Cycle (TBD)
Assignee: Marek Safar
Depends on:
Reported: 2017-01-31 18:32 UTC by Xavier Rigau
Modified: 2017-07-10 20:41 UTC (History)
11 users (show)

See Also:
Tags: xamexttriage xamextfollowed
Is this bug a regression?: ---
Last known good build:

Actual HTTP responses (105.13 KB, image/png)
2017-01-31 18:32 UTC, Xavier Rigau
Xamarin Studio errors (116.78 KB, image/png)
2017-02-02 21:30 UTC, Xavier Rigau

Description Xavier Rigau 2017-01-31 18:32:39 UTC
Created attachment 19668 [details]
Actual HTTP responses

In Mac OS X when a user is behind a web proxy the HTTP requests issued by HttpClientHandler don't work. They don't use the proxy settings and either timeout or if there is any server intercepting them you get the reply from the wrong host. For instance, in HP we use an automatic configuration script.

xMacMini:~ xavierr$ scutil --proxy
<dictionary> {
  ExceptionsList : <array> {
    0 : *.local
    1 : 169.254/16
  FTPPassive : 1
  HTTPEnable : 0
  ProxyAutoConfigEnable : 1
  ProxyAutoConfigURLString : http://autocache.hp.com

I reported it https://forums.xamarin.com/discussion/comment/249460 but I only got suggestions for workarounds.

-There is plenty documentation about the limitations of the Mono HTTP client.
-Xamarin Studio installer suffers the same problem and doesn't install behind my firewall
-The code we use in our app works untouched in .NET framework and WinRT in Windows.

The post done from inside the corporate network return a HTML response as in the attached picture.
Comment 2 Xavier Rigau 2017-02-02 21:30:43 UTC
Created attachment 19703 [details]
Xamarin Studio errors
Comment 3 Xavier Rigau 2017-02-02 21:31:03 UTC
Comment on attachment 19703 [details]
Xamarin Studio errors

-Xamarin Studio shows "Untrusted HTTP certificate detected" when it starts up behind a web proxy (wired)
-It doesn't if there is no proxy (wireless)

See 2nd attachment
Comment 4 Xavier Rigau 2017-02-02 21:31:10 UTC
-Xamarin Studio shows "Untrusted HTTP certificate detected" when it starts up behind a web proxy (wired)
-It doesn't if there is no proxy (wireless)

See 2nd attachment
Comment 5 Marek Safar 2017-02-06 14:46:49 UTC
Did you try to use CFNetworkHandler with .UseSystemProxy set to true ?

@chamons, I think the XS settings is quite misleading as it mentions iOS 6+ but no Mac.
Comment 6 Chris Hamons 2017-02-06 14:50:59 UTC
Good catch on IDE text, adding Vincent to let him know about that.
Comment 7 Vincent Dondain [MSFT] 2017-03-14 22:17:27 UTC
Not sure what's misleading in the XS setting, what are we talking about? Screenshot maybe?
Comment 8 Marek Safar 2017-03-23 09:58:44 UTC
Vincent, the handler settings indicates the handler can be used with iOS only which is not true
Comment 10 Marek Safar 2017-06-01 21:18:21 UTC
Still waiting on response whether https://bugzilla.xamarin.com/show_bug.cgi?id=52112#c5 fixes the issue for customer
Comment 11 Vincent Dondain [MSFT] 2017-06-07 20:47:19 UTC
@xavier can you please try to use CFNetworkHandler with .UseSystemProxy set to true?

Like Marek recommended, otherwise if you're not experiencing this issue anymore please close the bug.
Comment 12 Xavier Rigau 2017-06-07 21:04:06 UTC
No. Because: 

1) This won't fix the errors in Xamarin Studio/installer/updater.
2) The same code works with Windows HTTPS stack
3) This bug is not unique to HP or custom proxies; it is a Xamarin shortcoming. Xamarin's customers should not have to deal with this issue; fixing it is for the benefit of MS, HP and the Xamarin product.

The mono HTTP stack is just partially implemented see the code here https://github.com/mono/mono/blob/effa4c07ba850bedbe1ff54b2a5df281c058ebcb/mcs/class/System/System.Net/MacProxy.cs#L910 

switch (proxies[i].ProxyType) {
// unsupported proxy type (requires fetching script from remote url)

case CFProxyType.SOCKS:
// unsupported proxy type, try the next one

For more context speak with Rob DeRosa.
Comment 13 Miguel de Icaza [MSFT] 2017-06-09 20:13:34 UTC
Question, what Http client handler are you using?

The one in the link above for Mono's HTTP stack only comes into play when using the Mono HttpWebRequest.

On iOS/Mac, we support both a CoreFoundat-based handler for older systems, and an NSUrlSession based one, which is Apple's recommended API for HTTP downloads.

Does using the System.Net.Http.NSUrlSessionHandler solve this problem?
Comment 14 Xavier Rigau 2017-06-22 19:58:14 UTC
Happens with CFNetwork, NSUrlSession and default build settings. Please note that as a library developer I don't have control of this build setting since it is only for apps.
Comment 15 Chris Hamons 2017-06-23 15:25:00 UTC
Xavier - Can you post a snipped of code showing how you are invoking the networking code in question. There are a number of different classes / ways of doing it, and they may trigger different behaviors.

Note You need to log in before you can comment on or make changes to this bug.