Bug 41133

This is a major problem for us too. It happens with apps that were during the last few weeks.

Could you provide more details?

Are you running desktop httpclient, or android, or ios something else?

Does this happen with linker disabled?


There should not be anything random in this failure so I am not sure how to reproduce it yet. The method is called only when web request content headers are filled which is usually done by user code based on some conditions.

I don't beliebe that it has anything to do with the WebHeaderCollection class or the AddValue method at all.

I get the following error related to ServicePointManager:CloseConnectionGroup.

When I check the dll file in the .apl with ILSpy, the CloseConnectionGroup method is most certainly there. So it can't be a linker issue, can it?

Maybe something in the Mono runtime is getting corrupted and it doesn't find the method, even if it's there.

This error is URGENT because it affects ~ 15% of our customers and we cannot possibly do anything to work around it!



System.MethodAccessException: Method `System.Net.ServicePointManager:CloseConnectionGroup (string)' is inaccessible from method `System.Net.Http.HttpClientHandler:Dispose (bool)'

at System.Net.Http.HttpMessageHandler.Dispose () [0x00000] in <filename unknown>:0
at System.Net.Http.HttpMessageInvoker.Dispose (Boolean disposing) [0x0001c] in <filename unknown>:0 
at System.Net.Http.HttpClient.Dispose (Boolean disposing) [0x00023] in <filename unknown>:0
at System.Net.Http.HttpMessageInvoker.Dispose () [0x00000] in <filename unknown>:0
at Digitalkraft.Framework.TimeHelper+<getCurrentTimestamp>d__0.MoveNext () [0x001aa] in <filename unknown>:0

There androiud api versions of the user's phones are all over the place. 18, 19, 22, 23.

It also doesn't matter whether the app is compiled with the stable or beta version of Xamarin.Android.

It does seem to happend only if the app is running for at least a couple of hours. Which is why I think it's a bug deep in the mono runtime which prevents it from finding the method although it must have been executed before.

There are multiple classes in the System.Net.Http namespace which are affected. As far as I can tell, it has never been anything unrelated to HTTP.

The HttpClient's MessageHandler does NOT matter. It happens with and without ModernHttpClient.

Hi Marek,
we are using latest Xamarin Android (stable channel). We have no error reported on Xamarin iOS yet (only android version).

we use HttpClient which is defined inside portable library. This portable library is referenced from android application (no Xamarin Forms)


I could add additional info when our code sending exception to Xamarin Insights. But what information? 

I could send you our portable library source to email if you want ..... Or anythning else...

Thank you

This code is used quite commonly so I am still not sure what's exactly the trigger. It could be the PCL setup though.

Could you try to add e.g "Warning" header to your request and see if it fails.

e.g. headers.Warning.Add (new WarningHeaderValue (5, "agent", "\"txt\""));

It should fail with same exception but at different place.

Hi Marek,
in my case everything works without problem (no exception at different place)

client = new HttpClient(new HttpClientHandler() { AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip });
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));

client.DefaultRequestHeaders.Warning.Add(new System.Net.Http.Headers.WarningHeaderValue(5, "agent", "\"txt\"")); // Added Line

I tend to agree this is probably a race somewhere in JIT.

MethodAccessException happens only during JIT-ing. Which is usually once, here we are inside generic value-type which could play a part.

Second thing to consider is that the MethodAccessException would be correct in this case as both methods where this happens are internal but there is InternalsVisibleTo in System.dll which should cover that but could be related to the problem.

Same issue for us.

Mono service running on a RP2 (ARM)

Mono JIT compiler version 4.2.1 (Stable 4.2.1.102/6dd2d0d Thu Nov 12 10:06:49 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       normal
        Notifications: epoll
        Architecture:  armel,vfp+hard
        Disabled:      none
        Misc:          softdebug 
        LLVM:          supported, not enabled.
        GC:            

The process is polling a bunch of devices with HTTP

Sample Code:

using (var client = new HttpClient())
            {
                if (!string.IsNullOrEmpty(Username))
                {
                    client.DefaultRequestHeaders.Authorization = CreateBasicHeader(Username, Password);
                }

                // very low timeout - it should not take 3 second to process this request. If it does, the device must be offline.
                client.Timeout = TimeSpan.FromSeconds(3);

                var response = await client.GetAsync(_theUriHere);
                response.EnsureSuccessStatusCode();

                var responseContent = await response.Content.ReadAsStringAsync();
            }

This call is being made about once a second to 6 different devices. Takes about 6-12 hours before this error occurs.. and at that point the process must be restarted for the HTTPClient to work again.

160527 02:10:59,150 -System.String[]. Error: Method `System.Net.ServicePointManager:CloseConnectionGroup (string)' is inaccessible from m
ethod `System.Net.Http.HttpClientHandler:Dispose (bool)'

160527 02:11:00,110 -System.String[]. Error: Method `System.Net.ServicePointManager:CloseConnectionGroup (string)' is inaccessible from m
ethod `System.Net.Http.HttpClientHandler:Dispose (bool)'

160527 02:11:00,119 -System.String[]. Error: Method `System.Net.ServicePointManager:CloseConnectionGroup (string)' is inaccessible from m
ethod `System.Net.Http.HttpClientHandler:Dispose (bool)'

160527 02:11:00,127 -System.String[]. Error: Method `System.Net.ServicePointManager:CloseConnectionGroup (string)' is inaccessible from m
ethod `System.Net.Http.HttpClientHandler:Dispose (bool)'

That's exactly what I experienced, it happens if the app is running for at least a couple of hours, and it happens in that code that had been executed before. (so if the project is JIT-related, why is it JIT'ed again?)

The problem is that apps "run" often for many days, but are suspended by Android.

It must be something more part of the framework. I am running on an always-on Raspberry Pi 2, just polling json off 6 devices every second for data collection.

For what it's worth, we've seen this on Xamarin.Android for a couple of years to varying degrees.  Seems like way back when it was fairly rare.  Now it's happening with more frequency.

We define a descendant interface/class that inherits from HttpClient in our Android code and inject that into a *PCL* service class implementation that accepts an interface.  So our shared REST code is where this exception is thrown.

System.MethodAccessException: Method `System.Net.ServicePointManager:CloseConnectionGroup (string)' is inaccessible from method `System.Net.Http.HttpClientHandler:Dispose (bool)'

I hope this can help someone. We do not seem to be having that same problem with this alternative code. It does not seem to use those same classes (have not checked the source). It would be nice if the source problem was fixed. I have not tested this under PCL, but it does work under Mono / Ubuntu without that error coming up anymore.

            var r = WebRequest.Create(uri);
            if (!string.IsNullOrWhiteSpace(Username))
            {
                r.Credentials = new NetworkCredential(Username, Password);
            }
            r.Timeout = 3000;

            using (var response = await r.GetResponseAsync())
            {
                using (var stream = response.GetResponseStream())
                {
                    if (stream == null) return null;
                    using (var reader = new StreamReader(stream))
                    {
                        var data = await reader.ReadToEndAsync();
                    }
                }
            }

Hi Rodrigo, any news about our problem?

It is clear to me that you have a lot of work, but it often seems that your team rather preparing new things for Evolve 2017 etc.

Do you need some more info? How we can change priority of this problem (reports from our customers come every day)?

Hi everybody,
meanwhile we use following a little less painful solution

#if __ANDROID__
if ((ex is MethodAccessException) && (ex.Message.Contains("System.Net.WebHeaderCollection:AddValue")))
{
    Logger.Instance.SendReport(ex, "Application must be killed. Thank you Xamarin, thank you Rodrigo :-)");

    await Xamarin.Insights.Save();
    Java.Lang.JavaSystem.Exit(0);
}
#endif

David

This is still happening to me. Any other way except restarting app?

This is still happening to me. Any other way except restarting app?

My workaround above seems to have solved it for me. It should be fixed though :(

Without any sort of code to reproduce this we can't realistically fix the issue.

I did multiple reviews of the code related to how we initialize visibility related fields and consume the InternalsVisibleTo attribute but nothing glaring shows up.

The test program doesn't need to be small or show the issue straight away.

Is there any kind of data we could provide you if this happens in our apps? If I give you an app of ours that has this problem, you will probably not be able to reproduce it because it is so random.

But I could integrate any kind of error handling you need, which would be executed when it happens on one of our customer's devices.

Is there a way to programmatically create a memory dump in a catch block? Or what exactly would you do if you could reproduce the issue?

Hey guys,

We got one theory we'd like to validate. We want to know whether we're compiling the problematic method multiple times.

Travis, you mentioned you can repro this on you RP2, right? Can you run the RP2 service with -v and share the end of the log output when it crashes?

The initial output will be gigantic and you can ignore it, I'm interested at what happens close to the point where it crashes.

Let me see if I can make a sample app to run under the raspberry pi -- I have already coded out a solution for our commercial application, but would love for this to be fixed. Where would you like the verbose output sent?  Might be to large for this chat.

Travis, if you can comes with an app, we'll run it on our arm hardware to reproduce it.

The log output is probably going to be 1-2 Meg larger in total. You can compress and attach it to this bug report itself. If it contains private information, I'll create a private bug for you.

yes this error is very random. I just know this error from xamarin insight and happen when my mobile app try to send something to my server. In my test device is work ok but from Xamarin Insight it happened in Android M only for now

This is happening to me to. We're not using it from a PCL or anything, but simply a shared code project.
Android, latest stable.

For me, when it happens it happens pretty much directly, I do not have to run the app for hours to trigger it.

I can seem to trigger it sometimes (about 1 time per 30 tries or so), in the following scenario:
1. An activity is started, which starts up tasks with HttpClient requests.
2. Before any task has managed to finish run, or maybe even initialize and start(?), I immediately push a button to start another activity, which also starts tasks with HttpClient requests.

The error I'm getting is the System.Net.WebHeaderCollection: AddValue MethodAccessExceptionMethod. I haven't seen the System.Net.ServicePointManager:CloseConnectionGroup one yet.

I did the best I could to reproduce on the RP2. Attached is the test code, the logs and executable.  Sorry for the delay on this.

This test just hits 4 different websites pulling data, until it fails.

https://files.econnect.tv/share.cgi?ssid=0uV3gha

It does actually fail, but not always the same issue. Looks like some other bugs happen which cause the application to exit.  the output logs are in here. little big.

Tested on:

Mono JIT compiler version 4.2.1 (Stable 4.2.1.102/6dd2d0d Thu Nov 12 10:06:49 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       normal
        Notifications: epoll
        Architecture:  armel,vfp+hard
        Disabled:      none
        Misc:          softdebug 
        LLVM:          supported, not enabled.
        GC:            sgen

Created attachment 17154 [details]
Test

Test attached

Hey Marek,

What's the expected output? 

How do I run it?

I ran the monotest.exe from your zip file with 4.6.0 and got a lot of the following:

Error, but not the error we are looking for: One or more errors occurred.
Got data 12
Got data 10
Got data 10


Is that good or bad?

Your output is normal. That one error is just because sites it was pulling from must not be up anymore. but you can leave it running for a day or so, and see it fail eventually. The above reports show that it takes a long time for this to fail.  Feel free to modify the URLs and have it pull in more data. I just didn't want to DDoS some legit website.

The "Got Data x" shows how many bytes were received from the site.

Check out the very simple source file. Its only a few lines of code to attempt to repeat this problem.

I let it run for a few hours and got nothing.

*** Bug 44134 has been marked as a duplicate of this bug. ***

Hi Rodrigo,

You will see this happen over a longer period of time. If you have a spare raspberry pi 2 nearby -- let it run up to a week. Errors will trigger. 

I think that's why this takes so long to reproduce. People with long running processes seem to get hit by different exceptions.  

Appreciate you looking into it.

Travis

Hi, 

I'm the writer of the Bug 44134 that was marked as duplicate of this one.

My problem happened inside a PCL, used inside a Xamarin Droid app.
I tested the PCL multiple times in a console app, and (as far as I can remember), the bug never triggered. The only occurrences of the bug were in the Xamarin.Droid app, using the PCL in an async Task.

I hope this helps...

Anthony

Hi,
we have same architecture as Anthony. Xamarin Android project calls HttpClient defined inside PCL project. We use async tasks around HttpClient calls.

Our apllication crash many times per day (about 5% of users) on different phones with different OS versions (Android 4.1 - Android M)

Do you have acess to Raspberry pi? Coul'd you verify Travis code? We coul'd try to write something similar as Travis.

Thanks for more information

David

Guys, I recently installed the latest version of xamarin for VS(stable channel) AND uninstalled BitDefender completely as I was experiencing other problems deploying apps when BitDefender was installed. 

I am excited to report that it's been couple of days that I haven't received this message anymore. I am not sure which one fixed it but I have a feeling it was BitDefender uninstallation. Important: Disabling it wont cut it!! I realized this the hard way.

M.

Hi,
we start using AndroidClientHandler after months of waiting. We coul'd confirm that problem dissaper.

But we have another problems, because there is missing implementaion of Timeout and CancellationToken.

More info is there

https://bugzilla.xamarin.com/show_bug.cgi?id=44673

Hopefully it will help others

D.

*** Bug 46176 has been marked as a duplicate of this bug. ***

I'm suddenly seeing this with Xamarin 4.2.1.60

Is there any update on this? Rodrigo was given an app to repro the issue, and told it would take many days (or even 1-2 weeks), and yet said he couldn't repro after only a few hours. That was in August, I would've expected a more serious attempt at repro'ing (letting it run for 7-14 days) by now.

This is our number one crash report in our current release (2,000 crashes so far), and has been the highest occurring for many releases. The issue was reported a year and a half ago, it's disappointing to see so little attention paid to such a serious bug.

Removing the HttpClient (Managed HttpClientHandler) and using HttpWebRequest, HttpWebResponse instead fixed all the following issues.

Note: AndroidClientHander was not tested.

1. Threading race conditions, source of the many types of System.MethodAccessException
2. DNS issues
3. unexpended connection loss or inability to connect to any endpoint after process has started.
4. Slow and hanging downloads

async HttpWebRequest example:

var request = (HttpWebRequest) WebRequest.Create(uri);  

...

using (WebResponse response = await request.GetResponseAsync())
{
    using (var stream = response.GetResponseStream())
    {
        using (Stream fileStream = new FileStream(filePath, FileMode.Create))
        {
...

I've put a delay between 2 processes that were highly likely to start at the same time doing the same thing with the Microsoft.Azure.Devices.Client.Transport.HttpClientHelper originating from the Microsoft.Azure.Devices.Client.PCL NuGet package. This 'solved' it... for now.

Hard to reproduce, but it does exist.

Reopening.

I feel that this bug and the bug below are possibly related and lost of network connection in a thread during IO or just before IO is the best way to reproduce. In my previous post about removing the HttpClient, I also added a mechanism to prevent IO when a network connection is not available before stating the IO thread. This little change may have played a role.

https://bugzilla.xamarin.com/show_bug.cgi?id=45761

Zoltan came up with a repro with that I get a MethodAccessException crash instantly (not always, but pretty reliable):

https://gist.github.com/lewurm/9def48a0f9e1df9cf46e4d1a9d2affb7

Only on arm32/linux so far, no luck on amd64/darwin or amd64/linux, so it might be arm specific. reproduced on mono-4.8.0-branch/f11214c.

fix for master: https://github.com/mono/mono/pull/4277
fix for 4.8 (c9): https://github.com/mono/mono/pull/4278

The fix landed for 15.1 / mono 4.8.1.

Bernhard I have released android app with mono fix reported into your pull request 4278-4277. But issue is not solved. My crash report tool (Firebase) report (mostly for API 23) This is stacktrace 

Caused by android.runtime.JavaProxyThrowable: 
System.MethodAccessException: Method `System.Net.Http.HttpClientHandler:Dispose (bool)' is inaccessible from method `System.Net.ServicePointManager:CloseConnectionGroup (string)' at (wrapper managed-to-native) 
System.Object:__icall_wrapper_mono_throw_method_access (intptr,intptr) at System.Net.Http.HttpClientHandler.Dispose (System.Boolean disposing) [0x0001d] in <7508248e311245cb89d277b254c5e98a>:0 at Xamarin.Android.Net.AndroidClientHandler.Dispose (System.Boolean disposing) [0x00007] in <9f9b58e53f054b4a8ff2361fe40bc1c8>:0 
at System.Net.Http.HttpMessageHandler.Dispose () [0x00000] in <7508248e311245cb89d277b254c5e98a>:0 
at System.Net.Http.HttpMessageInvoker.Dispose (System.Boolean disposing) [0x0001c] in <7508248e311245cb89d277b254c5e98a>:0 
at System.Net.Http.HttpClient.Dispose (System.Boolean disposing) [0x00023] in <7508248e311245cb89d277b254c5e98a>:0 
at System.Net.Http.HttpMessageInvoker.Dispose () [0x00000] in <7508248e311245cb89d277b254c5e98a>:0 
at JR.SL.Services.JRBaseRestClient+<MakeRequestAsync>c__async2.MoveNext () [0x004cc] in <22ef47fadf794260ad9c35e9a6447864>:0 
--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <c7b255bd809f47538071732ebf2fd5d2>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <c7b255bd809f47538071732ebf2fd5d2>:0 
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <c7b255bd809f47538071732ebf2fd5d2>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <c7b255bd809f47538071732ebf2fd5d2>:0 
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () [0x00000] in

Hello Luigi,

afaik this fix didn't make into a stable release yet.  What Xamarin Android version/revision do you use?

Mono 4.8.0 (mono-4.8.0-branch/9ac5bf2) (64-bit)
=== Xamarin.Android ===
Version: 7.1.0.2

You need a release of Android with the fix, not of mono desktop.

Bernhard, maybe we can provide a link to an internal build of the next Xamarin.Android.

Luigi, are you on Mac or Windows?

Mac

Internal build for Xamarin Android on Mac OS that contains the fix:

https://bosstoragemirror.blob.core.windows.net/wrench/monodroid-mavericks-cycle9/95/9578cdcdb9cef07c6d11d62bd59d757b21308044/xamarin.android-7.1.0-41.pkg

md5=a7d9f6ab0c649cc9d9247d52715a4ea9


I highly recommend to wait for the next official stable release (C9), which is about to happen in the next days.

I will let you know if bug happens again.

Same project,
Xamarin.VisualStudio_4.3.0.784 throw exception:  System.MethodAccessExceptionMethod ... is inaccessible method from...
Xamarin.VisualStudio_4.2.2.11 is Ok.

Klark, can you provide more information?  What project? Full stacktrace?

Hi,
we switch back from AndroidClientHandler (which is future incomplete and full of bugs) back to default handler and we could confirm that problem disappear in C9 (after week of internal testing). 

We going to release our application to production environment next week. I will inform you if problem occurs again.

Thank you for fix. 
David

Hi David,

what's the situation with your application regarding this issue?


Thanks,
-Bernhard

Hi Bernhard,
our application is more than two weeks in production. Everything works like a charm.

We had received many exceptions in Xamarin Insights per day in dark times :-) (Before this fix)

We don't have similar exception in Insights again. 

From my point of view is this problem fixed and coul'd be closed.

Also from my side. It's only 5 days but there are no crash on Firebase console of this kind of issue.

Awesome news, thanks David and Luigi!

I mark it as resolved, feel free to reopen if there are issues.

As discussed on slack with Kyle White- This issue looks hard to reproduce based on comments 59 and 60 mark this bug as verified