Bug 18416 - System.Threading.SynchronizationLockException under very high load
Summary: System.Threading.SynchronizationLockException under very high load
Status: NEW
Alias: None
Product: Class Libraries
Classification: Mono
Component: System.Web (show other bugs)
Version: 3.2.x
Hardware: PC Windows
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2014-03-16 09:06 UTC by Lex Li
Modified: 2014-03-16 09:06 UTC (History)
1 user (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments

Description Lex Li 2014-03-16 09:06:30 UTC
During performance testing against xsp, sometimes exceptions are thrown frequently and the responses become pretty slow,

<!--
[System.Threading.SynchronizationLockException]: The current thread has not ente                                                                             red the lock in upgradable mode
  at System.Threading.ReaderWriterLockSlim.ExitUpgradeableReadLock () [0x00000]                                                                              in <filename unknown>:0
  at System.Web.Caching.Cache.Get (System.String key) [0x00000] in <filename unk                                                                             nown>:0
  at System.Web.Caching.InMemoryOutputCacheProvider.Get (System.String key) [0x0                                                                             0000] in <filename unknown>:0
  at System.Web.Caching.OutputCacheModule.OnResolveRequestCache (System.Object o                                                                             , System.EventArgs args) [0x00000] in <filename unknown>:0
  at System.Web.HttpApplication+<RunHooks>c__Iterator0.MoveNext () [0x00000] in                                                                              <filename unknown>:0
  at System.Web.HttpApplication+<Pipeline>c__Iterator1.MoveNext () [0x00000] in                                                                              <filename unknown>:0
  at System.Web.HttpApplication.Tick () [0x00000] in <filename unknown>:0
-->

If the load is stopped, xsp can recover from the slowness. Strangely if inserting Console.WriteLine() to slow down the request processing, the slowness does not occur.

We are using a 4 core Linux server with 2000 concurrent requests.

Is this caused by ReaderWriterLockSlim? We wrote a sample to test out ReaderWriterLockSlim and observed that the performance on Linux/Mono is much worse than on Windows/.NET.

===============
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace rwlck
{
    class Program
    {
        static int count;
        static ReaderWriterLockSlim rwLck = new ReaderWriterLockSlim();

        static void Main(string[] args)
        {
            for (var i = 0; i < 100; i++)
            {
                new Thread(Go).Start();
                new Thread(save1).Start();
            }

            Console.WriteLine("wait....");
            var t = DateTime.Now.Ticks;
            while (Interlocked.CompareExchange(ref count, 0, -1) < 1000 * 100)
            {
                Thread.Sleep(500);
            }
            Console.WriteLine("count:{0}, time:{1}ms",1000*100,(DateTime.Now.Ticks-t)/10000);
            Console.ReadKey();
        }


        static void Go()
        {
            for (var i = 0; i < 1000; i++)
            {
                Test();
            }
        }

        static void Test()
        {
            try
            {
                rwLck.EnterUpgradeableReadLock();

                Interlocked.Increment(ref count);

                var i = read();
                try
                {
                    rwLck.EnterWriteLock();
                    save();
                }
                finally
                {
                    rwLck.ExitWriteLock();
                }

            }
            finally
            {
                rwLck.ExitUpgradeableReadLock();

            }

        }

        static int read()
        {
            return count;
        }

        static int tt = 0;
        static void save()
        {
            //Console.WriteLine(count);
            //Thread.Sleep(1);
            tt++;
        }

        static void save1()
        {
            try
            {
                Thread.Sleep(1);
                rwLck.EnterWriteLock();
                save();
            }
            finally
            {
                rwLck.ExitWriteLock();
            }
        }

    }
}
================

More information about our load testing can be provided if required.

Note You need to log in before you can comment on or make changes to this bug.