Bug 16685 - Deadlock when using Windows Azure Storage libraries
Summary: Deadlock when using Windows Azure Storage libraries
Status: NEEDINFO
Alias: None
Product: Runtime
Classification: Mono
Component: io-layer (show other bugs)
Version: 3.2.x
Hardware: PC Linux
: --- normal
Target Milestone: ---
Assignee: Paolo Molaro
URL:
Depends on:
Blocks:
 
Reported: 2013-12-09 21:01 UTC by smith.cba
Modified: 2017-09-06 13:53 UTC (History)
4 users (show)

See Also:
Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Test case to reproduce (3.36 KB, text/plain)
2013-12-09 21:21 UTC, smith.cba
Details

Description smith.cba 2013-12-09 21:01:42 UTC
When using the latest Windows Azure Storage libraries (v2.1) I see deadlocks 100% of the time.  I've created a testcase where I have 15 Tasks, each doing an 17MB Upload, Download, Delete from Windows Azure Blob storage.  This works fine running with .NET 4.0 on Windows and with the older Mono 2.10.8.1.

Below is the backtrace.

Using Mono built from source (GitHub) on Ubuntu 13.04 server x64

$ /usr/local/bin/mono --version
Mono Runtime Engine version 3.2.7 (remotes/origin/mono-3.2.5-branch/9003740 Wed Dec  4 02:16:54 UTC 2013)
Copyright (C) 2002-2013 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen


(gdb) thread apply all bt

Thread 27 (Thread 0x7fcb747e5700 (LWP 39275)):
#0  0x00007fcb7732cf80 in sem_wait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626e38 in mono_sem_wait (sem=sem@entry=0x96f5a0 <finalizer_sem>, alertable=alertable@entry=1)
    at mono-semaphore.c:119
#2  0x00000000005a2d75 in finalizer_thread (unused=unused@entry=0x0) at gc.c:1073
#3  0x0000000000586942 in start_wrapper_internal (data=0x266d380) at threads.c:609
#4  start_wrapper (data=0x266d380) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25eedd0) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x2670ff0) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 26 (Thread 0x7fcb6ff7c700 (LWP 39276)):
#0  0x00007fcb7732e43d in nanosleep () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x000000000061aaa7 in SleepEx (ms=ms@entry=500, alertable=alertable@entry=1) at wthreads.c:842
#2  0x0000000000588673 in monitor_thread (unused=unused@entry=0x0) at threadpool.c:779
#3  0x0000000000586942 in start_wrapper_internal (data=0x2aa43d0) at threads.c:609
#4  start_wrapper (data=0x2aa43d0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25ef028) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x2b04520) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 25 (Thread 0x7fcb6fdff700 (LWP 39277)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x2b04ac0) at threads.c:609
#4  start_wrapper (data=0x2b04ac0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25ef0f0) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x2b04710) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 24 (Thread 0x7fcb6fbb5700 (LWP 39278)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25ef348) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68002650) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 23 (Thread 0x7fcb6f9b4700 (LWP 39279)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0b870, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6f9b1e78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb58002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6f9b1f20 in ?? ()
#11 0x00007fcb6f9b1ea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb768e5aa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 22 (Thread 0x7fcb6f7b3700 (LWP 39280)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0b710, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6f7b0e78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
---Type <return> to continue, or q <return> to quit---
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb5c002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6f7b0f20 in ?? ()
#11 0x00007fcb6f7b0ea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb768ecaa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 21 (Thread 0x7fcb6f5b2700 (LWP 39282)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25ef7f8) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68002b60) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 20 (Thread 0x7fcb6f3b1700 (LWP 39283)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0b450, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6f3aee78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb54002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6f3aef20 in ?? ()
#11 0x00007fcb6f3aeea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb768faaa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 19 (Thread 0x7fcb6f1b0700 (LWP 39284)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0b2f0, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6f1ade78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb48002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6f1adf20 in ?? ()
#11 0x00007fcb6f1adea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb76901aa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 18 (Thread 0x7fcb6efaf700 (LWP 39285)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25efca8) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68003070) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 17 (Thread 0x7fcb6edae700 (LWP 39286)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0b030, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6edabe78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb40002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6edabf20 in ?? ()
#11 0x00007fcb6edabea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb7690faa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 16 (Thread 0x7fcb6ebad700 (LWP 39287)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25effc8) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb680033d0) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 15 (Thread 0x7fcb6e9ac700 (LWP 39288)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f0158) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68003580) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 14 (Thread 0x7fcb6e7ab700 (LWP 39289)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0ac10, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6e7a8e78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb3c002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6e7a8f20 in ?? ()
#11 0x00007fcb6e7a8ea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb76924aa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 13 (Thread 0x7fcb6e5aa700 (LWP 39290)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0aab0, multiple=multiple@entry=0,
---Type <return> to continue, or q <return> to quit---
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6e5a7e78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb30002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6e5a7f20 in ?? ()
#11 0x00007fcb6e5a7ea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb7692baa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 12 (Thread 0x7fcb6e3a9700 (LWP 39291)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0x43e, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0x43e, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0x43e, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d0a950, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fcb6e3a6e78, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0x43e, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x00007fcb34002540 in ?? ()
#8  0x00007fcb769400b8 in ?? ()
#9  0x00007fcb76809040 in ?? ()
#10 0x00007fcb6e3a6f20 in ?? ()
#11 0x00007fcb6e3a6ea0 in ?? ()
#12 0x00007fcb76809040 in ?? ()
#13 0x00007fcb76932aa0 in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 11 (Thread 0x7fcb6e1a8700 (LWP 39292)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f0798) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68003c40) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 10 (Thread 0x7fcb6d985700 (LWP 39294)):
#0  0x00007fcb770514d3 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00000000005895e4 in tp_epoll_wait (p=p@entry=0x96f0a0 <socket_io_data>) at ../../mono/metadata/tpool-epoll.c:118
#2  0x0000000000586942 in start_wrapper_internal (data=0x7fcb2c06bec0) at threads.c:609
#3  start_wrapper (data=0x7fcb2c06bec0) at threads.c:654
#4  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f09f0) at wthreads.c:294
#5  0x000000000062b710 in inner_start_thread (arg=0x7fcb2c071b60) at mono-threads-posix.c:49
#6  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#7  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 9 (Thread 0x7fcb6d944700 (LWP 39295)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f148 <async_io_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f120 <async_io_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb2c06dc70) at threads.c:609
#4  start_wrapper (data=0x7fcb2c06dc70) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f0ab8) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb2c0545a0) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 8 (Thread 0x7fcb0c9fc700 (LWP 39303)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f5fe0) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68004660) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 7 (Thread 0x7fcaf75fa700 (LWP 39318)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f66e8) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68005420) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 6 (Thread 0x7fcaf6df6700 (LWP 39325)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f6ad0) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68005c50) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 5 (Thread 0x7fcaf63f1700 (LWP 39330)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f6eb8) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb680065e0) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 4 (Thread 0x7fcaf51e8700 (LWP 39344)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f7750) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb680076c0) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 3 (Thread 0x7fcaf4fe7700 (LWP 39345)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f7818) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68007740) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 2 (Thread 0x7fcaf5bed700 (LWP 39362)):
#0  0x00007fcb7732d071 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000626f1b in mono_sem_timedwait (sem=sem@entry=0x96f208 <async_tp+40>, timeout_ms=timeout_ms@entry=2000,
    alertable=alertable@entry=1) at mono-semaphore.c:82
#2  0x000000000058b292 in async_invoke_thread (data=0x0, data@entry=0x96f1e0 <async_tp>) at threadpool.c:1565
#3  0x0000000000586942 in start_wrapper_internal (data=0x7fcb680025c0) at threads.c:609
#4  start_wrapper (data=0x7fcb680025c0) at threads.c:654
#5  0x000000000061b7c1 in thread_start_routine (args=args@entry=0x25f9500) at wthreads.c:294
#6  0x000000000062b710 in inner_start_thread (arg=0x7fcb68007100) at mono-threads-posix.c:49
#7  0x00007fcb77326f8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007fcb77050e1d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 1 (Thread 0x7fcb77e5e7c0 (LWP 39274)):
#0  0x00007fcb7732aca4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000604c6b in _wapi_handle_timedwait_signal_handle (handle=handle@entry=0xc2b, timeout=timeout@entry=0x0,
    alertable=alertable@entry=1, poll=poll@entry=0) at handles.c:1588
#2  0x0000000000604c9b in _wapi_handle_wait_signal_handle (handle=handle@entry=0xc2b, alertable=alertable@entry=1)
    at handles.c:1533
#3  0x000000000061881d in WaitForSingleObjectEx (handle=0xc2b, timeout=timeout@entry=4294967295, alertable=alertable@entry=1)
---Type <return> to continue, or q <return> to quit---
    at wait.c:196
#4  0x0000000000584aef in mono_wait_uninterrupted (thread=thread@entry=0x7fcb76d08010, multiple=multiple@entry=0,
    numhandles=numhandles@entry=1, handles=handles@entry=0x7fff4b3f8cd8, waitall=waitall@entry=0, ms=ms@entry=-1, alertable=1)
    at threads.c:1497
#5  0x0000000000586279 in ves_icall_System_Threading_WaitHandle_WaitOne_internal (this=<optimized out>, handle=0xc2b, ms=-1,
    exitContext=<optimized out>) at threads.c:1629
#6  0x0000000041e3cf88 in ?? ()
#7  0x0000000002669960 in ?? ()
#8  0x00007fcb76ba5350 in ?? ()
#9  0x00007fcb76864210 in ?? ()
#10 0x00007fff4b3f8d90 in ?? ()
#11 0x00007fff4b3f8d00 in ?? ()
#12 0x00007fcb76864f88 in ?? ()
#13 0x00007fcb76864f20 in ?? ()
#14 0x00007fcb76ba5328 in ?? ()
#15 0xffffffffffffffff in ?? ()
#16 0x00007fcb74bb5c73 in System.Threading.WaitHandle:WaitOne (this=..., millisecondsTimeout=0, exitContext=false)
    at <unknown>:382
#17 0x00007fcb74bded91 in System.Threading.ManualResetEventSlim:Wait (this=..., millisecondsTimeout=-1, cancellationToken=0)
    at <unknown>:185
#18 0x00007fcb74bcd55c in System.Threading.Tasks.Task:WaitCore (this=..., millisecondsTimeout=-1, cancellationToken=0)
    at <unknown>:675
#19 0x00007fcb74bcd332 in System.Threading.Tasks.Task:Wait (this=..., millisecondsTimeout=-1, cancellationToken=0)
    at <unknown>:646
#20 0x00007fcb74bcd257 in System.Threading.Tasks.Task:Wait (this=...) from /usr/local/lib/mono/4.5/mscorlib.dll.so
#21 0x0000000041e3c0a6 in ?? ()
#22 0x00007fcb76864f88 in ?? ()
#23 0x00007fcb76864f88 in ?? ()
#24 0x000000000000000f in ?? ()
#25 0x0000000000000011 in ?? ()
#26 0x00007fcb76800c78 in ?? ()
#27 0x5b5b5b5b5b5b5b5b in ?? ()
#28 0x0120202020202020 in ?? ()
#29 0x00007fcb76868f48 in ?? ()
#30 0x00007fcb76864f20 in ?? ()
#31 0x000000000000000f in ?? ()
#32 0x0000000000000000 in ?? ()
(gdb)
Comment 1 smith.cba 2013-12-09 21:10:10 UTC
Also seen on Windows with Mono 3.2.3
Comment 2 smith.cba 2013-12-09 21:21:09 UTC
Created attachment 5625 [details]
Test case to reproduce

This test case requires the Azure Storage library 2.1
Comment 3 Rodrigo Kumpera 2013-12-10 16:45:09 UTC
Paolo, this looks related to the threadpool. Please take a look.
Comment 4 smith.cba 2013-12-10 17:45:32 UTC
I should have mentioned that the sources for the Azure storage linraries is available here:

https://github.com/WindowsAzure/azure-sdk-for-net


I also see *lots* of System.ObjectDisposedException exceptions when using the storage library, maybe this is related?


1386714822.036618 [Error]: An error has occurred and not retrying., Microsoft.WindowsAzure.Storage.StorageException: The object was used after being disposed. ---> System.ObjectDisposedException: The object was used after being disposed.
1386714822.036749   at System.Threading.WaitHandle.CheckDisposed () [0x00000] in <filename unknown>:0
1386714822.036811   at System.Threading.EventWaitHandle.Set () [0x00000] in <filename unknown>:0
1386714822.036848   at (wrapper remoting-invoke-with-check) System.Threading.EventWaitHandle:Set ()
1386714822.036885   at System.IO.Stream.EndRead (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0
1386714822.036921   at Microsoft.WindowsAzure.Storage.Core.Util.StreamExtensions.WriteToSync[NullType] (System.IO.Stream stream, System.IO.Stream toStream, Nullable`1 copyLength, Nullable`1 maxLength, Boolean calculateMd5, Boolean syncRead, Microsoft.WindowsAzure.Storage.Core.Executor.ExecutionState`1 executionState, Microsoft.WindowsAzure.Storage.Core.Util.StreamDescriptor streamCopyState) [0x00000] in <filename unknown>:0
Comment 5 Ludovic Henry 2017-09-06 13:53:18 UTC
Does this still reproduce with latest Mono? The threadpool implementation has changed drastically so the bug have certainly been fixed. Thank you

Note You need to log in before you can comment on or make changes to this bug.