When scanning the registers+stack for an SGEN collection, we do not currently scan the YMM (AVX) registers on any Intel platform.
We do scan / intend to scan the XMM registers ( https://github.com/mono/mono/pull/4127 , https://bugzilla.xamarin.com/enter_bug.cgi?product=Runtime ), but that patch has never supported AVX.
We have no evidence of any compiler on any platform using AVX to hold pointers. However, if compilers are using XMM, it is plausible they could be using YMM also either now or in future. For completeness we should eventually scan YMM also. My understanding (?) is XMM is a subset of YMM so a YMM scan if done correctly could possibly replace the XMM scan.
Marking this C10 because (unlike XMM) we have no evidence it is a current problem.
Intel's Cannondale will ship next year with AVX-512 which increases width to 512bits and the number of register to 32.
That's an additional 2k of data to copy and scan per thread.
It looks like our only sane option is for us to speed up coop.
It seems unfathomable we're the ones building libmono yet there's no way for us to determine whether the compilation our own build scripts execute are spilling into SIMD regs.